Conference article

University Performance Evaluation Approaches

Loukas N. Anninos
Department of Business Administration, University of Piraeus, Greece

Download article

Published in: 10th QMOD Conference. Quality Management and Organiqatinal Development. Our Dreams of Excellence; 18-20 June; 2007 in Helsingborg; Sweden

Linköping Electronic Conference Proceedings 26:111, p.

Show more +

Published: 2008-02-15

ISBN:

ISSN: 1650-3686 (print), 1650-3740 (online)

Abstract

During the last years; there has been extensive argumentation regarding university accountability; the evaluation of their performance (in the educational and administrative operation) and the publication of results with a view to more objective decision making (Ewell;1999; Banta; Borden;1994; Fuhrman;1999; 2003; Pounder; 1999; Wakim. Bushnell;1999; King; 2000; Goertz; Duffy; 2001; Welsh; Dey; 2002; Welsh; Metcalf; 2003; Bolton; 2003 Black; Briggs; Keogh; 2001). Decisions may be taken by individuals (eg students) aiming at choosing a university for studies; by the state aiming at a rational base to allocate resources and an imprint of higher education competitiveness or by the institutions themselves aiming at introducing changes and improvement wherever necessary. Moreover; universities do not constitute individualised organizational units but they operate and affect the wider economic and social system in which they belong. They are therefore accountable a) to the academic staff they employ (work in a suitable working climate and have great opportunities for scientific advancement); b) to the state (use of resources productively (efficiently and effectively) and c) the students and the society (comprehensive educational experience; scientific education and professional training to acquire quality of life) (Vidovich and Slee; 2001; Löfström; 2002; Corbett; 1992). Consequently; the evaluation of their performance proves to be a highly significant process for university institutions with many receivers of its results.

Baring in mind; a) university accountability towards the state and all stakeholders (that engage in institutional goal setting and operation and are influenced by their results); b) globalization that encourages the mobility of academic staff and students and hence stresses the need for international comparability of higher education systems; study programmes and degrees; c) the European objectives to create the area of European Higher Education which presupposes evaluation; d) global competition of higher education institutions in order to create attractive educational multi-cultural environments and the trend towards university collaboration and e) the opportunity to improve and eliminate institutional weaknesses; universities should have a suitable and reliable management system with processes and mechanisms of performance measurement that would allow comparisons and improvement (Wakim; Bushnell;1999; Pounder;1999; Al-Turki; Duffuaa; 2003; Diamond; 2002; Meyerson; Massy;1994; Welsh; Metcalf; 2003).

The effective management of any higher education system presupposes evaluation of results in institutional; departmental and study programme level. So; many evaluation approaches have been developed and successfully implemented globally with similarities and differences. Therefore; the present paper aims to present the different university performance evaluation approaches used internationally and examine the scientific correctness and suitability of the most common ranking systems; based on the presentation of issues regarding ranking systems that are raised in literature.

Keywords

Performance; Evaluation; Higher Education; Ranking Systems

References

Abbott; M.; Doucouliagos; C.; (2002); The efficiency of Australian universities: a data envelopment analysis; Economics of Education Review; Vol.22; No.1; pp.89-97

Alstete J W;(1995); Benchmarking in higher Education: Adapting best practices to improve quality; ASHE-ERIC Higher Education Reports No 5; The George Washington University

Al-Turki; U; Duffuaa; S.;(2003); Performance measures for academic departments; The International Journal of Educational Management; 17/7; pp.330-338

Banta; T.; Borden; V.; (1994); Performance indicators for accountability and improvement; New Directions for Institutional Research; No.82; pp.95-106

Black; S; Briggs; S; Keogh; W.;(2001); Service quality performance measurement in public / private sectors; Managerial Auditing Journal; 16/7; pp.400-405

Bolton; M.;(2003); Public sector performance measurement: delivering greater accountability; Work Study; 52/1; pp.20-24

Breu; T.M. and Raab; R.L.; (1994);Efficiency amd perceived quality of the nation`s Top 25 national universities and national liberal arts colleges: an application of DEA to higher education; Socio-Economic Planning Sciences; Vol.28; No.1; pp.35-45

Clarke; M.;(2002); Quantifying quality: what can the US news and World Report rankings tell us about the quality of Higher Education; Education Policy Analysis Archives; Vol.10; No.6

Corbett; D.; (1992); Australian public sector management (2nd ed) Sydney in Vidovich; L. and Slee; R.; (2001); Bringing universities to account?; Exploring some global and local policy tensions; Journal of Education Policy; Vol.16; No.5; pp.432

Danish Evaluation Institute; (2003); Quality procedures in European higher education; ENQA

Di Nauta; P.; Omar; P.J.; Schade; A.; Scheele; J.P.; (Eds); (2004); Accreditation models in higher education; ENQA

Diamond; R.M.;(2002); Field guide to academic leadership; Jossey Bass; pp. 225-240

Dichev; I.;(1999); How good are business school rankings?; The Journal of Business; Vol.72; No.2; pp.201-213

Dill; D and Soo; M.;(2004); Is there a global definition of academic quality? A cross national analysis of university ranking systems; Public Policy for Academic Quality; Background Paper

Ehrenberg; R.;(2003); Method or madness? Inside the USNWR college rankings; Paper presented at the Wisconsin Center for the Advancement of Postsecondary Education Forum on the Use and Abuse of College Rankings; ERIC

Epper; R.; (1999); Applying benchmarking to higher education; Change; pp.24-31

Ewell; P.T.;(1999); Linking performance measures to resource allocation: exploring unmapped terrain; Quality in Higher Education; Vol.5; No.3; pp..191-208

Fuhrman; S.H.;(1999); The new accountability; CPRE Policy Brief

Furhman; S.H.;(2003); Redesigning accountability systems for education; CPRE Policy Brief

Gander; J.P. (1995); Academic research and teaching productivities: a case study; Technological Forecasting and Social Change; Vol.49; pp.311-319

Gater; D.;(2003); Using national data in university rankings and comparisons; The Center

Gioia; D. and Corley; K.;(2002); Being good versus looking good: business school rankings and the Circean transformation from substance to image; Academy of Management Learning and Education; Vol.1; Issue 1

Goertz; M.E; Duffy; M.C.; (2001); Assessment and accountability across the 50 states; CPRE Policy Brief

Gose; B.;(1999); A new survey of “good practices” could be an alternative to rankings; Chronicle of Higher Education; Vol.49; Issue 9

Gottlieb; B.;(1999); Cooking the school books: How US News cheats in picking its best American colleges; Online

Hämäläinen; K.; Pehu-Voima; S.; Wahlen; S.; (2001); Institutional evaluations in Europe; ENQA

Hämäläinen; K.; Hämäläinen; K.; Jessen; A; Kaartinen-Koutaniemi;M.; Kristoffersen; D.; (2002); Benchmarking in the Improvement of Higher Education; European Network for Quality Assurance in Higher Education; p.7

Hattendorf; L.C.; (1996); Educational rankings of higher education: fact or fiction?; Paper presented at the International Conference on Assessing Quality in Higher Education; Queensland; Australia

Hossler; D.;(2000); The problem with college rankings; About Campus; Vol.5; No.1; pp.20-24

Johnes; J and Johnes; G.; (1995); Research funding and performance in UK university departments of economics: a frontier analysis; Economics of Education Review; Vol.14; pp.301-314

Johnes; J.; (1996); Performance assessment in higher education in Britain; European Journal of Operational Research; Vol.89; pp.18-33

Kao; C.; (1994); Evaluation of junior colleges of technology: the Taiwan case; European Journal of Operational Research; Vol.73; pp.487-492

King; A.;(2000); The changing face of accountability: monitoring and assessing institutional performance in higher education; The Journal of Higher Education; Vol.71; No.4;pp.411-431

Kocher; M.G .; Luptacik; M. and Sutter; M.; (2006); Measuring productivity of research in economics: a cross-country study using DEA; Socio-Economic Planning Sciences; Vol.40; pp.314-332

Löfström; E.; (2002); In search of methods to meet accountability and transparency demands in higher education: experience from benchmarking; Socrates Intensive Programme “Comparative Education Policy Analysis” Lake Bohinj; Slovenia; August pp 21-30

Lombardi; J.;Graig.; D.; Capaldi; E.;Gater; D.;(2000); The myth of number one: indicators of research university performance; The Center

Lombardi; J.;Graig.; D.; Capaldi; E.;Gater; D.;Mendonca; S.;(2001); Quality engines: the competitive context for research universities; The Center

MacGowan; B.;(2000); Those magazine rankings: lets beg them to stop; Professional School Counselling; Vol.3; Issue.4

Machung; A.;(1998); Playing the rankings game; Change; Vol.30; No.4; pp.12-16

Merisotis; J.; (2002); Summary report on the invitational roundtable on statistical indicators for the quality assessment of higher / tertiary education institutions: rankings and league table methodologies; Higher Education in Europe; Vol.27; No.4;pp.475-480

Meyerson; J.;Massy; W.;(1994); Measuring institutional performance in higher education; Peterson`s

Monks; J.; Ehrenberg; R.; (1999); US News and World Report Rankings: Why they do matter?; Change; Vol.31; pp.43-51

Muniz; M.; (2002); Seperating managerial inefficiency and external conditions in data envelopment analysis; European Journal of Operational Research; Vol. 143; pp.625-643

NORC (1997); A review of the methodology for the US News & World Report`s rankings of undergraduate colleges and universities; Washington Monthly

Post; T. and Spronk; J.; (1999); Performance benchmarking using interactive data envelopment analysis; European Journal of Operational Research; Vol.115; pp.472-487

Pounder; J.; (1999); Institutional performance in higher education: is quality a relevant concept? ; Quality Assurance in Education; Vol.7; Issue 3; pp.156-165

Provan; D.; Abercromby; K.;(2000); University league tables and rankings: a critical analysis; CHEMS;

Rickards; R.C.; (2003); Setting benchmarks and evaluating balanced scorecards with data envelopment analysis; Benchmarking: an International Journal; Vol.10; No.3; pp.226-245

Ridley; D.;Cuevas; M.;Matveev; A.;(2001); Transitions between tiers in US News and World Report rankings of colleges and universities; ERIC

Ruggiero; J.; Miner; J.; Blanchard; L.; (2002); Measuring equity of educational outcomes in the presence of efficiency; European Journal of Operational Research; Vol..142; pp.642-652

Sarrico; C.S.; Hogan; S.M.; Dyson; R.G. and Athanassopoulos; A.D.; (1997); Data envelopment analysis and university selection; Journal of the Operational Research Society; Vol.48;pp.1163-1177

Siemens; J.; Burton; S.; Jensen; T.; Mendoza; N.;(2005); An examination of the relationship between research productivity in prestigious business journals and popular press business school rankings; Journal of Business Research; Vol.58;pp.467-476

Sinuany-Stern; Z.; Mehrez; A.; Barboy; A.; (1994); Academic departments efficiency via DEA; Computers and Operations Research; Vol.21; pp.543-556

Stuart; D.; (1995); Reputational rankings: background and development; New Directions for institutional research; 88; pp.17-19

Thomson; N.;(2000); Cooking the school books (yet again). The US News college rankings gets phonier and phonier; On line

Trieschmann; J.; Dennis; A.; Nothcraft; G.; NIemi;A.;(2000); Serving multiple constituents in the business school: MBA program vs. research performance; Academy of Management Journal

Usher; A. and Savino; M.; (2006); A world of difference: a global survey of university league tables; Educational Policy Institute; Ontario; Canada

Van Dyke; N.; (2005); Twenty years of university report cards; Higher Education in Europe; Vol.30; No.2;

Van Raan; A.; (2005); Fatal attraction: conceptual and methodological problems in the ranking of universities by Bibliometric methods; Scientometrics; Vol.62; No1; pp.140-141

Vaugn; J.; (2002); Accreditation; commercial rankings and new approaches to assessing the quality of university research and education programmes in the United States; Higher Education in Europe; Vol.27; No.4; pp. 435-436

Vidovich; L. and Slee; R.; (2001); Bringing universities to account?; Exploring some global and local policy tensions; Journal of Education Policy; Vol.16; No.5; pp.431-432

Vlasceanu; L.; Grunberg; L.; Parlea; D.; (2004); Quality Assurance and Accreditation: a Glossary of Basic Terms and Definitions; UNESCO; pp.48-49

Wakim; N.; Bushnell; D.; (1999); Performance evaluation in a campus environment; National Productivity Review; Winter; pp.19-27

Walpole; M.;(1998); National rankings: ramifications for Academic Units; Paper presented at the Annual Meeting of the American Educational Association; ERIC

Webster; D.;(1992); Are they any good? Change; Vol.24; Issue 2

Webster; T.;(2001); A principal component analysis of the US News & World Report tier rankings of colleges and universities; Economics of Education Review; Vol.20; pp.235-244

Welsh; J.; Metcalf; J.; (2003); Administrative support for institutional effectiveness activities: responses to the new accountability; Journal of Higher Education Policy and Management; Vol.25; No.2; pp.183-192

Welsh; J.F.; Dey; S.;(2002); Quality measurement and quality assurance in higher education; Quality assurance in Education; Vol.10; No.1; pp.17-25

Citations in Crossref