Handle with care

Almost a decade after international university rankings first appeared, the obsession continues. If higher education is the engine of the economy, as political and university leaders like to argue, then its productivity, quality and status are vital indicators. But in the global economy, national pre-eminence is no longer sufficient to ensure success. Thus, rankings have focused attention on the attractiveness of nations and the talent-catching and knowledge-producing capacity of higher education. They have challenged national and institutional presumptions of the "world order" and generated a policy panic, with policymakers drawing simple correlations between league table positions, elite higher education and global competitiveness.

Over the years, governments around the world have embarked on massive programmes of higher education reform. Germany launched its Exzellenzinitiative in 2005, Malaysia presented its Higher Education Action Plan in 2007 and France kicked off Operation Campus in 2008. Taiwan, South Korea, India, Indonesia, Denmark and Japan - to name but a few - have introduced similar initiatives, restructuring their systems and institutions to match indicators identified by the rankings. Even countries with fewer resources are caught in the maelstrom: Nigeria, Sri Lanka and Vietnam worry about how rankings portray their universities and colour wider views of their countries.

League tables influence policy in other ways. A US study claims a strong correlation between ranking positions and state funding per student. Macedonia's Law on Higher Education automatically recognises qualifications from top 500 universities; Mongolia, Qatar and Kazakhstan restrict scholarships to students who win admission to the top 100 institutions, while Dutch immigration law prioritises for entry foreigners with qualifications from the top 150 universities. Singapore's Foreign Specialist Institute allows only the top 100 universities to collaborate with local institutions.

Of course, institutions are not immune from the impact of international comparisons. Universities around the world have adopted strategic plans and set targets to better align themselves with the rankings. A 2006 international survey revealed that 63 per cent of higher education leaders made strategic, organisational, managerial or academic decisions based on rankings; 50 per cent used them for publicity and official presentations; 50 per cent used rankings to monitor the performance of peer institutions, with 40 per cent considering an institution's rank before entering into discussions with them. Only 8 per cent took no action based on global comparisons.

League tables can serve a variety of purposes. Institutions may use them:

- as an explicit goal, for example, to gain or maintain a position within rankings

- as an implicit goal, for example, in declaring the aim to be "world class" or in the top tier

- as a performance indicator to measure achievement and set specific targets

- as a measure of success, for example, to validate particular strategies or actions.

International evidence shows that rankings are influencing universities' recruitment strategies and students' choices. US and UK studies repeatedly highlight how universities have adjusted entry requirements in order to raise the "quality" of students because of the knock-on consequences for reputation and application levels. High-achieving international students are most likely to use rankings; and engineering, business and science students refer to them more than other groups. Industry collaborators, employers and philanthropists also use rankings to inform decision-making.

Academics are not innocent victims in the rating game. They use rankings to identify partners and to help select research students. And, because careers are tied to institutional reputations, there is a chorus of approval applauding the validity of various indicators.

Questions are asked about whether rankings measure what we think they measure; this refers to the fact that they concentrate on research and pay lip service to education. It may be more important to ask whether rankings measure what counts. There is no objective set of criteria or weightings. They do not elucidate a basic truth; rather, the choice of indicators reflects the ranker's view of what is important.

The absence of internationally comparable and verifiable data skews rankings towards indicators and proxies that are, at best, imperfect measurements. Even in relation to scientific research, they can do damage, embracing a traditional concept of knowledge and its impact. There is no evidence that the "new kids on the block" - the European Commission-funded U-Multirank project and the partnership between Times Higher Education and Thomson Reuters - can overcome these or other fault-lines.

Despite all that, rankings have already made an impact on policy and institutional decision-making, enforcing three policy trends:

- accountability and transparency, which has led to the reification of indicators and proxies

- internationalisation and the "battle for talent", encouraging the adulation of particular types of academic output

- world-class excellence, a mantra underpinning the fetishisation of "world-class universities". This is usually accompanied by the demand to concentrate resources in fewer universities, what David Currie of the University of Ottawa calls the "Sheriff of Nottingham" model (for money is taken from the poor to fund the already rich). The public-policy imperative is lost in the (self-interested) belief that elite research universities have a bigger impact on society and the economy, or have higher quality.

Arguably, it is not rankings per se but their interpretation that is at fault - after all, most are commercial enterprises. The history of rankings shows that measuring the wrong things can produce distortions and perverse actions. The message to governments, institutions and individuals: caveat emptor.