Simulation software claimed to predict effect of management choices on ranking position

In a research project for the Higher Education Evaluation and Accreditation Council for Taiwan (HEEACT), Han-Lin Li, a professor at the National Chiao Tung University, has developed what he describes as “a rank simulation system for world universities”.

The software will eventually be made available worldwide, possibly on a commercial basis.

In a presentation on 3 November at a conference in Taipei, titled “International trends in university rankings and their impact on higher education policy”, Professor Li said his system could help a university to “allocate its resources optimally, to improve its rank”.

It could even model the impact of institutional mergers on ranking positions, he said.

“We need a system to help us know what kind of strategy we can use to get on the [rankings] list,” he told the audience of senior Taiwanese university administrators.

At present, the software can simulate the position of universities in Shanghai Jiao Tong University’s Academic Ranking of World Universities and HEEACT’s Performance Ranking of Scientific Papers for World Universities. It has not yet been able to model the Times Higher Education World University Rankings, as its new methodology was developed only in 2010 and there are insufficient historical data.

Professor Li demonstrated a simulation based on the Shanghai Jiao Tong ranking, which used six indicators, including the number of staff with a Nobel prize or Fields medal for mathematics, and the number of highly cited researchers.

He showed how an institution could see how much the recruitment of a single Nobel prizewinner could change its ranking position.

Institutions could also decide whether it would be better, in terms of a ranking position, to allocate resources to the hiring of highly cited researchers from other institutions, to focus on having a single article published in Science or Nature, or to provide incentives to staff to publish more in prominent journals.

The package allows users to compare themselves with competitors.

“We are interested in how we can assist the university to inform their strategy,” Professor Li said. “You can form your own strategy to win.”

Professor Li was asked by Times Higher Education, which presented details of its new World University Rankings methodology at the event, whether he is concerned that such a tool would encourage universities to focus on their rankings position against the interests of genuine improvement, to the detriment of real local and national needs.

He agreed that this was a concern. “It is very dangerous,” he joked. “It is just a simulation system, you need to specify your constraints carefully and I believe that each university [should have] their own strategy, a bit different from each other.”

In response to a question on the accuracy of the tool, Professor Li warned that it is only predictive. “We can give no guarantees about your success. It is a small research project, for what might be possible.”

A representative of HEEACT said that the tool is in an early stage of development and that more testing and discussions with software companies needed to take place before it is made widely available.

Times Higher Education presented a paper on its decision to produce a new ranking based on 13 indicators, instead of the six indicators used under a previous system between 2004 and 2009.

The move to a large number of indicators is designed to provide a better sense of the broad range of a university’s activities, as well as to reduce the scope for universities to manipulate the data, or change their strategic direction in arbitrary ways, in order to gain an advantage in the rankings.

phil.baty@tsleducation.com