100 Under 50 Rankings 2013 methodology

The Times Higher Education 100 Under 50 uses the same range of 13 performance indicators to rate institutions as the overall World University Rankings published every autumn but with a major ­difference: the weighting placed on subjective ­indicators of academic prestige has been reduced.

That given to our 11 objective performance indicators has been increased accordingly, although the weighting of our five headline performance ­categories remains the same:

• Research: volume, income and reputation (30 per cent)
• Citations: research influence (30 per cent)
• Teaching: the learning environment (30 per cent)
• International outlook: people and research (7.5 per cent)
• Industry income: innovation (2.5 per cent)

The decision to reduce the importance of reputation in the analysis was the result of feedback from our “platform group” of more than 50 expert advisers from around the world.

It was argued that older institutions can expect in general to enjoy greater global prestige based on their longer heritage of excellence. Older univer­sities have deeper, wider and more established alumni networks, with graduates more likely to hold senior positions in universities and society at large, all of which can bolster their reputations.

It was agreed that an analysis of younger institutions, designed to examine future potential as much as current excellence and to move away from heritage or legacy, should be based more on hard, objective ­performance indicators.

Therefore, the weighting given to ­reputation has been reduced from a third in the World University Rankings to just over a fifth in the 100 Under 50.

Research: volume, income and reputation
This category is made up of three ­indicators. First is a simple measure of a university’s research volume, scaled for institutional size, to give a sense of its productivity. We count the number of papers published in the ­academic journals indexed by ­Thomson Reuters per academic staff member to give a clear picture of each institution’s ­ability to get papers published in ­quality peer-reviewed journals.

This indicator is worth 9 per cent overall, up from 6 per cent in the World University Rankings, reflecting the reduced weight given to the reputation measures.

This category also looks at university research income, scaled against staff numbers and normalised for ­purchasing-power parity and for each university’s distinct subject profile. This indicator reflects the fact that research grants in science subjects are often bigger than those awarded for the highest-quality social science, arts and humanities research. This indicator is also weighted at 9 per cent, up from 6 per cent in the World University Rankings.

The final indicator in this category is based on the most recent results of our annual reputation survey. Thomson Reuters carried out its Academic ­Reputation Survey – a worldwide poll of experienced scholars – in spring 2012 (the 2013 poll has just closed and its data will be used to inform the World University Rankings 2013-14, to be published this autumn).

The poll examined the perceived prestige of institutions in both research and teaching, and the results for research are used in this category. There were 16,639 responses, ­statis­tically representative of global higher education’s geo­graphical and subject mix.
The research reputation indicator remains the most dominant measure in this category, despite a reduction in its weighting from 18 per cent in the overarching rankings to 12 per cent in the 100 Under 50 analysis.

Citations: research influence
In this indicator, we examine a university’s research influence by capturing the number of times its published work is cited by scholars around the world.

Worth 30 per cent of the overall score, this single indicator is the ­largest of the 13 employed to create the table – and its weighting remains identical to that employed in the World University Rankings.

The data are drawn from the 12,000 academic journals indexed by Thomson Reuters’ Web of Science database and include all indexed ­journals published in the five years between 2006 and 2010.

Citations made in the six years between 2006 and 2011 are also ­collected, thus improving the ­stability of the results and decreasing the impact of ­exceptionally highly cited papers on institutional scores.

The findings are fully normalised to reflect variations in citation volume between different subject areas. As a result, institutions with high levels of research activity in subjects with ­traditionally high citation counts do not gain an unfair advantage.

For institutions with relatively few papers, citation impact may be ­significantly boosted by a small ­number of highly cited papers, so only those institutions that have published at least 200 papers a year are included.

Teaching: the learning environment
This category employs five separate performance indicators designed to ­provide a clear sense of the teaching and learning environment of each ­institution from both the student and scholarly perspective.

Despite a reduction in weighting from the World University Rankings, the main indicator in this ­category is still based on the Academic Reputation Survey 2012.

The results of the survey with regard to teaching make up 10 per cent of the 100 Under 50 – down from 15 per cent in the World University Rankings.

Our teaching and learning category also employs a staff-to-student (total student numbers) ratio as a simple proxy for teaching quality – suggesting that where there is a low ratio of ­students to staff, the former will get the personal attention they require from faculty members.

It is worth 6 per cent of the 100 Under 50 score – up from 4.5 per cent in the World University Rankings to help fill the gap left by reputation’s reduced importance.

The teaching category also examines the ratio of PhD to bachelor’s degrees awarded by each institution. We believe that institutions with a high density of research students are
more knowledge-intensive, and that the ­presence of an active postgraduate community is a marker of a research-led teaching environment valued by undergraduates and postgraduates alike.

The PhD-to-bachelor’s ratio is worth 3 per cent of the 100 Under 50 scores (up from 2.25 per cent).

This category also uses data on the number of PhDs awarded by an institution, scaled against its size as measured by the number of academic staff.

As well as giving a sense of how committed an institution is to nurturing the next generation of academics, a high proportion of postgraduate research students also suggests the ­provision of teaching at the highest level that is attractive to graduates and good at developing them.

Undergraduates also tend to value working in a rich environment that includes postgraduates.

The indicator makes up 8 per cent of the score (up from 6 per cent in the World University Rankings).

The final indicator in the teaching category is a simple measure of ­institutional income scaled against academic staff numbers.

This figure, adjusted for ­purchasing-power parity so that all nations compete on a level playing field, ­indicates the general status of an ­institution and gives a broad sense of the infra­structure and facil­ities ­available.

This measure is worth 3 per cent, a marginal increase from the World University Rankings figure (2.25 per cent).

International outlook:people, research
Our international category looks at diversity on campus and how much each university’s academics collaborate with international colleagues on research projects – signs of how global an institution is in its outlook. This ­category is unchanged from the World University Rankings.

The ability of a university to ­compete in a global market for undergraduates and postgraduates is key to its success on the world stage; this ­factor is measured here by the ratio of international to domestic students.

This is worth 2.5 per cent of the 100 Under 50 list’s overall score.

As with competition for students, the top universities also operate in a tough market for the best faculty. So in this category we give a 2.5 per cent weighting to the ratio of international to domestic staff.

We also calculate the proportion of each university’s total research journal publications with at least one inter­national co-author and reward the higher volumes.

This indicator, which is also worth 2.5 per cent, is normalised to account for a university’s subject mix and uses the same five-year window that is employed in the “Citations: research influence” category.

Industry income: innovation
A university’s ability to reinforce ­industry with innovations, inventions and consultancy has become such an important activity that it is often known as its “third mission”, alongside ­teaching and research.

This category seeks to capture such knowledge transfer by looking at how much research income an institution earns from industry, scaled against the number of its academic staff.

It suggests the extent to which ­businesses are willing to pay for research and a university’s ability to attract funding in the competitive ­commercial marketplace – key indi­cators of quality.

However, because the figures ­provided by institutions for this ­indicator are relatively patchy, we have given it a low weighting: just 2.5 per cent.


For the 100 Under 50, it was decided that the foundation date of a university shall be either:

• The year it was founded (if it was purpose-built as a university)

• The date it attained degree-awarding powers (if the institution has changed status from another type of institution).

Much editorial judgement has been applied to take account of individual circumstances. Here we outline our approach to some common scenarios.

Mergers: The date of foundation is not the date of the merger or the renaming of the institution, but the date of the foundation of the dominant component in the merger. If that is unclear, we take the foundation date of the oldest component of the merger.

Demergers and spin-offs: If an institution that was once a campus or branch of a larger institution has become independent, we use the foundation date of the parent institution. If a merger is combined with an expansion to create a new institution, local feedback is used to make a judgement.

Institutions without degree-awarding powers: If the institution lacks degree-awarding powers (if it is, say, part of a larger federal system or group of universities such as the University of London in which the parent system awards degrees), then the foundation date is either: the date the institution was founded, if purpose-built as a university; or the date the first degree was awarded if it has changed from non-universi