By tweaking Times Higher Education’s rankings methodology, the 100 Under 50 provides a more accurate reading of the world’s best young universities
The Times Higher Education 100 Under 50 uses the same range of 13 performance indicators to rate institutions as the overarching World University Rankings published every autumn, but with a major difference: the weighting placed on subjective indicators of academic prestige has been reduced.
That given to our 11 objective performance indicators has been increased accordingly, although the weighting of our five headline categories remains the same:
- Research: volume, income and reputation (30 per cent)
- Citations: research influence (30 per cent)
- Teaching: the learning environment (30 per cent)
- International outlook: people and research (7.5 per cent)
- Industry income: innovation (2.5 per cent)
The decision to reduce the reputation weighting was the result of feedback from our “platform group” of more than 50 expert advisers from around the world.
It was argued that older institutions can expect in general to enjoy greater global prestige based on their longer heritage of excellence. Older universities have deeper, wider and more established alumni networks, with graduates more likely to hold senior positions in universities and society at large, all of which can bolster their reputations.
It was agreed that an analysis of younger institutions, designed to examine future potential as much as current excellence and to move away from heritage or legacy, should be based more on hard, objective performance indicators.
Therefore, the weighting given to reputation has been reduced from a third in the World University Rankings to just over a fifth in the 100 Under 50.
Research: volume, income and reputation
This category is made up of three indicators. First is a simple measure of a university’s research volume, scaled for institutional size, to give a sense of its productivity.
We count the number of papers published in the academic journals indexed by Thomson Reuters per academic staff member to give a clear picture of each institution’s ability to get papers published in quality peer-reviewed journals.
This indicator is worth 9 per cent overall, up from 6 per cent in the World University Rankings, reflecting the reduced weight given to the reputation measures.
This category also looks at university research income, scaled against staff numbers and normalised for purchasing-power parity and for each university’s distinct subject profile. This indicator reflects the fact that research grants in science subjects are often bigger than those awarded for the highest-quality social science, arts and humanities research. This indicator is also weighted at 9 per cent, up from 6 per cent in the World University Rankings.
The final indicator in this category is based on the most recent results of our annual reputation survey. Thomson Reuters carried out its Academic Reputation Survey – a worldwide poll of experienced scholars – in spring 2013 (the 2014 poll has just closed and its data will be used to inform the World University Rankings 2014-15, to be published this autumn).
The poll examined the perceived prestige of institutions in both research and teaching, and the results for research are used in this category. There were 10,536 responses, statistically representative of global higher education’s geographical and subject mix.
The research reputation indicator remains the most dominant measure in this category, despite a reduction in its weighting from 18 per cent in the overarching rankings to 12 per cent in the 100 Under 50 analysis.
Citations: research influence
In this indicator, we examine a university’s research influence by capturing the number of times its published work is cited by scholars around the world.
Worth 30 per cent of the overall score, this single indicator is the largest of the 13 employed to create the table – and its weighting remains identical to that employed in the World University Rankings.
The data are drawn from the 12,000 academic journals indexed by Thomson Reuters’ Web of Science database and include all indexed journals published in the five years between 2007 and 2011.
Citations made in the six years between 2007 and 2012 are also collected, thus improving the stability of the results and decreasing the impact of exceptionally highly cited papers on institutional scores.
The findings are fully normalised to reflect variations in citation volume between different subject areas. As a result, institutions with high levels of research activity in subjects with traditionally high citation counts do not gain an unfair advantage.
For institutions with relatively few papers, citation impact may be significantly boosted by a small number of highly cited papers, so only those institutions that have published at least 200 papers a year are included.
Teaching: the learning environment
This category employs five separate performance indicators designed to provide a clear sense of the teaching and learning environment of each institution from both the student and scholarly perspective.
Despite a reduction in weighting from the World University Rankings, the main indicator in this category is still based on the Academic Reputation Survey 2013.
The results of the survey with regard to teaching make up 10 per cent of the 100 Under 50 – down from 15 per cent in the World University Rankings.
Our teaching and learning category also employs a staff-to-student (total student numbers) ratio as a simple proxy for teaching quality – suggesting that where there is a low ratio of students to staff, the former will get the personal attention they require from faculty members.
It is worth 6 per cent of the 100 Under 50 score – up from 4.5 per cent in the World University Rankings to help fill the gap left by reputation’s reduced importance.
The teaching category also examines the ratio of PhD to bachelor’s degrees awarded by each institution. We believe that institutions with a high density of research students are more knowledge-intensive, and that the presence of an active postgraduate community is a marker of a research-led teaching environment valued by undergraduates and postgraduates alike.
The PhD-to-bachelor’s ratio is worth 3 per cent of the 100 Under 50 scores (up from 2.25 per cent).
This category also uses data on the number of PhDs awarded by an institution, scaled against its size as measured by the number of academic staff.
As well as giving a sense of how committed an institution is to nurturing the next generation of academics, a high proportion of postgraduate research students also suggests the provision of teaching at the highest level that is attractive to graduates and good at developing them.
Undergraduates also tend to value working in a rich environment that includes postgraduates.
The indicator makes up 8 per cent of the score (up from 6 per cent in the World University Rankings).
The final indicator in the teaching category is a simple measure of institutional income scaled against academic staff numbers.
This figure, adjusted for purchasing-power parity so that all nations compete on a level playing field, indicates the general status of an institution and gives a broad sense of the infrastructure and facilities available.
This measure is worth 3 per cent, a marginal increase from the World University Rankings figure (2.25 per cent).
International outlook: people and research
Our international category looks at diversity on campus and how much each university’s academics collaborate with international colleagues on research projects – signs of how global an institution is in its outlook. This category is unchanged from the World University Rankings.
The ability of a university to compete in a global market for undergraduates and postgraduates is key to its success on the world stage; this factor is measured here by the ratio of international to domestic students.
This is worth 2.5 per cent of the 100 Under 50 list’s overall score.
As with competition for students, the top universities also operate in a tough market for the best faculty. So in this category we give a 2.5 per cent weighting to the ratio of international to domestic staff.
We also calculate the proportion of each university’s total research journal publications with at least one international co-author and reward the higher volumes.
This indicator, which is also worth 2.5 per cent, is normalised to account for a university’s subject mix and uses the same five-year window that is employed in the “Citations: research influence” category.
Industry income: innovation
A university’s ability to reinforce industry with innovations, inventions and consultancy has become such an important activity that it is often known as its “third mission”, alongside teaching and research.
This category seeks to capture such knowledge transfer by looking at how much research income an institution earns from industry, scaled against the number of its academic staff.
It suggests the extent to which businesses are willing to pay for research and a university’s ability to attract funding in the competitive commercial marketplace – key indicators of quality.
However, because the figures provided by institutions for this indicator are relatively patchy, we have given it a low weighting: just 2.5 per cent.
Weights and measures
Universities are excluded from the 100 Under 50 list on the same basis as they are excluded from the World University Rankings: if they do not teach undergraduates; if they teach only a single narrow subject; or if their research output amounted to fewer than 1,000 articles between 2007 and 2011 (200 a year).
In some exceptional cases, institutions below the 200-paper threshold are included if they have a particular focus on disciplines with generally low publication volumes, such as engineering or the arts and humanities.
Scores
To calculate the overall rankings, “Z‑scores” were created for all datasets except for the results of the Academic Reputation Survey.
The calculation of Z-scores standardises the different data types on a common scale and allows fair comparisons between different types of information – essential when combining diverse information into a single ranking.
Each data point is given a score based on its distance from the mean average of the entire dataset, where the scale is the standard deviation of the dataset. The Z-score is then turned into a “cumulative probability score” to arrive at the final totals.
If University X has a cumulative probability score of 98, then a random institution from the same data distribution will fall below the institution 98 per cent of the time.
For the results of the reputation survey, the data are skewed in favour of a small number of institutions at the top of the rankings, so we add an exponential component to increase differentiation between institutions lower down the scale.
For the 100 Under 50, it was decided that the foundation date of a university shall be either:
• The year it was founded (if it was purpose-built as a university)
• The date it attained degree-awarding powers (if the institution has changed status from another type of institution).
Much editorial judgement has been applied to take account of individual circumstances. Here we outline our approach to some common scenarios.
Mergers: The date of foundation is not the date of the merger or the renaming of the institution, but the date of the foundation of the dominant component in the merger. If that is unclear, we take the foundation date of the oldest component of the merger.
Demergers and spin-offs: If an institution that was once a campus or branch of a larger institution has become independent, we use the foundation date of the parent institution. If a merger is combined with an expansion to create a new institution, local feedback is used to make a judgement.
Institutions without degree-awarding powers: If the institution lacks degree-awarding powers (if it is, say, part of a larger federal system or group of universities such as the University of London in which the parent system awards degrees), then the foundation date is either: the date the institution was founded, if purpose-built as a university; or the date the first degree was awarded if it has changed from non-university status.