Highly accomplished outcasts

Source: Alamy

Simon Pratt takes a closer look at the elite graduate schools and specialists excluded from the World University Rankings

University rankings can have far-reaching influence, and a change in ranking position can have a big impact. However, some institutions are excluded from the rankings completely not because of their performance, but rather because of their unusual characteristics.

In the Times Higher Education World University Rankings, institutions are excluded for three reasons: low research output; a single or very narrow disciplinary focus; or their status as graduate-only schools.

Robust comparisons of the performance of these institutions against the carefully balanced range of 13 indicators in the THE World University Rankings are challenging, and any attempt to include them in the overall tables would be imperfect and unrepresentative of their real performance.

It is a difficult decision to exclude universities from the rankings, and it is unfortunate that some excellent institutions have to miss out. However, it is impossible to assign these atypical institutions with positions that are meaningful or fair.

Thomson Reuters has worked with THE to help develop specific criteria for excluding universities from the overall rankings based on their data. By developing exclusion criteria, we have made the tables more robust and decreased year-on-year fluctuations in position.

But Thomson Reuters has a comprehensive dataset of hundreds of institutions well beyond the top 400 featured in the rankings, and we can use this data to look in depth at the strengths and weaknesses of all institutions, not just those featured in the tables.

On these pages we highlight the strengths of some world-class institutions that do not feature in the world top 400 list.

Graduate schools, of course, do not teach undergraduates, so they tend to have different demographics from typical universities. It is important to realise that although we can create indicators and indicator groups for such institutions, the data must be put in context and should be used only to make comparisons between similar institutions: holding them up against World University Rankings runners and riders is not recommended.

Among the graduate schools, we can subdivide the institutions into three types: those that concentrate on the social sciences; medical schools; and those institutions that focus on research and research-based teaching. The charts (below) show those institutions that can be considered “world class” – that is, those performing at a similar level to the ones that feature in the World University Rankings.

In general, graduate schools tend towards a narrow disciplinary focus and are relatively small. Logically, a narrow disciplinary focus and small size would limit the number of interactions an institution can have with the global academic community. This suggests that such institutions would be unlikely to excel in the Thomson Reuters academic reputation survey used to help create the THE World University Rankings.

However, despite these disadvantages, the University of California, San Francisco does very well in the survey and was ranked among the top 50 in the world in 2012. Additionally, the London School of Hygiene and Tropical Medicine, London Business School, Israel’s Weizmann Institute of Science and global business school Insead more than hold their own in reputation terms against their larger peers.

Most of these institutions also score well for research paper citations. They excel at their chosen missions, thus ensuring that they are considered among the best in the world by their peers.

In creating the graduate school table, the indicator used in the overall World University Rankings that looks at the ratio of undergraduate to doctoral degrees has been removed, with the weighting of other indicators in the “Teaching: the learning environment” group increased to compensate. However, all the other weightings are identical to the World University Rankings methodology. It is worth noting that these indicators may benefit the graduate schools (for example, their staff-to-student ratios are relatively high).

Specialist universities

Highly specialist institutions are excluded from the World University Rankings because of their very narrow disciplinary focus and/or low total research output, which makes citation analysis difficult. But when the institutions have sufficient output in one of the six specialist areas measured by the THE subject rankings, they are included accordingly. The profiles bring the institutions together in one place so that we can see how they compare.

When preparing the data for the subject tables, we employ methodological variations to make the rankings more robust. The different subjects have their own weighting schemata that are designed to highlight the specific characteristics of each subject area. For example, citation analysis is typically not considered to be as robust an indicator in the arts and humanities as it is in the hard sciences, and therefore its weighting is decreased.

Additionally, each individual indicator is calculated within the context of the subject area. The indicator scores represent where that institution falls compared with everyone else in the same subject discipline, and because the characteristics for the subjects are different, the scores will vary.

For these reasons, the data in the table are a fair representation of the performance within each subject area, but not a representation of how that institution might fare in the overall rankings.

Simon Pratt is product manager for institutional research at Thomson Reuters.

Click image to enlarge