A methodology to rate Chinese subjects

How can we expand on our World University Rankings to provide detailed analysis of university performance in China and beyond?

Rather than building a methodology that stands alone, we think it’s important that the methodology links closely to the World University Rankings – not least because it allows us to explore Chinese universities in the context of 1,361 universities from other systems.

So let’s start by thinking about the data sources for the World University Rankings. There are three separate sources of data:

• data provided directly by universities
• reputation data from our survey of academics across the world
• bibliometrics data on publications and citations from research

These data are provided at different levels of detail – for the university-provided data in 11 broad subject areas, and for the bibliometrics and reputation data at a much more detailed level. Where we don’t have the data at a sufficiently detailed level, we use data from the broader category into which a subject falls.

A detailed subject methodology

These data give us the ability to adapt our existing methodology. We start with the same five areas – teaching, research, citations, industry income, and internationalism.  These provide a unique broad view of performance, rather than focusing solely on bibliometrics.

Within the teaching area, we simplify the five metrics of the World University Rankings into three: teaching reputation; staff-to-student ratio; and the ratio of institutional income to the number of academic staff. This gives us a balance of the resources available to teach and the credibility of teaching as identified by other academics.

For research, we measure research reputation; the ratio of research income divided by number of academic staff; and the number of publications in the subject – a slight change from the World University Rankings, where we measure the publications per academic. But we also add an additional measure – one that indicates the number of related subjects where the university performs strongly. This reflects the interaction of detailed subjects.

Citations remains the field-weighted citation impact measure that we use within the World University Rankings – here using only those subjects from the bibliometric source that map to the appropriate Chinese subject.  For some subjects where publication and citation rates are lower than is acceptable from a statistical standpoint, we will not include citations.

Industry income is the amount of income derived from commerce or industry.

The three international metrics remain the same as in the World University Rankings: the proportion of students who are international; the proportion of academic staff who are international; and the proportion of academic publications that are co-authored with an international academic.

Rating inclusion

Because we are basing the rating on the World University Rankings, inclusion depends on two aspects.

• The breadth of the ratings – the number of subjects to be included – will depend on having sufficient publications in each subject, worldwide, within the worldwide bibliometric dataset.
• The number of universities included from China will depend on participation in the 2020 World University Rankings, and the relevant subject rankings.

Each university that qualifies will be compared and rated against universities from across the world. This is an important feature, and we hope it will give a true reflection of performance, not just within China, but also when compared with competitor universities across the globe.

duncan.ross@timeshighereducation.com