THE Europe Teaching Rankings 2019: methodology

Browse the full Times Higher Education Europe Teaching Rankings 2019 results

The Times Higher Education Europe Teaching Rankings use a balanced scorecard approach, with 14 individual performance indicators combined to create an overall score that reflects the broad strength of an institution.

The metrics, which focus on the student teaching and learning environment and student outcomes, have been designed to answer the questions that matter most to students and their families when making one of the most important decisions of their lives – who to trust with their education.

The performance indicators are grouped into four key areas:

Engagement – does the institution effectively engage with its students?
Resources – does the institution have the capacity to effectively deliver teaching?
Outcomes – does the institution generate appropriate outputs for students?
Environment – is the teaching and learning environment inclusive?

The methodology was modelled on those of our previous teaching-focused rankings: the Wall Street Journal / THE US College Rankings and the THE Japan University Rankings.

At its heart are the results of the THE European Student Survey, which feed into six of the 14 performance indicators and account for 45 per cent of universities’ final scores. The survey examines a range of key issues including students’ engagement with their studies, interaction with teachers, opportunities to develop skills and views on the quality of services at their institution.

Engagement – 35%

• Student engagement: 10%
• Student interaction: 10%
• Student recommendation: 10%
• Links to labour market: 5%

All the data in this pillar are gathered through the THE European Student Survey, carried out in partnership with market research provider Streetbees. The survey collected the views of more than 125,000 university students across 18 European countries on a range of issues relating to their higher education experience.

Students answered nine core questions on a scale of zero to 10 and provided background information about themselves. The survey was conducted online and respondents were recruited by Streetbees using social media and student representatives at individual universities. Respondents were verified as students of their reported institution through their email address and/or IP address.

At least 60 verified survey responses were required for a university to be included in the ranking. The data were rebalanced by gender to reflect the actual gender ratio of students at the institution. The scores were normalised at country/region level to minimise the impact of cultural (country) bias.

To capture engagement with learning, we looked at the answers to four questions: to what extent does the student’s university support critical thinking?; to what extent does the teaching support reflection on, or making connections between, the things that the student has learned?; to what extent does the teaching support apply the student’s learning to the real world?; and, finally, to what extent do the classes taken challenge the student?

To capture students’ opportunity to interact with others, we used responses to two questions: to what extent does the student have the opportunity to interact with faculty and teachers?; and to what extent does the university provide opportunities for collaborative learning?

The student recommendation metric is based on the question: “If a friend or family member were considering going to university, based on your experience, how likely or unlikely are you to recommend your university to them?”

To measure links to the labour market, students were asked to what extent their institution provided opportunities for them to learn about and make connections to the world of work.

Resources – 20%

• Staff-to-student ratio: 7.5%
• Papers-to-staff ratio: 7.5%
• Quality of services: 5%

By looking at the ratio of academic staff members to students (both full-time equivalent), we get an overall sense of whether an institution has enough teachers to teach. It also gives a broad sense of class sizes and how likely it is that a student will receive the individual attention required for them to succeed at university. These data come from institutions’ submissions, institutions’ websites and statistical agencies.

To measure productivity, we count the number of papers published in the academic journals indexed by Elsevier’s Scopus database between 2013 and 2017 per academic staff, scaled for institutional size. Scholars who are experts in their academic fields and engaged in pushing the boundaries of knowledge can significantly enhance a student’s educational experience by demonstrating, for example, the power of real-world problem-solving.

The third metric is based on the average scores of two questions in the THE European Student Survey, which asked students to assess their access to learning materials and the quality of their learning environment.

Outcomes – 20%

• Academic reputation: 10%
• Graduation rate: 5%
• Skills development: 5%

This section looks at universities’ records in delivering successful outcomes for students – an increasingly important mission given the growing concerns in many countries over the affordability and value of higher education.

The most prominent indicator in this category looks at a university’s reputation for teaching excellence among its peers, based on the responses to our 2018 and 2019 THE Academic Reputation Survey. These surveys attracted more than 21,000 total responses. We used only teaching votes provided by academics associated with European institutions.

The graduation rate measures the share of students that graduate within five years, first degrees only. It shows whether institutions within a country have a strong record in supporting students to complete their degree programmes.

The skills development metric is based on a question in the THE European Student Survey, which asked students whether the teaching at their university supported them to learn skills that will be useful in the world of work.

Environment – 25%

• Gender balance of academic staff: 7%
• Gender balance of students: 7%
• Proportion of international students: 5.5%
• Erasmus+ student mobility: 5.5%

This category looks at the gender balance of students and academic staff at a university to give a sense of whether the institution provides a diverse and inclusive learning environment for students.

It also includes two internationalisation metrics. The measure on international students indicates whether a university is able to attract talent from across the world and offer a multicultural campus where students from different backgrounds can learn from one another.

The indicator on Erasmus+ student mobility measures the proportion of students who have participated in the European Union’s Erasmus+ student exchange programme. This indicates the ability of students to benefit from learning outside their home country.

Exclusions

The ranking only includes universities in countries that are full members of both the European Higher Education Area and the Organisation for Economic Cooperation and Development, and that are classified as a “developed economy” by the United Nations.

Universities are eligible for inclusion only if they offer courses and programmes that lead to officially recognised higher education degrees such as bachelor’s degrees and master’s degrees or equivalent; are focused on more than one narrow subject area; and have at least 3,000 enrolled students at bachelor’s or equivalent level. Institutions must also receive a minimum of 60 verified responses in the THE European Student Survey to be included.

In cases where institutions have not provided the required data, we have assigned the minimum score for the relevant indicators. We have excluded institutions if this has not been possible or if the data for a particular institution are too sparse.

Data sources

Data come from a variety of sources: direct submissions from institutions, the THE European Student Survey, the THE Academic Reputation Survey and Elsevier’s bibliometric dataset. We also used data from national statistical agencies such as the UK Higher Education Statistics Agency (Hesa). Our data are, in most cases, normalised so that the value we assign in each metric can be sensibly compared with other metrics.

Notable methodology changes

• The way we calculate graduation rate has changed since last year to accommodate the wider range of education systems being considered. We previously measured the share of students that graduate within the nominal length of their course of study. We now measure the share of students that graduate within 5 years.
• We have included two new metrics in the Environment pillar: Erasmus+ student mobility, and proportion of international students. The metric weightings for the other two metrics in the Environment pillar (gender balance of academic staff and students) has subsequently changed.
• We no longer have a metric measuring career preparation in the Engagement pillar. 

The calculation of the THE Europe Teaching Rankings has been subject to independent audit by professional services firm PricewaterhouseCoopers (PwC). Read more about the PwC/Times Higher Education Europe Teaching Rankings 2019 report here.