Europe Teaching Rankings: FAQs

General

What
will the rankings do that other rankings do not already do?

University league tables do not do a great job of capturing and rewarding excellence in teaching.

While the Times Higher Education World University Rankings are designed for research-intensive global universities and are dominated by indicators of research excellence, THE’s data team has pioneered new teaching-led rankings, focusing on teaching excellence and student success.

A major innovation was THE’s Student Survey, a poll of students at European institutions that explores not their satisfaction with university facilities, services and tuition, but rather their engagement with learning. A series of questions examine the extent of students’ immersion in the intellectual and social life of their institutions, how accessible they find their professors, and if they would recommend their college experience to others.

Who will THE be ranking?

The aim is to evaluate multilevel and multi-faculty institutions in nine European countries that have a minimum of 3,000 students enrolled at International Standard Classification of Education (ISCED 6), bachelor’s degree or equivalent level.

What is the time frame?

Institutional data collection starts in January 2020 and will conclude at the end of February 2020. During this period, data will be collected from institutions via the THE data collection portal.

The THE European Student Survey will run from October 2019 to the end of February 2020. 

When
will the rankings be published?

The THE Europe Teaching Rankings 2020 will be published in July 2020.

Inclusion criteria

What
are the inclusion criteria?

The institution must:

• be located in a country that is a full member of the European Higher Education Area (EHEA) and classified as a “developed economy” based on United Nations (UN) country classifications
• be located in an Organisation for Economic Cooperation and Development (OECD) country
• offer courses and programmes leading to an officially recognised higher education degree such as a bachelor’s award or equivalent degree
• be focused on more than one subject area       
• have a minimum of, or more than, 3,000 students enrolled (at ISCED 6, bachelor’s or equivalent level)
• have a minimum of 80 valid student survey responses
• have submitted institutional data via the THE data collection portal

Will all participating institutions be ranked?

Inclusion will depend on the quality of data available and the completeness of datasets. For example, we might exclude institutions if we do not receive sufficient valid responses from the student survey, or if direct submissions are not sufficiently complete. We also reserve the right to use editorial judgement in exceptional cases. However, our intention is to provide a ranking of all qualifying institutions.

Methodology

What weight will you apply to each metric?

Our current weighting distribution is:

Engagement (35%)
Student engagement: 10%
Student interaction: 10%
Student recommendation: 10%
Links to labour market: 5%

Resources (20%)
Staff-to-student ratio: 7.5%
Papers-to-staff ratio: 7.5%
Quality of services: 5%

Outcomes (20%)
Academic reputation: 10%
Graduation rate: 5%
Skills development: 5%

Environment (25%)
Gender balance of academic staff: 7%
Gender balance of students: 7%
Proportion of international students: 5.5%
Erasmus+ student mobility: 5.5%

Do you have
a detailed description of all the data fields?

Yes. We are providing a data collection portal guide that explains the key aspects of the data collection process, including data field definitions and further personnel-related definitions.

Are you considering measuring the impact of schools’ research and teaching on business, organisational and policy practice?

The focus of these rankings will be teaching quality, rather than research excellence, and asking students specific questions about their learning experience. We are, however, open to adding external measures of teaching quality if the right data can be gathered. Indicators that are universal (ie, comparable across all countries) and likely to be either (1) gathered by all schools, or (2) found in reliable public datasets (such as those from the OECD or World Bank indicators) would be good sources. 

On the subject of research, measuring an institution’s research production is possible and we do so in one of our metrics. Here, we try to capture whether the teaching is research-informed and whether teachers contribute to knowledge and science.

Is your rankings process verified independently?

The Europe Teaching Rankings 2020 will use PricewaterhouseCoopers (PwC) to provide an assurance audit to validate that our data gathering and analyses are correct and correspond to our methodology. The 2019 ranking received a similar PWC audit, which can be found here.

Data collection

General

What are your data sources?

Data will be sourced from:

• direct submissions from institutions through our portal for the majority of our Resources and Environment metrics 
• student survey: asking students for feedback on a variety of Engagement questions. This is conducted for THE by Streetbees and with the help of universities themselves. 
• Academic Reputation Survey: providing teaching reputation data for the Outcomes pillar
• Elsevier for all bibliometric data
• public sources: the OECD, World Bank, European Tertiary Education Register (ETER), statistical agencies, ministries of education

How do you validate data submitted by universities?

Data will be cross-checked against external sources, at our discretion, and we reserve the right to investigate institutions where we believe inappropriate data collection or submission has taken place.

Student survey

Who will
the student survey target?

If the institution decides to run their own campaign, one of the following options has to be chosen:

• all students who are over 18 years old
• all undergraduates who are over 18
• a random sample of all students who are over 18
• a random sample of all undergraduates who are over 18.

Is there a minimum requirement in terms of responses from the student survey?

Yes. In order to be considered for participation in the THE Europe Teaching Rankings an institution needs to have 80 or more valid responses.

How will
the student survey be administered?

The THE European Student Survey is an international study of the university experience for students across Europe conducted for THE by Streetbees. The campaign is run by StreetBees, which contacts the institution and recruits the respondents.

However, THE also offers institutions the option to run their own campaign in parallel with Streetbees. This helps to ensure a valid sample can be collected (a requirement for entry into the ranking). Alternatively, this year we invite all institutions to self-distribute the student survey. If you agree, StreetBees will not run an additional campaign.

Some countries (and any universities with fewer than 5,000 undergraduates, according to ETER) cannot be targeted by Streetbees. These universities will need to self-distribute the survey to be included in the ranking.

Which countries can Streetbees run the survey in?

Streetbees is able to run the student survey for institutions based in:
Germany, Spain, Portugal, Ireland, France, Italy, the UK and the Netherlands.

This means that any universities based in the following countries will need to help us to distribute the survey to be included in the ranking: Hungary, Denmark, Finland, Lithuania, Greece, Czech Republic, Poland, Slovakia, Slovenia, Latvia, Switzerland, Estonia, Iceland and Luxembourg.

What is the process for universities to distribute the survey?

1. We ask a representative (primary contact) from the university to complete a confirmation form, confirming that they agree to use one of our approved sampling methods.This form is available here.
2. This person will then receive an email from us containing three URLs. The first will provide a template to use for any planned email campaign. The second URL is a link to the survey, to be used exclusively in direct email campaigns to students. The third URL will be another link to the survey which can be posted on a university website or social media page.
3. Universities are then free to send out the emails or post the URL whenever they want throughout the survey period. The surveys are collected by us, and we do all data handling on our end so no further action is required.

How can I sign up to self distribute the survey?

Please visit our Europe Teaching Rankings 2020 - Student Survey Distribution page here.

What questions
will you ask students?

The aim is to ask a core set of questions to find out about students’ engagement with their learning. A series of questions examines the depth of students’ immersion in the intellectual and social life of their institutions, how accessible they find their professors, and if they would recommend their college experience to others.

Will you share the survey instrument with institutions?

Yes. We will share the questions (the survey template) with institutions when requested to do so.