THE Student Experience Survey 2018 methodology

This year, the annual results are based on the responses of 20,251 undergraduates, who were asked to describe how their university contributed to a positive or negative experience on a seven-point scale, from “strongly agree” to “strongly disagree”. Questions have remained unchanged since 2005 to allow comparisons across years.

Data are collected from October to June from full-time, UK-based undergraduates. There is a one-year time lag between collection and publication. 

The survey is linked to YouthSight’s student omnibus surveys and respondents can participate once a year. Only full-time undergraduate members of YouthSight’s 140,000-strong opinion panel community can take part.

More on the 2018 Student Experience Survey

Student Experience Survey 2018: the results 
Student Experience Survey 2018: Higher education sector still raises the bar in a turbulent year  
Student Experience Survey 2018: Protecting free speech while keeping students safe 
Student Experience Survey 2018: Loughborough says its students’ ‘get-up-and-go’ attitude is the key to its success 
Student Experience Survey 2018: Staffordshire’s 24-hour campus that works for all 
Student Experience Survey 2018: Keeping mental health in mind 
Student Experience Survey 2018: the rising stars of higher education 
Student Experience Survey 2018: ‘At over £9K a year, is a degree worth it?’

For each measure, a mean “agreement score” is created per institution. Each score for the 21 formulated measures is then weighted depending on how strongly it correlates with the 22nd “measure” recommendation. 

For each institution, the sum of the weighted mean scores is divided by the sum of the weights to calculate a weighted average mean. This is then indexed to provide an overall score (out of 100). 

A selection of composite scores have been created to allow institutions to see how they are performing in different areas of student experience. 

As the number of responses per institution broadly reflects institution size, it is not always possible to achieve a large enough sample to provide statistically robust data at all institutions. The compromise is to set a minimum threshold of 50 responses per institution before inclusion. In total, 116 institutions were included, with an average sample size of 175.

As with any survey, there is an “error bar” linked to each institution’s sample size and variance. On average, for the SES, a difference of 3.7 points in the overall score is required to produce a significant difference between institutions. So, at the top and bottom of the rankings, institutions have to move only a few places to see a significant difference, but in the middle, where scores are bunched together, rankings need to shift by 30-40 places to see a significant difference.

For the first time this year, three-year rolling average data have been published, which aggregate results for a period of three consecutive years, starting with the years 2009 to 2011. 

This allows for a very large sample, which reduces the impact of sampling errors in any one given year. The next time period adds one year and loses one year – for example, the period 2009-11 is followed by 2010-12, and so on. In this way, we are able to show the long-term trends at the level of individual higher -education institutions and the overall sector.  

We have supplied a rolling three-year -verage for the overall scores, for the composite scores and for each individual measure. Only institutions that achieve at least 150 interviews across a three-year rolling period have been included in calculations. 

Ashleigh Gillan 
Research manager
YouthSight