NSS 'yea-saying' may be due to image-conscious universities

Growing numbers of university leavers may be giving the same response to every question on the National Student Survey because of “overzealous promotion” by institutions concerned about their public profile, a study suggests.

In 2014, 6.1 per cent of those completing the NSS online gave the same answer to every question – a phenomenon known as “yea-saying” – up from 1 per cent in 2005. The majority of these responses comprise wholly positive ratings.

A report to the UK’s higher education funding councils by polling organisation Ipsos MORI, which conducts the NSS, says that the “worrying” trend “may continue”, because yea-saying is even more prevalent among graduates who complete the survey on a smartphone, a category that is expected to grow.

One interpretation is that yea-saying reflects students trying to answer the survey as quickly as possible, and Ipsos MORI found evidence to support this.

Yea-sayers completed the 2014 survey in an average time of five and a half minutes, compared with eight and a half minutes for respondents who gave more varied responses. One in five yea-sayers completed the survey in under two minutes, compared with only 3 per cent of the rest of the sample.

But other evidence suggests there might be something more to it, as although yea-sayers were less likely to answer optional questions than other respondents, some 68 per cent still did, and more than half gave open-ended comments.

Ipsos MORI says that institutions are “aware of the importance of the survey and the visibility of the results, especially in the media and league tables”. This, the report says, “may encourage some overzealous promotion” of the NSS, and “could lead students to rate their institutions more generously”.

“Strong NSS scores at the institutional level can benefit both students and institutions themselves…there may therefore be some incentive on the part of both to encourage or give positive ratings,” the report says.

One possible solution would be to introduce negatively worded questions, to encourage more considered responses, but in a pilot students found these questions confusing.

A review chaired by Janet Beer, vice-chancellor of the University of Liverpool, instead recommends changes to the design of the survey interface. A “drag and drop” method of answering questions, or a warning if the same answer is given too many times, may be introduced. 

chris.havergal@tesglobal.com