Satisfaction and its discontents

One of the striking features of a highly centralised system of higher education, such as that of the UK, is that the introduction of new targets and modifications to the quality assurance framework can have a dramatic impact in a very short space of time. When the National Student Survey was introduced in 2005, few colleagues imagined that, just several years down the road, finessing and managing its implementation would require the employment of an entirely new group of quality-assurance operatives. At the time, the NSS was seen by many as a relatively pointless public-relations exercise that would have only a minimal effect on academics' lives. It is unlikely that even its advocates would have expected the NSS to acquire a life of its own and become one of the most powerful influences on the form and nature of the work done in universities.

The frenetic chain of emails crowding lecturers' in-boxes and demanding that they hassle their students to complete the online questionnaire indicate that it is "that time of year" again. The annual ritual of imploring undergraduates to fill in the NSS is upon us, and up and down the country staff are exhorted to be on their best behaviour and to do the business. Multiple emails are also dispatched to students to inform them that their views are really very important and that, in any case, if their name is drawn they can win a "fantastic prize!". Some universities hold special NSS weeks and devote considerable resources to publicity campaigns involving posters, postcards, plasma screens, lectures and online promotions. Sometimes it appears that universities spend as much time worrying about what students think of them as undergraduates spend worrying about grades.

One quality-assurance operative explains to me that it is crucial to "front-load the survey". Apparently, the early responders tend to be more positive than the procrastinating students who are fed up with being harassed for weeks by their desperate lecturers. However, he also has a clever scheme to deal with the laggards. He outlines a complex plan of action that includes classroom visits, low-grade bribes (vouchers, coffee mugs and chocolate bars) and phone calls. When he registers my look of incomprehension towards this, he rounds on me: "Don't you know that the survey will be used to allocate funding and impose penalties?"

It seems that the strategic objective of "front-loading" the survey is widely shared in universities. One recently appointed social-science lecturer, who has had the misfortune to be landed with the task of being her department's NSS coordinator, informs me about the law of diminishing returns. According to her analysis, about 30 to 35 per cent of the students fail to respond to the exhortation to fill in the online survey during the first few weeks of the campaign. In the end, most holdouts relent and fill in the forms, "just so we'll stop bothering them". But she suspects that some of these students will exact revenge for being pestered by giving her department lower satisfaction scores.

For many colleagues involved in the management of the NSS, the project represents a major claim on their time. They face significant pressure to ensure that the completion rates are high. In some new universities, departments that achieve the highest response rate are rewarded with a financial bonus; conversely, those that fail to achieve their targets face the wrath of their administrators.

What is remarkable is that the impact of the NSS on university life goes way beyond the annual campaign to get students to participate in the survey. The idea that you live or die by the NSS is now widely internalised, particularly by recently appointed staff. One young lecturer told me that, at first, she was "amazed" at the central role allocated to the NSS by her university. During the 18 months of her employment she has twice been asked to reorganise her module to enhance student satisfaction. Although at first sceptical about the merits of this, she now accepts that there is no choice but to embrace it. After several departmental meetings and discussion with colleagues, she has come to realise that the reputation of her institution is intimately linked to student satisfaction.

There is in the NSS something deeply intrusive and destructive that distinguishes it from other attempts to audit academic life. Since the consolidation of auditing culture in academia, I have encountered numerous quality-assurance initiatives focusing on teaching and research, as well as institutional reviews. Auditing instruments such as the old research assessment exercise have had a significant impact on scholarship and the quality of professional academic life. There is little doubt that, on balance, the RAE had a negative influence on scholarship. Its main achievement has been the proliferation of paperwork and the expansion of pointless publications. But although the RAE had a disorienting effect and increased workloads, it touched us externally for the most part. In contrast, the NSS has as its target the modification of everyday academic life. The exhortation to raise student-satisfaction rates conveys an implicit demand to alter academic identity and behaviour. Unlike previous auditing measures, the NSS does not merely demand accountability but directly challenges the identity of a scholar. It possesses a corrosive immediacy that encourages the subordination of education and scholarship to the arbitrary imperative of student satisfaction.

Numerous colleagues have rightly argued that the measuring of improvements in student experience is highly flawed. It is widely recognised that student satisfaction is not an accurate measure of the quality of education. The surveys provide no insight about the quality of the experience of the education received. The responses do not amount to a statement about standards of teaching, scholarship and intellectual development but rather reveal the subjective proclivities of different groups of students. So the surveys indicate that female students tend to be more positive about their university life than their male counterparts. Older students tend to be more satisfied than younger ones, and black, Asian, mixed race and other ethnic groups are less positive than those from white ethnic groups.

From a sociological point of view, the results say more about the attitudes of different constituencies of students towards the world in general than about their experience of university life. At best, the NSS provides an indication of whether or not students enjoyed their experience as undergraduates. That such a subjective evaluation is routinely recycled as a statement about the quality of a course is understandably a source of frustration to many academics.

However the real problem with the NSS is not that it has the potential for misrepresenting the strengths and weaknesses of an institution to the outside world; it is its corrosive impact on the internal life of academia. The inexorable consequence of the current obsession with student experience is the adoption of a risk-averse and defensive approach towards the provision of undergraduate courses.

The subordination of so much of academic life to the imperative of cultivating student satisfaction invariably leads to a role reversal between the authority of the academic and that of the student. When it comes to the measurement of the quality of student experience, what counts is not the judgement of an academic but the opinion of a student. In such circumstances, the question of "what do students want?" frequently trumps that of "what do students need?". The necessity of giving students what they want is expressed most dramatically in the modifying of systems of assessment and feedback. Progressive marking schemes (for which read less rigorous) and the regular offering of cheerful comments about essays no matter how poor they may be are now integral to the life of many lecturers at insecure institutions. There was a time when the idea that what students want is not always what they need was a view endorsed by serious academics. Those who raise this view today are likely to court ridicule and be accused of being an out-of-touch, elitist snob.

Probably the most damaging outcome of the NSS is the contribution that it makes to flattering students. The official party line towards student criticism and complaint is to promise to improve the "experience". One obvious way of improving the experience is by lowering the bar and making life less complicated and, invariably, less challenging for students. All it takes is for a few students to complain about course material to provoke the administration to instruct teaching staff to get their act together and provide more resources - lecture notes, readings, model essays - so that undergraduates are spared the hassle of going to a lecture or visiting a library. As one of my colleagues confides, "I cannot help but feel that we are actually encouraging students to think of themselves as children."

The cumulative effect of all these little modifications to coursework is to gradually erode the distinction between a school pupil and a university student. The model of teaching that is slowly creeping into university life is one in which undergraduates are perceived as biologically mature pupils who require constant direction and guidance. The idea of a university student as someone who is supposed to engage in independent study and self-directed work at least some of the time is implicitly and often explicitly questioned by current practice.

Consider the paper "Reconceptualising assessment feedback: A key to improving student learning?", by Chris Beaumont, Michelle O'Doherty and Lee Shannon, which was published last September in Studies in Higher Education.

The authors claim that the "fault line" between secondary and higher education represents a major problem for new undergraduates. Their solution to this problem is to minimise the difference in expectation between the two sectors by getting universities to treat their students as school pupils - at least for a year. They suggest that the first year of university life ought to be a transitional phase - a year of induction - subject to the standardised orientation of a school curriculum.

Rising to the occasion, Usman Ali, vice-president (higher education) of the National Union of Students, asserts that you can never have too much of a good thing. He insists that "induction should be much more than just the first two weeks of the first year". He states that induction should be integrated into coursework through all years of study. The premise of this call to minimise the distinction between secondary and tertiary education is the conviction that, since young men and women lack the capacity for self-directed independent study, they will respond to their university experience from the standpoint of a school pupil.

If higher education is perceived as merely a continuation of schooling, many practices associated with the pursuit of scholarship and academic learning are experienced as a negative dimension of undergraduate life. So Beaumont has denounced the practice of refusing to accept drafts of assignments from first-year undergraduates as "indefensible". Of course, from the perspective of a 14-year-old pupil, such outrage is entirely understandable. But from the point of view of academic teaching, asking a first-year student to write a complete essay is a legitimate expectation. As a devoted practitioner of this "indefensible" practice, I have concluded that it is far better to have a discussion about the subject of the essay with students before gently directing them to the library than to short-circuit the process of discovery through discussing early drafts with them. By the time they are 18, it is better that they face a challenging intellectual experience than merely be given smiley faces for making token efforts.

From the perspective of academic study, the very term "student satisfaction" is an irrational one. Why? Because students need to be placed under intellectual pressure and challenged to experience the intensity of problem-solving. Such an engagement, as Stefan Collini has argued in his most recent book, What Are Universities For?, does not always lead to a happy experience. Not a few individuals at the receiving end of a Socratic dialogue feel provoked and angry. Today, this ancient philosopher would not make much of a showing in a student satisfaction survey. So the question worth asking is: "Ought the satisfaction of the student be one of the central objectives of the university?" From the perspective of the development of a stimulating and creative academic life, the answer must be a resounding "no". The moment that students are invited to regard their experience as more important than academic education, their intellectual development is likely to be compromised.

Believers, apostates, agnostics: the cult of the NSS

The higher education White Paper of 2003, The Future of Higher Education, promised the introduction of an annual national student survey of teaching quality as part of the Labour government's plans to make students "intelligent customers".

It was an idea that met with a less-than-enthusiastic reception from staff. "Most higher education institution staff we interviewed felt that (a national survey) could add little to internal feedback mechanisms, and there is quite widespread resistance to the introduction of such a survey," a team of researchers and consultants reported in 2003.

A number of students' unions also boycotted the survey when it began in 2005, including those at the universities of Oxford and Cambridge. Student leaders argued that the survey was of limited use and unfairly burdened students, who are asked to complete the questionnaire in their final year.

As the data began to appear in university league tables compiled by national newspapers, the results of the survey grew in influence and the NSS was treated increasingly seriously by university leaders.

For example, in 2007 and after a poor institutional showing in a league table, the vice-chancellor of Anglia Ruskin University wrote to staff to tell them that they needed to ensure "that students are aware of the relation between the grades that they give [in the NSS] and, thanks to league tables, the perceived value of their degrees".

At the same time, concerns about universities trying to influence the results began to surface. In 2008, a lecturer at Kingston University advised students to inflate the marks they gave their university in the NSS or risk losing out in the jobs market. "If Kingston comes bottom, the bottom line is that no one is going to want to employ you because they'll think your degree is shit," the lecturer said.

Also that year, staff at London Metropolitan University's business school were instructed to show students a slide repeatedly at the start of teaching sessions telling students to "seriously reflect" on their responses to the survey questions because the survey "impacts on the reputation of your university, your teaching department and your award".

Writing a letter to Times Higher Education about London Met's exhortations landed Lee Harvey, who was then director of research and evaluation at the Higher Education Academy, in hot water. In the letter, Harvey called the NSS "a hopelessly inadequate improvement tool". He was suspended from his post by Paul Ramsden, who was head of the HEA at the time, and then resigned.

In 2010, however, a report by a team from the Institute of Education, led by Ramsden, found that the NSS was "valued as a means by which institutional managers can encourage improvement".

The same report, Enhancing and Developing the National Student Survey, warned against using results from the NSS in league tables. The data could not be used "responsibly" to compare different subject areas, for example, the researchers found.

The usefulness of the NSS continues to be debated. Last year, Duna Sabri, visiting research fellow at the Centre for Public Policy Research, King's College London, told a conference in London that her studies of the NSS showed that it was a poor indicator of quality. Dr Sabri said that it was nonetheless revered by university leaders and argued that it had gained a disproportionate influence.

"I have never come across an instrument that has more function heaped upon it or importance imbued in it than the NSS," she said. "There is an almost religious belief in the power of the NSS to enhance experience."

Rebecca Attwood