Verdict on QAA reform: amateurish and daft

The government has been criticised for a "high level of amateurishness" in formulating proposals on quality assurance in the higher education White Paper.

The suggestion that student surveys could trigger reviews was also dismissed as "daft".

As part of plans to move the work of the Quality Assurance Agency towards a "risk-based" system, the policy document proposes that the use of institutional reviews could depend on an "objective set of criteria and triggers", including student satisfaction.

The White Paper does not specifically mention the National Student Survey - which has been criticised in the past for its reliability in measuring quality - as a potential trigger.

But Peter Williams, former chief executive of the QAA, said that there would be serious question marks if satisfaction ratings were used to determine the need and frequency of institutional reviews.

He said that although surveys such as the NSS might be a useful tool in some respects, they were completely unsuitable for use in a quality assurance regime because of the concentration of most scores into a narrow band of results.

"Student satisfaction is not a criterion for good quality and is certainly not a criterion for assessing academic standards," he said, adding that the authors of the White Paper appeared to have "no real idea of what quality assurance is for".

"I get the impression that there is a high level of amateurishness in the White Paper's treatment of the quality assurance question."

He also warned that the reputation of English higher education would suffer if a risk-based system led to some universities not being inspected regularly.

"If there is an extended period of time during which universities will not be subject to any kind of review, then this is going to seriously damage the UK's status in the world of higher education," said Mr Williams, who was head of the QAA from 2002 to 2009.

Roger Brown, professor of higher education policy at Liverpool Hope University, said he had "never heard anything so daft in all my life" in response to the idea of student satisfaction triggering reviews. Survey scores, he said, were "desperately unreliable" and the only "secure basis" for maintaining quality was to have a clear set of sanctions for wayward universities by, for example, threatening the removal of degree-awarding powers.

Anthony McClaran, current chief executive of the QAA, said the detail of how risk might be measured in any future framework would be subject to the consultation on regulation that the government is expected to launch in a few weeks.

However, he said he could envisage risk being measured in several ways: bodies such as the Office of the Independent Adjudicator could flag specific concerns, but indicators such as student satisfaction, a university's finances or its governance could also prompt alerts.

"If there were consistently poor scores or declining scores on student satisfaction for a particular institution, that might trigger a closer examination," he said.

He added that indicators would have to be "robust" and he stressed that the QAA already made sure it substantiated evidence when inspections were triggered under its current procedure for investigating student concerns about a university.

simon.baker@tsleducation.com