Standards are rising

Top research universities argue that only they should be funded to produce top-quality research. Here chairs of assessment panels in the Research Assessment Exercise claim that standards are rising throughout the system.

IBERIAN & LATIN AMERICAN LANGUAGES

At the beginning of the assessment stage of the process, chair Roger Walker, professor of Spanish at Birkbeck College, asked each of his eight-strong panel to offer an initial estimate of the score for each of the 31 submissions, based on their first examination of them.

"I recorded the scores they suggested on a chart and it came in handy later. For instance, I could say 'You wanted to give this department a 3b at the start and now you're arguing for a 4 - what has changed your mind?' It was a very useful reference point," he said.

He found that his panel - renamed after the mild diplomatic gaffe of calling it Spanish including Portuguese in the 1992 exercise - had few difficulties in reaching a consensus. But submissions were being regraded right up until the last moment: "We moved two up at our last meeting."

He was happy with the seven-point rating scale, but argues strongly that any further extension would involve panels in too many fine judgements: "We found that the most difficult line to judge was between 3a and 4 rather than between 3a and 3b."

He was impressed with the standard: "The quality of submissions was higher and there is little doubt that the absolute standard of work is also getting better."

After sitting on panels for the past two exercises, he does not expect to be invited in 2000, but has one suggestion for those who conduct it: "One of the great difficulties for panel members was getting acc- ess to the publications listed for the academics included in the exercise. I can see the problems of asking for copies to be supplied with submissions, it would involve HEFCE in a huge warehousing job, but something of the sort would help."

He said his panel managed to read about 90 per cent of the publications submitted, well up on 1992.

ECONOMICS & ECONOMETRICS

David Hendry, chair of the economics and econometrics panel and professor of econometrics at Oxford University said: "It was an unbelievable amount of work. I think HEFCE will have difficulty persuading people to join the panel next time, especially if they continue to offer such a derisory fee. We had 4,000 articles, books and papers and each were supposed to be read twice. I had to take six months out of the year and my backlog is horrendous. It has badly affected my own research."

On the positive side, he said that he has learned a tremendous amount about other research and the committee was delighted at the quality, especially from the younger members of the profession.

But he adds: "The upward drift of grades could be detrimental to overall quality. The net result is the exact opposite of what HEFCE is trying to achieve. HEFCE wants to channel most of the money into the top end, but the upward drift will mean less money at the top."

BIOCHEMISTRY

Biochemistry scored the highest average rating of any of the 69 units of assessment. But John Coggins of Glasgow University and chair of the panel, said: "The research councils tell us that biochemistry is one of the strongest subjects in British universities." Even so he would like broader units of assessment to be used in the next exercise.

Results were much better than in 1992, an outcome he says reflects university policies: "They have invested a lot of money in biochemistry. In 1992 there were a lot of extremely bright young people who had been recently appointed and were still to start producing seriously. We have seen the benefits over the past four years."

GENERAL ENGINEERING

Improved grades in general engineering in this assessment exercise stemmed from a combination of better research and more selective submissions, according to Brian Rowe, chair of the assessment panel and an engineering professor at Liverpool John Moores University.

"They cut off the tail of the poorer research, removing it from the exercise," he said. "It is a way for universities to self-select the way researchers will be funded and is probably better than trying to direct funds only into institutions funded through some arbitrary decision."

Professor Rowe said new universities, in particular, had been much more selective than in 1992.

SOCIAL WORK & SOCIAL POLICY

New universities have made substantial progress in improving their research ratings in social work and social policy, said the chair of the panel covering the two disciplines.

Nicholas Deakin, of Birmingham University, said new institutions now routinely appeared in the upper categories for social policy and administration.

He said many performed poorly in 1992 because they were totally unprepared, but now knew what was expected of them. While there were many ratings of 1 in the previous RAE, this time there were none in social work and only 2 institutions earned a 1 for social policy.

DRAMA, DANCE & THE PERFORMING ARTS

Standards have risen across the board in drama, dance and the performing arts, said panel chairman, Martin Banham, professor of English at Leeds University.

He felt the RAE to be necessary, well-conceived and conscientiously executed. But he would like to see more thought given to methods of assessing non-traditional subjects such as drama.

"Quite often on this kind of exercise there is an assumption of scientific programmes of research and research methods," he said. If it goes ahead again, there needs to be continuing thought given to the needs of humanities and not just the traditional humanities.

"It is a changing world, particularly with many of the new universities coming into the frame. The nature of research in the different subject areas needs to be examined in a fresh way."

HISTORY

Michael Prestwich, of Durham University, chair of the history panel, said: "I would ban the use of the word 'vibrant'. Everyone said they had a 'vibrant research culture' and I don't know what this means. In asking for information on the research environment, it would be better to ask more specific questions."

Like many panel chairs he pointed to difficulties in getting hold of published material. "Even panel members in Oxford and Cambridge had problems.

"Many of the publications submitted were published close to the census date and took time to get into libraries."

And there were occasional difficulties with the rating scale. "While the concept of international as opposed to national standing may be recognisable to chemists, it is not one that historians use much."

COMMUNICATION, CULTURAL & MEDIA

Communication, cultural and media studies was new to the list of research assessment panels this year and its chairman, Philip Schlesinger, feels it was a necessary one.

While media and cultural studies has been one of the boom areas in teaching in recent years, the assessment exercise revealed valuable research was also being done. Six of the 35 departments assessed received a grade 5.

Schlesinger, professor of film and media at Stirling University, and a member of the library and communications studies panel in the last RAE, said: "Britain is a world leader in this area and it's high time media and cultural studies received the recognition it deserves."

He described standards as on a par with the last assessment with the advance of the new universities proving that good research should be encouraged everywhere. But he said a four-yearly assessment cycle was not a good idea. First, because institutions started gearing themselves up for the next assessment just two years after the last.

Second, because it restricted development of new talent; and finally because it distorted the whole university system for the year before submissions were made.

He admitted finding evidence of a transfer market in academics to boost research ratings but said it was not a dominant feature.

"Now that we have it, some sort of research assessment exercise, is bound to take place again," he said. "But in the fourth round we must think carefully about how wide to do it, the costs involved and the opportunities foregone."