Poisonous exercises

A dartboard and a map could do better than the RAE in targeting excellence, says Martin Cohen.

The fifth research assessment exercise is just getting under way. The last exercise cost £.3 million and, by its organisers' own account, had a number of undesirable side effects. Its critics say that is a lot of money. It is certainly a lot of money to make things worse. A dartboard and a map of Britain might be a more effective way of targeting research money at centres of excellence.

So what has been the problem with the past few RAEs?

Problem one is the damage to and distortion of the production, publication and dissemination of knowledge. Publishers, who are outside the system and may therefore be considered "objective", report that the quality of book proposals and manuscripts has plummeted as thousands of academics reach for their lecture notes in a bid to stave off a low research rating. Libraries have begun to stop stocking journals and theses. When every academic department has at least two journals to its name, it becomes rather expensive to keep up with the "scholarly" output.

Problem two is research orthodoxy - that is, preferring conventional, stereotypical research in big, established departments over innovative, unconventional and novel activities anywhere. Despite complaints last time about the absence of any "objective" criteria, the latest RAE will again use the "subject expert" method.

Problem three is the people carrying out the assessment. There are two types of panel members: donkeys and vipers. The donkeys take away piles of learned submissions, journal papers and unreadable books. The vipers dine with principals, professors and vice-chancellors, busy preparing for the time when the grades are fixed and scores are settled.

Who chooses them? This exercise, like all the others, will be carried out primarily by using the Order of the Brown Nose method - otherwise known as peer review. The OBN permeates the exercise. The panel chairmen are selected over a quiet dinner with the previous chairman. The panels are selected by these chairmen on the basis of "recommendations". And the final thinking behind the assessments is kept secret from the public.

Problem four is the division of research into traditional subject areas: chemistry; art and design; education; philosophy. There are no good reasons to accept these boundaries other than administrative convenience. Certainly, research does not necessarily fit any one category. But interdisciplinary work implies the end of the system of experts, and this is unthinkable.

Problem five is the effect on "non-assessed activity" such as teaching, always the poor cousin of research in universities. As far as teaching goes, the motto of most lecturers is "little and not often - and lots of reading weeks". The RAE has reinforced this attitude, as the Dearing report noted.

The RAE marginalises research that cannot be entered for assessment - the sort of activity that bears fruit slowly or looked a very long shot at the outset; the sort of activity that most needs seeding. Meanwhile, time is consumed "preparing" for the RAE.

Problem six is the cattle market of academics with publications and the expensive jobs merry-go-round of "established researchers". A related problem is awarding top grades to a department that has "entered" just a few of its staff. Does it really make sense to call this a "top-performing" research centre? Of course not. But there is no clear solution, so the nonsense continues.

What, then, can be done?

A random distribution might seem hit and miss, but the sine qua non of the present ludicrous exercise is precision. The model is, however, hopelessly imprecise. The experts do not know what they need to know to decide between many projects. They may not even be competent researchers themselves - just good at sitting on committees. In the research assessment exercise, all the prejudices of the British university system are entrenched. It is elitism and amateurism at a time when UK universities are slipping further behind.

Martin Cohen is editor of The Philosopher , an interdisciplinary researcher and author of 101 Philosophy Problems , (Routledge, 1999) and Political Philosophy published by Pluto Press (spring 2001).