Nancy Rothwell

Will 2008 really see the last RAE?Until then, there'll be plenty of discussion... and ideally some research as well

It is two years until we get into the submission and assessment process for the next research assessment exercise, and much longer before we hear the outcome, study the league tables and, crucially, discover how much funding each university submission has won. In spite of the timescale, the RAE seems to be featuring heavily in the academic press (not least in The Times Higher ), and in the minds and activities of universities.

The RAE has always been important and served a useful purpose.

It has made universities more rigorous in managing research resources and assessing quality, and has provided an independent review and national comparator. When it began in 1986 it was much needed and ensured that universities knew what they were doing and how well they were doing it.

Questions have been raised at each successive exercise about its value and a further question might address how much it diverts academic staff from doing the research that is to be assessed. Whether the 2007-08 RAE is the last such exercise remains to be seen, but it is to be hoped that some of the assessment processes are now deeply embedded in the sector and will survive even if the RAE does not.

The Higher Education Funding Council for England has always been anxious that the RAE should not change the behaviour of the academic sector and has worked to adjust the rules to ensure this and to limit the capacity for game playing. But previous experience suggests that its attempts have been less than successful. Academics are clever at adjusting what they do to fit "the rules". We hear of RAE strategies, internal RAE reviews and plans for hiring, firing and diverting staff from research towards teaching to enhance ratings. Perhaps most surprising is that staff should change where they publish from prestigious journals to less notable ones ( The Times Higher , July ). In the end, academics will be judged by a group of peers doing the best they can. The true details of the assessment will depend largely on the views and efforts of these panel members.

The next RAE has seen big changes after a major review by Sir Gareth Roberts. The shift from a single grade for each unit to a profile of outputs is welcome. It avoids the "cliff-edge" effect of just achieving or just missing a coveted high grade, sometimes with devastating effect. But one unintended consequence is that hiring "research stars" is likely to have more benefit than in the 2001 RAE, where one international leader in a reasonable-sized unit was unlikely to have had a major impact on the overall grade. The star system is also proving confusing to those struggling to distinguish between three internationally rated grades (2, 3 and 4*). Indeed, the inclusion of an international point of reference in 2* opens up the danger that the "gold standard" may be eroded.

There is much debate about what it takes to make the dizzy heights of the 4* grade, again with consequences for standards if panels adopt different yardsticks. We may also need to do a bit of explaining to our international colleagues. The details of RAEs past were probably a mystery to them, but most understood that 5 and 5* were very good, 3 and 4 not so good. Now it's different.

It will be interesting to see the extent of responses to the panels'

working methods, and any changes. Much of the detail has been welcomed: recognition of young staff, importance of interdisciplinary research, focus on outputs (publications) rather than on inputs (research funding). But some concerns are being discussed, not least the significant discrepancy in the relative weightings different panels will place on output versus esteem.

Take, for example, an individual or a group whose work on biomaterials might be considered by panel 14 (biological sciences) or panel 29 (metallurgy and materials). Panel 14 will weight output as 75 per cent and esteem as 5 per cent of final grade (a ratio of 15:1). By contrast, for panel 29, the weightings are 50 per cent and 30 per cent (a ratio of 1.7:1). It is hard to see how a consensus can be reached in such cases.

Some panels will also have difficulty assessing outputs independently of individuals. We have all seen 4* publications on which the individual in question is just one of a large team of authors. There are many other talking points, not least the appointment of some panel members from units rated 4 or below at the last RAE.

There is a long way to go before the final results of RAE 2007-08 are known, but it is important that we do not waste too much valuable research time on crystal-ball gazing and mock exercises, and do not lose sight of the important issues - the long-term future of UK research in the international arena.

Nancy Rothwell is MRC research professor in the faculty of life sciences at Manchester University.