Next RAE may demote 5*s

Most university departments with top 5* research ratings will find themselves relegated to second-division status under plans for the next research assessment exercise, it has been revealed.

The first detailed study of how new finer gradings of research standards will impact on the academic sector also exposes wide gaps in the quality of research in different disciplines.

The analysis by Stuart McLeay, a Bangor University researcher, shows for the first time the likely pecking order that will emerge between departments and broad subject areas in the revamped RAE for 2008.

The study, exclusively revealed by The Times Higher , is based on the performance of academics in the previous RAE but interprets how they are likely to be rated under the new assessment system.

It uses previously unpublished data from the 2001 RAE on the proportion of research that was at an international, national or unclassified level to show how fine distinctions between the quality of research under the new system are likely to play out.

According to the analysis, the average 5* department is likely to find itself rated three out of a maximum of four under the scheme proposed by the higher education funding councils, while even an above-average 5* department is unlikely to make the top grade.

Only a select few departments, where virtually all staff are well-established world-class researchers stand a chance of achieving the best rating.

The results of the 2008 RAE will be expressed as quality profiles showing the proportions of research work that meet four new levels of excellence.

The funding councils have said that the top two grades will roughly equate to the top and bottom halves of research at international level, while it is expected that the bottom two grades will be allotted to national-level research.

But Professor McLeay estimates that in the 2001 RAE 34 per cent of research was assessed as world class, while 56 per cent was at national level and 10 per cent was not good enough to be classified.

Translated into the new system, he predicts that this means the average 5* department might expect a standardised score of about 3, while the average 5-rated department will score about 2.3, and a 4-rated department about 1.8.

Those classified as 3a can expect to hover around the 1.4 mark in 2008; the average 3b will get about 0.9; 2s will score only about 0.4; and 1s will hardly register at 0.1.

Professor McLeay applied the same analysis to the RAE units of assessment - broad subject areas - alongside taking numbers of research-active staff into account. This showed that even in the top performing subject - mineral and mining engineering - the "grade-point average" of an academic will rate only as 2.58 under the new system.

The average researcher in the worst performing unit of assessment - nursing - is likely to manage a rating of 1.44.

Professor McLeay said: "What is clear from my study is that there are substantial differences already in the proportion of international work undertaken across the different units and the percentage of work that did not get classified.

"One of the big differences the new system will make is that this information will become clearer, whereas under the old system it was never released."

The analysis of performance in the 2001 RAE also includes a breakdown of average research income earned per full-time academic in six broad subject categories according to their department's RAE rating.

Average annual research income for academics in 5* departments ranged from £128,000 in medical science down to £3,000 in languages and area studies.

Professor McLeay has also estimated the average income academics in top-rated departments are likely to bring in under the new system. The figures are: £133,000 in medical science; £107,000 in science and mathematics; £105,000 in engineering and computer science; £23,000 in social science and education; £8,000 in arts and humanities; and £3,000 in languages and area studies.

tony.tysome@thes.co.uk

Details: http://sbard.bangor.ac.uk/mcleay/rae/index.htm

SUBJECT RANKINGS This table estimates the average rating for academics in different subject areas under the new rating system for the 2008 research assessment exercise. It also shows the proportion of academics in subjects with research judged to be of international and national standing, and work unclassified in the 2001 RAE.   % research unclassified % research national
standard % research international standard Number
of staff Average rating
(out of four) under
2008 criteria Top ten subjects Mineral and mining engineering 1 44 55 82 2.58 Pre-clinical studies 4 49 47 159 2.39 Classics and ancient history 4 51 45 337 2.33  Anatomy 4 53 43 137 2.31 Clinical laboratory sciences 69 52 43 1,090 2.29 Celtic studies 7 49 45 92 2.29 Law 5 52 43 1,288 2.29 Asian studies 5 53 43 129 2.28 Physics   54 42 1,640  2.28 Accounting and finance 5 53 42 211 2.   Bottom ten subjects Built environment 16 59  25 588 1.77 Social work 16 58 26 364 1.77 Library and information management 18 56 26 296 1.76 Art and design 13 64 23 1,639 1.76 Other studies allied to medicine 18 55 26 982 1.75 Education 16 61 24 1,964 1.74 Communication and media studies 24 54 22 345 1.58 Drama, dance and performing arts 25 54 21 383 1.55 Environmental sciences 50 23 533 1.55 Nursing 56 17 562 1.44