Fair assessment? 2

The letters on "game-playing" for this research assessment exercise and the financial consequences of the result (September 29) have not separated two important issues: how to rank the outcomes given by the "Roberts vector" of proportions of output in each of five classes; and how the Higher Education Funding Council for England will divide funding on the basis of the outcome, presumably taking account of the proportion of staff submitted.

It has been clear since the Roberts proposals were accepted that ranking departments on the day after the announcement would be tricky and that, in the absence of any summing method across the grades, they would probably be listed by the proportion of 4*s awarded. It is obvious that this would be unfair. The problem is that there is no agreed method of summing the star proportions to get an overall ranking, and until Hefce declares one, we are all in the dark.

The outcome last time was non-linear, with high grades counting for far more than lower grades. For example, departments rated 5 got infinitely more than five times that given to departments rated 1, since the latter got nothing.

When a ranking method is declared, the issue of proportions of staff submitted, to give a fair financial outcome, is easy: the (notional) outputs of non-submitted staff can be added to the unclassified column as a proportion of the whole department, a column that will be funded at a zero rate. If that is done, any funding algorithm that is acceptable across the grades will deal also equitably with the proportion of staff submitted.

Yorick Wilks.
Sheffield University