Differential impact 1

The latest Times Higher Education devoted an editorial and two articles to concerns about the proposal to include "impact" as a factor in assessing the quality of research and scholarship under the research excellence framework ("First, let's find out if it will fly" et al, 17/24 December).

The impression gained was that academics are united in their opposition to introducing impact and view the proposal with alarm.

Impact was described as at best difficult to measure and at worst a thinly disguised ploy to commercialise the universities. Academics in contrast were portrayed as noble souls engaged in the pursuit of truth and wisdom. The truth is more complicated.

Surely the impact of research as described by the Higher Education Funding Council for England in terms of social, health, economic, environmental, cultural and/or quality of life is important.

The key questions revolve around what are the alternatives and how impact is assessed. Reducing the percentage given to impact will increase the percentage given to outputs. These will be assessed primarily by citations, which are highly problematic and tend to favour established disciplines. Further, citations reflect the use of work by others in the field rather than by society as a whole. In this light, impact would seem a good counterbalance to a reliance on bibliometrics.

However, the assessment of impact in the Hefce proposals may be too crude. Assigning a single score to it would be dangerous.

In some disciplines, such as economics, education and medicine, impact is likely to be evident and a proxy for high quality. In others, such as astronomy, evolutionary biology or pure mathematics, impact might be almost irrelevant or evident only over many decades.

As academics working in a department of palliative care, policy and rehabilitation, we find it hard to envisage a programme of research on dying or disabled people that would not have some element of intended social impact. At the same time, we understand how colleagues who study hominid evolution, black holes or number theory might find a 25 per cent weighting unworkable.

However, in our own field we would argue that even 25 per cent could be deemed to be too small. What is needed is a system that can take account of these differences, either in the assessment of outputs and impact or via a differential weighting.

Irene J. Higginson, Richard Siegert
Department of palliative care, policy and rehabilitation, King's College London.