On the subject of research .. 2

It has been suggested that citations should be counted over five years or ten years, but it should be longer.

It can take a year between submission and publication in quality journals. A paper that is seminal in that it stimulates further work can wait up to two years before it gains a citation, particularly if it stimulates experimental work. As such, if a short period is chosen a significant portion of papers may well not have gained citations. Alternatively, the citation count could be applied to papers published, say, one or two years before the census date.

A more important issue is the selection of a period that results in a reliable assessment of the significance of published work. In the case of academic impact, care should be taken to guard against "bandwagon" effects, where for short periods there is high interest with short bursts of high-citation activity. It may be subsequently found that the work has no lasting significance and the bandwagon rolls on. A recent example would be the frenetic but ultimately futile activity in relation to cold fusion.

If a citations time-frame captured only the short period of high activity, it would give a false impression of the significance of the work. A longer period would detect the decline of interest in the topic.

It is even more challenging to make an accurate assessment of work with applications significance. Citation counts cannot be expected to measure such impact and indeed can be highly misleading.

Two examples spring to mind: Josephson junctions and high-temperature super-conductivity. Both are of great scientific interest and both have led to Nobel prizes. But the relevant discoveries were also accompanied by excited claims about future applications that have not yet materialised. For both there will be sustained periods of high citation activity, but ironically these in part signify a failure to carry forward the work to applications.

Commercial application of research would normally be based on protected intellectual property and hence the filing and licensing of patents offers some kind of metric in this respect. But a gestation period of 20 years is typical for the transfer of technology from the laboratory to the marketplace. Can this be accommodated within the RAE?

Alan Shore, School of Electronic Engineering Bangor University.