Research intelligence - A measure of humanities?

It is widely recognised that when it comes to measuring performance in the humanities and social sciences, citations - the number of times an academic's work is cited by their peers - work either poorly or not at all.

For a start, the main bibliometric databases that tally up citation counts - Thomson Reuters' Web of Science and Elsevier's Scopus - consist mainly of international journals written in English, with scant regard for regional and non-English titles, although the companies are working on this.

Another problem is that, as opposed to the sciences where journal papers are the norm, many non-science scholars write books instead.

It comes as no surprise that in the forthcoming research excellence framework, arts, humanities and many social science subject panels are not even expected to ask that citation data be made available to help guide assessments. However, a new European project could change all that.

The project, sponsored by four national funding agencies - the UK's Economic and Social Research Council, the French National Research Agency (ANR), the German Research Foundation (DFG) and the Netherlands Organisation for Scientific Research (NWO) - is investigating whether it is possible to develop a database capable of capturing a wider range of research outputs from the humanities and social sciences. It could then be used to determine the academic impact of the work, for example by counting citations.

The group overseeing the project is led by Ben Martin, professor in the Science and Technology Policy Research (SPRU) unit at the University of Sussex. A draft document outlining the idea, Towards a bibliometrics database for the social sciences and humanities - a European scoping project, was presented to scholars for the first time at a meeting in Berlin last month.

The project is a response to governmental pressure for more accountability in research funding, Professor Martin said. But it also aims to provide an alternative to the European Science Foundation's much-loathed European Reference Index for the Humanities (ERIH), which has caused a storm in the academy by categorising journals in three classes (A, B and C) depending on their global significance.

This has created a de facto hierarchy that some institutions and funders are said to be unofficially using to decide promotions and grants.

That the ERIH also encroaches on social science journals appears to be the main reason for the ESRC's desire to develop another approach.

Professor Martin said that funders wanted to explore an alternative to the ERIH and investigate "whether in principle it might be possible to construct a more useful and realistic database".

This would include not only international and regional journal articles, but also books and chapters aimed at both academic and general audiences, "grey literature" such as policy reports, and even non-published output such as performance.

Works included would probably have to be peer-reviewed, which would pose problems given that some top humanities journals rely on editorial judgment, and book publishers and journals could be graded according to quality. The project could be rolled out in phases, with journals first, followed by books and then the other elements.

No easy task

The hurdles to success, however, are huge, and Professor Martin stressed that nothing has been set in motion yet. The group plans to present recommendations to the funders at the end of the year, one of which is likely to call for a pilot to test the scheme's feasibility.

But it is unclear whether the benefits of implementation would outweigh the costs.

Professor Martin said: "It would be perhaps five to ten years before we have usable, reliable results."

Opposition from academics also seems to be brewing. Judi Loach, director of Cardiff University's Graduate School in Humanities, who attended the Berlin meeting on behalf of A-Hug, a group that brings together the UK's learned societies in the arts and humanities, described the proposals as "potentially more dangerous than the ERIH".

"No one in the humanities agrees we need bibliometrics. We don't think it is appropriate to our disciplines," she said.

"The worry is that (the database) will be easier to slide through than the ERIH because the proposals have more academic rigour."

She also lamented the lack of academic consultation, the failure to involve the Arts and Humanities Research Council, and the choice of the SPRU to lead the project.

"(SPRU) has nothing to do with the humanities," she said.

zoe.corbyn@tsleducation.com

www.sussex.ac.uk/Units/spru/esf/.