UK university launches responsible metrics guide

Loughborough University has launched a formal policy on the responsible use of research metrics.

The statement, which includes 10 elements to consider when evaluating research, looks at how the university can use metrics data sensibly.

James Wilsdon, professor of research policy at the University of Sheffield and chair of the Higher Education Funding Council for England’s review on responsible metrics, called Loughborough’s statement the most comprehensive he had seen to date.

But he added that using the principles in hiring, promotion and evaluation criteria would be a bigger challenge than signing up to the manifesto.

The evaluation of research is increasingly dependent on the use of metrics – data-driven statistics that are used to give an indication of research quality. Commonly used metrics include bibliometrics, such as citation counts and journal impact factors.

Metrics are now often used in funding decisions, the recruitment and promotion of staff and to measure the visibility of research. But critics fear that number crunching is not able to give the full picture of research quality.

Loughborough’s research policy manager, Lizzie Gadd, said: “There are lots of things that affect citation rates, some of which you can normalise for, such as the age, field and type of publication. And some of which you can’t, such as the gender and career stage of the author, the type of research and the newness and size of the discipline.”

There have been several attempts to encourage the research community to use metrics more responsibly, such as the San Francisco Declaration on Research Assessment and the Leiden Manifesto for Research Metrics. These are voluntary codes that universities and funders can follow.

Hefce's independent review, published in 2015, also called for a more responsible use of metrics.

In response to these pressures, Loughborough established in 2016 a small working group of academics and administrators to consider the issues surrounding the use of metrics. The result was the new policy.

The policy is based heavily on the Leiden Manifesto, adapting the key principles for Loughborough's context. The principles include using metrics at the same time as expert assessment, making the collection of data and analytic processes open, transparent and simple, and ensuring that career stage, gender and discipline are taken into account when interpreting metrics.

Professor Wilsdon said: “There have been a series of calls for universities and research funders to adopt more responsible uses of metrics.

“But signing up to manifestos is the easy part: the bigger challenge is to embed such approaches in institutional systems, and in the criteria used for hiring, promotion and evaluation.

“A few universities have taken steps towards this, but Loughborough’s policy is the most comprehensive I’ve seen to date, and sets a new benchmark that I hope other universities will follow."

holly.else@tesglobal.com