Big science: sizing up a colossus

View the full World University Rankings 2015-2016 results

Authorship and its impact is one of the primary benchmarks of an academic’s success – influencing an individual’s career advancement and reputation, and increasingly used by funding agencies and regulators to evaluate research excellence at the departmental, institutional, national and international level.

But it has become a real challenge across the world.

A widely held assumption is that the authors listed on a paper are responsible for the new knowledge that a paper contains. It’s fair to say that this isn’t universally true; certain groups have been excluded from credit, while others have been included, despite a lack of involvement. In addition, there is no standard way of acknowledging the importance of specific individuals within papers with multiple authors. Different disciplines take different approaches, and many have no agreed approach.

In recent years, however, a new and more profound challenge has arisen, for understandable reasons. In the world of big science, where projects may last for years before results can be generated, and where teams of thousands are involved in moving towards the end goal, a new category of paper has developed. With multi-author papers there has been a regulated and concrete approach: all qualified participants are listed as authors, alphabetically, and in a way that prevents decomposition of the author list. These projects rely on teamwork, and the team is the unit that gets celebrated. This resolves the need to find ways of recognising those academics who are involved in a project, but who aren’t fortunate enough to be part of the far smaller team who analyse the results and write the papers.

For example, as the effort into particle physics has grown, driven by the need to delve deeper into the subatomic world, so academic involvement has expanded. From the initial synchrotron accelerators (such as the Bevatron), generating in the region of 6 GeV (gigaelectron volt), to the European Organisation for Nuclear Research’s (Cern) Large Hadron Collider’s new target of more than 13 TeV (teraelectron volt), the size and cost of projects has increased. This, in turn, results in the teams involved getting bigger.

The scale and range of work at the highest level across multiple disciplines is unparalleled in the history of science. High energy physics, alongside other big science projects, is a crucial part of the academic landscape. It has captured imaginations in a way that other subjects struggle to do. Physicists may not be happy with the term “God particle”, but it has clearly gained them many column inches of press.

The type of papers emerging from these projects – in particle physics, predominantly, but also increasingly in other disciplines – is qualitatively different from those in the rest of academia. For those interested in understanding academic excellence they pose a problem, and one for which there isn’t a perfect solution. Their difference means that conventional ways of evaluating papers and their citations can become distorted.

In the case of the overall Times Higher Education World University Rankings, we have always been interested in describing how universities perform at an institutional level – even in our subject-level rankings we do not break down performance below six broad areas (arts and humanities; clinical, pre-clinical and health; physical sciences; life sciences; engineering and technology; and social sciences). Beyond this, we try not to make assumptions about the validity of different research or different types of knowledge, and we count each author of a multi-author paper equally. This serves us well for relatively small numbers of authors, but proves challenging at the university level when the number of authors grows, especially in institutions with a strong focus in one specific area.

In the century ahead, perhaps billions of students will grace the halls – either physical or virtual – of universities and millions will advance the progress of human understanding. From time to time, one branch of knowledge or another will come to dominate according to the counting methodology in use at the time.

The impact of including these kilo-author papers is not always immediately obvious. However, our analysis has shown that for some institutions, those where a large proportion of their cited papers are in the kilo-author category, it has resulted in an over-emphasis on their performance. This is clearly not appropriate in a ranking designed to reflect overall performance, where from a data perspective removing any one subject area should not significantly change the picture of university performance.

This year, we decided to reduce this distortion by not including kilo-author papers in the bibliometric part of our ranking.

To be clear: this means that we include 11.3 million papers, published between 2010 and 2014, in the analysis for the World University Rankings, and we exclude 649 papers that have more than 1,000 authors. This amounts to the removal of 0.006 per cent of the papers analysed, and accounts for 0.04 per cent of all the 50 million citations to those papers.

We recognise that our decision, while making a very small difference to the overall rankings of most institutions, is a compromise.

Over the next year, we will consult widely to establish a framework for appropriate measurement of kilo-author papers, with the intention of reincorporating them in the next iteration of the rankings in a way that is fairer to all disciplines. As always, we welcome dialogue with any stakeholders engaged in academic publishing and research, in order to further develop our rankings.

Duncan Ross
Data and analytics director, Times Higher Education
duncan.ross@tesglobal.com
Twitter: @duncan3ross