What we value about research needs to change

We have no shortage of research activities in today’s digital age, when innovation is a top priority for most nations.

Indeed, every year, according to Australia’s chief scientist, Alan Finkel, more than a quarter of a million new PhDs are created globally. Since 2011, China alone has added more than a million people to its research workforce.

Through toil and patience, researchers have uncovered valuable insights alongside occasional breakthroughs throughout their demanding careers. But the time has come for change. Not in individuals’ dedication to scientific research but rather in the way that research endeavours are assessed for the purpose of funding approval, as well as promotion and tenure.

For many years, under the entrenched system of assessing research on the basis of metrics – predominantly the number of publications and citations – academics have often been tempted to comply rather than make full use of their inquisitive minds.

Being productive has always been crucial to researchers’ career development, but strong calls for tighter control over the process have been made repeatedly – most recently at the sixth World Conference on Research Integrity held at the University of Hong Kong in early June.

That up to 700 academics, government officials, journal editors and funding agency representatives from around the world attended the conference underlines the international community’s deep concern about how we should go about scientific pursuits.

Finkel and other plenary speakers made it clear that with the millions of research articles published each year, much more needed to be done to ensure robust, relevant and reliable research. The long-held, traditional assessment criteria are not necessarily conducive to responsible or impactful endeavours.

Removing the predominant focus on quantity of output means introducing disincentives for questionable practices such as selective reporting, using poor methodology, publishing in journals with zero citations, or dividing up authorship so that individual researchers get more publications.

Changes are already happening in some countries. The Irish Health Research Board recently implemented a strict requirement that every investigator be able to prove that they have completed a research integrity training course. The Wellcome Trust, the UK charity devoted to biomedical research, is also implementing broad quality and integrity policies.

According to Maura Hiney, the head of post-award and education at the Irish Health Research Board, another plenary speaker at WCRI, the European University Association recently published a joint statement with Brussels-based Science Europe – an association of European Research Funding Organisations and Research Performing Organisations – saying they are trying to broaden assessment criteria in the university sector.

The moves are in keeping with the principles contained in the Singapore Statement developed at the second WCRI, held in 2010 – namely honesty, accountability, professional courtesy and fairness, and duty of care at all stages of the research process and for all actors.

Hiney rightly pointed out the need also for transparency, both in the research process and in the presentation of results.

To secure public trust in the wide-ranging, often time-consuming, efforts funded with public or donors’ money, researchers have an obligation to communicate clearly and openly, for example the objective and limitations of their research, and to engage with end users when possible.

For the sake of societal benefits, for example, it would be desirable to transform findings into products or services, be it a longer-lasting hip replacement or software that can diagnose diabetic blood in the elderly before transmitting the result to a pharmacy to give patients a new prescription.

At the conference, it was encouraging to see the unveiling of the Hong Kong Principles for Assessing Researchers: Fostering Research Integrity, written by a group of concerned academics as guiding principles to help institutions improve the current system of faculty incentives and rewards.

These principles involve:

• assessing researchers (and institutions) on responsible practices from conception to delivery, including the development of the research idea, research design, methodology and execution, and effective dissemination
• valuing the accurate and transparent reporting of all research, regardless of the results, and rewarding honest and transparent reporting
• valuing the practices of open science – such as open methods, materials and data – when feasible
• valuing a broad range of research and scholarship, such as replication, synthesis and meta-research
• and valuing a range of other contributions to responsible research and scholarly activity, such as peer review for grants and publications, mentoring, outreach and knowledge transfer.

Peer review training for academics and the incorporation of research integrity courses into the graduate curriculum are important steps towards beefing up the research culture. As noted by David Moher, an associate professor in the School of Epidemiology and Public Health at the University of Ottawa: “The only way science progresses is that people peer-review other people’s works.”

But we also need the support of ranking agencies. It will be hard to drive changes in ranking-obsessed institutions if the agencies take little heed of the principles advocated above.

Indeed, institutions should earn an acclaimed reputation not by churning out publications but rather by producing relevant and trustworthy research that improves mankind’s well-being.

Mai Har Sham is associate vice-president (research) at the University of Hong Kong.