Covid-19 has exposed the dirty secrets of medical science

Unscientific practices are a long-standing scandal in medical research. Medical studies are often impossible to replicate and colossal sums of research money are lost to avoidable research waste. The coronavirus pandemic has amplified these weaknesses and exposed the vulnerability of health research to distortions of hype, bias and commercial interests.

The vulnerability of the research system was exposed early in the pandemic when a flawed study on the medicine hydroxycloroquine led to months of uncertainty and many patients treated without benefit before adequate research emerged. Medicine’s dirty secret is out. We need to take this opportunity to raise the bar.

The public should demand much more from medical research, the vast majority of which is funded by their taxes and charitable donations. Policymakers and doctors talk about empowering patients to make their own decisions. If we take that commitment seriously, we should not expect patients to blindly trust research – we should ensure research findings are robust, reliable and open to all.

Changing research culture is possible and some hard-won reforms have brought improvement. But huge problems remain. The incentive system for research is very poorly aligned to reliable scientific discovery. It is often impossible to find out if those involved in research have conflicts of interest. Journals privilege positive or novel findings, skewing the published scientific record. For most types of research, there is little expectation that scientists preregister their methods – leaving open the possibility the results have been cherry-picked and are unreliable. Even when studies are registered and published, the detailed documentation necessary to understand how the research was carried out is frequently unavailable.

The scale and complexity of the challenges are daunting. Instead of setting out to fix health research overnight, we should take inspiration from the successes of reforming social movements. More than a century after the achievement of women’s suffrage and 50 years after the US Civil Rights Act, the causes of gender and racial equality are far from satisfied. But these successful campaigns for reforms were important for the transition towards a fairer society.

As part of an international collaboration, we have proposed three actions that we believe should be adopted to help reform health research.

The first is the mandatory registration of conflicts of interest. All those involved in research, including editors, peer reviewers and patient groups, should declare their interests on a central accessible register. This could easily be indexed through existing researcher IDs and linked to all publications.

The adoption of registered reports by academic journals is the second important step. These reports are a publication format in which authors submit study design and detailed methods prior to data collection or analysis. If they satisfy peer review, journals commit to publication regardless of whether the subsequently obtained results are statistically significant or “noteworthy”. This format has the potential to greatly reduce biased reporting, but only about 1 per cent of journals indexed by the US National Library of Medicine accept registered reports.

Finally, there is also a pressing need for mandatory registration of publicly funded research. Comprehensive documentation, such as protocols, analytic code and, when possible, study data for all publicly funded research should be made available on a repository affiliated to the World Health Organization. Making such information easily and openly accessible will allow more rigorous evaluation and help ensure that findings are reproducible.

These measures have been selected as achievable actions that would deliver immediate improvements to health research. We believe they represent a minimum level of transparency that patients and the public should be able to expect from the research they fund and rely upon. The ideas are not novel – we need to push for basic and well understood measures that will improve research quality.

Some researchers ask why we should have to change what we do because of a few rotten apples – who won’t stop breaking the rules anyway. The answer is that, unfortunately, the vast majority of failings in research are not a matter of fraud. The problems are much more widespread and insidious.

Layers of dysfunctional incentives to produce fast, “positive” and novel results are compounded by the lack of ability and capacity of others to verify analyses or reproduce results. Peer review, while important, cannot by itself counterbalance these forces, and relying on the supposedly “self-correcting mechanisms” of science is probably naive. Although particularly egregious errors are sometimes rectified, inaccurate reporting of studies is common.

Such inaccuracies are unrealistically burdensome for peers to routinely identify, and, even when asked, journals and authors frequently ignore calls for correction. These deep systemic biases require system-wide improvements beyond calls for education and injunctions for researchers to “be better”.

As we begin to emerge from a pandemic that has so clearly revealed the disappointing state of medical research, we need to own up to the problems and advocate for specific solutions.

Stephen Bradley is a GP and a researcher investigating cancer diagnosis at the University of Leeds (disclosure statement). He is also helps organise the University of Leeds ReproducibiliTea journal club. Jessica Butler is a research fellow at the University of Aberdeen Centre for Health Data Science and the local lead for the UK Reproducibility Network. Chris Chambers is professor of cognitive neuroscience at Cardiff University and a member of the UK Reproducibility Network’s steering group.