Poor design of alternative medicine research is an ethical issue

Research in alternative medicine has boomed during the 30 years I have been studying it. In 1991, the Medline databank of medical research listed 90 papers in the category of “complementary and alternative” medicine; in 2021, the figure had risen to 1664. Many of these articles report positive findings, too. Alternative medicine bears the superficial hallmarks of a flourishing field.

However, once you carefully consider the plausibility and quality of the research – which I do regularly on my blog – you are bound to get disillusioned. The vast majority of the positive studies of alternative medicine lack scientific validity.

This, of course, raises the question of who is responsible for this situation. In the first instance, the blame clearly lies with the researchers and their supervisors. Then there is the funder of the project. Yet studies of alternative medicine are often conducted without external funding or with support from private donors, therefore bypassing the scrutiny of grant committees.

However, since 1991, all research involving human subjects has been required to undergo an independent assessment of its ethical acceptability. You might argue that this is an entirely separate issues, but I disagree. Poor-quality research is ethically unacceptable. It is a waste of resources. It abuses the cooperation of the study participants. It prompts patients to make wrong and potentially dangerous therapeutic choices. And, ultimately, it undermines trust in science.

Hence, ethics committees that approve poor-quality alternative medicine studies also bear some culpability for their flaws. Unfortunately, it is my impression that studies of alternative medicine are considered to be less serious than other research and are thus judged less stringently by research ethics committees (REC)s, which see their main task as preventing harm to research subjects.

The possibilities of designing studies of alternative medicine such that they generate a false-positive finding are almost endless. For instance, a recent trial tested whether visceral osteopathy (VO) is effective for polycystic ovary syndrome (PCOS). Thirty women received either VO plus a low-calorie diet or the diet alone with no further therapy. The authors found that the VO group had “greater improvement in menstrual pain, irregularities, and premenstrual symptoms”.

Another trial tested acupuncture as a treatment for obesity. Fifty women were treated with acupuncture, while the control group received no treatment at all. The authors found that acupuncture “promoted a reduction in abdominal waist circumference, supra-iliac and abdominal skin folds, and percentage body and abdominal fat”.

A third study tested an Ayurvedic medicine, AYUSH-64, on Covid infections. Eighty patients received either the remedy plus conventional care or conventional care alone. The authors found that “AYUSH-64 as adjunct to standard care is safe and effective in hastening clinical recovery in mild to moderate COVID-19.”

It is easy to see how these conclusions might fool us. All three studies are “randomised clinical trials” and, thus, might seem reliable. Yet, they have major design flaws. Most importantly, none of them made any attempt to account for placebo effects. Arguably, with implausible treatments, it is more likely that the results have nothing to do with the tested therapies but are entirely due to such effects.

In all these instances, the flaws would have been apparent already when the protocols were submitted for ethical review. Yet the RECs evidently did approve them. This means that the committees failed in their duty of ensuring the ethical acceptability of these research projects.

The issue could be trivial if it arose only in isolated cases. However, the literature of alternative medicine is littered with fatally flawed studies. In fact, it is hard to find a trial of alternative medicine that is rigorous among the abundance of studies with serious design flaws arriving at unjustifiably positive conclusions.

If RECs would ensure that studies with serious design flaws are amended or aborted, these problems could be avoided. At present, RECs across the world rarely include experts on research methodology and seem too lenient with studies of alternative medicine.

Therefore, I urge RECs to make sure they are equipped to adequately check whether research projects are properly set up to find a conclusive answer to the research questions they purport to address. For studies testing alternative treatments, this scrutiny should be just as stringent as for the rest of medical research.

Edzard Ernst is emeritus professor of complementary medicine at the University of Exeter.