Ask any natural health practitioner and they will tell you that they know their methods work, because they see results in their clinics every single day. But their approaches are not widely accepted because of a lack of evidence from randomised, controlled clinical trials (RCTs) verifying the effects. The reasons? In short, RCTs are both prohibitively expensive and unsuited to evaluating multi-modality approaches or interventions in real-world environments. Enter comparative effectiveness research (CER).

Limitations of RCTs

In order to achieve the unbiased results for which they are so esteemed, RCTs must be performed under very strict, ‘ideal’ conditions. However, because healthcare is practised very differently in the real world, often combining many different approaches along with varying levels of patient engagement, the extent to which patients, practitioners and healthcare decision-makers can rely on RCT evidence when making important decisions is highly questionable.

More than that, as pointed out by Marcia Angell MD, a past editor of one of the world’s most prestigious and generally pro-pharmaceutical journals, the New England Journal of Medicine, RCTs are often biased. Dr Angell offers various reasons for this in her blockbuster 2004 book, The Truth About the Drug Companies: How They Deceive Us and What To Do About It, but most of the problems stem from the fact that RCTs are overwhelmingly funded by Big Pharma, the prime group with a resource base large enough to afford them.

Efficacy vs. effectiveness

This observation is nothing new. In healthcare research, there is a crucial distinction between the terms ‘efficacy’ and ‘effectiveness’. Efficacy is, “The ability of an intervention to produce the desired beneficial effect in expert hands and under ideal circumstances” (RCTs),while effectiveness is, “The ability of an intervention to produce the desired beneficial effect in actual usage” (real world).

Implications for natural healthcare

Natural healthcare particularly suffers from the inability of RCTs to describe the real world, partly because it is based upon creating personalised programmes for individual patients, and partly because many natural healthcare disciplines utilise several strategies in each patient. Traditional Chinese medicine (TCM) practitioners, for example, will prescribe multi-herb formulations, acupuncture, Qi gong, moxibustion and others either alone or in combination, according to patient need.

Enter comparative effectiveness research – and Claudia Witt

Comparative effectiveness research (CER) may provide the answer to this quandary. CER as applied to complementary and alternative medicine (CAM) is a speciality of Professor Claudia M Witt of the Institute of Social Medicine, Epidemiology, and Health Economics, Charité University, Berlin, Germany.

Of course, the approach also is gaining increasing recognition as a means of comparing conventional treatments. CER is particularly applicable for complex, chronic conditions, such as type 2 diabetes.

The US Institute of Medicine defines CER as, “The generation and synthesis of evidence that compares the benefits and harms of alternative methods to prevent, diagnose, treat and monitor a clinical condition or to improve the delivery of care”. Prof Witt points out that, unlike RCTs, CER, “Compares two or more health interventions...in order to determine which of these options works best for which types of patients”.

In effect, CER may well be the best tool we currently have to help healthcare providers decide which treatments or healthcare approaches are most appropriate for particular patient groups. The methodology is equally adept at dealing with individual modalities, such as acupuncture, herbal medicine or nutritional interventions, as it is with multiple modalities, where various approaches are combined – diet and lifestyle modification, for example. For us, CER’s powerful ability to study these multi-modality approaches – especially when compared with uni-directional pharmaceutical interventions – may help create better outcomes for patients.

Guidance for researchers

Of course, researchers can’t just jump into the CER field with their eyes closed and expect high-quality results. So-called ‘evidence guidance documents’ (EGDs) are designed to address the major elements of study design, such as patient inclusion, settings and comparators. Prof Witt recently published an EGD for prospective acupuncture studies – the first of its kind in natural medicine – and is working on another for TCM as a whole.

What are we waiting for?

There are two very important lessons that the natural health community can take from CER. One is that there are very good reasons why the feted RCT should not be regarded as the gold standard when trying to establish the most effective treatment regimen for particular patient groups with specific conditions. Second, performing trials that contribute to CER is both easier and cheaper than setting up classical RCTs. While recognising the importance of randomisation, CER also sees the value of alternatives such as dynamic allocation and – most importantly – does not automatically demote observational data to the bottom of an evidence hierarchy.

The result is that CER, “Improve[s] the external validity of clinical research to enable decision makers to make informed decisions [and] contribute to a more strategic use of limited research resources and more consistency in trial design”. CER offers a powerful hand to natural health researchers drowning in a sea of negative publicity stemming from a perceived lack of evidence. If CER makes trials far easier and cheaper to set up and run than ever before, and if a wider range of evidence can be included in the analysis, what’s stopping researchers from getting multiple trials up and running as soon as possible? 

CER could well be the solution to an old dilemma. It’s time for natural healthcare to grasp the nettle.

What next?

• We’d be very interested to hear from any researchers currently using, or planning to use, CER to compare CAM modalities, especially any researchers intending to compare CAM modalities with conventional ones. Please contact our Science Unit at [email protected] so that we might learn more about your work and use it to inform our advocacy strategies with governments. At present, and almost without exception, governments are wedded to RCTs as the only reliable form of evidence
• Similarly, if you’re a natural health practitioner, why not consider getting involved in the world of clinical research – along CER lines? Data from your day-to-day interactions with patients could form an important part of a future trial, designed to demonstrate the effectiveness of your chosen modality or modalities under real-world conditions
• If you’re a patient or client who has received CAM treatments, don’t be surprised if your personal experiences are at odds with published critiques of CAM methods or various forms of pro-pharmaceutical propaganda – which include articles in many national newspapers! As we showed in our 2007 analysis of the work of Prof Edzard Ernst, responsible for the largest body of negative evidence on CAM, RCTs are of limited value in healthcare decision-making.

ANH-Europe Homepage

ANH Good Science campaign page