We could take any of the many uses for which acupuncture is promoted as an example – for example, Bell’s Palsy (BP). BP is paralysis usually of one side of the face caused by inflammation of the facial nerve where is passes through a long bony canal to exist the skull. Within this bony pathway the nerve has no room to expand, and so the swelling caused by the inflammation compresses and damages the nerve, causing facial weakness.
How might sticking thin needles into the body affect the course of this illness? It can’t, is the obvious answer. Acupuncture.com offers an answer, however:
“The treatment principle used in this patient's case was to use a few distal and several local acupuncture points to eliminate wind, regulate qi and blood in the Shaoyang and Yangming channels in order to remove obstruction and restore facial nerve function.”
Dysfunction of the facial nerve is caused by a bit of excess wind, you see. There’s nothing superstitious about that perfectly cromulent explanation.
What does the actual clinical evidence have to say about the effectiveness of acupuncture for BP? A 2010 Cochrane review is very revealing.
“Poor quality caused by flaws in study design or reporting (including uncertain method of randomization, allocation concealment and blinding) and clinical differences between trials prevented reliable conclusions about the efficacy of acupuncture.”
When reviewing the acupuncture literature a few revealing patterns in the evidence emerge. One is that, among the few high quality clinical trials, the evidence shows that there is no difference in outcome regardless of where you place the needles, or even if you place the needles. In other words, the best evidence shows that acupuncture does not work.
Reviews and meta-analyses show either no effect or at best a tiny clinical effect, which is insignificant. More importantly, such tiny effect sizes are not reliably real. Clinical trials are not precise and unbiased enough to weed out systematic effects that can cause small residual, and completely illusory, effect size.
This is a pattern we are familiar with from many areas of critical analysis. Whenever the phenomenon of interest is likely not real (ESP, homeopathy, or human energy fields, for example) quality studies show no effect, moderate studies show a tiny effect that is easily due to error, and poor quality studies are all over the place but mostly positive. This is what the literature looks like when the phenomenon is not real – and this is what the acupuncture literature looks like.
The real question is this – why are there so many horribly poor quality acupuncture studies? At this point it is almost medical malfeasance to perform yet another crappy acupuncture study, the results of which are worthless to science (but highly useful to those who wish to promote a bogus therapy). The literature is flooded with these poor studies, causing nothing but noise and confusion (and work for reviewers).
It does allow for proponents to make unsubstantiated claims and have lots of references, which appear to back up their claims – references to worthless studies that show exactly what the researchers want them to show.
Acupuncture has a particular problem with poor quality studies as biases. A 1998 review found that 100% of acupuncture studies published in China (and also Russia) were positive – 100%. That’s compared to 75% of acupuncture studies published in the UK. That is extremely statistically unlikely, even if acupuncture worked (which it likely doesn’t), and is very compelling evidence for profound systematic bias in those countries (with Japan and Taiwan being almost as bad). I would like to see an update to the review, but I doubt the problem of systematic bias has been entirely solved.
In addition there are a great deal of biased and flawed systematic reviews of acupuncture coming out of China –biased reviews of biased studies.
Despite all of this clear bias, the evidence that acupuncture works for anything (when looked at objectively) is convincingly negative.
The most recent published study of acupuncture for Bell’s Palsy has been receiving a great deal of press, all glowingly reporting how acupuncture works for Bell’s Palsy. This study compared acupuncture to “intense” acupuncture in which the needles were inserted until they achieved “de qui” which is a certain sensation on the part of the person receiving the acupuncture. The results are reported as positive, but there are two good reasons to be skeptical of this study.
The first is that, when you read through the complete description of the study you will find the notable absence of the word “blinded.” There is no mention of any blinding in the study, which makes sense since the defining feature of the treatment group is a subjective experience of the subjects. Unblinded clinical studies are practically worthless.
The second problem with this study is that it comes out of China. While that might not sound politically correct, the published evidence shows clear, and in fact complete, bias toward acupuncture in Chinese research. The burden of proof is now on Chinese researchers to show that they have taken steps to minimize this bias before any acupuncture research out of China should be taken seriously. Lack of proper blinding is also not the only problem with acupuncture research coming out of China, which tends to be plagued with poor randomization and other significant methodological flaws.
Essentially we have an unblinded acupuncture study out of China showing that magic works. This is not exactly what I would characterize as compelling evidence.
Even if we set aside the issue of proven bias toward acupuncture, the study is of poor quality and does not meet the generic criteria for well-established science. For any new phenomenon to be considered reasonably demonstrated we would need to see rigorous studies with both statistically significant and clinically significant effects that are reliably independently replicated. This is a standard and fair threshold for acceptance, even for plausible claims. Acupuncture has not come anywhere near this threshold for any indication, and I would not consider it a plausible claim.
Steven Novella, M.D. is the JREF's Senior Fellow and Director of the JREF’s Science-Based Medicine project.