The subject line might strike you as a tad obvious, but a new study published by the journal, BMJ Acupuncture in Medicine, appears to claim that this is exactly what you should do, and asserts that their latest study proves this. Many media outlets in the US have been promoting this, but not so much in the UK where the BBC got a subject matter expert in to review and tell them what was really going on.
TL;DR; … basically no, the claimed results are not backed up by any actual data.
Side note: The journal “BMJ Acupuncture in Medicine” is not the BMJ, but instead is a specific journal that focuses on Acupuncture.
First the Basics – What exactly is acupuncture and why is it popular?
Most of us are perhaps familiar with the concept. This is where you basically stick thin steel needles in and thus, supposedly, manipulate qi, an alleged life-force that is part of Traditional Chinese medicine.
What might not be appreciated by many is that acupuncture, as it is now practised in the west, only became popular in 1970s. The basis for that popularity rests upon one entirely false claim …
Its revival in the West was largely a result of a single anecdote promulgated by journalist James Reston in the New York Times after he had acupuncture in Beijing for postoperative pain in 1971. Despite his eminence as a political journalist, Reston had no scientific background and evidently did not appreciate the post hoc ergo propter hoc fallacy, or the idea of regression to the mean.
After Reston’s report, acupuncture quickly became popular in the West. Stories circulated that patients in China had open heart surgery using only acupuncture.6The Medical Research Council (UK) sent a delegation, which included Alan Hodgkin, to China in 1972 to investigate these claims, about which they were skeptical. The claims were repeated in 2006 in a British Broadcasting Corporation TV program, but Simon Singh (author of Fermat’s Last Theorem) discovered that the patient had been given a combination of 3 very powerful sedatives (midazolam, droperidol, fentanyl) and large volumes of local anesthetic injected into the chest. The acupuncture needles were purely cosmetic.
Well yes, the BBC got it wrong then and started it all, so it is good to see that they learned from that.
Is there any evidence that acupuncture works for anything?
Short answer … no.
Clinical research can never prove that an intervention has an effect size of zero. Rather, clinical research assumes the null hypothesis, that the treatment does not work, and the burden of proof lies with demonstrating adequate evidence to reject the null hypothesis. So, when being technical, researchers will conclude that a negative study “fails to reject the null hypothesis.”
Further, negative studies do not demonstrate an effect size of zero, but rather that any possible effect is likely to be smaller than the power of existing research to detect. The greater the number and power of such studies, however, the closer this remaining possible effect size gets to zero. At some point the remaining possible effect becomes clinically insignificant.
In other words, clinical research may not be able to detect the difference between zero effect and a tiny effect, but at some point it becomes irrelevant.
What David and I have convincingly argued, in my opinion, is that after decades of research and more than 3000 trials, acupuncture researchers have failed to reject the null hypothesis, and any remaining possible specific effect from acupuncture is so tiny as to be clinically insignificant.
In layman’s terms, acupuncture does not work – for anything.
There is a new study that claims to prove it works
Acupuncture may alleviate babies’ excessive crying
Needling twice weekly for 2 weeks reduced crying time significantly
Acupuncture may be an effective treatment option for babies with infantile colic—those who cry for more than 3 hours a day on 3 or more days of the week—reveals research published online in Acupuncture in Medicine.
Read on within that press release and you might indeed conclude that it is real evidence.
However, a review of the actual study itself reveals that this not not true at all.
Professor David Colquhoun explains …
The paper made the most elementary of all statistical mistakes. It failed to make allowance for the jelly bean problem.
The what????
XKCD explains the Jelly Bean problem like this …
So what does that actually mean?
It works like this. If you test, again and again and again and again, then random chance means that you will get a few hits. These are false positives.
Overall in the study they did 24 tests and out of these, they found that just three gave P value of 0.05, so they focused down on just these three false positives and declared it a win.
The term used to describe this is P-hacking and is a well-known statistical problem.
The key question is this. If you perform a study, and then declare the results, how do you establishthat what you declare is a meaningful observation and not simply random chance?
There is a rather interesting article that digs into that question here in detail. David also has a paper here that explores the impact of the false discovery rate when you use p=0.05 to suggest that you have made a discovery. You simply can’t do this because statistically you will be wrong at least 30% of the time.
However, we need not revet to statistical analysis to work out that sticking needles in a baby is not exactly going to be a good way to get it to stop crying. Treatment for Colic is itself a tad dubious. It is not strictly speaking an illness, but instead is a clinical syndrome. The term means that a baby has been crying for three hours or more, so deploying Acupuncture for Colic is not exactly curing an illness. There is a lot of scope for other things causing the baby too stop crying such as a cuddle. It is going to stop quite naturally and so the possibility that what was done being the actual cause of the resolution is about as dubious as claiming that because I cleaned my loft and it then rained I can concluding that loft cleaning causes rain.
In other words, within tests like this you will get random hits, it is inevitable.
So does Acupuncture work?
After over 3,000 trails that demonstrated nothing credible, I’m not buying into the idea when faced with a bit of rather dubious P-hacking, and neither should you.
The Interview
Professor David Colquhoun, who wrote the spectator article, was interviewed on the BBC Radio 4 Today program along with Dr Cummings the journal editor who is very much an acupuncture believer …
The text under the following clip explains …
I was ased to give an opinion on a dire paper published by the BMJ journal, Acupuncture in Medicine. The statistical analysis was so bad that even Mike Cummings, an editor if the journal that pubished it, admitted that the paper failed to show anything useful. He didn’t explain why, that being the case, they’d published it.
There are some technical details in the Science Media Centre handout. at http://www.dcscience.net/Science-Medi…
The most important comments were “What’s the first thing that any responsible parent thinks of if their baby is crying? Stick needles into it!”. And of course the appropriate control group would have been to cuddle the baby.
Further Reading
- Professor Edzard Ernst – Acupuncture study on colicky kids shows the opposite of what the authors conclude
- Professor David Colquhoun – Acupuncture isn’t the answer to a crying baby
- Dr Steven P. Novella of Science Based Medicine – Acupuncture for Infantile Colic
- Professor Edzard Ernst – Acupuncture and the disgraceful Dr Cummings
- The actual BMJ Acupuncture in Medicine paper – Effect of minimal acupuncture for infantile colic: a multicentre, three-armed, single-blind, randomised controlled trial (ACU-COL)
- The press release associated with the above paper
Final Quote
Despite these rather obvious – some would say fatal – flaws, the editor of ACUPUNCTURE IN MEDICINE (AIM) thought this trial to be so impressively rigorous that he issued a press-release about it. This, I think, is particularly telling, perhaps even humorous: it shows what kind of a journal AIM is, and also provides an insight into the state of acupuncture research in general.
The long and short of it is that conclusions about specific therapeutic effects of acupuncture are not permissible. We know that colicky babies respond even to minimal attention, and this trial confirms that even a little additional TLC in the form of acupuncture will generate an effect. The observed outcome is most likely unrelated to acupuncture.
Professor Edzard Ernst, writing about this study here.