A Navy neurologist’s credulous venture into acupuncture advocacy serves as a useful case study. Here are twelve mistakes he made rambling down the garden path of self-delusion.
He posted a three-part article on “The Power of Acupuncture” (Hopkins 2011–2012) on Navy Medicine Live, the official blog of Navy and Marine Corps Health Care. It is a prime example of how even the most intelligent, educated person can ramble step by step down the garden path into self-delusion. Dr. Hopkins’s story can serve as a useful lesson to all of us.
Background
Acupuncture has been increasingly accepted in military circles. The Air Force is teaching its doctors “battlefield acupuncture” (Gorski 2008) based on the faulty evidence of one Air Force doctor, Col. Richard Niemtzow. The Army is using it to treat PTSD. The Navy offers it too.
Hopkins says that after forty years of practicing neurology, “It was only natural to begin thinking about something else.” (Why? Boredom? And why pick acupuncture?) When he got an email from his Specialty Leader announcing the opportunity for Navy doctors to learn how to do acupuncture, he submitted his application that same day. He was undoubtedly impressed that this training was being offered by the Navy, lending it the imprimatur of authority. His prior impression of acupuncture was that it was a “mysterious tool” that seemed to work, and instead of asking the critical questions, he said he was looking for “a fundamental scientific understanding of acupuncture” and asking to see the supporting research and data.
Mistake #1: Prior prejudice. It sounds like he already had a favorable opinion of acupuncture and was predisposed to accept it.
Mistake #2: Confirmation bias. He was looking only for confirmation rather than also looking for any disconfirming research. He wanted to understand how it worked; he was not asking whether it worked.
He says his intellect was actively engaged by the teacher, a “charismatic master acupuncturist,” who laid a neurophysiologic foundation for how acupuncture works.
Mistake #3: Getting information from a biased source. A master acupuncturist is hardly likely to present a balanced picture of the evidence for and against the source of his livelihood. In calling the teacher “charismatic,” he might have suspected that he could be influenced by that charisma to accept things he would not have accepted as readily from a dry, objective presentation of scientific evidence about acupuncture’s validity.
Hopkins was told about local physiologic changes in tissues stimulated by needles. He was not told that non-needle “acupuncture” (with electrical stimulation through intact skin or with simple touching with toothpicks) had been shown equally effective, even when acupuncture points were avoided. He was told that needles caused reversal of tissue acidosis. (This is a claim I don’t remember hearing before, and I think it is based on a couple of Chinese studies on animals. Even if true, its clinical relevance would be questionable.) He was told about the “gate control” hypothesis but was not told that after half a century of investigation it has not been accepted as the explanation for acupuncture’s effects. He was told about brain MRI findings and endorphin release but was not told that those same findings can be elicited by placebo pills. I see them as evidence of the mechanism for acupuncture’s placebo effects; he interprets them as evidence that acupuncture “resets normal controls within the autonomic nervous system and maintains CNS homeostasis,” though it’s not clear what that even means; it sounds to me like typical alternative medicine pseudoscientific doublespeak.
Mistake #4: Cherry-picking the literature. The charismatic master acupuncturist snowed Hopkins with every shred of data that might possibly support a physiologic mechanism for acupuncture, even providing “an extensive reference library.” Did he include the studies showing that it doesn’t matter where you put the needles? Did he list the high-quality trials showing that sham acupuncture works just as well? Did he list all the negative systematic reviews or Edzard Ernst’s recent systematic review of systematic reviews (Ernst et al. 2011) of acupuncture for pain? It is obvious that he cherry-picked the literature to support his claims. This is easy to do: plenty of low-quality studies of acupuncture have been published.
In Part Two of his article, Hopkins actually asks if the clinical effects might be due to placebo. He wonders how we would know, since there is “no honest way” to do a properly controlled double-blind study. His teacher says it is better to go by the functional outcome rather than by patient reports of pain levels. So far, so good. But then Hopkins throws science out the window and never mentions placebo again. He actually writes, “There is nothing like personal experience to convince one of an effect. It is a bit like not requiring a double blind placebo controlled cross-over study to establish that an open parachute is more effective than a closed one.”
This sounds like it was written by someone ignorant of science and logic rather than by a neurologist. His analogy is a clichéd fallacy: We don’t accept the effectiveness of parachutes because we have had personal experience jumping out of planes. I think he meant to say that not every claim requires proof by placebo-controlled trials, which is true but irrelevant here. We don’t need to do controlled trials to find out if it is effective to stop hemorrhages or set broken bones, but we do require controlled trials to find out if acupuncture works. It’s true that personal experience is the best way to convince someone there is an effect, but it’s useless for determining whether there really is an effect. To correct his statement: There is nothing like solid scientific evidence to convince a scientist who knows better than to accept personal experience as evidence.
Mistake #5: Not understanding why science is necessary. It’s hard to believe that was written by someone who has gone through medical school and residency training. It’s a sad indictment of our educational system.
The instructor asks for volunteers and Hopkins offers himself as a guinea pig. The instructor treats him for his Raynaud’s disease, telling him he believes it is due to prior cervical injury. As a neurologist, Hopkins should know that the term “Raynaud’s disease” refers only to idiopathic cases and if the condition is secondary to some instigating factor, it is called “Raynaud’s syndrome.” Also, while repetitive trauma from vibrating tools like jackhammers and prior injuries to the hands or feet have been recognized as causes of Raynaud’s, “cervical injury” has not. There are studies showing that acupuncture is more effective for Raynaud’s than drugs or than no treatment, but they are not convincing because they didn’t use placebo control groups. After the treatment, Hopkins’s symptoms resolved, and he became a believer.
Mistake #6: Relying on his personal experience. True believers ask us to “try it yourself” and they say, “I saw it with my own eyes.” We have ample evidence that seeing something with our own eyes is often misleading, and that trying something for yourself can interfere with your ability to objectively assess the evidence.
Mistake #7: The post hoc ergo propter hoc fallacy. Hopkins assumes that because his symptoms improved after the treatment, they improved because of the treatment. He doesn’t consider that there might be alternative explanations or confounding factors. (For example, emotional stress is a known trigger for Raynaud’s symptoms.)
His classmates were treated for various conditions including nerve abnormalities and bladder inflammation, and they all “benefited.” He calls acupuncture a “safe and inexpensive tool that has been time-tested for several thousand years.”
Mistake #8: Relying on the personal experience of others. Testimonials abound for even the quackiest of quack treatments. No matter how many anecdotes we manage to accumulate, the plural of anecdote is not data; that’s why we do science.
Mistake #9: The ancient wisdom fallacy. Hopkins is completely wrong about acupuncture being several thousand years old (Kavoussi 2010), and even if it were that old, length of use is no indication of truth. Astrology has been around for longer than acupuncture; does he think its validity has been proven because it has been “time-tested”? Bloodletting to balance imaginary bodily humors was “time-tested” for many centuries, but it turned out to do more harm than good.
Hopkins devotes Part Three of his article to recounting how he has implemented acupuncture in his practice. He has given over one thousand acupuncture treatments for everything from headaches to prostatitis. (One wonders why a neurologist would be treating prostatitis.) He claims a 90 percent success rate with many spectacular responses, and even uses the word “miracle.” He admits that some patients don’t respond, saying it is “never clear why.” (I think I can guess why!) He reports improvements in control of diabetes and hypertension, less need for medication, better sleep, etc. He concludes that “It is now evident to me that there truly is a great benefit to acupuncture.” He says it is safe (although the Ernst study [2011] documented rare but serious adverse effects including death). He says there are no contraindications, but numerous lists of contraindications can be found on the Internet both on acupuncture websites and on mainstream medical websites. He says the only time he would not use it was if the patient didn’t want it. He recommends that anyone caring for patients should consider adding this tool to his or her kit.
Mistake #10: Relying on uncontrolled observations. His patients improved, but how many of them would have improved without any treatment or with a credible placebo that offered some of the nonspecific treatment effects of acupuncture?
Mistake #11: Proselytizing on the basis of his own uncontrolled observations. Now that Hopkins has convinced himself, he wants to persuade others by simple assertions and by the same kind of unreliable “evidence” that convinced him in the first place.
Mistake #12: Not doing his own research. Hopkins might have checked PubMed and found a neat new study (White et al. 2012) confirming previous evidence that acupuncture is no more effective than placebo. It showed that patients were more likely to improve if they believed in acupuncture and believed they got the real thing rather than a placebo, regardless of which one they actually got. He might have read what Yale neurologist and CSI Fellow Steven Novella (2007) wrote after independently researching the literature on acupuncture for himself [see also Novella’s column, “What Is Acupuncture?” Skeptical Inquirer, July/August 2011]. He might have read the many negative systematic reviews, such as the one (Madsen et al. 2009) showing that a “small analgesic effect of acupuncture was found, which seems to lack clinical relevance and cannot be clearly distinguished from bias” or the systematic review of systematic reviews by Edzard Ernst and others (2011) showing “numerous contradictions and caveats.” He might have read the many skeptical articles on science blogs. He might have read The Skeptic’s Dictionary entry on acupuncture. He might have consulted Quackwatch’s affiliate Acupuncture Watch. Even just reading the acupuncture article on Wikipedia might have raised some doubts in his mind.
At this point, even if he were willing to look at the great mass of disconfirming evidence, he would probably not be capable of judging it objectively. Once someone has become a true believer on the basis of personal experience there is rarely any hope, especially when belief is reinforced by social support and grateful patient feedback. Let’s hope he doesn’t go on to seek training from a charismatic homeopath or a reiki master.
I can understand why many doctors are less skeptical than they should be about most of the alternative medicine information they encounter: they are used to having predigested, accurate scientific information presented to them by experts. They were (sadly) not taught to question what their teachers said in medical school and residency. They were taught about evidence-based medicine, but they were not taught to take prior probability into account: they are willing to accept the results of controlled studies even when they are incompatible with the rest of scientific knowledge. They rely on published information in medical journals, but they may not realize that half of the studies they read are wrong (Ioannidis 2005). Even if they are good at critically evaluating scientific medical information, they may not be used to critically analyzing information from the realm of alternative medicine. They may not have learned to recognize the common logical fallacies and the pitfalls.
The One Big Mistake: Not Following the SkepDoc’s Rule of Thumb. My rule, which applies to critical thinking in every sphere: before accepting any claim, find out who disagrees with it and why. Once you fully understand the arguments on both sides, only then are you qualified to judge whether the claim is credible (and it will usually be glaringly obvious which side makes more sense). What if a jury listened to the prosecution but not to the defense?
When examined as a whole, the available evidence fits the hypothesis that acupuncture is an elaborate placebo system. Using placebos on patients is unethical. As a retired Air Force colonel and as a physician, I am saddened to see acupuncture infiltrate the military health care system. And I am saddened to see how Dr. Hopkins’s faulty thinking led him astray.
Our brains evolved for success in survival as hunter-gatherers on the plains of Africa, not for a modern world of science and computers. We prefer stories to statistics, personal anecdotes to scientific studies. System 1 thinking (fast, emotional, and intuitive) is the default mode; it comes more naturally to us and requires much less effort than System 2 thinking (slow, deliberative, and logical). Not everyone reads the Skeptical Inquirer. Not everyone can overcome the natural tendencies of our flawed brains. Not everyone wants to try. But those who do can learn from Dr. Hopkins’s bad example.
Note
This is adapted from an article originally posted on the Science-Based Medicine blog under a different title (Hall 2012).
References
Ernst, Edzard, Mysong Soo Lee, and Tae-Young Choi. 2011. Acupuncture: Does it alleviate pain and are there serious risks? A review of reviews. Pain 152(4): 755–64.
Gorski, David. 2008. Battlefield acupuncture? (blog entry). Science-Based Medicine (December 15). Available at http://www.sciencebasedmedicine.org/index.php/acupuncture-invades-the-military/.
Hall, Harriet. 2012. Acupuncture, the Navy, and faulty thinking (blog entry). Science-Based Medicine (January 10). Available at http://www.sciencebasedmedicine.org/index.php/acupuncture-the-navy-and-faulty-thinking/.
Hopkins, Elwood. 2011–2012. The power of acupuncture (blog entry in three parts). Navy Medicine Live (December 22, December 29, and January 5). Available at http://navymedicine.navylive.dodlive.mil/archives/1550; http://navymedicine.navylive.dodlive.mil/archives/1577; http://navymedicine.navylive.dodlive.mil/archives/1604.
Ioannidis, J.P.A. 2005. Why most published research findings are false. PLoS Medicine 2(8):e124. Available at http://www.plosmedicine.org/article/info:doi/10.1371/journal.pmed.0020124.
Kavoussi, Ben. 2010. Acupuncture and history: The “ancient” therapy that’s been around for several decades (blog entry). Science-Based Medicine (October 18). Available at http://www.sciencebasedmedicine.org/index.php/acupuncture-and-history-the-ancient-therapy-thats-been-around-for-several-decades/.
Madsen, M.V., P.C. Götzsche, and A. Hróbjartsson. 2009. Acupuncture treatment for pain: Systematic review of randomized clinical trials with acupuncture, placebo acupuncture, and no acupuncture groups. British Medical Journal 338:a3115. Available at http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2769056/.
Novella, Steven. 2007. Does acupuncture work or not? (blog entry) Neurologica (September 25). Available at http://theness.com/neurologicablog/index.php/does-acupuncture-work-or-not/.
White, P., F.L. Bishop, C. Scott, et al. 2012. Practice, practitioner, or placebo? A multifactorial, mixed-methods randomized controlled trial of acupuncture. Pain 153(2): 455–62.