What is the Scientific Contribution when the Study Methodology has so Many Limitations?

carlo 2018 96The following is a letter written by Dr. Giovanardi, President of the Italian Federation of Acupuncture Societies. This was submitted to the editor of the Journal of the American Medical Association (JAMA) and the authors of the study on IVF published in JAMA in May 2018 regarding acupuncture to support IVF therapy. JAMA decided not to publish this saying they had received several letters in response to that study and could not publish them all.

Dear Editor and corresponding Author,

I have thoroughly read the article Effect of Acupuncture vs Sham Acupuncture on Live Births Among Women Undergoing In Vitro Fertilization by Caroline A. Smith et al. published by JAMA in May, 2018, and would like to make some observations.

The authors themselves identify five limitations to their study; however, in spite of them, they reach a definite and clearly stated conclusion (‘These findings do not support the use of acupuncture when administered at the time of oviarian stimulation and embryo transfer to improve the rate of live births’) which leaves no doubt as to the total inefficacy of acupuncture to improve birth outcomes among women undergoing IVF. Such a strong statement, therefore, is being supported by a trial, the limitations of which have been identified but not overcome.

I am sure we all agree on one point, namely that a methodologically incorrect article brings about misleading results which affect the choices of both medical doctors and patients; that’s why I wonder how the conclusions reached can be useful to acupuncturists and patients who decide to choose acupuncture as a therapy.

However, this is not the main issue I’d like to focus on.

My analysis starts with the generally accepted principle that no skin stimulation is inert, which implies that ‘sham acupuncture’ cannot be inert: any skin stimulation brings about central and peripheral responses (Birch S., A review and analysis of placebo treatments, placebo effects, and placebo controls in trials of medical procedures when sham is not inert, J Altern Complement Med, 2006). Effects have been observed when using non-needle sham, penetrating sham needles and non-penetrating sham needles, although they were different in size (MacPherson H., Influence of control group on effect size in trials of acupuncture for chronic pain: a secondary analysis of an individual patient data meta-analysis, PLos One, 2014).

The authors say that sham acupuncture can lead to weak physiological effects and the sham needle has been shown to stimulate different brain activity than acupuncture: this means, they recognize and accept the fact that sham treatment is not inert.

At the same time, their study addressed whether acupuncture was more efficacious than a placebo. This appears to be a contradiction in terms: how can sham acupuncture be defined as a placebo treatment? Placebo means being inert, by definition, whereas sham acupuncture is not, according both to the literature and to the article. Effects may be minimal, as researchers assume, but as a matter of fact such outcomes cannot be assessed, and therefore any speculation is possible.

Moreover, the authors affirm that they used sham points at locations away from known acupuncture points and with no known function, to minimize physiological effects.

First, if they decide to ‘minimize’ the effects, they admit that the effects are not only present but also remarkable and – let me repeat it –their impact cannot be assessed. Second, according to the Traditional Chinese Medicine, there are no points with no function, the whole skin being able to trigger physiological responses, as it is clearly revealed by the functions of the cutaneous regions called pí bù in Chinese (Maciocia G., The foundations of Chinese medicine: a comprehensive text for acupuncturists and herbalists, 1989; Ching N., The fundamentals of acupuncture, 2016).

By proceeding this way, a cutaneous stimulation, such as acupuncture, has been compared with another manual therapy involving skin stimulation and with widely recognized neuroendocrine effects; moreover, sham being a manual technique, various different effects may be associated with its administration.

Minimal, superficial, sham, or 'placebo' acupuncture have been thoroughly investigated: it has been demonstrated that light touch of the skin stimulates mechanoreceptors coupled to slow conducting unmyelinated (C) afferents resulting in activity in the insular region, but not in the somatosensory cortex. Activity in these C tactile afferents has been suggested to induce a 'limbic touch' response resulting in emotional and hormonal reactions. It is likely that, in many acupuncture studies, control procedures that are meant to be inert are in fact activating these C tactile afferents. (Lund I., Are minimal, superficial or sham acupuncture procedures acceptable as inert placebo controls?, Acupunct Med, 2006).

Because of the risk – might we say certainty? - that the sham is not an inert placebo but rather an active treatment that may affect the pregnancy outcome, using sham acupuncture as the control may unnecessarily confuse rather than clarify the interpretation of the effects of IVF adjuvant acupuncture (Manheimer E., Selecting a control in IVF and acupuncture RCTs: how sham controls may unnecessarily complicate the RCT evidence base, Fertil Steril, 2011).

Sham acupuncture may introduce bias against the treatment being tested and I share Lund’s point of view when she affirms that minimal acupuncture is not valid as an inert placebo-control despite its conceptual brilliance (Lund I., Minimal acupuncture is not a valid placebo control in randomised controlled trials of acupuncture: a physiologist’s perspective, Chin Med, 2009).

In conclusion, an important risk of bias exists in this study and what it actually lacks is a true control group (a care alone group). As already suggested by Zheng (Zheng C.H., Effects of acupuncture on pregnancy rates in women undergoing in vitro fertilization: a systematic review and meta-analysis, Fertil Steril, 2012), sham acupuncture (Streitberger control) may not be an inactive control. More positive effects from using acupuncture in IVF can be expected if an appropriate control and more reasonable acupuncture programs are used.

I do hope my contribution can lead to a fruitful debate providing useful insight into CAM therapies; I appreciate your attention to this important matter and remain at your disposal for any comments you may have.

Very truly yours,

Carlo Maria Giovanardi, MD
President of the Italian Federation of Acupuncture Societies (F.I.S.A.)
President of the Association of Medical Acupuncturists of Bologna (A.M.A.B.)