The study evaluated the efficiency of electronic cigarette use in a randomized, double blind clinical trial. Two nicotine concentrations were used, and a control group which received non-nicotine electronic cigarette was also included. The control group is not in reality a placebo group, because they received a product which mimics the act of smoking and provides visual and sensory stimulation; thus, it handles several aspects of the addiction to smoking, but does not provide nicotine. Including such a control group leads to underestimation of the true effects on the treatment group. Moreover, no psychological support was provided, which is a key issue and may explain much (if not all) of the efficacy of other smoking-cessation methods like NRTs [1].

It is extremely important that the authors used intention-to-treat analysis. The failure rate was significant, something that is common in studies of smoking cessation. The fact that all participants were not willing to quit further raises the expected lost-to-follow up rate, as is the use of an old and inefficient electronic cigarette device (the study started in 2009, technology has significantly evolved since then). Treating all these participants as failures in the final analysis was the correct thing to do, but is also another reason for underestimating the true effects of electronic cigarettes.

Yates correction for continuity [1] is a method to correct for continuity in chi-squared statistics. However, it tends to overcorrect the p-value. Moreover, it is chiefly used when at least one cell of the table has an expected count smaller than 5, and this is not the case in this study. It has also been suggested that this correction is unnecessary even when the sample size is low [2]. In fact, it has been supported that Yates correction “produces a test that is so conservative as to be almost useless” [3], and several other studies have found such a correction inappropriate [4,5].

Considering the use of an old and inefficient electronic cigarette device, the inherent lack of placebo available for electronic cigarette studies and the inclusion of unmotivated smokers (also supported by their overall good health status, which would further decrease their incentive to quit), it is extremely difficult to discard the results of this study.


1. Moore D, Aveyard P, Connock M, Wang D, Fry-Smith A, Barton P. Effectiveness and safety of nicotine replacement therapy assisted reduction to stop smoking: systematic review and meta-analysis. BMJ. 2009;338:b1024.
2. Yates, F. Contingency table involving small numbers and the χ2 test. Journal of the Royal Statistical Society 1934;1: 217–235.
3. Sokal RR, Rohlf FJ. Biometry: The Principles and Practice of Statistics in Biological Research. Oxford: W.H. Freeman 1981, ISBN 0-7167-1254-7.
4. Grizzle JE. Continuity correction in the χ2 test for 2 × 2 tables. The American Statistician 1967;21:28–32.
5. Maxwell EA. Analysis of contingency tables and further reasons for not using Yates correction in 2 × 2 tables. Canadian Journal of Statistics 1976; 4:277–290.
6. Conover WJ. Some reasons for not using the Yates continuity correction on 2×2 contingency tables. Journal of the American Statistical Association 1974;69:374–376.

The commenter, Stanton Glantz, is the American Legacy Foundation Distinguished Professor of Tobacco Control at UCSF, and a tobacco control activist, including being a founder of Americans for Nonsmokers' Rights: http://profiles.ucsf.edu/...

In his critique of this study, Dr. Glantz makes two main arguments:

1. "There is not a control group of people who were not using e-cigarettes that would allow assessment of spontaneous quit rates. By not having a true control group that would account for spontaneous quitting without using e-cigarettes one cannot say anything about whether e-cigarettes affected quitting."

2. "The [second] problem is that the authors failed to include the required Yates correction in their calculation of the chi-square test statistic and associated p value. Recalculating the test properly yields p = 0.07, which is no longer statistically significant. Thus, the correct conclusion is that there is no statistically significant difference between the nicotine and non-nicotine e-cigarettes."

Let's take each of these arguments in turn.

First, it is true that the study did not include any control group. However, that doesn't necessarily mean that we have no idea what the quit rate would have been for smokers with no interest in quitting in the absence of electronic cigarettes. We can simply examine the ample scientific evidence reporting one-year quit rates among smokers in the absence of intervention.

However, what we really want is the one-year quit rate among smokers with no interest in quitting. To be conservative, we can use the one-year quit rate among smokers who do want to quit (and try to quit), which is about 3%.

Clearly, the quit rate observed in the high-nicotine electronic cigarette group (13%) is considerably larger than the 3% observed for the population of smokers who want to quit and make quit attempts. The natural quit rate among smokers with no desire to quit would be substantially lower than 3%.

In fact, the paper provides population-based data showing that in Italy, the spontaneous quit rate during the study period was a dismal 0.02%.

So this first argument does not invalidate the study's conclusion that electronic cigarettes did help achieve smoking cessation for some smokers. There is no scenario under which you would observe 13% of non-motivated smokers quitting within one year without any intervention.

It is also important to point out that the comparison group Dr. Glantz requests is not possible. One could not ethically conduct a clinical trial of smoking cessation in which one group is assigned to receive no intervention.

What is really needed is a head-to-head comparison of electronic cigarettes c ompared to nicotine replacement therapy (NRT).

Dr. Glantz's second argument is that the study conclusions are invalid because of failure to apply the Yates correction. This argument is invalid, however, because it is a diversion from the actual research question. It is a diversion because there is no placebo in this trial. Had this study compared electronic cigarettes with a placebo, then the appropriate analysis would indeed be to statistically compare the cessation rates in the e-cigarette vs. placebo groups. But there was no placebo. The group receiving electronic cigarettes without nicotine was in fact an intervention arm. Even without nicotine, electronic cigarettes have been shown to reduce the craving to smoke. And in fact, in this trial, the most remarkable finding was that in the zero nicotine e-cigarette group, 14% either quit or cut down by more than half. This is remarkable for a group of smokers who had no intention to quit and who received no non-tobacco cigarette nicotine for an entire year.

The paper actually acknowledges the lack of statistical significance in quit rates between the three study arms at one year. But this doesn't mean the treatment wasn't effective. It reflects the small sample size of the study along with the fact that there was some effect among the 0 nicotine e-cigarette group.

Dr. Glantz is absolutely right that the correct comparison to make to evaluate the results of the study is how the observed quit rate (about 9% for all three groups combined) compares to the spontaneous one-year quit rate among the population of smokers, except that one would have to restrict that to the population of smokers who have no interest in quitting.

If Dr. Glantz can provide data demonstrating that more than 2% of smokers with no interest in quitting achieve smoking cessation over a one-year period, then his argument about the lack of validity of this study would be reasonable. But short of such data, it cannot be denied that the results achieved in this study are better than what one would have obtained with no intervention.

Michael Siegel, MD, MPH
Professor
Department of Community Health Sciences
Boston University School of Public Health