The continual battles over research relating to tobacco harm reduction intensified in May. Two papers, published on consecutive days, attacked the scientific credibility of a study that claimed the health risks of vaping are similar to many of those of combustible cigarettes.
That study, titled “Population-Based Disease Odds for E-Cigarettes and Dual Use versus Cigarettes,” was published in February 2024 in the New England Journal of Medicine.
Based on meta-analysis of 107 past studies, it associated vaping with increased odds of cardiovascular disease, stroke and metabolic dysfunction—to about the same extent as smoking. It also associated vapes with increased risks of asthma, chronic obstructive pulmonary disease (COPD) and other diseases; and associated “dual use”—a person using both combustible cigarettes and vapes—with higher risk of disease compared to smoking alone.
Prominent researchers soon challenged its conclusions, calling them, “premature and reflect[ing] a serious misinterpretation of the epidemiologic evidence.”
Its authors were led by Dr. Stanton Glantz, a retired professor of medicine at the University of California, San Francisco. Over the years, Glantz’s regular anti-vape findings have repeatedly seen him clash with researchers who find benefits in tobacco harm reduction (THR); some of his past research in the field has been retracted.
“There is a need to reassess the assumption that e-cigarette use provides substantial harm reduction across all cigarette-caused diseases, particularly accounting for dual use,” Glantz et al. wrote.
Soon after the study’s publication, a number of prominent researchers challenged its conclusions, calling them, “premature and reflect[ing] a serious misinterpretation of the epidemiologic evidence.”
There’s little doubt that the Glantz study garnered more attention and influence than the subsequent objections. But two new papers by leading THR researchers, each pointing to specific problems with the study, might help redress the balance.
One, published in the Internal and Emergency Medicine journal on May 12, was led by Dr. Brad Rodu, professor of medicine at the University of Louisville in Kentucky, who said Glantz’s research had “serious deficiencies.”
The other study, published in the Harm Reduction Journal on May 13 and coauthored by Dr. Konstantinos Farsalinos, a cardiologist and research fellow at the Onassis Cardiac Surgery Center in Athens, Greece, similarly found that “the conclusions of Glantz et al. are not supported by the currently available evidence.”
“Such dramatic claims require substantial evidence, which the Glantz et al. study failed to provide.”
But why now, more than a year after the original publication? Meticulous re-analysis or new analysis takes significant time; in addition, both new papers had to wait to be peer-reviewed.
Caution and checks are clearly desirable with scientific publications. Unfortunately, media attention tends not to last that long, when clickbait headlines, often based on studies that have not been published or peer-reviewed, can immediately skew public perceptions.
Glantz claimed that for cardiovascular disease, stroke and metabolic dysfunction, the odds for disease associated with current use of vapes or cigarettes were similar.
“Such dramatic claims require substantial evidence, which the Glantz et al. study failed to provide,” Rodu told Filter.
Rodu said he identified three major failings in Glantz’s research. For one, he said the study created disease categories that were “incomprehensible.”
“Everyone knows that smoking contributes to serious, even fatal heart and circulatory diseases,” Rodu said. “No legitimate study would combine such less serious conditions as erectile dysfunction (ED) and haemorrhoids with heart attacks and heart failure, but Glantz et al. added ED—why? Because the ED study contributed the largest odds of any of the source studies, thus sustaining the author’s claim that vaping is as bad as smoking.”
“Studies reporting non-specific respiratory symptoms and asthma were classified as COPD studies, and a study asking about erectile dysfunction was classified as a heart disease study.”
Farsalinos said that his own re-analysis of the systemic review identified studies that reported “irrelevant” diseases.
“For example, studies reporting non-specific respiratory symptoms and asthma were classified as COPD studies, and a study asking about erectile dysfunction was classified as a heart disease study,” he told Filter.
Farsalinos said that by re-analyzing the data with the appropriate disease classifications, he and his colleague found that the level of risk reduction from vaping compared to smoking “was very much similar with the risk reduction we expect from quitting smoking without using any alternative nicotine product.”
Another flaw Rodu identified was with the cross-sectional source studies used, which he noted contained no information about when people started smoking or vaping, or when they got sick.
“We previously published research documenting that these studies are unreliable and should never be used or cited,” he said, describing the problem as “garbage in, garbage out.” A meta-analysis based on flawed source studies is inevitably flawed itself.
“The original study counted 69 COPD cases among current vapers, but its authors ignored the fact that all but one were current or former smokers when they were diagnosed.”
Glantz’s analysis, for instance, included a longitudinal study that contributed the largest odds for vaping and COPD.
“The original study counted 69 COPD cases among current vapers, but its authors ignored the fact that all but one were current or former smokers when they were diagnosed,” Rodu pointed out.
Farsalinos’s study examined the accuracy of Glantz’s findings by checking whether the studies included in the meta-analysis were relevant to the outcomes examined, and performing new meta-analysis using studies with relevant outcomes.
“Our re-analysis of the systematic review by Glantz et al. identified serious flaws in their research methodology, which compromised the reported conclusions and study interpretation,” he said.
“Evidence accumulated over the years is contradictory to his claims,” Farsalinos added. “In fact, one of the most recent studies provides undisputed evidence about substantial health benefits from switching to vaping.”
Other studies have meanwhile shown, contrary to Glantz’s conclusions, that dual use has benefits; that reducing the number of cigarettes consumed per day can significantly improve health, besides the reality that dual use is often part of a transition to smoking cessation.
Glantz may be one of the most prominent producers of THR-related research that’s criticized for flaws and unjustifiable conclusions, but he’s far from alone. So why do so many academics seem to be getting it wrong?
“The data contain major biases, such as confounding by smoking history and possible reverse-causality.”
“It boils down to limitations of the data we have: observational and usually cross-sectional,” Dr. Arielle Selya, a behavioral scientist who has published her own THR-related work, told Filter.
“The data contain major biases, such as confounding by smoking history and possible reverse-causality,” she continued. “There are ways of analyzing more carefully to reduce (but not eliminate) these biases, but most underlying studies don’t take those extra steps.”
Farsalinos, for his part, pointed to “ideological opposition” to THR and nicotine use within the public health community. “This, combined with the associated confirmation bias, can lead to the expression of strong positions that are not supported by the available evidence,” he said.
Rodu said he and his colleagues are all too aware of the flood of defective studies on vaping, many funded by the United States National Institutes of Health, which he and others have tried to counter.
“But it is very difficult,” he explained, “as it takes months to analyze such studies; there is little-to-no funding to support the work; it is accomplished only at the expense of original research; because this isn’t ‘original’ research, it is given little credit; and few journals will publish such critiques.”
“For all these reasons,” Rodu disclosed, “my colleague and I are ending our research program here at the University of Louisville.”
Photograph by Lindsay Fox via Wikimedia Commons/Creative Commons 3.0
Dr. Selya is an employee of Pinney Associates, Inc. which consults to Juul Labs and Philip Morris International (PMI) on non-combustible nicotine products. She also individually provides consulting services on behavioral science to the Center of Excellence for the Acceleration of Harm Reduction, which received funding from the Foundation for a Smoke-Free World (now Global Action to End Smoking). Her commentary in this article is her own. The Influence Foundation, which operates Filter, has received grants from Juul Labs (previously), PMI and Global Action to End Smoking. Pinney Associates CEO Joe Gitchell has made personal donations to The Influence Foundation. Filter’s Editorial Independence Policy applies.
