IJCA-15527; No of Pages 2
International Journal of Cardiology xxx (2012) xxx–xxx
Contents lists available at SciVerse ScienceDirect
International Journal of Cardiology
journal homepage: www.elsevier.com/locate/ijcard
Letter to the editor
Hast thou slain the Jabberwock of funnel plot asymmetry? From the meta-analysis of
smoking bans for reduction of acute myocardial infarction
Hisato Takagi ⁎, Masao Niwa, Yusuke Mizuno, Shin-nosuke Goto, Takuya Umemoto
for ALICE (All-Literature Investigation of Cardiovascular Evidence) Group
Department of Cardiovascular Surgery, Shizuoka Medical Center, Shizuoka, Japan
a r t i c l e
i n f o
Article history:
Received 12 October 2012
Accepted 1 November 2012
Available online xxxx
Keywords:
Acute myocardial infarction
Funnel plot
Meta-analysis
Publication bias
Smoking ban
Meyers et al. [1] performed a systematic review and a metaanalysis in 2009 to determine the association between public smoking
bans and risk for hospital admission for acute myocardial infarction
(AMI). The authors included only AMI cases (some investigators
supplied additional data), except where the case definition was acute
coronary syndrome (ACS), which required an elevated serum troponin.
More recently (in 2010), Mackay et al. [2] undertook another metaanalysis of the effects of smoke-free legislation. The case definitions
used in the studies included in the meta-analysis [2], however, varied:
e.g. myocardial infarction, ACS, coronary heart disease including angina
and heart failure, and myocardial infarction plus ischemic heart disease.
Thus, the study by Meyers et al. [1] has been the most comprehensive
meta-analysis of smoking bans for reduction in exclusive AMI incidence
to date.
Using 11 reports from 10 study locations in the meta-analysis by
Meyers et al. [1], AMI risk decreased by 17% overall (random-effects
rate ratio [RR], 0.83; 95% confidence interval [CI], 0.75 to 0.92). A
serious statistical concern in the meta-analysis, however, is the funnel
plot asymmetry shown graphically, indicating either publication bias
or heterogeneity that cannot be explained by a random-effects metaanalysis. Therefore, we assessed the funnel plot asymmetry not only
graphically but also mathematically using an adjusted rank-correlation
test without continuity correction, according to the method of Begg and
⁎ Corresponding author at: Department of Cardiovascular Surgery, Shizuoka Medical
Center, 762-1 Nagasawa, Shimizu-cho, Sunto-gun, Shizuoka 411-8611, Japan. Tel.: +81
559752000.
E-mail address: [email protected] (H. Takagi).
Mazumdar [3]. All analyses were conducted using Comprehensive
Meta-Analysis version 2 (Biostat, Englewood, NJ). There was marginally
nonsignificant funnel plot asymmetry (p=0.064). Because of this, we
undertook a sensitivity analysis using the trim and fill method [4],
which conservatively imputes hypothetical negative unpublished studies
to mirror the positive studies that cause funnel plot asymmetry. The
imputed 3 studies produce a symmetrical funnel plot (Fig. 1). The pooled
analysis incorporating the hypothetical studies showed a statistically
nonsignificant 3% reduction in AMI risk in the random-effects model
(RR, 0.97; 95% CI, 0.87 to 1.08).
A funnel plot is a simple scatter plot of the intervention effect estimates from individual studies against some measure of each study's
size or precision. In the absence of bias the plot should approximately
resemble a symmetrical (inverted) funnel [5]. When there is bias, this
would lead to an asymmetrical appearance of the funnel plot with a
gap in a bottom corner of the graph e.g. because smaller studies without
statistically significant effects remain unpublished. The effect calculated
in a meta-analysis would tend to overestimate the intervention effect in
this situation [6,7]. The more pronounced the asymmetry, the more
likely it is that the amount of bias would be substantial. Publication
bias need not lead to asymmetry in funnel plots. In the absence of any
intervention effect, selective publication based on the p value alone
would lead to a symmetrical funnel plot in which studies on the extreme left or right are more likely to be published than those in the
middle [5]. This could bias the estimated between-study heterogeneity
variance. It has been argued that visual interpretation of funnel plots is
too subjective to be useful. In particular, it was found that researchers
had only a limited ability to correctly identify funnel plots from metaanalyses subject to publication bias [8]. There remains a concern that
visual interpretation of funnel plots is inherently subjective. When
review authors are concerned that small study effects are influencing
the results of a meta-analysis, they may want to conduct sensitivity
analyses in order to explore the robustness of the meta-analysis'
conclusions to different assumptions about the causes of funnel plot
asymmetry [5].
The ‘trim and fill’ method aims both to identify and correct for
funnel plot asymmetry arising from publication bias [4]. The basis of
the method is to (1) ‘trim’ (remove) the smaller studies causing funnel
plot asymmetry, (2) use the trimmed funnel plot to estimate the true
‘centre’ of the funnel, then (3) replace the omitted studies and their
missing ‘counterparts’ around the centre (filling). Performing a metaanalysis including the filled studies derives an adjusted intervention
effect as well as providing an estimate of the number of missing studies.
0167-5273/$ – see front matter © 2012 Elsevier Ireland Ltd. All rights reserved.
http://dx.doi.org/10.1016/j.ijcard.2012.11.028
Please cite this article as: Takagi H, et al, Hast thou slain the Jabberwock of funnel plot asymmetry? From the meta-analysis of smoking bans for
reduction of acute myocardial ..., Int J Cardiol (2012), http://dx.doi.org/10.1016/j.ijcard.2012.11.028
2
H. Takagi et al. / International Journal of Cardiology xxx (2012) xxx–xxx
There was no funding source for this study. The corresponding
author had full access to all the data in the study and had final responsibility for the decision to submit for publication.
For no authors there is any condition that may represent a potential conflict of interest.
The authors of this manuscript have certified that they comply
with the Principles of Ethical Publishing in the International Journal
of Cardiology (Shewan and Coats 2010; 144:1–2).
References
Fig. 1. Funnel plot with trim and fill. Open circles and an open rhombus denote identified
studies and their summary measure, respectively. Closed circles and a closed rhombus
denote estimated missing studies after adjustment for funnel plot asymmetry and the
summary measure incorporating the hypothetical studies, respectively.
The trim and fill method requires no assumptions about the mechanism
leading to publication bias, provides an estimate of the number of missing studies, and also provides an estimated intervention effect ‘adjusted’
for the publication bias (based on the filled studies) [5]. It is, however,
built on the strong assumption that there should be a symmetric funnel
plot. Further, there is no guarantee that the adjusted intervention effect
matches what would have been observed in the absence of publication
bias, since we cannot know the true mechanism for publication bias [5].
The present re-assessment adjusting for the publication bias suggests
that smoking bans in public places and workplaces may not be associated with a reduction in AMI incidence. Although the ‘corrected’ intervention effect estimate from the trim and fill method should be interpreted
with caution because of the afore-mentioned limitations, the results of
the meta-analysis by Meyers et al. [1] are not compelling. An association
between smoking bans and reduced AMI incidence may be due to not
only bans themselves (high levels of compliance with bans [9,10] and
reduced environmental exposure to tobacco smoke [9–13]) but also
beneficial changes in other factors: e.g. decreased smoking prevalence
and sales of tobacco [9,14,15] and improved air quality [10,11,16].
[1] Meyers DG, Neuberger JS, He J. Cardiovascular effect of bans on smoking in public
places: a systematic review and meta-analysis. J Am Coll Cardiol 2009;54:1249-55.
[2] Mackay DF, Irfan MO, Haw S, Pell JP. Meta-analysis of the effect of comprehensive
smoke-free legislation on acute coronary events. Heart 2010;96:1525-30.
[3] Begg CB, Mazumdar M. Operating characteristics of a rank correlation test for
publication bias. Biometrics 1994;50:1088-101.
[4] Duval S, Tweedie R. Trim and fill: a simple funnel-plot-based method of testing
and adjusting for publication bias in meta-analysis. Biometrics 2000;56:455-63.
[5] Sterne JA, Egger M, Moher D, editors. Chapter 10: addressing reporting biases. In:
Higgins JP, Green S, editors. Cochrane Handbook for Systematic Reviews of Intervention. Version 5.1.0 (updated March 2011). The Cochrane Collaboration, 2011.
Available from www.cochrane-handbook.org.
[6] Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta-analysis detected
by a simple, graphical test. BMJ 1997;315:629-34.
[7] Villar J, Piaggio G, Carroli G, Donner A. Factors affecting the comparability of
meta-analyses and largest trials results in perinatology. J Clin Epidemiol 1997;50:
997–1002.
[8] Terrin N, Schmid CH, Lau J. In an empirical evaluation of the funnel plot, researchers
could not visually identify publication bias. J Clin Epidemiol 2005;58:894-901.
[9] Gallus S, Zuccaro P, Colombo P, et al. Smoking in Italy 2005–2006: effects of a
comprehensive National Tobacco Regulation. Prev Med 2007;45:198-201.
[10] Semple S, Creely KS, Naji A, Miller BG, Ayres JG. Secondhand smoke levels in Scottish
pubs: the effect of smoke-free legislation. Tob Control 2007;16:127-32.
[11] Gorini G, Gasparrini A, Fondelli MC, et al. Environmental tobacco smoke (ETS)
exposure in Florence hospitality venues before and after the smoking ban in
Italy. J Occup Environ Med 2005;47:1208-10 [author reply 1210].
[12] Cesaroni G, Forastiere F, Agabiti N, Valente P, Zuccaro P, Perucci CA. Effect of the
Italian smoking ban on population rates of acute coronary events. Circulation
2008;117:1183-8.
[13] Centers for Disease Control and Prevention (CDC). Indoor air quality in hospitality
venues before and after implementation of a clean indoor air law—Western New
York, 2003. MMWR Morb Mortal Wkly Rep 2004;53:1038-41.
[14] Centers for Disease Control and Prevention (CDC). Reduced hospitalizations for
acute myocardial infarction after implementation of a smoke-free ordinance—
City of Pueblo, Colorado, 2002–2006. MMWR Morb Mortal Wkly Rep 2009;57:
1373-7.
[15] Galeone D, Laurendi G, Vasselli S, Spizzichino L, D'Argenio P, Greco D. Preliminary
effects of Italy's ban on smoking in enclosed public places. Tob Control 2006;15:
143.
[16] Valente P, Forastiere F, Bacosi A, et al. Exposure to fine and ultrafine particles from
secondhand smoke in public places before and after the smoking ban, Italy 2005.
Tob Control 2007;16:312-7.
Please cite this article as: Takagi H, et al, Hast thou slain the Jabberwock of funnel plot asymmetry? From the meta-analysis of smoking bans for
reduction of acute myocardial ..., Int J Cardiol (2012), http://dx.doi.org/10.1016/j.ijcard.2012.11.028