Potentially Large yet Uncertain Benefits
A Meta-analysis of Patent Foramen Ovale Closure Trials
Georgios D. Kitsios, MD, MS, PhD; David E. Thaler, MD, PhD; David M. Kent, MD, MS
U
Downloaded from http://stroke.ahajournals.org/ by guest on July 28, 2017
ntil recently, the evidence base for patent foramen ovale
(PFO) closure with a device to prevent future cerebrovascular events in patients with cryptogenic stroke and presumed paradoxical embolism consisted of a large body of >50
observational studies that cumulatively showed substantial
and unequivocal benefits with this intervention.1 On the basis
of this observational data and the compelling pathophysiologic rationale, PFO closure to prevent stroke recurrence had
become routine in many centers, and off-label closure of PFOs
hampered enrollment in ongoing trials.
However, within the past year, the first true clinical experiments on PFO closure were published, and 3 consecutive randomized clinical trials (RCTs) investigating 2 different devices
(STARFlex in CLOSURE-I and Amplatzer PFO Occluder in
RESPECT and PC trial) failed to show evidence of statistically significant benefit in their primary outcome.2–4 Yet,
despite the consistently null overall outcome of these trials,
on closer inspection, their results seem well calibrated to sustain the controversy; in particular, the results of the RESPECT
and PC trial, published simultaneously, seem highly suggestive of benefit for mechanical closure over medical therapy.5
Thus, we aimed to explore more thoroughly the results from
the available randomized evidence via meta-analysis, which
can sometimes overcome the uncertainty in individual trials
through their pooling.6
We considered the outcome of recurrent stroke as our primary outcome (as opposed to also including transient ischemic attacks), given its more standardized definition and its
long-term clinical import. We additionally explored composite
outcomes in sensitivity analyses, and apart from the intentionto-treat analysis, we also synthesized the per-protocol results
of the trials to appreciate the effects of crossovers and dropouts. For the statistical synthesis, we considered the generally
accepted random effects model,7 which accounts for possible
between-trial treatment-effect heterogeneity. In sensitivity
analysis excluding the CLOSURE-I trial, we also explored a
(less conservative) fixed effects model for the 2 RCTs using
the same device, appropriate only under a strong assumption
of a single true effect of closure using the Amplatzer device.
First, we examined the absolute event rates of recurrent
stroke, incidence rates with their 95% exact Poisson confidence intervals (95% CIs) in device and medical groups
separately (Figure 1). By random effects meta-analysis, the
summary incidence rate of stroke was relatively low in both
groups; in the device groups, it was 0.76 (95% CI, 0.30–1.96)
per 100 person-years versus 1.30 (95% CI, 0.94–1.81) per
100 person-years in medical groups. Statistically significant
heterogeneity of incidence rates in the device groups (I2 for
heterogeneity=78%; P=0.01) may reflect between-trial differences in study design. Important differences may have
included variation in the stringency of patient selection,
medical therapy received on either arm, the presence of true,
device-specific effects of closure, or differences in outcome
ascertainment methods (which were, for example, much more
assiduous in RESPECT than in the PC trial). In fact, a concerning signal for differential referral for outcome adjudication in the device group was noted in the PC trial,5 which may,
in part, contribute to the exceedingly low incidence rate in this
trial.
Next, we conducted a series of meta-analyses for the treatment effects estimated as hazard ratios. In our primary results
for the outcome of stroke, closure showed a nonsignificant
45% reduction in risk with all 3 RCTs combined (summary
hazard ratio, 0.55 [95% CI, 0.26–1.18]; Figure 2). The results
for the outcome of stroke were also nonsignificant when
the 2 Amplatzer device trials were combined without the
CLOSURE-I trial included, or when the per-protocol results
were synthesized (Table). Yet the absence of significance is
not universally the case across all analyses. Importantly, for
the composite primary outcome for which each RCT was
powered (nonidentically defined across studies), a borderline
statistically significant effect was found for the intentionto-treat analysis only (summary hazard ratio, 0.67 [95% CI,
0.44–1.00]). The treatment effect was larger still and more
statistically significant when the 2 Amplatzer device trials
(RESPECT and PC trial) were synthesized with a fixed effects
model (summary hazard ratio, 0.41 [95% CI, 0.19–0.88]),
although a random effects model is probably more appropriate given differences in the design and conduct of these trials.
No significant effects were found for the composite of stroke
or transient ischemic attack.
We further explored sources of potential treatment-effect
heterogeneity with subgroup meta-analyses based on available data from RCTs for their primary outcomes (Figure 3).
Received April 11, 2013; final revision received April 11, 2013; accepted May 29, 2013.
From the Institute for Clinical Research and Health Policy Studies (G.D.K., D.M.K.) and Department of Neurology (D.E.T., D.M.K.), Tufts Medical
Center, Boston, MA; and Department of Internal Medicine, Lahey Hospital and Medical Center, Burlington, MA (G.D.K.).
Correspondence to David M. Kent, MD, CM, MSc, Institute for Clinical Research and Health Policy Studies, Tufts Medical Center, Box 63, 800
Washington St, Boston, MA 02111. E-mail [email protected]
(Stroke. 2013;44:2640-2643.)
© 2013 American Heart Association, Inc.
Stroke is available at http://stroke.ahajournals.org
DOI: 10.1161/STROKEAHA.113.001773
2640
Kitsios et al PFO Closure Meta-analysis 2641
Downloaded from http://stroke.ahajournals.org/ by guest on July 28, 2017
Figure 1. Incidence rates of stroke (95% confidence intervals) in the device and medical treatment groups of 3 randomized clinical trials with
their corresponding meta-analytic summaries.
CLOSURE-I indicates Evaluation of the Starflex
Septal Closure System in Patients with a Stroke
and/or Transient Ischemic Attack Due to Presumed
Paradoxical Embolism through a Patent Foramen
Ovale; RESPECT, Randomized Evaluation of Recurrent Stroke Comparing PFO Closure to Established
Current Standard of Care Treatment; and PC Trial,
Clinical Trial Comparing Percutaneous Closure of
the Patent Foramen Ovale Using the Amplatzer PFO
Occluder with Medical Treatment in Patients with
Cryptogenic Embolism.
However, no significant tests of interaction were found (all P
values >0.10) and CIs were wide, indicating substantial uncertainty around available estimates, despite some interesting yet
nonsignificant signals, such as the case of substantial shunt
on echocardiography. Nonetheless, our analyses show that
biological plausibility of paradoxical embolism (eg, younger
age or presence of atrial septal aneurysm) does not necessarily
translate into clinically detectable effect modification.
Our meta-analyses are limited by the fact that the primary composite outcome definitions (and also the actuarial
component events) varied across studies, and thus, interpretation of some of our summary estimates should be viewed
only as exploratory. Furthermore, our analyses for potential treatment-effect modifiers are restricted to subgroups
analyzed by the individual component trials because this
meta-analysis is based on reported results, not individual
patient data.
The main message of these analyses is that the uncertainty
of the individual trials is not resolved by combining their
results through meta-analysis. The wide CIs of the individual trials are transmitted in the main results of the combined
analysis and appear broad enough to accommodate the mutually conflicting viewpoints on the benefits of closure. The
uncertainty is only underscored by the conflicting results of
different meta-analytic approaches.
Despite the absence of formal statistical significance in
our main analysis, the magnitude of these summary effect
sizes is reminiscent of the large effect sizes seen in observational studies.1 In particular, meta-analyses of the Amplatzer
device trials provided strong summary effects that fell just
short of the significance threshold. Larger sample sizes
and longer follow-ups could have potentially solidified the
benefits of PFO closure. However, the very low numbers of
available data points (eg, a single stroke event in the device
group of the PC Trial) makes these results extremely vulnerable to even slight shifts of event numbers between groups,
either because of any type of bias or because of random
error, and the effect estimate might attenuate substantially
(or even completely) with more outcomes. Thus, closing the
path for further randomized evidence on the basis of this
promising signal leaves substantial uncertainty about the
best options for our patients.
Figure 2. Forest plot for the
meta-analysis of hazards ratios
of stroke of mechanical closure
vs medical treatment from 3
randomized clinical clinical
trials.
2642 Stroke September 2013
Table. Meta-analysis Results for the Hazard Ratio of Stroke and Additional Outcomes
All Devices (No. of Studies)
Amplatzer Only RCTs (n=2)
Outcome
Random Effects Model
Random Effects Model
Fixed Effects Model
Stroke (ITT)
0.55 (0.26–1.18), n=3
0.38 (0.14–1.02)
0.41 (0.19–0.88)
Stroke/TIA (ITT)
0.69 (0.43–1.13), n=2*
NA
NA
Composite primary outcome† (ITT)
0.67 (0.44–1.00), n=3
0.54 (0.29–1.01)
0.54 (0.29–1.01)
Composite primary outcome† (PP‡)
0.57 (0.32–1.02), n=3
0.44 (0.17–1.12)
0.44 (0.22–0.89)
Stroke (PP‡)
0.52 (0.16–1.70), n=2§
NA
NA
Values represent summary HR (95% CI). Statistically significant results (P<0.05) are shown in bold. CI indicates confidence interval; HR, hazard ratio; ITT, intention to
treat; PP, per protocol; RCT, randomized clinical trial; and TIA, transient ischemic attack.
*Data contributed by CLOSURE-I and PC trial only.
†Composite primary outcome definitions: CLOSURE-I: stroke or TIA, all-cause death within 30 days, neurological death >30 days; RESPECT: stroke or TIA, all-cause
death within 30 to 45 days (device group, whichever comes first) or 45 days (medical group), all primary outcomes that occurred in RESPECT were strokes; PC trial:
stroke or TIA, all-cause death and peripheral embolism.
‡Per-protocol analysis corresponds to analysis of outcomes with treatments as received by patients. Data for synthesis here correspond to the per-protocol analyses
of CLOSURE-I and PC trial and the as treated analysis of RESPECT.
§Data contributed by CLOSURE-I and RESPECT only.
Downloaded from http://stroke.ahajournals.org/ by guest on July 28, 2017
Survivors of cryptogenic stroke are typically of young
age, with the index event in itself being a harbinger of mortality8 and recurrence potentially catastrophic; whatever risk
is conferred by PFO in those with an index event presumably persists for life and accrues over time. Withholding an
intervention that can potentially cut this risk in half for the
sake of statistical purity can seem unjustified and unethical.
However, any risks of an implanted device in the septum of
a heart would also be near-permanent and accrue over time,
including device-related thrombus formation or increase in
the risk of atrial fibrillation. What we owe patients, then, is
not just the option to close their hole with an appropriate
device, but better information in a reasonable time frame
about the risks and benefits of doing so. Unfortunately, in
this case, meta-analysis is no substitute for more randomized data.
Sources of Funding
This study was partially funded by grants UL1 RR025752, R01
NS062153, and R21 NS079826, all from the National Institutes of
Health.
Disclosures
Drs Kent and Thaler have consulted for WL Gore Associates. Dr Thaler
is a consultant to AGA Medical Corporation. The other author reports
no conflict.
Figure 3. Forest plot of subgroup meta-analyses results for the primary component outcomes of the 3 randomized clinical trials. Subgroup meta-analytic summaries (hazard ratios with corresponding 95% confidence intervals) and P values for the tests of interaction are
displayed in the 2 columns on the right side of the graph, respectively. ASA indicates atrial septal aneurysm; and TIA, transient ischemic
attack.
Kitsios et al PFO Closure Meta-analysis 2643
References
1. Kitsios GD, Dahabreh IJ, Abu Dabrh AM, Thaler DE, Kent DM. Patent
foramen ovale closure and medical treatments for secondary stroke prevention: a systematic review of observational and randomized evidence.
Stroke. 2012;43:422–431.
2. Furlan AJ, Reisman M, Massaro J, Mauri L, Adams H, Albers GW, et al;
CLOSURE I Investigators. Closure or medical therapy for cryptogenic
stroke with patent foramen ovale. N Engl J Med. 2012;366:991–999.
3. Carroll JD, Saver JL, Thaler DE, Smalling RW, Berry S, MacDonald LA, et
al; RESPECT Investigators. Closure of patent foramen ovale versus medical therapy after cryptogenic stroke. N Engl J Med. 2013;368:1092–1100.
4. Meier B, Kalesan B, Mattle HP, Khattab AA, Hildick-Smith D, Dudek
D, et al; PC Trial Investigators. Percutaneous closure of patent foramen
ovale in cryptogenic embolism. N Engl J Med. 2013;368:1083–1091.
5. Messé SR, Kent DM. Still no closure on the question of PFO closure. N
Engl J Med. 2013;368:1152–1153.
6. Lau J, Antman EM, Jimenez-Silva J, Kupelnick B, Mosteller F, Chalmers
TC. Cumulative meta-analysis of therapeutic trials for myocardial infarction. N Engl J Med. 1992;327:248–254.
7. DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin
Trials. 1986;7:177–188.
8.Rutten-Jacobs LC, Arntz RM, Maaijwee NA, Schoonderwaldt HC,
Dorresteijn LD, van Dijk EJ, et al. Long-term mortality after stroke
among adults aged 18 to 50 years. J Am Med Assoc. 2013;309:
1136–1144.
Key Words: foramen ovale, patent ◼ meta-analysis ◼ patent foramen ovale
closure ◼ secondary prevention ◼ stroke
Downloaded from http://stroke.ahajournals.org/ by guest on July 28, 2017
Potentially Large yet Uncertain Benefits: A Meta-analysis of Patent Foramen Ovale
Closure Trials
Georgios D. Kitsios, David E. Thaler and David M. Kent
Downloaded from http://stroke.ahajournals.org/ by guest on July 28, 2017
Stroke. 2013;44:2640-2643; originally published online July 18, 2013;
doi: 10.1161/STROKEAHA.113.001773
Stroke is published by the American Heart Association, 7272 Greenville Avenue, Dallas, TX 75231
Copyright © 2013 American Heart Association, Inc. All rights reserved.
Print ISSN: 0039-2499. Online ISSN: 1524-4628
The online version of this article, along with updated information and services, is located on the
World Wide Web at:
http://stroke.ahajournals.org/content/44/9/2640
Permissions: Requests for permissions to reproduce figures, tables, or portions of articles originally published
in Stroke can be obtained via RightsLink, a service of the Copyright Clearance Center, not the Editorial Office.
Once the online version of the published article for which permission is being requested is located, click
Request Permissions in the middle column of the Web page under Services. Further information about this
process is available in the Permissions and Rights Question and Answer document.
Reprints: Information about reprints can be found online at:
http://www.lww.com/reprints
Subscriptions: Information about subscribing to Stroke is online at:
http://stroke.ahajournals.org//subscriptions/