Reaching certainty:
A descriptive study of “stable” Cochrane Reviews and coming to firm conclusions
Hilda Bastian, National Center for Biotechnology Information, National Library of Medicine, NIH, USA
Lars G. Hemkens, Basel Institute for Clinical Epidemiology and Biostatistics, University Hospital Basel, Basel, Switzerland
Background
Results
According to the Cochrane Handbook, a “stable” review is
“one that is highly likely to maintain its current relevance for
the foreseeable future.”
TABLE. Reviews declared “stable” with firm conclusions
(New trials would be unlikely to change conclusions)
•
•
•
•
•
•
•
•
•
Objectives
To monitor the extent and reasons for Cochrane reviews
(CR) being designated stable, and to describe the reviews
designated by the CR authors as having a conclusion that
would be unlikely to change with new trials (called here a
“firm conclusion”).
•
•
Methods
We had previously identified six categories of Cochrane
reviews designated “stable” by review groups (CRGs):
1. Little or no further evidence expected (“No data
expected”)
2. Intervention discontinued or superseded (“Intervention
discontinued”)
3. Update required, but in more than 2 years (“Longer
update interval”)
4. Review has been or will be superseded by a new review
(“Review superseded”)
5. Will search every 2 years, but only update if a new trial
appears (“Monitor”)
6. New trials would be unlikely to change conclusions (“Firm
conclusions”)
Stable reviews were identified in the February 2013 Issue of
the Cochrane Database of Systematic Reviews (CDSR).
Data on the year and reasons of the designation “stable”
were extracted from the “What’s new” section. When
reasons were not clearly reported there, the abstract,
discussion and conclusions sections were searched. Both
authors agreed on categories. The conclusions and body of
evidence of reviews with firm conclusions were described.
•
FIGURE. Reasons given for declaring Cochrane reviews
stable (n = 154)
There were 180 stable reviews among 5137 “nonwithdrawn” intervention and diagnostic CRs in the CDSR
(3.5%).
•
•
•
•
Selegeline for Alzheimer's disease
Short vs long antibiotics for acute otitis media
Myobacterium immunotherapy for tuberculosis
Thiamine for Alzheimer's disease
Immediate vs deferred AZT for HIV
D-cycloserine for Alzheimer's disease
Condoms to reduce heterosexual HIV transmission
DOT (directly observed therapy) for tuberculosis
Male circumcision to prevent heterosexual HIV
acquisition in men
Nonoxynol-9 for preventing heterosexual HIV
transmission to women
Nonoxynol-9 for preventing heterosexual STI
transmission to women
H-pylori eradication vs non-eradication therapy to
prevent gastric bleeding
Oral vs IV rehydration for children with dehydration
from gastroenteritis
Calcium antagonists for Duchenne muscular dystrophy
Electronic mosquito repellents to prevent mosquito
bites and malaria infection
Microwave therapy for cervical entropion
Conclusions
The number of reviews being designated stable has been
increasing in recent years. Reasons for the designation
were often not explained in the “What’s new” section, and a
total of 26 reviews gave no clear reason in the review for the
categorizing reviews as stable (14%).
The use of the “stable” category is determined on a case-bycase basis, with highly variable practice among CRGs.
Reasons for the decision to designate a review “stable” are
often poorly reported.
Where a reason was given (154 reviews), the most common
category was a belief that future trials were unlikely (36%)
(see Figure).
Cochrane authors rarely both conclude that new trials would
be unlikely to change the conclusions of their reviews and
designate the review as “stable”. They may be more likely to
do so when there is an absence of evidence of effect.
Only 16 reviews (Table) were designated as having firm
conclusions, mostly because there was no evidence of
benefit, or any benefits were either clinically unimportant or
not sustained (11/16 reviews, 69%).
Partially supported by the NIH intramural research program.