© International Epidemiological Association 2002
Printed in Great Britain
International Journal of Epidemiology 2002;31:125–127
Number and size of randomized trials reported
in general health care journals from
1948 to 1997
Steve McDonald,a,b Margaret Westby,a Mike Clarke,a Carol Lefebvrea and the Cochrane Centres’ Working
Group on 50 Years of Randomized Trials
Background Randomized trials are important for controlling selection biases, and where sufficient numbers of participants are involved, have the potential to yield reliable
estimates of treatment effects.
Methods
We investigated trends in the number and size of randomized trials reported
in general health care journals from 1948 to 1997. From the handsearching of
18 general health care journals we collected data on the number of reports of
randomized trials in each journal per year, and the number of participants in each
trial.
Results
A total of 5503 reports of trials were identified in 18 general health care journals.
More than a third appeared in the British Medical Journal. The peak period for trial
reports was the mid 1980s, with more in 1986 than any other year (242). By the
mid 1990s the number per year had declined by a third. Trials with fewer than
100 participants accounted for most of the reports (69%). In spite of the overall
decline in the number of trial reports, those involving 100 participants or more
continued to increase throughout the period studied.
Conclusions The continued increase in the number of larger trials reported is encouraging,
especially if it represents an increase in the size of trials more generally. Further
research is needed to determine whether the trends over time identified here are
reflective more of trends in the actual conduct of, rather than simply the reporting, of randomized trials.
Keywords
Randomized controlled trials, periodicals
Accepted
3 October 2001
In 1948 the British Medical Journal (BMJ) published the
landmark trial of streptomycin for pulmonary tuberculosis.1
Although not the earliest reported randomized trial,2,3 it is
credited with setting the standard for the conduct and reporting
of modern randomized trials, and marks the earliest year from
which members of the Cochrane Collaboration routinely search
for trials.4
Efforts by the Cochrane Collaboration to identify randomized
trials have resulted in over 310 000 references being made
available in The Cochrane Controlled Trials Register.5,6 Cochrane
centres have contributed to this activity by handsearching
general health care journals, a procedure in which the complete
contents of a journal are read to identify reports of trials. This
study presents an analysis of the number and size of randomized trials reported in a sample of these journals.
a UK Cochrane Centre, NHS Research & Development Programme, Oxford
OX2 7LG, UK.
b Current address: Australasian Cochrane Centre, Monash Institute of Health
Services Research, Melbourne, VIC 3168, Australia.
Group participants: Gerd Antes (German Cochrane Centre), Xavier Bonfill
(Iberoamerican Cochrane Centre), Frank Davidoff (Annals of Internal Medicine),
Peter Gotzsche (Nordic Cochrane Centre), Margaret Haugh (French Cochrane
Centre), Marjan Loep (Dutch Cochrane Centre), Philippa Middleton
(Australasian Cochrane Centre), Elizabeth Pienaar (South African Cochrane
Centre) and Vanna Pistotti (Italian Cochrane Centre).
Correspondence: Steve McDonald, Australasian Cochrane Centre, Monash
Institute of Health Services Research, Monash Medical Centre, Locked Bag
29, Clayton, VIC 3168, Australia. E-mail: [email protected]
Methods
To ensure a range of countries and languages was represented,
each Cochrane centre involved in the study selected one general health care journal from one or more of the countries for
which it acts as the reference centre. Where possible, this was
the journal of the national medical association (or equivalent).
Journals were handsearched for the 50 years from 1948 to
1997 for all reports of trials in which participants were
randomly (or quasi-randomly) assigned to alternative forms of
125
126
INTERNATIONAL JOURNAL OF EPIDEMIOLOGY
Table 1 Number of randomized trial reports in general health care journals
Journal name
(period searched 1948–1997 unless stated)
Country of publication
Language of publication
USA
English
580
UK
English
2016
Annals of Internal Medicine
BMJ
Deutsche Medizinische Wochenschrift
No. of trial reports
Germany
German
254
Duodecima
Finland
Finnish
25
Irish Medical Journal
Ireland
English
82
Sweden
Swedish
168
319
Lakartidningena
Medical Journal of Australia
Australia
English
Medicina Clinica
Spain
Spanish
97
Minerva Medica
Italy
Italian
116
Nederlands Tijdschrift voor Geneeskunde
New Zealand Medical Journal
Presse Medicale
Saudi Medical Journal (1979–1997)
Schweizerische Medizinische Wochenschrift (1967–1997)
South African Medical Journal
Tidsskrift for den Norske Laegeforeninga
Ugeskrift for Laegera
Wiener Medizinische Wochenschriftb (1976–1997)
Netherlands
Dutch
103
New Zealand
English
137
258
France
French
Saudi Arabia
English
19
Switzerland
German
168
South Africa
English
347
Norway
Norwegian
161
Denmark
Danish
573
Austria
German
80
a Data on the number of randomized participants not extracted. Reports of randomized trials published as letters included, reports of quasi-randomized trials
not included.
b Data on the number of randomized participants extracted for 1985–1997 only.
care. Trial reports published as letters, conference abstracts and
news items were excluded from this study. We analysed data on
the annual number of trial reports identified in each journal,
and the number of participants in each trial.
Results
Data for 18 journals were supplied by 10 Cochrane centres.
Fifteen were searched for the entire period from 1948 to 1997.
Seven journals were published in English, three in German, and
one each in Danish, Dutch, Finnish, French, Italian, Norwegian,
Spanish and Swedish (Table 1).
A total of 5503 trial reports were identified from a total of
more than 800 journal years of searching. The largest single
contribution was from the BMJ (2016 reports, 37%). The peak
period for trial reports was the mid 1980s, with more in 1986
than any other year (242). By the mid 1990s the number had
declined by a third to levels similar to the 1970s (Figure 1). This
general pattern was seen in each journal, with the peak year for
each mostly occurring during the 1980s.
The number of randomized participants was extracted from
4527 (82%) of the trial reports, representing a total of more
than 3.7 million people. The largest trial (1 910 093 children)
was reported in the BMJ in 1957.7 To investigate trends in
size, trials were grouped into those that randomized fewer
than 100 participants (for ease of reference these are deemed
‘small’), and those that randomized 100 or more (‘large’ trials).
Most (3139, 69%) of the trials were small.
The number of reports of large trials per year increased from
a mean of 9 in the mid 1950s to 44 in the mid 1980s. In contrast
to the pattern for small trials, this increased into the 1990s
(to 62). The increase in the 1990s was primarily a feature of two
journals (Annals of Internal Medicine and the BMJ) which account
for 76% of all large trials identified between 1990–1997. The
number of large trials appearing in the other journals was
either declining or the trend was uncertain. The proportion of
trials defined as large increased over time. During the 1950s and
1960s a third of the trials were large. This proportion fell to a
quarter by the mid 1980s but then increased, so that just over
half the trials had 100 or more participants by the mid 1990s.
Discussion
Figure 1 Number of trial reports per year according to trial size over
time
Our study found that, in spite of a decline in the number of trial
reports appearing in general health care journals over the last
10–15 years, there was an increase throughout the 50-year
period in the number of trials involving 100 or more participants.
TRIALS REPORTED IN HEALTH CARE JOURNALS 1948–1997
The downturn in trial reports seen in these journals is
unlikely to reflect a wider trend to conduct fewer randomized
trials. A search of MEDLINE, for example, using the publication
type term Randomized-Controlled-Trial reveals a large year-onyear increase since 1966 in the number of studies retrieved. In
addition, a survey of 2000 controlled trials in schizophrenia,8
found largely in psychiatric journals by hand- and electronicsearching, revealed a steady increase in the annual number of
trials over 50 years. More plausible explanations are the growth
in the number of specialist journals, and the changing nature of
general journals. A publication analysis of trials conducted by
the UK Medical Research Council shows a trend towards reporting in specialist journals (I Chalmers, personal communication).
For the period 1948–1967, 98% (59/60) of MRC trials were
reported in the BMJ or the Lancet, but this was only 20%
(32/160) for 1978–1997. Competition from specialist journals,
particularly those with an international readership, is likely
to be most keenly felt by the general journals with smaller
circulations.
Decreases in the number of trial reports in general journals
may indicate changing editorial policies and author preferences.
The emergence of systematic reviews may have affected the
number of trials being published in general journals, and the
trend towards larger trials may suggest that editors are becoming more selective in the trials they publish. Authors themselves
may be by-passing general journals, especially if the trial is small
and of only limited interest, or deciding that their research will
receive greater exposure in a leading specialist journal.
The trend towards an increasing number and proportion of
‘large’ trials is encouraging, although this is only apparent in
the Annals of Internal Medicine and the BMJ. The number of trials
involving over 1000 participants has also increased in these two
journals. This might suggest that editors are becoming more
selective in the trials they decide to publish, but also, hopefully,
indicates a realization by researchers and funding bodies of
127
the need to conduct trials that are of a sufficient size to detect
moderate but important treatment effects.
Our study had several potential limitations. First, our sample
size was small and omitted other major general journals such
as JAMA and the Lancet. Second, we acknowledge that 100 is an
arbitrary marker of trial size and that other factors besides
absolute sample size numbers affect power calculations. Third,
we did not attempt to correct for potential confounding resulting from changes in the number of pages and/or number of issues
published per year in these journals. Finally, a small number of
trials might have been counted in the analyses more than once,
for example, where multiple reports of the same trial appeared
in two or more journals.
Conclusion
Our findings have revealed a decline in the number of trial
reports in general health care journals, but also a tendency for
the trials that do appear to be larger. Further research is needed
to determine whether these findings are reflective more of trends
in the conduct of trials or in their reporting.
Acknowledgements
Financial support: The handsearch of the European journals
included in this study formed part of a European Union project
under the BIOMED-1 Program [contract number BMH1-CT941289] which was co-ordinated at the UK Cochrane Centre.
The authors thank Daniel Galandi, Kirsten Lone Jensen, Joan
Marti and Jane Nguyen for contributing to the data extraction,
Jim Halsey for running the statistical analyses, Iain Chalmers for
permission to use unpublished material, and the many handsearchers who between them read through a combined total
of over 800 years of published papers. MW is supported by a
Daphne Jackson Fellowship.
KEY MESSAGES
•
The 1980s was the peak period for trial reports and was followed by a downturn in the 1990s.
•
Trials with fewer than 100 participants accounted for over two-thirds of the 5500 reports.
•
In spite of the decline in the number of trial reports, the continued increase in the number of larger trials is
encouraging.
•
The downturn in the trial reports seems most likely to be due to a growth in the number of specialist journals
and the changing nature of general journals, rather than a reflection of fewer randomized trials being conducted.
References
1 Medical Research Council Streptomycin in Tuberculosis Trials
5 Lefebvre C, Clarke MJ. Identifying randomised trials. In: Egger M,
Committee. Streptomycin treatment for pulmonary tuberculosis.
Br Med J 1948;ii:769–82.
Davey Smith G, Altman DG (eds). Systematic Reviews in Health Care:
Meta-analysis in Context. 2nd Edn. London: Br Med J Publishing, 2001.
2 Royal College of Physicians of Edinburgh. Controlled trials from history.
6 The Cochrane Controlled Trials Register. In: The Cochrane Library.
(www.rcpe.ac.uk/controlled_trials) Accessed 30 September 2001.
3 Doll R. Controlled trials: the 1948 watershed. Br Med J 1998;317:
1217–20.
4 Bero L, Rennie D. The Cochrane Collaboration. Preparing, maintain-
ing, and disseminating systematic reviews of the effects of health care.
JAMA 1995;274:1962–64.
Oxford: Update Software, 2001. Updated quarterly.
7 Medical Research Council Poliomyelitis Vaccines Committee. The
assessment of the British vaccine against poliomyelitis. Br Med J
1957;i:1271–77.
8 Thornley B, Adams C. Content and quality of 2000 controlled trials in
schizophrenia over 50 years. Br Med J 1998;317:1181–84.