Papers
Results
What is already known on this topic
We identified 292 eligible trials including a total of
258 315 participants that evaluated 75 different
strategies for increasing response to postal questionnaires (for references see bmj.com). The trials identified
were not restricted to medical surveys: one third were
medical, epidemiological, or health related; one quarter
were psychological, educational, or sociological; and two
fifths were marketing, business, or statistical.
All tests for selection bias were significant (P < 0.05)
in five strategies: monetary incentives, varying length of
questionnaire, follow up contact with non-respondents,
saying that the sponsor will benefit if participants return
questionnaires, and saying that society will benefit if participants return questionnaires. Tests were not possible
in 15 strategies where fewer than three trials were
included. The method of randomisation was not known
in most of the eligible trials. Where information was
available, the quality of the concealment of participants’
allocation was poor in 30 trials and good in 12 trials. The
figure shows the pooled odds ratios and 95% confidence
intervals for the 40 different strategies in which the combined trials included more than 1000 participants.
We found significant heterogeneity among trial
results in 17 out of the 31 strategies that included more
than one trial. For trials of monetary incentives the
heterogeneity among the results was significant
(P < 0.0001): the odds of response initially increased
with an increasing amount of money. This effect, however, levelled off when the amount greatly increased.
Heterogeneity was also found among the results of
trials examining the effect of questionnaire length
(P < 0.00001): with increasing length of the questionnaire the odds of response decreased
Postal questionnaires are widely used in the collection of data in
epidemiological studies and health research
Non-response to postal questionnaires reduces the effective sample
size and can introduce bias
What this study adds
This systematic review includes more randomised controlled trials than
any previously published review or meta-analysis on questionnaire
response
The review has identified effective ways to increase response to postal
questionnaires
this systematic review. Some strategies will require
additional materials or administrative time, whereas
others can be implemented at little extra cost.
We have presented odds ratios for methodological
reasons,4 but the practical implications of the odds
ratio for a strategy may be difficult to interpret without
knowing the response at baseline without the strategy.
If the size of the effect that would be expected if a specific strategy were used is an important consideration
for researchers, the data used in this review may be
accessed through the Cochrane Library, where they will
be updated regularly.5
We thank Peter Sandercock, Iain Chalmers, and Catherine
Peckham for their help and advice with the study.
Contributors: see bmj.com
Funding: The study was supported by a grant from the
BUPA Foundation and a Nuffield Trust short term fellowship.
Competing interests: None declared.
Discussion
Before these results are implemented, several methodological issues must be considered.
Identification and inclusion of all relevant trials
We estimate that our search strategy retrieved nearly all
eligible trials (estimated sensitivity 95%; 84% to 99%)
and that we missed very few relevant records during
screening.3 We excluded some trials because we could
not confirm that participants had been randomly allocated to intervention or control groups, and we have
not examined whether the results of these trials differ
systematically from the included trials. Tests for
selection bias were significant in five strategies.
Although these results may be due to true heterogeneity between trial results rather than bias in the
selection of trials,2 we cannot rule out the possibility of
selection bias having an effect on the results.
Methodological quality of trials
In our review, information on allocation concealment
was unavailable for most of the included trials. If they
had inadequate concealment, this may have biased the
results, which is unlikely in this context because the
researchers making the allocations would find it difficult
to predict propensity to respond to a questionnaire.
Conclusions
Researchers can increase response to postal questionnaires by using the strategies shown to be effective in
BMJ VOLUME 324
18 MAY 2002
bmj.com
1
2
3
4
5
Armstrong BK, White E, Saracci R. Principles of exposure measurement in
epidemiology. Monographs in epidemiology and biostatistics. Vol 21. New York:
Oxford University Press, 1995:294-321.
Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta-analysis
detected by a simple graphical test. BMJ 1997;315:629-34.
Edwards P, Clarke M, DiGuiseppi C, Pratap S, Roberts I, Wentz R. Identification of randomised controlled trials in systematic reviews: accuracy
and reliability of screening records. Stat Med (in press).
Engels EA, Schmid CH, Terrin N, Olkin I, Lau J. Heterogeneity and statistical significance in meta-analysis: an empirical study of 125
meta-analyses. Stat Med 2000;19:1707-28
Edwards P, Roberts I, Clarke M, DiGuiseppi C, Pratap S, Wentz R, Kwan I.
Methods to influence response to postal questionnaires. Cochrane Library.
Issue 2. Oxford: Update Software, 2002.
(Accepted 3 December 2001)
Corrections and clarifications
This Week in the BMJ
We inadvertently gave the dates the wrong way round in the key to the
figure that accompanied the summary of the paper by Chiara Pandolfini
and Maurizio Bonati (9 March, pp 582-3). This gave the impression that
the quality of health information on the internet had decreased, whereas
in fact it had improved.
Canada sees a bigger role for private companies in health care
In the news article by David Spurgeon (2 February, p 259) we were
wrong to say that hip replacements have been contracted to private
clinics in Alberta. Although standards for facilities that seek to provide
overnight postoperative care and major arthroplasties were approved
recently by the College of Physicians and Surgeons of Alberta, no
private surgical facility in Alberta has been approved (yet) to provide
those services.
1185