The Clinician's Illusion
Patricia Cohen, PhD, Jacob Cohen, PhD
There are several diseases, lncludlng schlzophrenla, alcohollsm, and oplab addlctlon, tor whlch the long-term prognosls Is Subject to disagreement between cllnlclans and re-
(Arch Gen Psychlatry 1984;41:117&1182)
Cohen), New York.
St, New York, NY 10032 (Dr P. Cohen).
1178
Arch Gen Psychiatry-Vol41, Dec 1984
findings. Current research reports i
not allow a proper consideration and
Recommendations for future researc
n more striking is the view of clinicians,
by the public, of opiate addiction as an
for most, if not all, users. This view was
adicted by Robins and associates,1T*"who
of recidivism of juvenile delinquents and
as well. They may also be found, although
sser degree, in certain more strictly phys-
heri"a, Arch Gen Psychiatry-Vol41,
lasts a year is far more likely to appear on any given day
than one whose illness lasts only two months, in fact, six
times more likely; and a patient whose illness lasts for ten
years is ten times more likely to appear than one whose
illness lasts only one year. For this example, each of the few
patients making up the 2% of the Bopulation of longest
duration (X=128)is 64 times more likely to appear than
each of the 40% of shortest duration (X =2), 32 times more
likely to appear than each of the 25% of next shofist
duration (X=4), and so on.
The outcome of this durationdependent differential
probability of being encountered on rounds is that the
distribution of illness duration for the rounds sample is
markedly different from that of the population made up of
people who ever contract the illness. By weighting the
percentages in each duration interval in Fig 1 by its mean
and recomputing the'percentages for the weighted distribution, the duration distribution for the rounds sample of Fig
2 is produced. Perhaps most startling is the outcome of this
differential weighting for the typical current clinic group;
nearly a quarter of the group is made up of the 2% of the
population who have the longest duration, ie, the worst
40,
1
I
m
30-
0
'S
M-
8
Duration Mean=lOJ mo
I
I
I
10-
I
0-
C3
Duration
ndings are characterized by great
of the illness. Other characterislong duration possible) are that
Mean, m0
2
6 t o 4 2 24t0<48
296
3 to <6
12 to <24 48 to <96
4
8
16
64
32
128
Fig 1.-Distribution of illness duration in population. (Sum of
X times percent ~ l u e for
s seven intervals in Fig 1 is 1,068. This is
now divisor for weighted percentages. For example, percent of
lowest interval for rounds sample is 2[40]/1.068=7.5%; for next
highest, 4[25]11,068= 9.4%; etc.)
Fig 2.-Distribution of illness duration in clinic sample from same
population.
40-
of the illnesses discussed, a
Duration Mean= 50.8 mo I
I
I
I"
0
f
::Ezddl
<3
0
DlJrati0n
mo
Mean, mo 2
Dec 1984
6to <I2
3 to c6
4
24tO 4 8
48 to <96
12 to <24
8
16
296
32
64
128
Clinician's Illusion-Cohen & Cohen
1179
prognosis. Furthermore, the ultimate average duration for
the daily clinic rounds sample seen by our hypothetical
clinician will be more than four years (ie, 50.8 months), in
stark contrast to the true population average duration of
less than one year (10.7 months). Small wonder that the
clinician will view any truly population-based findings as
grossly discrepant from daily experience. Note again that
the phenomenon does not depend on the shape of the
distribution but only on its relative variability.
I t i s useful to examine what happens to this illusion if a
duration requirement is inserted into the diagnostic criteria, as has been done for some af the diseases discussed
previously. Suppose in our previously mentioned example
that those ill less than six months were excluded from the
diagnosis. For the remaining 35%of the original population,
the mean duration is now 25.4 months. A clinic sample of
cases so diagnosed would have a mean duration of 60.5
months, still a large discrepancy.
How then can one correct the observed rounds sample for
this distortion of its population representation? Since each
case's chance of being included in the sample is proportional
to its illness duration, one needs to weight each case
inversely by its duration to reproduce the population distribution. Although we do not know what the eventual duration of each case will be, we do know the duration up to the
time of observation, and Freeman and Hutchisona provide a
detailed exposition of how to use durations to date of the
clinic sample to estimate the distribution of illness duration
for the entire population.
In the previously mentioned example, we assumed a
constant probability of treatment throughout the illness.
Should this not be the case, the bias may be increased or
decreased. If treatment is most likely for the short duration
cases, the bias will be lessened, if most likely for the long
duration, it will increase. Probably the latter is most
frequent in real clinical populations. Again, it is a question
of the ratio of probabilities. In our example, if the longest
duration group were twice as likely to be treated relative to
the shortest, they each would be 3.28(rather than 64) times
more likely to appear in the sample than a member of the
shortest duration group.
One necessary consequence of this bias in the representation in the sample is a bias with regard to the presence of
any condition or symptom more characteristic of long-term
than short-term treatment of patients. Therefore, it may be
impossible to distinguish between a characteristic affecting
the duration of illness (failure to recover) from one causally
related to the illness.
In sum, the clinician's illusion is the attribution of the
characteristics and course of those patients who are currently ill to the entire population contracting the illness. To
put the issue in terms familiar to epidemiologists, it is the
consequence of using a prevalence sample as a substitute for
an incidence sample.
OTHER SOURCES OF BIAS IN
SAMPLES FROM CLINICAL SETTINGS
There are, of course, several other reasons why samples of cases
in treatment settings are likely not to be representative of the
entire population of persons experiencing the condition in question. Although they have been discussed in other places, it is useful
to consider them herein because they are likely to add to the
illusion identified previously.
First, those who recover from a given episode of the condition
without entering the formal treatment system, that is, by using
personal and social resources, are likely to have a better prognosis
by virtue of the stability of the curative factors. Unlike those who
seek professional help, their help will not disappear when they
1180
Arch Gen Psychiatry-Vol41, Dec 1984
conclusions.
This problem should not be confused with that of the distortion
relationships between characteristics that is typical when sample
are selected on the basis of these characteristics." When,
samples and the populations whence they come.
FACTORS RELATING TO THE MAGNITUDE
OF THE CLINICIAN'S ILLUSION
In addition to characteristics of patients and treatment seleeti
that lead to bias in treated samples, it is worthwhile to identi
factors that are likely to maximize the impact of the clinician
illusion. The disparity between what the clinician actually observes and the true state of the population will be greatest when
2. Disease manifestations fluctuate, symptoms are episodic, and
treatment is sporadic and handled by multiple clinical services Or
clinicians. Under these conditions, it is impossible for clinicians to
know whether a "cure" is permanent and whether the patient is
receiving help in a new setting. Since recovered patients do not
return to the clinic whereas relapsed patients do, it may be unclear
whether recovery occurs at
Clinician's Illusion-Cohen & Cahen
THE CLINICIAN'S ILLUSION AND DISPAR~T~ES
AMONG RESEARCH FINDINGS
to overestimate the number of new cases. This
COMMENT
The phenomenon described herein as the clinician's illusion should be understood in the context of other illusions;
that is, as a natural consequence of a combinationof certain
human perspectives and information-processing tendencies." Thus, it is not intended to be a pejorative term and
rch Gen Psychiatry-Vol41,
I I
Dec 1984
Clinician's Illusion-Cohen & Cohen
1181
matters relevant to the natural history of chronic diseases.
This issue arises with particular potency in the area of
National Institute of Mental Health.
Albe*Shkard, MD,gave eneoUrSgementtoarticle.
References
1182
Arch Gen Psychiatry-Vol41, Dec 1984
the
of