2
Editorial
Editorial: Transient Ischemic Attacks and Aspirin,
Stroke and Death; Negative Studies and Type II Error
Downloaded from http://stroke.ahajournals.org/ by guest on June 16, 2017
OF ALL SCIENTIFIC RESEARCH, clinical studies
with their large number of recognized and unrecognized variables demand the most sophisticated and rigorous hypothesis testing and experimental design. Because of the small numbers and special charcteristics of
a single center, large prospective double-blind cooperative studies have become necessary. Unfortunately,
when multiple centers and investigators with different
patient populations are involved, the probability of
defects in design increases. Not unexpectly, the cooperative studies concerning the effect of the antiplatelet
aggregating agent acetylsalicylic acid (aspirin) on
stroke and death in patients with transient ischemic
attacks (TIA) are no exception.12 Nevertheless, despite different weaknesses, past multiple center studies
have always shown significant differences in favor of
aspirin over placebo.34 In this issue, two additional
prospective double-blind controlled cooperative studies are reported.5,6 For the first time one of these, the
Danish study,5 did not conclude that aspirin was effective in reducing stroke and death. Although the French
study6 was primarily a study of stroke and not TIAs, it
did demonstrate a significant decrease in recurrent
stroke and myocardial infarction in those who were
treated with either aspirin or the combination of aspirin
and dipyridamole. Thus, the results are similar to those
of the Canadian3 and the United States4 cooperative
studies.
Does the failure of the Danish study to show any
favorable affect of aspirin mean that the other studies
are wrong? If not, does it mean that Danes are different
from Frenchman, Canadians and Americans? If neither of these alternatives are true, should one assume
that the Danes were studying a different population of
patients? If we analyze these studies, certain charactersitics suggest that none of the previous possibilities
need necessarily be true. First, because of the high
likelihood of Type II error in the Danish study, one
should not conclude that, because it does not show a
difference, none exists. In addition, there are other
reasons that this study might have different results
from the other cooperative studies. First, a number of
risk factors were identified by the investigators that
might affect outcome. They concluded that, as the
occurrence of these factors was not significantly different between the placebo and aspirin groups, randomization was successful. One could look at this in a
different way. Diseases and risk factors listed in the
paper thought to be unfavorable, as determined in a
previous study in the same hospitals7 or by the Hospital
Frequency Study of TIA,8 are summarized (table 1).
Withdrawal from the study before an end point was
considered to be favorable as once withdrawn the patient was no longer at risk. Although it is true that for
each factor there is no statistical difference between the
groups, in 15 of the 17 the trend is in favor of placebo.
Though these factors may not be completely independent as required for the binomial distribution, if each
had an equal chance of falling into either treatment
group, the probability of 15 or more falling into one
group would be about one in 400. Although the binomial formula is not entirely appropriate, one must
still assume that the odds are quite high that the results
in each group are not primarily related to the therapy
prescribed. Also, using the binomial formula for the
French, American and Canadian studies, the favorable
risk factors occur randomly.
Additional differences from other studies are noted
in both the Danish and the French studies. For example, in the Danish trial 306 patients presented to the
study but 99 were excluded before randomization. Of
the 207 that were randomized, four were dropped after
randomization because the diagnosis was not correct.
This left 203 patients of which 101 received aspirin
and 102 placebo. From this point on, the authors analyzed all events on the basis of these numbers. Yet a
substantial percentage of patients in each group discontinued the trial before an end point occurred and,
therefore, were no longer at risk. This resulted in a
very small number of patients remaining in the study
(77 in the aspirin and 71 in the placebo group). Furthermore, half of the patients did not have platelet
aggregation studies at six months and, of those who
did, 10% who were assigned to the aspirin group yielded no aspirin effect in their blood samples. High withdrawal rates were also present in some of the other
studies but in those with large numbers this was less
critical. In the Danish study, only 67% of those taking
aspirin had TIAs as compared to 76% of those taking
placebo. All of these observations are confounding,
indicating an unusual selection factor and an unusual
drop out.
Both the Danish and French studies are interesting
as they do not show any sex subgroup differences. The
Canadian Study3 noted that the favorable response to
aspirin in men was significantly different than in females and a review of the American Study4 and the
Cooperative Hospital Frequency Study of TIA8 supported the Canadian observation. In each of these studies women had fewer strokes and lower mortality than
men regardless of treatment.15 Data from Hospital
Frequency Studies suggest that this might be primarily
limited to older age men and one possibility was that
older men had a greater loss of prostacyclin function so
EDITORIAL, TIAS AND TYPE II ERROR/Dyken
TABLE 1 Percentage of Patients in Treatment Group with Risk
Factors
Risk Factor
Downloaded from http://stroke.ahajournals.org/ by guest on June 16, 2017
Age 60 years or more
Male sex
History of
myocardial infarction
intermittent claudication
hypertension
smoking
oral contraceptives in Females
Attacks
single
24-72 Hours
Carotid territory
Polycythemia
Hyperlipidemia
Platelet hyperaggregability
Atrial fibrillation
Cardiac enlargement
Relevant carotid stenosis
Withdrawal from study
Treatment
Acetylsalicylic
Placebo
acid
52.9*
69.6*
57.4
76.2
6.9*
11.8*
26.5*
64.3*
29.0*
10.9
17.8
27.7
75.3
37.5
48.0
22.5*
72.5*
1.0*
34.4
31.3*
2.0*
8.9*
12.0*
30.4*
47.5*
29.7
81.2
3.0
27.4*
40.9
4.0
9.0
36.1
23.8
*Denotes a difference favoring this group.
that the system was unbalanced. Another possibility is
that because of the small number of events in females,
larger numbers would need to be entered to reach statistical significance and if the evolution of the disease
process is low it might take much longer than the
average two to three years of follow-up performed in
most cooperative studies for the effect to become apparent. The possibility exists that the lack of sex difference in the French study could be because 84% of the
patients entering already had a more serious event,
stroke, and that the time factor would be much less of a
contributor. Of course, the Danish Study did not show
any overall difference and it is not surprising that subgroups also did not show a difference.
Regardless of these problems, the Danish Study
should be published and be available for reference
because of the paucity of cooperative prospective studies. This critique is not intended to single it out as
having more defects than other studies (which is not
true), but to identify the differences so that they can be
considered in analyses or as a base for planning future
studies.
The most serious potential problem with the Danish
Study is that, as many outstanding clinicians are not
well based in statistical techniques, they might conclude after reading this paper that because no differences were noted between those treated with aspirin
and placebo that the study indicates that aspirin is not
effective. A number of excellent reviews in texts and
journals have pointed out the high likelihood of Type II
3
error (concluding that no real difference exists when it
does) due to the small number of patients in many
clinical studies.9-14
The aim of any randomized controlled trial is to
estimate the real response rate for a treatment and to
compare this to an estimate of the real rate for a control. Differences commonly are accepted as real when
a small likelihood exists that they occur by chance.
Each investigator must determine what risk will be
taken before the study begins. Commonly, one accepts
one chance in twenty (alpha equal to 0.05) or one in a
hundred (alpha equal to 0.01). These are the probabilities of a Type I error (concluding differences are real
when actually they are by chance). On the other hand if
no statistically significant differences occur, do we
assume that no real differences exist? If we do and
there really is a difference, a Type II error has been
made. To decrease the magnitude of a Type II error,
before any study is initiated investigators must estimate the true response rates and what changes in these
would be of practical importance at a clinical level.
Once this has been done, they then must determine the
acceptable level of a Type I error, and then estimate the
power (1 - Type II error) of the trial to detect differences with reasonable confidence. For instance, using
the Danish study as an example, it was assumed from
previous studies that 20% of untreated patients would
have a disabling stroke or die. If we accept that a onefourth reduction to 15% would be of practical clinical
importance and we would like to accept a probability
of less than 0.05 of making a Type I error, then we are
in a position to estimate how many patients would be
required in each group to detect this difference if it
truly exists. For instance, if we wanted the power of
the test to be 0.90 or a 90% chance of seeing a difference if it actually existed, we could then calculate how
many patients would be required in each group. For
this example, if we expect to have 90% chance of
seeing a difference of 20% to 15% with an alpha level
of 0.05, then 2,580 patients or 1,290 in each treatment
group for a two-tailed test, would be required; or for a
one-tailed test a total of 2,132 or 1,066 in each group.13
Even a 50-50 chance of seeing the difference (power
0.5) would require a total of 1,042 for a two-tailed test
or 776 for a one-tailed test. Similarly, to see a one-half
reduction from 20% to 10% would require 610, 510,
270 or 210 total patients. Even if there had been no
withdrawals in the Danish Study, 203 patients would
not have been expected to show a difference if one
existed. Therefore, one cannot conclude from this
study that as no statistically significant difference was
seen that, indeed, no real difference exists. To draw
these conclusions, the study would have required
somewhere between three to ten times the number entered. Fortunately, the authors do not draw these conclusions but the unsophisticated reader might. Indeed
the authors point out that, even though a small difference in favor of placebo was observed, because of the
very small numbers in the study, this observation is
quite consistent with there being a true benefit of aspirin of as much as 50% reduction in events.
4
STROKE
This study should not be ignored as the previous
prospective studies also have their defects. Regardless, if one assumes the Danish study is comparable to
the American and Canadian studies and we combine all
aspirin treated groups and all placebo groups, the statistical tests accepted by each study would still show a
difference in favor of aspirin at much less than a 0.05
level.
In conclusion, two additional prospective doubleblind studies have been added to the literature. One
gives additional support to previous studies. The second shows no statistical differences which would be
expected even if a difference exists, because of the
small number of patients entered. These studies have
defined their population well, making it easy to recognize chance bias and, therefore, are available for reference and for future analysis.
Downloaded from http://stroke.ahajournals.org/ by guest on June 16, 2017
Mark L. Dyken, M.D.
Department of Neurology
Indiana University School of Medicine
Indianapolis
Indiana 46223
References
1. Dyken ML: Assessment of the Role of Antiplatelet Aggregating
Agents in Transient Ischemic Attacks, Stroke and Death. Stroke
10: 602-604, 1979
2. Dyken ML: Transient Ischemic Attacks and Stroke: Aspirin Trials.
In Thrombosis and Atherosclerosis. Editors Bang NU, Glover JL,
Holden RW, Triplett DA. Year Book Medical Publishers, Chicago-London, pp 393-404, 1982
3. Canadian Cooperative Study Group: Randomized Trial of Aspirin
and Sulfinpyrazone in Threatened Stroke. In N Engl J Med 299:
53-59, 1978
4. Fields WS, Lemak NA, Frankowski RF, Hardy RJ: Controlled
V O L 14,
No
1, JANUARY-FEBRUARY
1983
Trial of Aspirin in Cerebral Ischemia. Stroke 8: 301-316, 1977
5. S0rensen PS, Pedersen H, Marquardsen J, Petersson H, Heltberg
A, Simonsen N, Munck O, Andersen LA: Acetylsalicylic Acid in
the Prevention of Stroke in Patients With Reversible Cerebral Ischemic Attacks. A Danish Study. Stroke (current issue)
6. Bousser NG, Eschwege E, Haguenau M, Lefauconnier JM, Thibult N, Touboul D, Touboul PJ: The French "A.I.C.L.A." Randomized Trial of Aspirin and Dipyridamole in the Secondary Prevention of Atherothrombotic Cerebral Infarction. Stroke (current
issue).
7. Simonsen N, Christiansen HD, Heltberg A, Marquardson J, Pedersen HE, S0rensen PS: Long-Term Prognosis After Transient Ischemic Attacks. Acta Neurol Scandinav 63: 156-168, 1981
8. Conneally PM, Dyken ML, Futty DE, Poskanzer DC, Calanchini
PR, Swanson PD, Price TR, Haerer AF, Gotshall RA: Cooperative
Study of Hospital Frequency and Character of Transient Ischemic
Attacks. VIII. Risk Factors. JAMA 8: 742-746, 1978
9. Aleong J, Bartlett DE: Improved Graphs for Calculating Sample
Sizes When Comparing Two Independent Binomial Distributions.
Biometrics 35: 875-881, 1971
10. Cohen J: Statistical Power Analysis for the Behavioral Sciences,
Academic Press, New York, 1977
11. DerSimonian R, Charette J, McPeek B, Mosteller F: Reporting on
Methods in Clinical Trials. N Engl J Med 306: 1332-7, 1982
12. Feinstein LAR: Sample size and the other side of "statistical significance". In Clinical Biostatistics, C.V. Moseby Co., St. Louis,
pp 320-334, 1977
13. Fleiss JL: Determining Sample Sizes Needed to Detect the Difference Between Two Proportions, pp. 22-34. Tables A.3, Sample
Sizes Per Group for Two-Tailed Tests on Proportions, pp. 176—
194. In Statistical Methods for Rates and Proportions, John Wiley
and Sons, New York, 1973
14. Freiman JA, Chalmers TC, Smith H, Keubler RR: The Importance
of Beta, the Type II Error and Sample Size in the Design and
Interpretation of the Randomized Control Trial Survey of 71 "Negative" Trials. N Engl J Med 299: 690-694, 1978
15. Dyken ML: Antiplatlet Aggregating Agents in Transient Ischemic
Attacks and the Relationship of Risk Factors. In Prophylaxis of
Venous, Peripheral, Cardiac and Cerebral Vascular Diseases with
Acetylsalicylic Acid. Editors Breddin K, Loew D, Uberla K, Dorndorf W, Marx R. F.K. Schattauer Verlag Stuttgart — New York,
pp 141-148, 1980
Transient ischemic attacks and aspirin, stroke and death; negative studies and Type
II error.
M L Dyken
Downloaded from http://stroke.ahajournals.org/ by guest on June 16, 2017
Stroke. 1983;14:2-4
doi: 10.1161/01.STR.14.1.2
Stroke is published by the American Heart Association, 7272 Greenville Avenue, Dallas, TX 75231
Copyright © 1983 American Heart Association, Inc. All rights reserved.
Print ISSN: 0039-2499. Online ISSN: 1524-4628
The online version of this article, along with updated information and services, is located
on the World Wide Web at:
http://stroke.ahajournals.org/content/14/1/2.citation
Permissions: Requests for permissions to reproduce figures, tables, or portions of articles originally
published in Stroke can be obtained via RightsLink, a service of the Copyright Clearance Center, not
the Editorial Office. Once the online version of the published article for which permission is being
requested is located, click Request Permissions in the middle column of the Web page under Services.
Further information about this process is available in the Permissions and Rights Question and Answer
document.
Reprints: Information about reprints can be found online at:
http://www.lww.com/reprints
Subscriptions: Information about subscribing to Stroke is online at:
http://stroke.ahajournals.org//subscriptions/