CONFIDENCE
INTERVALS
DR.S.SHAFFI AHAMED
ASST. PROFESSOR
DEPT. OF F & CM
BACKGROUND AND NEED OF CI
---Statistical analysis of medical
studies is based on the key idea
that, we make observations on a
sample of subjects from which the
sample is drawn.
---If the sample is not representative
of the population we may well be
misled and statistical procedures
cannot help.
.
---Even a well designed study can
give only an idea of the answer
sought , because of random
variation in the sample.
---And the results from a single
sample are subject to statistical
uncertainty, which is strongly
related to the size of the sample.
-- The quantities( single mean,
proportion, difference in means,
proportions, OR, RR, Correlation,----------) will be imprecise estimate
of the values in the overall
population, but fortunately the
imprecision can itself be estimated
and incorporated into the
presentation of findings.
--- Presenting study findings directly on the
scale of original measurement together
with information on the inherent
imprecision due to sampling variability,
has distinct advantages over just giving ‘pvalues’ usually dichotomized into
“significant” or “non-significant”.
“THIS IS THE RATIONALE FOR
USING CI”
Confidence Intervals for Reporting
Results
 “[Hypothesis tests] are sometimes overused and
their results misinterpreted.”
 “Confidence intervals are of more than
philosophical interest, because their broader use
would help eliminate misinterpretations of
published results.”
 “Frequently, a significance level or pvalue is
reduced to a ‘significance test’ by saying that if the
level is greater than 0.05, then the difference is ‘not
significant’ and the null hypothesis is ‘not
rejected’….The distinction between statistical
significance and clinical significance should not be
confused.”
But why do we always see
95% CI’s?
 “Duality” between confidence intervals and
pvalues
 Example: Assume that we are testing that for a
significant change in QOL due to an intervention,
where QOL is measured on a scale from 0 to 50.
 95% confidence interval: (-2, 13)
 pvalue = 0.07
 It is true that if the 95% confidence interval overlaps
0, then a t-test testing that the treatment effect is 0
will be insignificant at the alpha = 0.05 level.
 It is true that if the 95% confidence interval does not
overlap 0, then a t-test testing that the treatment
effect is 0 will be significant at the alpha = 0.05
level.
-THE BRITISH MEDICAL JOURNAL
-THE LANCET
-THE MEDICAL JOURNAL OF
AUSTRALIA
-THE AMERICAN JOURNAL OF PUBLIC
HEALTH
-THE BRITISH HEART JOURNAL
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Different Interpretations of the
95% confidence interval
 “We are 95% sure that the TRUE
parameter value is in the 95% confidence
interval”
 “If we repeated the experiment many many
times, 95% of the time the TRUE
parameter value would be in the interval”
 “Before performing the experiment, the
probability that the interval would contain
the true parameter value was 0.95.”
-- In a representative sample of 100 observations
of heights of men, drawn at random from a large
population, suppose the sample mean is found
to be 175 cm (sd=10cm) .
-- Can we make any statements about the
population mean ?
-- We cannot say that population mean is 175 cm
because we are uncertain as to how much
sampling fluctuation has occurred.
-- What we do instead is to determine a range of
possible values for the population mean, with
95% degree of confidence.
-- This range is called the 95% confidence interval
and can be an important adjuvant to a
significance test.
 In general, the 95% confidence interval is given
by:
Statistic ± confidence factor x S.Error of statistic
In the previous example, n =100 ,sample mean =
175, S.D., =10, and the S.Error =10/√100 = 1.
Therefore, the 95% confidence interval is,
175 ± 1.96 * 1 = 173 to 177”
That is, if numerous random sample of size 100 are
drawn and the 95% confidence interval is
computed for each sample, the population mean
will be within the computed intervals in 95% of
the instances.
Sampling Distributions
0 5 10 15
sem = 0.47
=
25
n
=
0 5 10 15 20 25
n
50
sem = 0.23
2.
0
2.
5
3.
0
3.
5
4.
0
2.
0
2.
5
3.
0
3.
5
4.
0
s a mp s
=
100
n
0 5 10 15 20
sem = 0.17
0 10 20 30
n
s a mp s
=
500
sem = 0.10
2.
0
2.
5
3.
0
3.
5
4.
0
2.
0
2.
5
3.
0
3.
5
4.
0
s a mp s
s a mp s
General formula for 95% confidence
interval for single mean
x  196
. sem
 Notes:
s
x  196
.
n
 sample size must be sufficiently large for
non-normal variables.
 how large is large? depends on
skewness of variable
 VERY often people use 2 instead of 1.96.
Example: A study was carried out to determine the
effect, if any, of pesticide exposure on blood
pressure. A random sample of 100 men was
selected from a group of agriculture workers
known to have been exposed to pesticides. 100
randomly selected workers with no such
exposure comprised the control group. Their
mean SBP and Sd., were given as 145 mm Hg, 20
mm hg (exposed group) and 120 mm hg, 15 mm
Hg (non exposed group). Calculate the 95% and
90% confidence intervals for the true difference
in mean SBP between the exposed and non
exposed populations .
95 % C.I. for Difference in
Means
2
1
2
2
s s
( x1  x2 )  196
.

n1 n2
The 90% confidence interval for the difference in
means :
25 ± 1.645 (2.5) = 20.9 to 29.1
The 95 % confidence interval for the difference in
means:
25 ± 1.96 (2.5) = 20.1 to 29.9
Thus we can be 90% (or 95%) certain that the true
mean difference in systolic blood pressure
between the exposed and non-exposed
populations lies between 21 and 29 mm Hg (20
and 20 mm Hg). Notice that the both the intervals
does not include zero so the data are not
compatible with no difference in mean systolic
blood pressure.
95% Confidence Intervals for
Proportions
 Socinski et al., Phase III Trial Comparing a Defined Duration
of Therapy versus Continuous Therapy Followed by
Second-Line Therapy in Advanced-Stage IIIB/IV Non-SmallCell Lung Cancer JCO, March 1, 2002.
 Patients and Methods: Arm A (4 cycles of carboplatin at an
AUC of 6 and paclitaxel), Arm B (continuous treatment with
carboplatin/ paclitaxel until progression). At progression,
patients from each arm receive second-line weekly
paclitaxel at 80mg/m2/week.
 Results: 230 Patients were randomized (114 in arm A and
116 in Arm B). Overall response rates were 22% and 24%
for arms A and B. Grade 2 to 4 neuropathy was seen in 14%
and 27% of Arm A and B patients, respectively.
95% Confidence Intervals for
Proportions
 What are 95% confidence intervals for the response rates
in the two arms?
 (1  p
)
p
n
 standard error of a sample proportion is
 An equation for confidence interval for a proportion:
p (1  p )
p  196
.
n
 Assumptions:
 n is reasonably large
 p is not “too” close to 0 or 1
 rule of thumb: pn > 5
Example: Response Rate to
Treatment
p  196
.
p (1  p )
n
 Arm A:
0.22  196
.
0.22( 0.78)
 ( 014
. ,0.30)
114
0.24  196
.
0.24( 0.76)
 ( 016
. ,0.32)
116
 Arm B:
Example: Grade 2 to 4
Neuropathy
p  196
.
p (1  p )
n
 Arm A:
014
.  196
.
014
. ( 0.86)
 ( 0.08,0.20)
114
0.27  196
.
0.27( 0.73)
 ( 019
. ,0.35)
116
 Arm B:
95% Confidence Interval for Difference in
Proportions
( p1  p 2 )  196
.
p1 (1  p1 ) p 2 (1  p 2 )

n1
n2
What is the 95% confidence interval for
the difference in rates of neuropathy in
arms A and B?
0.27( 0.73) 014
. ( 0.86)
( 0.27  014
. )  196
.

 ( 0.03,0.23)
116
114
APPLICATION
OF
CONFIDENCE
INTERVALS
Example: The following finding of
significance in a clinical
trial on 178 patients.
non-
Treatment Success
Failure
Total
A
76 (75%)
25
101
B
51(66%)
26
77
Total
127
51
178
Chi-square value = 1.74 ( p > 0.1)
(non –significant)
i.e. there is no difference in efficacy between the
two treatments.
--- The observed difference is:
75% - 66% = 9%
and the 95% confidence interval for the
difference is:
- 4% to 22%
-- This indicates that compared to treatment B,
treatment A has, at best an appreciable
advantage (22%) and at worst , a slight
disadvantage (- 4%).
--- This inference is more informative than just
saying that the difference is non significant.
Example:
Treatment Success
Failure
Total
A
49 (82%)
11
60
B
33 (60%)
22
55
Total
82
33
115
The chi-square value = 6.38
p = 0.01 (highly significant)
The observed difference in efficacy is
82 % - 60% = 22%
95% C .I. = 6 % to 38%
This indicates that changing from treatment
B to treatment A can result in 6% to 38%
more patients being cured.
Again, this is more informative than just
saying that the two treatments are
significantly different.
Example: Disease Control Program
Consider the following findings pertaining to
case-holding in the National TB control program
Completed
Treatment
Year
Program
1987
Routine 276
(46%)
324
600
1988
Special
288
600
312
(52%)
Failed to Total
Complete
The chi square = 4.32, p-value = 0.04 (significant)
The impact of special motivation program
= 52 % - 46% = 6% in terms of improved
case-holding.
The 95% C.I. = 0.4 % to 11.6%, which indicates
that the benefit from the special motivation
program is not likely to be more than 11.6%.
This information may helps the investigator to
conclude that the special program was not really
worthwhile, and that other strategies need to be
explored, to provide a greater magnitude of
benefit.
CHARACTERISTICS OF CI’S
--The (im) precision of the estimate is
indicated by the width of the
confidence interval.
--The wider the interval the less
precision
THE WIDTH OF C.I. DEPENDS ON:
---- SAMPLE SIZE
---- VAIRABILITY
---- DEGREE OF CONFIDENCE
EFFECT OF VARIABILITY
 Properties of error
1. Error increases with smaller sample size
For any confidence level, large samples reduce the margin
of error
2. Error increases with larger standard Deviation
As variation among the individuals in the population
increases, so does the error of our estimate
3. Error increases with larger z values
Tradeoff between confidence level and margin of error
Not only 95%….
 90% confidence interval:
NARROWER than 95%
x  165
. sem
 99% confidence interval:
WIDER than 95%
x  2.58sem
C.I. (degree of
confidence)
Z -value
90%
1.64
95%
1.96
98%
2.33
99%
2.58
Interval width (error) increases with
Increased confidence level
Higher confidence levels have
Higher z values
Figure 8-10 and 8-11
Error is high in small samples
Other Confidence Intervals
 Differences in means
 Response rates
 Differences in response rates
 Hazard ratios
 median survival
 difference in median survival
 OR, RR, Correlation, Regression
 Non – parametric tests
Recap
 95% confidence intervals are used to quantify
certainty about parameters of interest.
 Confidence intervals can be constructed for any
parameter of interest (we have just looked at
some common ones).
 The general formulas shown here rely on the
central limit theorem
 You can choose level of confidence (does not
have to be 95%).
 Confidence intervals are often preferable to
pvalues because they give a “reasonable range”
of values for a parameter.
Excellent References on Use of
Confidence Intervals in Clinical Trials
 Richard Simon, “Confidence Intervals for
Reporting Results of Clinical Trials”, Annals of
Internal Medicine, v.105, 1986, 429-435.
 Leonard Braitman, “Confidence Intervals Extract
Clinically Useful Information from the Data”,
Annals of Internal Medicine, v. 108, 1988, 296-298.
 Leonard Braitman, “Confidence Intervals Assess
Both Clinical and Statistical Significance”, Annals
of Internal Medicine, v. 114, 1991, 515-517.
THANK YOU