2/20/2015
Confidence Intervals
STAT 250
Dr. Kari Lock Morgan
Confidence Intervals:
Bootstrap Distribution
statistic ± ME
Sample
Sample
...
Sample
SECTIONS 3.3, 3.4
• Bootstrap distribution (3.3)
• 95% CI using standard error (3.3)
• Percentile method (3.4)
Statistics: Unlocking the Power of Data
Sample
Population
Sample
Sample
Margin of Error (ME)
(95% CI: ME = 2×SE)
Sampling Distribution
Calculate statistic
for each sample
Lock5
Reality
Standard Error (SE):
standard deviation of
sampling distribution
Statistics: Unlocking the Power of Data
Lock5
Your best guess for the population
is lots of copies of the sample!
One small problem…
… WE ONLY HAVE ONE SAMPLE!!!!
• How do we know how much sample
statistics vary, if we only have one
sample?!?
Sample repeatedly from
this “population”
BOOTSTRAP!
Statistics: Unlocking the Power of Data
Lock5
Sampling with Replacement
Statistics: Unlocking the Power of Data
Lock5
Suppose we have a random sample
of 6 people:
• It’s impossible to sample repeatedly from
the population…
• But we can sample repeatedly from the
sample!
• To get statistics that vary, sample with
replacement (each unit can be selected
more than once)
Statistics: Unlocking the Power of Data
Lock5
Statistics: Unlocking the Power of Data
Lock5
1
2/20/2015
Bootstrap Sample:
Sample with
replacement from the original sample, using
Remember: sample
the same sample size.
size matters!
Original
Sample
Original
Sample
A simulated “population” to sample from
Statistics: Unlocking the Power of Data
Lock5
Bootstrap Sample
Statistics: Unlocking the Power of Data
Reese’s Pieces
Lock5
Bootstrap Sample
• How would you take a bootstrap sample from
your sample of Reese’s Pieces?
Your original sample has data values
18, 19, 19, 20, 21
Is the following a possible bootstrap sample?
18, 19, 20, 21, 22
a) Yes
b) No
Statistics: Unlocking the Power of Data
Lock5
Statistics: Unlocking the Power of Data
Bootstrap Sample
Bootstrap Sample
Your original sample has data values
Your original sample has data values
18, 19, 19, 20, 21
18, 19, 19, 20, 21
Is the following a possible bootstrap sample?
Is the following a possible bootstrap sample?
18, 19, 20, 21
18, 18, 19, 20, 21
a) Yes
b) No
Statistics: Unlocking the Power of Data
Lock5
a) Yes
b) No
Lock5
Statistics: Unlocking the Power of Data
Lock5
2
2/20/2015
Original
Sample
Sample
Statistic
Bootstrap
Sample
Bootstrap
Statistic
Bootstrap
Sample
Bootstrap
Statistic
.
.
.
Bootstrap
Sample
.
.
.
Mercury Levels in Fish
• What is the average mercury level of fish
(large mouth bass) in Florida lakes?
Bootstrap
Distribution
•Key Question: How much can statistics vary
from sample to sample?
Bootstrap
Statistic
Statistics: Unlocking the Power of Data
• Sample of size n = 53, with 𝑥 = 0.527 ppm.
(FDA action level in the USA is 1 ppm, in
Canada the limit is 0.5 ppm)
Lock5
Lange, T., Royals, H. and Connor, L. (2004). Mercury accumulation in
largemouth bass (Micropterus salmoides) in a Florida Lake. Archives of
Environmental Contamination and Toxicology, 27(4), 466-471.
Statistics: Unlocking the Power of Data
Lock5
Bootstrap Sample
Mercury Levels in Fish
You have a sample of size n = 53. You sample with
replacement 1000 times to get 1000 bootstrap
samples.
What is the sample size of each bootstrap sample?
(a) 53
(b) 1000
Statistics: Unlocking the Power of Data
Lock5
Bootstrap Distribution
Statistics: Unlocking the Power of Data
Lock5
Why “bootstrap”?
You have a sample of size n = 53. You sample with
replacement 1000 times to get 1000 bootstrap
samples.
“Pull yourself up by your bootstraps”
How many bootstrap statistics will you have?
• Lift yourself in the air simply by pulling up on
the laces of your boots
(a) 53
(b) 1000
Statistics: Unlocking the Power of Data
• Metaphor for accomplishing an “impossible”
task without any outside help
Lock5
Statistics: Unlocking the Power of Data
Lock5
3
2/20/2015
Sampling Distribution
BUT, in practice we
don’t see the “tree” or
all of the “seeds” – we
only have ONE seed
Bootstrap Distribution
What can we
do with just
one seed?
Population
Bootstrap
“Population”
µ
Statistics: Unlocking the Power of Data
µ
𝑥
Lock5
Estimate the
distribution
and variability
(SE) of 𝑥’s
from the
bootstraps
Statistics: Unlocking the Power of Data
Golden Rule of Bootstrapping
Lock5
Center
•The sampling distribution is centered around the
population parameter
Bootstrap statistics are to the
original sample statistic
as
the original sample statistic is to
the population parameter
• The bootstrap distribution is centered around
the
a) population parameter
b) sample statistic
c) bootstrap statistic
d) bootstrap parameter
•Luckily, we don’t care about the center… we care
about the variability!
Statistics: Unlocking the Power of Data
Lock5
Statistics: Unlocking the Power of Data
Standard Error
Confidence Intervals
• The variability of the bootstrap statistics
is similar to the variability of the sample
statistics
• The standard error of a statistic can be
estimated using the standard deviation of
the bootstrap distribution!
Statistics: Unlocking the Power of Data
Lock5
Bootstrap
Sample
Sample
statistic ± ME
Bootstrap
Sample
Bootstrap
Sample
...
Bootstrap
Sample
Bootstrap
Sample
Margin of Error (ME)
(95% CI: ME = 2×SE)
Bootstrap Distribution
Calculate statistic
for each bootstrap
sample
Lock5
Statistics: Unlocking the Power of Data
Standard Error (SE):
standard deviation of
bootstrap distribution
Lock5
4
2/20/2015
Mercury Levels in Fish
Mercury Levels in Fish
• What is the average mercury level of fish
(large mouth bass) in Florida lakes?
SE = 0.047
• Sample of size n = 53, with 𝑥 = 0.527 ppm.
(FDA action level in the USA is 1 ppm, in
Canada the limit is 0.5 ppm)
0.527  2  0.047
(0.433, 0.621)
We are 95% confident
that average mercury
level in fish in Florida
lakes is between 0.433
and 0.621 ppm.
•Give a confidence interval for the true average.
Lange, T., Royals, H. and Connor, L. (2004). Mercury accumulation in
largemouth bass (Micropterus salmoides) in a Florida Lake. Archives of
Environmental Contamination and Toxicology, 27(4), 466-471.
Statistics: Unlocking the Power of Data
Lock5
Statistics: Unlocking the Power of Data
Same process for every parameter!
Estimate the standard error and/or a
confidence interval for...
• proportion (𝑝)
• difference in means (µ1 − µ2 )
• difference in proportions (𝑝1 − 𝑝2 )
• standard deviation (𝜎)
• correlation (𝜌)
• ...
Generate samples with replacement
Calculate sample statistic
Repeat...
Statistics: Unlocking the Power of Data
Lock5
Hitchhiker Snails
 A type of small snail is very widespread in Japan, and
colonies of the snails that are genetically very similar
have been found very far apart.
 How could the snails travel such long distances?
 Biologist Shinichiro Wada fed 174 live snails to birds,
and found that 26 were excreted live out the other end.
(The snails are apparently able to seal their shells shut
to keep the digestive fluids from getting in).
 What proportion of these snails ingested by birds
survive?
Yong, E. “The Scatological Hitchhiker Snail,” Discover,
October 2011, 13.
Lock5
Hitchhiker Snails
Give a 95% confidence interval for the proportion of
snails ingested by birds that survive.
Statistics: Unlocking the Power of Data
Lock5
Body Temperature
What is the average body temperature of humans?
www.lock5stat.com/statkey
a) (0.1, 0.2)
b) (0.05, 0.25)
c) (0.12, 0.18)
d) (0.07, 0.18)
Statistics: Unlocking the Power of Data
Lock5
Shoemaker, What's Normal: Temperature, Gender and Heartrate, Journal of Statistics
Education, Vol. 4, No. 2 (1996)
Statistics: Unlocking the Power of Data
Lock5
5
2/20/2015
Other Levels of Confidence
Percentile Method
• For a P% confidence interval, keep the middle
P% of bootstrap statistics
• What if we want to be more than 95%
confident?
• How might you produce a 99% confidence
interval for the average body temperature?
• For a 99% confidence interval, keep the
middle 99%, leaving 0.5% in each tail.
• The 99% confidence interval would be
(0.5th percentile, 99.5th percentile)
where the percentiles refer to the bootstrap
distribution.
Statistics: Unlocking the Power of Data
Lock5
Best
Guess at Sampling
Distribution
• For a P%
confidence
interval:
www.lock5stat.com/statkey
P%
2
3
Middle 99% of bootstrap statistics
Observed
Statistic
4
5
Lock5
Body Temperature
Bootstrap Distribution
Lower
Bound
Statistics: Unlocking the Power of Data
Upper
Bound
6
7
8
We are 99% sure that the average body temperature is between
98.00 and 98.58
Statistic
Statistics: Unlocking the Power of Data
Lock5
Statistics: Unlocking the Power of Data
Lock5
Mercury and pH in Lakes
Level of Confidence
• For Florida lakes, what is the correlation
between average mercury level (ppm) in fish
taken from a lake and acidity (pH) of the lake?
Which is wider, a 90% confidence interval or a
95% confidence interval?
𝑟 = −0.575
(a) 90% CI
(b) 95% CI
Give a 90%
CI for 
Lange, Royals, and Connor, Transactions of the American Fisheries
Society (1993)
Statistics: Unlocking the Power of Data
Lock5
Statistics: Unlocking the Power of Data
Lock5
6
2/20/2015
Mercury and pH in Lakes
Bootstrap CI
Option 1: Estimate the standard error of the
statistic by computing the standard deviation of
the bootstrap distribution, and then generate a
95% confidence interval by
www.lock5stat.com/statkey
statistic  2  SE
Option 2: Generate a P% confidence interval as
the range for the middle P% of bootstrap statistics
We are 90% confident that the true correlation between average
mercury level and pH of Florida lakes is between -0.702 and -0.433.
Statistics: Unlocking the Power of Data
Lock5
Statistics: Unlocking the Power of Data
Lock5
Bootstrap Cautions
Mercury Levels in Fish
• These methods for creating a confidence
interval only work if the bootstrap distribution is
smooth and symmetric
SE = 0.047
0.527  2  0.047
(0.433, 0.621)
• ALWAYS look at a plot of the bootstrap
distribution!
Middle 95% of bootstrap
statistics
Statistics: Unlocking the Power of Data
• If the bootstrap distribution is skewed or looks
“spiky” with gaps, you will need to go beyond
intro stat to create a confidence interval
Lock5
Bootstrap Cautions
Statistics: Unlocking the Power of Data
Statistics: Unlocking the Power of Data
Lock5
Bootstrap Cautions
Lock5
Statistics: Unlocking the Power of Data
Lock5
7
2/20/2015
Number of Bootstrap Samples
• When using bootstrapping, you may get a
slightly different confidence interval each time.
This is fine!
Summary
 The standard error of a statistic is the standard
deviation of the sample statistic, which can be
estimated from a bootstrap distribution
 Confidence intervals can be created using the standard
error or the percentiles of a bootstrap distribution
• The more bootstrap samples you use, the more
precise your answer will be.
 Increasing the number of bootstrap samples will not
• For the purposes of this class, 1000 bootstrap
samples is fine. In real life, you probably want to
take 10,000 or even 100,000 bootstrap samples
 Confidence intervals can be created this way for any
Statistics: Unlocking the Power of Data
Lock5
change the SE or interval (except for random
fluctuation)
parameter, as long as the bootstrap distribution is
approximately symmetric and continuous
Statistics: Unlocking the Power of Data
Lock5
To Do
 Read Sections 3.3, 3.4
 Do HW 3.3, 3.4 (due Friday, 2/7)
Statistics: Unlocking the Power of Data
Lock5
8