7. Samples and
observational studies
The Practice of Statistics in the Life Sciences
Third Edition
© 2014 W.H. Freeman and Company
Objectives (PSLS Chapter 7)
Samples and observational studies

Observation versus experiment

Population versus sample

The role of randomness in sampling

The simple random sample (SRS)

Other probability samples

Sample surveys

Comparative observational studies
Observational versus experimental studies
Observational study: record data on individuals without attempting to
influence the responses.
In 1992, several major medical organizations said that women should take
hormones such as estrogen after menopause, because women who took
hormones seemed to reduce their risk of a heart attack by 35% to 50%.
Experimental study: Deliberately impose a treatment on individuals
and record their responses. Influential factors can be controlled.
By 2002, several studies concluded that hormone replacement does not reduce
the risk of heart attacks. These studies had assigned women to either hormone
replacement or to dummy pills. The assignment was done by a coin toss.
A 2013 Gallup study investigated how phrasing affects the opinions of Americans
regarding physician-assisted suicide. Telephone interviews were conducted with a
random sample of 1,535 national adults. Using random assignment, 719 heard the
question in Form A and 816 the one in Form B.
Form A: When a person has a disease that cannot be cured, do you think
doctors should be allowed by law to end the patient’s life by some painless
means if the patient and his or her family request it?
Form B: When a person has a disease that cannot be cured and is living in
severe pain, do you think doctors should or should not be allowed by law to
assist the patient to commit suicide if the patient requests it?
70% of those given Form A answered “should be allowed”, compared with only 51%
of those given Form B. What type of study is this?
A. Observational study.
B. Randomized experiment.
C. Neither. This is just anecdotal evidence.
Confounding
Two variables are confounded when their effects on a response
variable cannot be distinguished.
Observational studies often fail to yield clear causal conclusions,
because the explanatory variable is confounded with lurking variables.
CONFOUNDING?
Population versus sample

Population: The entire group
of individuals in which we are
interested but can’t usually
assess directly

Sample: The part of the
population we actually examine
and for which we do have data
Population
Sample

A parameter is a number
summarizing a characteristic
of the population.

A statistic is a number
summarizing a characteristic of
a sample.
The role of randomness in sampling
How do you select the individuals/units in a sample?

Voluntary response sampling: individuals choose to be involved

Convenience sampling: ask whoever is around (mall, street) or take
the next 10 units

Probability sampling: individuals or units are randomly selected;
the sampling process is unbiased
Ann Landers summarizing responses of readers: 70% of
(~10,000) parents wrote in to say that having kids was not
worth it—if they had to do it over again, they wouldn’t.
But a random sample showed that 91% of parents WOULD have kids again.
What do you think explains such drastically different responses?
Would you expect very different responses on the
potential legalizing of marijuana if you asked the first
people you saw on the parking lot of a university or the
first people you saw on the parking lot of a church?
The simple random sample
A Simple Random Sample (SRS) is made of randomly selected
individuals. Each individual in the population has the same probability of
being in the sample. All possible samples of size n have the same
chance of being drawn.
How to choose an SRS?

Draw from a hat (lottery style)

Flip a coin

Use a table of published random numbers (Table A)

Use software that generates random numbers
Choosing a simple random sample with Table A
We need to select a random sample of 5 from a class of 20 students.
1) List and number all members of the population, which is the class of 20.
2) The number 20 is two digits long.
3) Parse the list of random digits into numbers that are two digits long. Here
we chose to start with line 103, for no particular reason.
45 46 71 17 09 77 55 80 00 95 32 86 32 94 85 82 22 69 00 56
45 46 71
52 71
17 09
13
77 55 80 00 95 32 86 32 94 85 82 22 69 00 56
88 89 93
07
46
02 …
4) Choose a random sample of size 5 by reading through the
list of two-digit numbers, starting with line 103 and on.
5) The first five random numbers matching numbers assigned
to people make the SRS.
The first individual selected is Ramon, number 17. Then
Henry (09). That’s all we can get from line 103.
We then move on to line 104. The next three to be
selected are Moe, George, and Amy (13, 7, and 02).
• Remember that 1 is 01, 2 is 02, etc.
• If you were to hit 17 again before getting five people, don’t
sample Ramon twice—you just keep going.
01 Alison
02 Amy
03 Brigitte
04 Darwin
05 Emily
06 Fernando
07 George
08 Harry
09 Henry
10 John
11 Kate
12 Max
13 Moe
14 Nancy
15 Ned
16 Paul
17 Ramon
18 Rupert
19 Tom
20 Victoria
Other probability samples
A stratified random sample: make sure your sample has x,y,z% of
individuals of certain types

America's State of Mind report was based on a probability
sample of Medco's de-identified database of members with 24
months of continuous insurance enrollment. Sampling was stratified by age
group and sex to match the demographics of the whole customer base.
A multistage sample: select your final sample in stages, by
sampling within a sample within a sample

The National Youth Tobacco Survey administered in schools uses a sampling
procedure to generate a nationally representative sample of students in grades
6–12. Sampling is probabilistic and consists of selecting:
1) Counties as Primary Sampling Units (PSU).
2) Schools within each selected PSU.
3) Classes within each selected school.
Sample surveys
A sample survey is an observational study that relies on a random
sample drawn from the entire population.

Opinion polls are sample surveys that typically use voter registries or
telephone numbers to select their samples.

In epidemiology, sample surveys are used to establish the incidence
(rate of new cases per year) and the prevalence (rate of all cases at
one point in time) of various medical conditions, diseases, and lifestyles.
These are typically stratified or multistage samples.
Some survey challenges

Undercoverage: Parts of the population are systematically left out.

Nonresponse: Some people choose not to answer/participate.

Wording effects: Biased or leading questions, complicated/
confusing statements can influence survey results.

Response bias: Fancy term for lying or forgetting (especially on
sensitive/personal issues). Can be exacerbated by survey method (in
person vs. by phone or online).
How bad is nonresponse?


The Census Bureau’s American Community Survey (ACS): ~ 2.5%
Via mail with reminders. Response is mandatory.
University of Chicago’s General Social Survey (GSS): ~ 30% - In person.

Pew Research Center methodology survey
up to ~ 90% in 2012

Private polling firms such as SurveyUSA:
~ 90% as of 2002 (stopped showing after that)
Phone (with interviewer or automated call)
or online.
1995-2002
A 2013 Gallup study investigated how phrasing affects the opinions of Americans
regarding physician-assisted suicide. Telephone interviews were conducted with a
random sample of 1,535 national adults. Using random assignment, 719 heard the
question in Form A and 816 the one in Form B.
Form A: When a person has a disease that cannot be cured, do you think
doctors should be allowed by law to end the patient’s life by some painless
means if the patient and his or her family request it?
Form B: When a person has a disease that cannot be cured and is living in
severe pain, do you think doctors should or should not be allowed by law to
assist the patient to commit suicide if the patient requests it?
Question wording resulted in a substantial difference in opinions: 70% of those
given Form A answered “should be allowed”, compared with only 51% of those
given Form B.
Some examples of
possible response bias
Comparative observational studies

Case-control studies start with 2 random samples of individuals with
different outcomes, and look for exposure factors in the subjects’ past
(“retrospective”).
Individuals with the condition are cases, and those without are controls.
Good for studying rare conditions. Selecting controls is challenging.

Cohort studies enlist individuals of common demographic, and keep
track of them over a long period of time (“prospective”). Individuals who
later develop a condition are compared with those who don’t.
Cohort studies examine the compounded effect of factors over time.
Good for studying common conditions. Very expansive.
Aflatoxicosis epidemics
Aflatoxins are secreted by a fungus found in damaged
crops and can cause severe poisoning and death.
The Kenya Ministry of Health investigated a 2004 outbreak of aflatoxicosis resulting
in over 300 cases of liver failure. A sample of 40 case-patients and 80 healthy
controls were asked how they had stored and prepared their maize.
The case-patients were randomly selected from a list of individuals admitted to a
hospital during the 2004 outbreak for unexplained acute jaundice.
Control individuals were selected to be as similar to the case-patients as possible,
yet randomly selected.
 Preliminary data suggested that soil, microclimate, and farming practices
might have played a role, but not age or gender.
 For each case-patient, two individuals from the patient’s village with no
history of jaundice symptoms were randomly selected.
The Nurses’ Health Study is one of the largest prospective
observational studies designed to examine factors that may
affect major chronic diseases in women.
Since 1976, the study has followed a cohort of over 100,000 registered nurses.
Every two years, they receive a follow-up questionnaire about diseases and
health-related topics. Response rate: ~ 90% each time.
2007 report on age-related memory loss:
About 20,000 women ages 70+ had completed telephone interviews every two
years to assess their memory with a set of cognitive tests. One of the findings:
the more women walked during their late 50s and 60s, the better their memory
score was at age 70 and older.
However, we cannot unambiguously conclude that walking has a protective
effect against memory loss.