Chapter 3:
Producing Data
(Part 2)
Dr. Nahid Sultana
Chapter 3: Producing Data
Introduction
3.1 Design of Experiments
3.2 Sampling Design
3.3 Toward Statistical Inference
3.4 Ethics
3.2 Sampling Design
 Sample survey and Sample Design
 Voluntary Response Sample
 Simple Random Sample
 Stratified Samples
 Undercoverage and Nonresponse
3
Sample survey and Sample design
 Sample surveys are an important kind of observational study.
 The sample is the part from which we draw conclusions about the
whole population.
 The design of a sample survey refers to the method used to
choose the sample from the population.
 Poor sample designs can produce misleading conclusions.
 In reporting the results of a sample survey it is important to
include all details regarding the procedures used.
 The proportion of the original sample who actually provide usable
data is called response rate.
The response rate should be reported for all surveys.

4
Sample survey and Sample design (Cont..)
Example: The National Data Evaluation Center (NDEC) Web site
says that there are 13,823 RR (Reading Recovery) teachers. The
researchers send a questionnaire to a random sample of 200 of these.
The population consists of all 13,823 RR teachers, and the sample is
the 200 that were randomly selected.
If only 150 of the teachers who were sent questionnaires
provided usable data, then response rate = ?
The response rate would be 150/200, or 75%.
Sample survey and Sample design (Cont..)
The design of a sample is biased if it systematically favors certain
outcomes.
Convenience sampling: Choosing individuals who are
easiest to reach . (Just ask whoever is around)
Example: “Man on the street” survey (now very
popular with TV “journalism”))
Which men, and on which street?
 Ask about gun control or legalizing drug “on the street” in Berkeley
or in some small town in Idaho and you would probably get totally
different answers.
 Even within an area, answers would probably differ if you did the
survey outside a high school or a country western bar.
6
Sample survey and Sample design (Cont..)
A voluntary response sample consists of people who choose
themselves by responding to a general appeal.
 Voluntary response samples show bias because people with
strong opinions, especially negative opinions, are most likely to
respond.
For example, an email survey to 100 persons were sent out on a
certain topic.
 Chances are only those who are strongly for or against will
reply. Others who don't bother to reply will offer no comments which will tend to distort the accuracy of the survey.
Simple Random Samples
Random sampling, the use of chance to select a sample, is the
central principle of statistical sampling.
A simple random sample (SRS) ) is made of randomly selected
individuals.
 Each individual in the population has the same probability of being
in the sample.
All possible samples of size n have the same chance of being
drawn.
The simplest way to use chance to select a sample is to place names
in a hat (the population) and draw out a handful (the sample).
In practice, we use random numbers generated by a computer or
calculator to choose samples. If you don’t have technology handy,
you can use a table of random digits.
8
Other Sampling Designs
The basic idea of sampling is straightforward:
 take an SRS from the population and
 use your sample results to gain information about the population.
Stratified samples: slightly more complex form of random sampling
A stratified random sample is essentially a series of SRSs performed
on subgroups of a given population.
 The subgroups are chosen to contain all the individuals with a
certain characteristic. (called Strata)
For example:
 Divide the population of UD students into males and females.
 Divide the population of Dammam by major ethnic group.
 Divide the cities in KSA as either urban or rural based on criteria
of population density.
9
Stratified samples (Cont…)
The SRS taken within each group in a stratified random sample need
not be of the same size.
For example:
 A stratified random sample of 100 male and 150 female UD
students.
A stratified design can produce more exact information than an SRS
of the same size by taking advantage of the fact that individuals in the
same stratum are similar to one another.
Strata for sampling are similar to blocks in experiments.
10
Cautions About Sample Surveys
Bias: Tendency to systematically favors certain outcomes over
others.
Sources of bias:
 Under-coverage
 Non-response
 Response
 Question Wording
11
Cautions About Sample Surveys (Cont…)
Bias due to Under-coverage
Occurs because some groups in the population are left out the
sample is chosen.
Example: A survey of households excludes:
Homeless who can’t be found.
People who have extremely busy lives.
Subjects who are in hospitals, nursing homes, motels etc.
Under-coverage is often a problem with convenience samples.
12
Cautions About Sample Surveys (Cont…)
Bias due to Non-response
Occurs when an individual chosen in the sample refuses to provide
answers or can’t be contacted .
Example:
If you mailed out a survey to 100 people and only 80 answered,
those 20 that didn't respond are not being reported for and are
causing a non response bias.
Bias due to Response
A systematic pattern of incorrect responses in a sample survey leads
to response bias.
This is particularly important when the questions are very personal
(e.g., “How much do you drink?”) or related to the past.
13
Cautions About Sample Surveys (Cont…)
Bias due to wording of questions
The wording of questions is the most important influence on the
answers given to a sample survey.
Confusing or leading questions can introduce strong bias, and even
minor changes in wording can change a survey’s outcome.
Example:
How do Americans feel about government help for the poor?
Only 13% think we are spending too much on “assistance to the poor,” but
44% think we are spending too much on “welfare.”
How do the Scots feel about the movement to become independent from
England?
Well, 51% would vote for “independence for Scotland,” but only 34%
support “an independent Scotland separate from the United Kingdom.”
It seems that “assistance to the poor” and “independence” are nice, hopeful
words.
“Welfare” and “separate” are negative words.
14