Chapter 7
Sampling, Significance Levels, and
Hypothesis Testing
Three scientific traditions critical to
experimental research
Sampling
Significance levels
Hypothesis testing
Copyright c 2001 The McGraw-Hill Companies, Inc.
1
Population and Sample
Population – all units (people or things)
possessing the attributes and characteristics
of interest
Sample -- subset of a population
Sampling frame -- subset of units that have
a chance to become part of the sample
Researchers study the sample to make
generalizations back to the population
Copyright c 2001 The McGraw-Hill Companies, Inc.
2
Defining the Population
Choose the dimensions or characteristics
meaningful to the hypothesis or research
question
Must be at least one common characteristic
among all members of a population
Must develop procedure to ensure
representative sampling
Copyright c 2001 The McGraw-Hill Companies, Inc.
3
Addressing Generalizability
Extent to which conclusions developed from data
collected from sample can be extended to its
population
Sample is representative to the degree that all units
had same chance for being selected
Representative sampling eliminates selection bias
 Characteristics of population should appear to the same
degree in sample
Representativeness can only be assured through
random sampling
Copyright c 2001 The McGraw-Hill Companies, Inc.
4
Probability Sampling
The probability of any unit being included
in the sample is known and equal
When probability for selection is equal,
selection is random
Also known as random sampling
Sampling error will always occur
Copyright c 2001 The McGraw-Hill Companies, Inc.
5
Types of Probability Sampling
Simple random sampling
 Simplest and quickest
Systematic sampling
 If used on a randomly ordered frame, results in truly
random sample
Stratified random sampling
 Random sampling within all subgroups
Cluster sampling
 Random sampling within known clusters
Copyright c 2001 The McGraw-Hill Companies, Inc.
6
Nonprobability Sampling
Does not rely on random selection
Weakens sample-to-population
representativeness
Used when other techniques will not result
in an adequate or appropriate sample
Used when researchers desire participants
with special experiences or abilities –
including qualitative research
Copyright c 2001 The McGraw-Hill Companies, Inc.
7
Nonprobability Sampling
Techniques
Convenience sample
Volunteer sample
Inclusion/exclusion sample
Snowball sample
Network sample
Purposive sample
Quota sample
Copyright c 2001 The McGraw-Hill Companies, Inc.
8
Sample Size
Number of people/units for whom you need
to collect data
Determined prior to selecting sample
Less than the number you ask to participate
The larger the sample relative to the
population, the less error or bias
Copyright c 2001 The McGraw-Hill Companies, Inc.
9
Comparisons of Sample Size
to Population
Population
Size
Sample
Size
Population
Size
Sample
Size
100
80
1,000
278
200
132
5,000
357
500
217
50,000
384
Copyright c 2001 The McGraw-Hill Companies, Inc.
10
Significance Levels
The researcher sets the significance level, or
p, for each statistical test
The degree of error the researcher finds
acceptable in a statistical test
An estimate of what would happen if the
study were actually repeated many times
Generally .05 is accepted level
Copyright c 2001 The McGraw-Hill Companies, Inc.
11
Significance Levels
.05 significance level = 5 out of 100
findings that appear to be valid will be due
to chance
Also known as the alpha level or p
If p > .05, the finding is nonsignificant
If p is  .05, the finding is significant or
real
Copyright c 2001 The McGraw-Hill Companies, Inc.
12
Hypothesis Testing
Hypothesis states the expected relationship
or difference between two or more variables
Alternative hypothesis presented in report
Null is statistically tested
Act of decision making based on the
significance level
Decision based on comparison between p set
before study to p produced by statistical test
Copyright c 2001 The McGraw-Hill Companies, Inc.
13
Hypothesis Testing
Belief in the null hypothesis continues until
there is sufficient evidence to the contrary
If p for statistical test exceeds significance
level, null is retained (p > .05)
If p for statistical test is  .05 then
alternative hypothesis is accepted
Copyright c 2001 The McGraw-Hill Companies, Inc.
14
Error in Hypothesis Testing
Use level of
significance to
reject null
Use level of
significance to
retain the null
In reality, the null
hypothesis is true
In reality, the null
hypothesis is false
Type I error –
Null is rejected
even though it is
true
Decision 2 –
Null is retained
when it is true
Decision 1 –
Null is rejected
when it is false
Copyright c 2001 The McGraw-Hill Companies, Inc.
Type II error –
Null is retained
even though it is
false
15