Chapter 14
Questionnaires,
Survey Research,
and Sampling
@ 2012 Wadsworth, Cengage Learning
Topics
1.
2.
3.
4.
5.
Question Construction and Formats
Methods of Administering a Survey
Sampling
Sample Size
Reliability and Validity
@ 2012 Wadsworth, Cengage Learning
Question Construction
and Formats
@ 2012 Wadsworth, Cengage Learning
Types of Questions
• Open-ended question: has no fixed answer
but allows the respondent to answer in any
manner
• Advantages:
– Does not impose the researcher’s point of view on
the respondent
– Person being interviewed may give you
information that you had not considered
– Offer an important way to begin a survey
@ 2012 Wadsworth, Cengage Learning
Types of Questions (cont’d.)
• Limitation: difficult to know how to translate
the responses into categories that fit the
original research hypothesis
• Fixed-alternative question: limits the number
of responses that a participant can make
• Example:
@ 2012 Wadsworth, Cengage Learning
Types of Questions (cont’d.)
• Funneling
– When a researcher begins a line of questioning
with an open-ended question and then follows
this with more specific items
• Choose responses that fit with the category of
the question being asked
– If the question has to do with frequency
– Then appropriate answers might be “never,”
“rarely,” “sometimes,” “often,” and “very often”
@ 2012 Wadsworth, Cengage Learning
Types of Questions (cont’d.)
• Likert-type item example:
• Semantic differential example:
@ 2012 Wadsworth, Cengage Learning
General Considerations
• Be as clear as possible
• Avoid unwanted ambiguity
• Double-barreled questions
– Combine two separate questions into one
– Difficult to code
• Negative and double negative questions
– Take longer to process than positive questions
– More prone to misinterpretation
@ 2012 Wadsworth, Cengage Learning
General Considerations (cont’d.)
• Consider whether you are setting up an
overall response set
• Ask only what you need to know
• Consider the order in which the questions are
asked
• Consider what you are asking:
– From your own perspective
– From the perspective of those responding
@ 2012 Wadsworth, Cengage Learning
General Considerations (cont’d.)
• Create a context in which the person can
answer the question
• Avoid leading in developing questions
• Pretest the survey
@ 2012 Wadsworth, Cengage Learning
Special Techniques for
Reactive Questions
• Random response method (Warner, 1965)
– Give each of a large group a survey and a coin and
– Tell them to answer:
• Yes when the coin lands on heads
• No when the coin lands on tails
• How do we determine the honest response?
– Subtract the expected probability from the
percentage of yes responses and
– Divide by the expected probability
@ 2012 Wadsworth, Cengage Learning
Methods of
Administering a Survey
@ 2012 Wadsworth, Cengage Learning
Face-to-Face Interviews
• Advantage:
– Achieve a better idea of the psychological set from
which the person is answering the question
• Disadvantages:
– Expensive
– Interviewer can bias the results
– Respondent could give different answers face-toface than with a written questionnaire
@ 2012 Wadsworth, Cengage Learning
Telephone Interviews
•
•
•
•
Establish rapport
Determine motivation
Clarify questions and responses
More cost-effective than a face-to-face
interviewer
• To increase response rates:
– Begin with human contact and then switch to a
computer-directed one
@ 2012 Wadsworth, Cengage Learning
Mail Questionnaires
• Advantage: can be sent almost anywhere for
the cost of postage
• Disadvantage: may not always be returned
• Return rate
– Very low in comparison to the completion rate of
face-to-face interviews
– 50% should be considered acceptable
@ 2012 Wadsworth, Cengage Learning
Computer Questionnaires
• Some of the same problems as mail surveys
• Advantages:
– Computer format is more enjoyable and faster
than a paper survey
– Allows for easier branching
@ 2012 Wadsworth, Cengage Learning
Internet Research
• Almost unlimited access to individuals from
around the world
• Targets individuals with particular interests
• Problems:
– Not knowing exactly who completed your survey
and under what conditions (Nosek & Banaji, 2002)
– Not everyone has access to the Internet
@ 2012 Wadsworth, Cengage Learning
Surveys: Conclusion
• Trade-off between response rate and cost
• If it is important to have a high rate of return
on your surveys
– Face-to-face or telephone interviews
• When cost is an important factor and you
need a large number of responses
– Mail or Internet survey
• Consider why you are doing the research
@ 2012 Wadsworth, Cengage Learning
Sampling
@ 2012 Wadsworth, Cengage Learning
Sampling
• Basic idea: learn about the characteristics of a
population by studying a sample
• Define the population
• The more the sample reflects the larger
population
– The more confidence we have in any inference we
make from our smaller sample to the larger
population
@ 2012 Wadsworth, Cengage Learning
Probability Sampling
• Simple random sampling: each member has
an equal chance of being selected
• Systematic sampling:
– List the total population
– Then choose every nth person listed
• Stratified random sampling: ensures that
certain people are represented systematically
in our sample
@ 2012 Wadsworth, Cengage Learning
Probability Sampling (cont’d.)
• Cluster sampling:
– Randomly select a certain number of population
units and then
– Enlist the participation of the people who live in
those units
• Multistage sampling: relies on sampling at
different stages in the process
@ 2012 Wadsworth, Cengage Learning
Nonprobability Sampling
• Convenience sampling: use the people who
are available to us
• Quota sampling: sets up a number of specific
types of people
• Snowball sampling: used most often when no
list of the population exists
@ 2012 Wadsworth, Cengage Learning
Sample Size
@ 2012 Wadsworth, Cengage Learning
Sample Size
• Determine the size of the sample by asking
two questions:
– How many people are available to be used in the
survey, and how homogeneous are these people?
– How much difference can we tolerate between
the characteristics found in the sample and those
of the overall population?
@ 2012 Wadsworth, Cengage Learning
Sample Size (cont’d.)
• Amount of sampling error that is acceptable
– Related to the original purpose of the survey
• General formula for the sample size:
@ 2012 Wadsworth, Cengage Learning
Reliability and Validity
@ 2012 Wadsworth, Cengage Learning
Reliability and Validity
• A survey is reliable if:
– It can be given a number of times and produce the
same results
• To increase reliability, consider:
– The wording of questions
– Pretesting the survey
@ 2012 Wadsworth, Cengage Learning
Reliability and Validity (cont’d.)
• The survey should accurately reflect what is
being examined
• Validity can be increased through:
– Sampling procedures
– Techniques to ensure increased response rates
@ 2012 Wadsworth, Cengage Learning
Summary
• In developing questions for a survey, it is
important to be clear
• To ensure that the results from a single group
can be applied to other groups, sampling
techniques are used
• Sample size required is related to confidence
level, variation in the population, and the
desired precision (amount of error)
@ 2012 Wadsworth, Cengage Learning