5.4 Creating Survey Questions
Surveys
• Very common for data collection
• Types:
–
–
–
–
Face-to-face interviews
Telephone surveys
Internet surveys
Mail-in surveys
• All have questions to answer
Creating Survey Questions
• Difficult!
• Must pilot/test draft questionnaire
– with people who will not form part of survey
sample
• Primary data:
– Collected by researcher
• Secondary data:
– Collected from other sources (e.g. internet)
Creating Survey Questions
• Should be
–
–
–
–
Relevant to the study
Readable (use familiar words)
Specific
Simple
Things to Avoid
• Jargon (slang)
• Abbreviations or acronyms
– Abbreviation, not abbrev.
– Jarvis Collegiate Institute, not J.C.I.
• Negatives
– “Do you think we should we do this”, not “Do
you think we should not do this?”
Things to Avoid
• Leading respondents
– “Should the office continue its heinous practice of
forcing students to come to detentions?”
– “Given the amount of CO2 they produce, rate your
opinion of SUVs on a scale from 1 to 10.”
• Double-barrelled questions (asking two things at
once)
– “Do you walk or take the bus?”
• Insensitivity: belittling or insulting
Question Styles
• Open questions
– Respondents answer in their own words
– E.g.: “What changes should be made in the school to
ensure better student achievement?”
• Closed questions
– Respondents given limited number of responses to
chose from
– Response categories should be
• Mutually exclusive: should not be able to choose more than
one unless you can “choose as many as apply”
• Exhaustive: all possibilities listed
Open vs. Closed
• Open Pros
– Respondent can interpret
and answer question any
way they choose
• Open Cons:
– Not necessarily consistent
– Difficult to
analyse/interpret answers
– Wide variety of
possibilities
• Closed Pros
– Faster and easier to
answer
– Faster and easier to code
and analyse
– Provides consistency
• Closed Cons
– Restricts answers to
choices given
– Options may bias choice
Ex: Inconsistent Answers
Difficulties with open questions:
“How much orange juice do you think this bottle
contains?”
• One orange & a little
water and sugar
• 25 % orange juice, 75%
carbonated water
• Juice of ½ dozen oranges
• 3 oz
• Full strength
• ¼ c. orange juice
•
•
•
•
•
•
None
Not much
Don’t know
A pint
Most of it
About 1 ½ glass
Ex: Inconsistent Answers
For a uniform response, sometimes you need to state what
kind of response is needed.
millilitres
“This bottle contains 250 mL orange drink. How many
mL of this drink would you say are orange juice?”
millilitres
Question Styles
• 4 main types:
–
–
–
–
Information
Checklist
Ranking
Rating
Information Questions
• Solicit information
• Usually fill-in-the-blank & open
What kind of car do you own?
Model _________________
Make __________________
Year _____
Checklist Questions
• Closed
• Exhaustive
What level of license do you hold?
 G2
□ G1
□G
Note that these are mutually
exclusive response categories.
□ None
Another Checklist Question
Which of the following brand names is your
one)
all that
apply)
favourite? (Check only
 Honda □ Toyota □ Nissan □ GM
 Other (please state): ________________
 None of the above/I don’t have a favourite
Problems?
• Unclear if should only choose one
• What if your favourite is not up there or you
don’t have a favourite (not exhaustive)
Ranking Questions
• Orders preferences
Rank the following car concerns in order of
importance from 1 to 4: (1 is low, 4 is high)
____ price
____ appearance
____ maintenance
____ gas
costs
consumption
Is 1 high or low?
Rating Questions
How satisfied are you with your car?
 Very satisfied  Satisfied
 Dissatisfied
 Very Dissatisfied
• Can also assign a numerical value
On a scale from 0 to 10, 0 being very
dissatisfied and 10 being very satisfied, how
satisfied are you with your car? _____
• Could use a line: 0 1 2 3 4 5 6 7 8 9 10
Creating a Questionnaire
• Introduction
–
–
–
–
Informative
Stimulates respondent interest
Should answer: “Why should I answer this?”
State what will be done with the results
• Begin with easy and interesting questions
• Questions should flow and read well
Testing a Questionnaire
• Fantastically important! Do not skip!
• Identifies problems in survey
• Discovers poor wording/ordering
– People might be unwilling/unable to answer a
question
• Identifies errors in layout and instructions
Testing a Questionnaire
• Suggests additional response categories that can
be pre-coded
– If 10 people suggest that Ford should be added to
favourite car…
• Provides preliminary idea of length of interview
• Provides preliminary idea of refusal problems
– Should we adjust survey size to account for nonresponse?