Introducing Bias? An Evaluation of a Referendum Ballot
Results
Abstract
This poster evaluates a referendum on congestion charges in the
municipality of Gothenburg, Sweden, taking place in September 2014.
The politicians in Gothenburg have decided to use an unbalanced
question wording on the ballot, presenting an introductory text with
their own justification of the congestion charges, as well as a question
wording asking whether the municipality should “continue with the
congestion charges”. Previous research has been in disagreement on
the effects of balancing survey questions. Whilst Schuman & Presser
(1981) and Narayan & Krosnick (1996) argue that balancing questions
change the distribution of the answers, Shaeffer et al. (2005) argue that
minimally balanced questions do not change this distribution. Perhaps
the politicians sway the opinion by not providing a balanced or
minimally balanced question wording.
Experimental Design
Stimuli Difference by Ability
The results show, in contrast to what survey methodology would
suggest, that an unbalanced positive introduction text did not
affect the number of positive respondents.
Using a 2x2 full factorial experimental design on a probability sample of
Gothenburg residents, comparing the ballot with or without the
introductory text as well differences in wording (continue vs. abolish the
congestion charges), this study evaluates the ballot text in the referendum.
Further, changing the question wording to a negative word
(abolish) instead of a positive (continue) did not introduce bias in
the form of acquiescence.
Experiment group 1 and 3: Full introduction (“continue” and “ be abolished” stimuli
not bolded in actual experiment)
The City of Gothenburg has together with the Swedish Government, the Region
Gothenburg, the Region Halland, and the Region Västra Götaland agreed on the
West Swedish Package.
However, when comparing only low ability respondents, getting
the No Intro + Abolish stimuli compared to the Intro + Abolish
stimuli showed a significant effect. However, compared to the real
ballot text (Intro + Continue), none of these groups were
significantly different.
The congestion charge that was introduced in Gothenburg 2013 is a part of that
agreement and serves three different purposes:
Lower the congestion of cars, improve the environment, and jointly finance the
West Swedish Package. The package includes, amongst other things, a new
Götaälv bridge, a commuter train tunnel, a new river connection, and
investments in public transportation.
The local council has, after a people’s initiative decided to hold an advisory
referendum on the question.
Do you think that the congestion charges should [continue/be abolished] in
Gothenburg after the election of 2014?
Yes
No
s
Experiment group 2 and 4: No introduction (“continue” and “be abolished” stimuli
not bolded in actual experiment)
Do you think that the congestion charges should [continue/be abolished] in
Gothenburg after the election of 2014?
Yes
No
s
Conclusion
Stimuli Difference by Preference
Hence, we conclude, that when it comes to hot topics as the
referendum on congestion charges in Gothenburg, the survey
methodology usual suspects (unbalanced questions and
acquiescence) that introduce satisficing behavior does not seem
to bias the distribution of answers.