The Political Relevance of
Irrelevant Events
E THAN C.
BUSBY, J AMES
N. D RUCKMAN , A LEXANDRIA F REDENDALL
N ORTHWESTERN U NIVERSITY
N OVEMBER 21, 2015
Where we’re headed
INTRODUCTION
OUR STUDY
FINDINGS
CONCLUSIONS
Where we’re headed
◦
◦
◦
◦
Motivation
Study details
Findings
Conclusions
INTRODUCTION
OUR STUDY
FINDINGS
CONCLUSIONS
Irrelevant Event Effects
Do events disconnected from politics influence people’s political opinions?
INTRODUCTION
OUR STUDY
FINDINGS
CONCLUSIONS
Irrelevant Event Effects
Do events disconnected from politics influence people’s political opinions?
Event
Influences…
Source
Daily climate fluctuations
Beliefs about global warming
Zaval et al 2014
Shark attacks
Vote for Woodrow Wilson
Achen and Bartels 2002
Weather
Mood and tolerance for risky
candidates
Bassi 2013
Sporting wins/losses
Incumbents’ vote share
Healy, Malhotra, and Mo 2010
Random lottery win/loss
Preference for current system
(incumbent)
Huber, Hill, and Lenz 2012
Wins/losses in March Madness
Approval of President Obama
Healy, Malhotra, and Mo 2010
INTRODUCTION
OUR STUDY
FINDINGS
CONCLUSIONS
Irrelevant Event Effects
Bigger picture
INTRODUCTION
OUR STUDY
FINDINGS
CONCLUSIONS
Irrelevant Event Effects
Bigger picture
◦ Democracy and representation
◦ Most portrayals of representative democracy presume citizens base preferences over outcomes political actors can control
◦ Has implications for theories about representation including retrospective voting (Healy and Malhotra 2013) and
anticipatory representation (Mansbridge 2003)
INTRODUCTION
OUR STUDY
FINDINGS
CONCLUSIONS
Irrelevant Event Effects
Bigger picture
◦ Democracy and representation
◦ Most portrayals of representative democracy presume citizens base preferences over outcomes political actors can control
◦ Has implications for theories about representation including retrospective voting (Healy and Malhotra 2013) and
anticipatory representation (Mansbridge 2003)
◦ Public opinion in general
◦ Changes how interested researchers and policy makes think about these evaluations
◦ What happens if a hurricane, sporting event, or raid of especially vicious sharks occurs before a survey or an election?
INTRODUCTION
OUR STUDY
FINDINGS
CONCLUSIONS
Our study
INTRODUCTION
OUR STUDY
FINDINGS
CONCLUSIONS
Our study
THE EVENT: 2015 College Football Playoff National Championship game: The Ohio State University
(OSU) against the University of Oregon (UO). OSU won the game, 42 to 20
◦ We randomly sampled students from the online directories of both schools
◦ Within each school, students were randomly assigned to be surveyed before (pre-game group) or
after (post-game group) the actual game.
◦ Subjects took the study in a narrow time frame
◦ Subjects were recontacted a week after their original participation
INTRODUCTION
OUR STUDY
FINDINGS
CONCLUSIONS
Our study
THE EVENT: 2015 College Football Playoff National Championship game: The Ohio State University
(OSU) against the University of Oregon (UO). OSU won the game, 42 to 20
◦ The main outcome variables were approval for President Obama, evaluation of the economy, and
satisfaction with one’s university
◦ Also considered the decay of the effect of the game
◦ Measured one mechanism (mood) and social contagion
INTRODUCTION
OUR STUDY
FINDINGS
CONCLUSIONS
Expectations
INTRODUCTION
OUR STUDY
FINDINGS
CONCLUSIONS
Findings
INTRODUCTION
OUR STUDY
FINDINGS
CONCLUSIONS
Findings
5.93
6
5.35
5
4.63
4.18
4
4.56
4.3
4.12
3.04
3
5.24
3.38
2.71 2.57
Pre-game
Post-game
2
*p≤0.05
**p≤0.01
One-tailed tests
1
0
OSU*
UO*
Presidential approval
(1-7)
OSU**
UO
State of the economy
(1-5)
INTRODUCTION
OSU**
UO**
College satisfaction
(1-7)
OUR STUDY
FINDINGS
CONCLUSIONS
Over Time Effects
Remember that at the recontact, all of the subjects had experienced the treatment (game
outcome)
Initial pre-game attitudes represent the normal, uninfluenced opinions
Initial post-game attitudes represent the opinions after experiencing the effect of the game
INTRODUCTION
OUR STUDY
FINDINGS
CONCLUSIONS
Presidential Approval
4.9
4.7
4.5
4.3
4.1
Note: Pre and post
reference if the
subjects were
initially surveyed
before or after the
football game
3.9
3.7
3.5
Initial survey
OSU Pre
OSU Post
INTRODUCTION
Recontact
UO Pre
OUR STUDY
UO Post
FINDINGS
CONCLUSIONS
Presidential Approval: Lasting Effect
4.9
4.7
4.5
4.3
4.1
Note: Pre and post
reference if the
subjects were
initially surveyed
before or after the
football game
3.9
3.7
3.5
Initial survey
OSU Pre
OSU Post
INTRODUCTION
Recontact
UO Pre
OUR STUDY
UO Post
FINDINGS
CONCLUSIONS
Presidential Approval: Temporary Effect
4.9
4.7
4.5
4.3
4.1
Note: Pre and post
reference if the
subjects were
initially surveyed
before or after the
football game
3.9
3.7
3.5
Initial survey
OSU Pre
OSU Post
INTRODUCTION
Recontact
UO Pre
OUR STUDY
UO Post
FINDINGS
CONCLUSIONS
Presidential Approval: Actual Results
4.9
4.7
4.5
4.3
4.1
Note: Pre and post
reference if the
subjects were
initially surveyed
before or after the
football game
3.9
3.7
3.5
Initial survey
OSU Pre
OSU Post
INTRODUCTION
Recontact
UO Pre
OUR STUDY
UO Post
FINDINGS
CONCLUSIONS
Conclusions
Effects from Unrelated Events
INTRODUCTION
OUR STUDY
FINDINGS
CONCLUSIONS
Conclusions
Effects from Unrelated Events
◦
◦
◦
◦
These events influence unrelated political opinions and substantively relevant attitudes
Expected results on mechanism of mood
Possibility for spread through social networks
The political effects seem short-lived
INTRODUCTION
OUR STUDY
FINDINGS
CONCLUSIONS
Conclusions
Interesting questions that remain:
INTRODUCTION
OUR STUDY
FINDINGS
CONCLUSIONS
Conclusions
Interesting questions that remain:
◦ Why do economic evaluations behave differently than presidential approval?
◦ How does the timing of the event influence its lasting effect?
◦ What is the relative impact of these events as compared to other things that shape preferences?
INTRODUCTION
OUR STUDY
FINDINGS
CONCLUSIONS
Extras
Design Requisite
Our Study
Elected officials cannot influence the event: there
could be no reasonable action taken by elected
officials that could affect the event’s occurrence,
nature, or outcome.
The event should be an occurrence in the real world.
The 2015 College Football Playoff National Championship game, pitting The Ohio State
University against the University of Oregon.
Individuals under study must experience the event.
Undergraduate students at the competing schools (University of Oregon and The Ohio State
University).
Effects on (status quo) political and economic
attitudes.
Clear causal inference that the event affected
individuals’ political attitudes.

Presidential approval.

Evaluation of the economy.
An experiment that randomly assigns respondents from each school to

a pre-game survey, or

a post-game survey.
Changes between average pre-game and average post-game attitudes indicate a causal effect
of (losing/winning) the game.*
The game occurred on January 12, 2015.
The event’s connection to political attitudes should be Survey advertised as a study of the “social, economic, and political attitudes of college
outside of the individuals’ conscious awareness.
students” (no mention of the football game).
Mood mechanisms
Positive and Negative mood measured with the PANAS scale.
Secondary effects



Relevant opinions: satisfaction with university.
Social Contagion: posting on Facebook
Durability: Re-measurement one week after the game.
ES: REQUIREMENTS
ES: MEASURES
ES: OVER-TIME
Extras
Sample notes:
◦ About 1800 names were sampled from each school
◦ T1 Response rates were (N=414):
◦
◦
◦
◦
OSU pre-game: 10.77%
OSU post-game: 13.04%
UO pre-game: 10.40%
UO post-game: 11.22%
◦ At T2 (N=243)
◦
◦
◦
◦
OSU pre: 66.67%
OSU post: 63.30%
UO pre: 52.38%
UO post: 53.98%
Procedure notes:
◦ Pre-game invitation sent January 10th; two reminders also sent. Closed on January 12th
◦ Post-game invitation sent January 13th; two reminders sent. Closed January 15th
◦ $5 Amazon gift card incentive for completing wave 1; given and additional $2 for wave 2
ES: REQUIREMENTS
ES: MEASURES
ES: OVER-TIME
Extras
Main DVs:
◦ Presidential Approval  How much do you disapprove or approve of the way President Obama is handling his
job as President? (7 point scale, higher scores = higher approval)
◦ Evaluation of the Economy  What do you think about the state of the economy these days in the United States?
(5 point scale, higher scores = better assessments)
◦ Satisfaction with University  To what extent are you unsatisfied or satisfied with your decision to attend XXX?
(7 point scale, higher scores = increased satisfaction)
Mood
◦ The box below contains a number of words that describe different feelings and emotions... Indicate to what
extent you feel this way right now, that is, at the present moment. (5 point scale, higher scores = feeling more
that way)
◦ 4 items that reflect a positive mood (enthusiastic, proud, interested, and elated)
◦ 9 items for a negative mood (afraid, worried, anxious, angry, bitter, hatred, contempt, resentful, sad).
ES: REQUIREMENTS
ES: MEASURES
ES: OVER-TIME
Extras
Facebook prompt:
We are interested in how your friends react to your feelings. If you use Facebook, are you
willing to post about how you currently feel on your Facebook page and include a link to our
study? (yes/no)
If you are willing to post, you can simply post comments about how you feel with the following
statement:
“I am posting this as part of my participation in a study by researchers at Northwestern University. If
you would like to participate in a part of that study you can follow this secure and encrypted link:
XXXX. Participation would entail completing a brief survey and you would then be entered into a
drawing for one of twenty $25 gift cards to Amazon.”
ES: REQUIREMENTS
ES: MEASURES
ES: OVER-TIME
Extras
4
3.49
3
3.03
2.79
2.43
2.16
2
1.82
1.71
1.8
Pre-game
Post-game
1
*p≤0.05
**p≤0.01
One-tailed tests
0
OSU**
UO**
Positive Mood
(1-4)
OSU
UO**
Negative Mood
(1-9)
ES: REQUIREMENTS
ES: MEASURES
ES: RESULTS
Extras
15.5
16
Facebook posting (%)
14
12
9.5
10
8
9.4
7.3
Pre-game
Post-game
6
4
2
0
OSU*
UO
ES: REQUIREMENTS
ES: MEASURES
ES: RESULTS
Extras
Presidential Approval Over Time
4.93
(1.63; 69)
4.9
Approval Score
4.7
Note: Time 1 and Time
2 refer to the initial
and recontact waves.
Pre and post reference
if the subjects were
initially surveyed
before or after the
football game
4.6
(1.59; 53)
4.74
(1.55; 53)
4.57*
(1.54; 61)
4.5
4.3
4.22
(1.61; 58)
4.03**
(1.34; 69)
4.18
(1.79; 61)
4.1
3.9
3.98
(1.62; 58)
3.7
3.5
**p ≤ .01, *p ≤ .05, for one-tailed
tests of T1 vs. T2
UO Pre
Time 1
Time 2
Time
UO Post
OSU Pre
ES: REQUIREMENTS
OSU Post
ES: MEASURES
ES: RESULTS
Extras
4
Evaluation of the Economy Over Time
3.8
Evaluation Score
3.6
Note: Time 1 and Time
2 refer to the initial
and recontact waves.
Pre and post reference
if the subjects were
initially surveyed
before or after the
football game
3.4
3.45
(1.09; 69)
3.1**
(1.02; 69)
3.2
2.95
(1.05; 58)
3
2.8
2.6
3.05
(.98; 58)
2.9*
(1.03; 61)
2.89*
(1.01; 55)
2.66
(.96; 61)
2.64
(1.03; 55)
2.4
2.2
2
**p ≤ .01, *p ≤ .05, for
one-tailed tests of T1 vs.
T2
UO Pre
Econ T1
Econ T2
Time
UO Post
OSU Pre
OSU Post
ES: REQUIREMENTS
ES: MEASURES
ES: RESULTS
Extras
6
Satisfaction with University Over Time
5.88
(1.70; 66)
5.8
5.5
(1.64; 56)
Satisfaction Score
5.6
Note: Time 1 and Time
2 refer to the initial
and recontact waves.
Pre and post reference
if the subjects were
initially surveyed
before or after the
football game
5.79
(1.41; 66)
5.7
(1.37; 56)
5.35
(1.43; 55)
5.4
5.49
(1.22; 55)
5.2
5
4.79
(1.93; 61)
4.8
4.6
4.53…
4.4
4.2
4
**p ≤ .01, *p ≤ .05, for onetailed tests of T1 vs. T2
UO Pre
Coll Sat T1
Coll Sat T2
Time
UO Post
OSU Pre
ES: REQUIREMENTS
OSU Post
ES: MEASURES
ES: RESULTS