Voter Turnout Sensitivity Analysis:
Towards a More Parsimonious
Combinatorial Likely Voter Model
Presentation to 2015 AAPOR Conference
May, 2015
© 2015 Ipsos. All rights reserved. Contains Ipsos' Confidential and Proprietary information and
may not be disclosed or reproduced without the prior written consent of Ipsos.
 Typically ‘Likely Voter Models’ include a 5 or 6 item summated index from
which “cut points” are defined, which typically correspond to the historical
turnout rates for a given type of election (Presidential or Midterm).
 Recent work has explored the challenges of this model, specifically the ‘coarse’
nature of the index, which does not allow for as much precision as needed to
approximate expected turnouts.
 The problem is especially acute in Primary or low-turnout Midterm elections
like 2014, where only 25% - 35% of the population cast a vote.
 The problem has been in part addressed via the use of logistic regression
models that allow for more discrimination between intervals.
2
 However, the need for a simple question battery allowing for sufficient
granularity at low-turnout elections persists.
 Our paper will undertake a sensitivity analysis for a broad range of turnout
cutpoints from over 30,000 interviews conducted in the run-up to the 2014
Midterm elections, for generic congressional ballot voting intention measures.
 We will re-analyze the data in order to assess both question sensitivity and the
full Likely Voter battery, in an attempt to create a combinatorial model that is
more parsimonious.
 To measure performance, we will employ the Average Absolute Difference
between the survey estimates and election results.
3
Ipsos Likely Voters Question Battery – 2014
Question
Q1. Are you currently registered to vote, or not?
Responses
Yes
No
Don’t know / Refused
Q2. Sometimes things come up and people are not able
to vote. In the [last] election, did you happen to vote?
Yes
No
Don’t know / Refused
ASK IF ‘NO’
Q2A. Why not?
Did not live in state/district at the time
Too young to vote at the time
Not registered, too busy, something came up
Other (SPECIFY)
Don’t know / Refused
Q3. Did you happen to vote in any of these other
elections?
2006 midterm congressional election
2008 presidential election
2010 midterm congressional election
None of these [EXCLUSIVE RESPONSE]
Q4. In November [year], the next [election type] election
will be held. Using a 1-to-10 scale, where 10 means you
are completely certain you will vote and 1 means you are
completely certain you will NOT vote, how likely are you
to vote in the upcoming [election type] election?
10 – Completely certain
9-6
5-2
1-Completely certain will not vote
Don’t know
Q5. How much interest do you have in following news
about the upcoming [election type] election?
A great deal
Quite a bit
Only some
Very little
No interest at all
Don’t know / Refused
Score
Voter
Non-voter
Voter
Non-voter
Voter
Non-voter
Voter
Non-voter
Voter
Non-voter
Voter
Non-voter
4
Likely Voter Index Formulation
• Ipsos General Election Likely Voter Formulation
• Summated index:
ln(vote/not vote = Past voting(Voted 2006 + Voted 2008 + Voted 2010 + Voted 2012) +
Prob.of Voting 2014 + Interest in election
• Easy to implement.
• Provides resolution for turnout levels from 40-70%.
• Ipsos Primary Election Likely Voter Formulation
• Logistic regression index:
• More complicated implementation.
• Relies on partisan sentiment – acceptable for primaries, less so for
general.
• Provides resolution for turnout levels from 10-30%.
5
• Problem:
• Ipsos has no unified likely voter model approach
for both primaries and general elections.
• Lack of resolution when turnout is between 30%
and 40% ( a la the 2014 midterm election).
• Solution:
• Unified likely voter approach providing resolution
across entire spectrum of scenarios.
6
Likely Voter Battery Components
Voter
Registration
Likelihood to Vote
Likelihood to
vote
Base LV
Model
Interest in
election
Past Election
Behavior
10%
20%
30%
40%
50%
60%
Partisan ID
intensity
70%
80%
90%
100%
Percentile of Vote Eligible Public
7
Ipsos Standard – Summated Likely Voter Index – 2014
• Constructed by combining Q1+Q2+Q3+Q4+Q5
• Each variable has equivalent weight
Likelihood to Vote
Always votes
Primary turnout range
Never votes
Midterm turnout
range
Presidential
turnout range
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentile of Vote Eligible Public
8
Alternative One – “Kitchen Sink” Additive
• Constructed by combining Q1 + Q2/3 + Q4 + Q5
+ States Preference + Intensity of Partisan ID
Likelihood to Vote
Base LV Model
Alt-1 LV
Full Model
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentile of Vote Eligible Public
9
Alternative Two – Stated Interest Only
• Constructed by combining Q4 + Q5
Likelihood to Vote
Base LV
Model
Alt-2 LV
Interest Only
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentile of Vote Eligible Public
10
Alternative Three – Historic Only
• Constructed by combining Q1+Q2/3
Likelihood to Vote
Base LV
Model
Alt-3 LV
Historic Only
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentile of Vote Eligible Public
11
Alternative Four – Historic + Stated Intention
• Constructed by combining Q1+Q2/3 + Q4
Likelihood to Vote
Base LV
Model
Alt-4 LV Historic +
Stated intention
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentile of Vote Eligible Public
12
•
Accuracy = proximity to actual 2014 election results.
•
Measure accuracy by comparing absolute average error of
findings vs. actual results.
•
Use standard practice of dropping respondents who say “don’t
know” or “refuse” and rebase to 100%.
13
Impact on Poll Results
Absolute average error
50%
6
6
3
3
4
3
2
46
46
50
50
47
51
49
47
46 Democrats
3.3
6
8
8
6
3
6
6
6
51
48
42
42
47
46
45
47
48 Republicans
Other
Ipsos Data
Reuters / Ipsos Poll: Oct-Nov 2014
14
•
Likely voter model significantly improves accuracy of election
polling.
•
Historic election behavior is the better measure of likely voter.
•
However, still underreport Republican vote share in 2014.
15
•
Hypothesis 2: Dropping respondents who show up as highly
likely to vote but are “don’t know” or “refuse” loses actual
voters.
•
Challenge: How to impute vote preference on voters who have
no stated preference?
•
Potential solution: code “don’t know” or “refuse” likely voters as
voting for the incumbent party for their district.
16
Thinking about the elections in 2014, if the election for U.S. Congress were held today,
would you vote for the Democratic candidate or the Republican candidate in your district
where you live?
Democratic Republican
Other
Not voting DK/ Refused
Party ID of Democrat
53%
36%
44%
46%
43%
the office
holder of
Republican
47%
64%
56%
54%
57%
the 113th
Congress
113th Congress
"US House 2012" by Mr. Matté File:113th U.S. Congress House
districts.
17
Impact on Poll Results – Don’t Know Other to Incumbent
Absolute average error
50%
5
5
2
3
3
2
2
46
46
49
49
46
51
46
47
46 Democrats
3.3
6
8
8
6
3
6
6
6
51
48
46
45
48
46
48
49
49 Republicans
Other
Ipsos Data
Reuters / Ipsos Poll: Oct-Nov 2014
18
Implications
1. Interest in election news provides no significant addition to our
likely voter model and can be removed.
2. A likely voter model consisting of voter registration, past voting
behavior and stated intent to vote in the upcoming election is
as accurate as the “full” Ipsos model.
3. All Ipsos likely voter models continue to understate Republican
vote share.
4. Coding likely voters with no stated preference to support the
incumbent for their district improves estimates.
19
Thank you
Chris Jackson
Ipsos Public Affairs
[email protected]
20