How Have The Polls
Changed Since 2015?
John Curtice
British Polling Council
Performance of Final Polls
38
40
35
34
34
31
30
25
20
13 13
15
9
10
8
5
5
4
6
6
0
Con
Lab
UKIP
Poll Average
Lib Dem
Green
Others
Result
Based on 10 polls whose fieldwork did not end before 5.5; Polls by Opinium; YouGov; Survation; ComRes; Populus;
Ashcroft, Ipsos MORI: BMG Research, Panelbase; ICM Research
Inquiry’s Key Recommendations
• Turnout
• ascertain who has voted by post;
• identify better ways of who is more likely to vote
• Don’t Knows
• review ways of imputing likely voting behaviour
• Sampling/Representativeness
• improve representativeness of who is interviewed;
• in setting quotas and weighting targets investigate use
new variables associated with responding to polls and
vote choice
• benchmark against random probability surveys
Overview
• Nearly every company has made at least some
change to their methods.
• Most have addressed to some extent all of the
principal recommendations in the Sturgis Report,
albeit not necessarily in the same way
• Some have also made changes that do not flow
directly from the report
% voted
Age Gap in Turnout
100
90
80
70
60
50
40
30
20
10
0
55 58 54
56 57
64
68
75 78 72
80 83
84
77
88
78
64
43
18-24
25-34
BSA
35-44
BES
45-54
MORI Estimate
55-64
65+
Anticipated Age Gap in Turnout
Estimated Gap 18-24 vs 65+
45
40
35
30
25
20
15
10
5
0
39
35 35
34
27
22
24
21
18
15
12
9
10
4
MORI Post Ashcroft
Elec
ComRes
Certain to Vote
* 18-24 vs 60+
ICM
Ipsos MORI Populus
Average Propensity
YouGov*
The Widened Age Gap in Party
Support
Change in % vote since 2010
20
17
15
10
10
8
6
5
3
4
1
0
18-24
-5
-10
Source: BES 2010 and 2015
25-34
-3
35-44
-3
45-54
55-64
-2
65+
-5
Con
Lab
-7
Sampling/Representativeness - 1
• Secure greater inclusion of those who are less
engaged in politics (YouGov)
• Changes of mode
• ICM from phone to online
• ComRes all polls online
• Quota/Weight data by (BES) interest in politics
(ICM; YouGov) or overall estimated turnout (Kantar)
or newspaper readership (Ipsos MORI)
• Weight data so that % in each demographic (esp.
age) group represents % of voters (BES) rather than
% population (ORB; Panelbase?)
Sampling/Representativeness - 2
• Quotas/weighting by education (Ipsos MORI;
Kantar; ORB)
• Change quota/weighting age bands to secure more
older voters (Kantar; YouGov)
• Changes to quotas/weighting by past vote/party
identification etc. (ICM; Opinium; YouGov)
• Weighting by 2016 EU Referendum vote (Opinium,
Panelbase; YouGov)
Identifying Who Will Vote - 1
• Weight data so that estimated turnout in different
demographic groups is in line with modelling of
turnout in 2015; drop asking people how likely they
are to vote (ComRes)
• Use reported likelihood to vote in a model of the
relationship between likelihood and turnout in
2015 (Kantar)
• Weight data so that estimated turnout in each
demographic group matches a variety of estimates
for 2015 (ICM)
Identifying Who Will Vote - 2
• Ask/use additional questions apart from reported
likelihood of voting (ICM, Ipsos MORI; YouGov)
• Change use/wording of likelihood of voting
question (Ipsos MORI; Opinium)
• Not ask postal voters likelihood of voting (Kantar)
Shy Voters?
• Introduce ‘squeeze’ question (Kantar)
• Introduce/extend imputation of Don’t Knows etc
(ICM; Kantar?)
Conclusion - 1
• Biggest change is the application of greater effort to
ensuring a more accurate estimate of key
differences in turnout
• Some effort to contact the less politically engaged
• Much greater attention to weighting/modelling data to
reflect patterns of turnout/engagement at previous
elections, sometimes with reference to the 2015
random probability BES
• Somewhat less reliance on a likelihood to vote question
alone
Conclusion - 2
• Also some changes designed (inter alia) to improve
representativeness of those who do vote
• Greater use of past vote weighting
• Some use of weighting by education and EU referendum
vote
• Many of the changes clearly informed by an
understanding of what went wrong in 2015 (thanks,
not least, to the Sturgis report)
• But inevitably whether they will prove sufficient
will only be known on June 9!