here - Stuart Soroka

Read, Seen
or Heard
A Text-Analytic
Approach to
Campaign
Dynamics
Gallup, Michigan, Georgetown Working Group: Frank Newport, Lisa
Singh, Stuart Soroka, Michael Traugott, Andrew Dugan
Slides presented at the Center for Political Studies, Institute for Social Research, University of Michigan, October 5 2016.
Background
There is a good deal of research capturing
trends in government/candidate approval, and
vote intentions.
We are often able to connect these trends to
campaign events, and/or shifts in media
content.
We are rarely able to directly capture the
information that voters remember.
Background
Whereas most data sources focus on the
effects of campaign information, i.e., vote
intentions,
or the provision of campaign information,
i.e., media coverage,
we capture the reception of information, i.e.,
what respondents recall having read, seen or
heard about the candidates.
Data
The Gallup U.S. Daily* interviews roughly 500
people by per night, and asks what
respondents have read, seen or heard about
Hillary Clinton / Donald Trump in the past
several days.
Verbatim responses are recorded by
interviewers.
* The Gallup U.S. Daily interviews adults aged 18 and older living in all 50 states and the District
of Columbia using a dual-frame design, which includes both landline and cellphone numbers.
Gallup samples landline and cellphone numbers using random-digit-dial methods.
% Having Read, Seen and Heard Something
about Clinton or Trump
% Non−Empty Responses, 3−day Mean
100
90
RNC
convention
DNC
convention
9-11
80
70
First
debate
60
50
~41,000 interviews
40
July 10
Sept 1
Sept 26
(debate)
Rolling 3-day (lagged) averages, unweighted.
Words Distinguishingclinton
Read, Seen and Heard
about Clinton and Trump
ads
children
untrustworthy
facebook department
mostly congress
corruption
deleted done campaigning
private found questions
criminal classified
response saw ella investigation timkaine time
jail her trail berniesanders problems
playshe
released dishonestsupporters
got
memorial
information bengazi
regarding
server crook
corrupt debate
secretary
truth benghazi
issues charges
interview
money
situation still
billclinton
state well fun
mikepence
tryingbaby flood
putin trip
wentbuilding border
war mouth
bornstatement
idiot
black
wants
gold
made church soldier
negative
people man flip
calling build russia speech isis
racist plan remarks
meeting making family obama wall
son like rally mind
foundervisitsaid going tax wife
paid twitter parents Top 160 words that are
changed president
policy
talking
most uniquely associated
illegal changing amendment
doesnt
second
with responses about
louisiana
american
bigot manager
african
Clinton or Trump, font size
missuniverse paulryan
heart
tedcruz make america
women
weighted by frequency,
hack veterans military stance change
based on all responses,
mccain statements business
July 11-Oct 3, using
states lost rnc way star wifes take
limited synonym recoding
arizona
americans
and lemmatization.
scandal
dnc
fbi
liar
health
email
get stupid
detroit
muslim
mexico
country
immigration
media
makeamericagreat
khan
economy
foundation
Weekly Trends in Read, Seen and Heard
Jul 10−16
email berniesanders
Jul 17−23
email convention liar
Jul 24−30
Jul 31−6
Aug 7−13
Aug 14−20
Aug 21−27
Aug 28−3
Sep 4−10
Sep 11−17
Sep 8−24
Sep 25−1
Oct 2−
fbi liar scandal
speech going
convention email speech
email liar
dnc president
conventionspeech campaign
email liar
email
tax foundation
scandal
foundation liar health going
email
foundation liar
email
email
scandal health
foundation liar fbi scandal
health
health
liar scandal fbitalking
email issues campaign liar
debate email liar newyork
debate
debate email tax
liarcampaign
email liar health
night
Top 5 words, font size
weighted by frequency,
based on all responses,
July 11-Oct 3, using
limited synonym
recoding and
lemmatization.
Weekly Trends in Read, Seen and Heard
Jul 10−16
mikepence dallas going shooting pick
Jul 17−23
convention speech
Jul 24−30
russia email convention speech
Jul 31−6
muslim family son paulryan convention
Aug 7−13
isis amendment obama second people
Aug 14−20
campaign
Aug 21−27
immigration
wife national
going
speech isis people louisiana
campaign people louisiana trying
Aug 28−3
mexico immigration
Sep 4−10
mexico immigration speech
Sep 11−17
Sep 8−24
Sep 25−1
Oct 2−
going
speech president going
going president
health obama born
talking speech
talkingdebatenewyork obamapeople
debate
tax
missuniverse
women people going
debate paid missuniverse returns
Top 5 words, font size
weighted by frequency,
based on all responses,
July 11-Oct 3, using
limited synonym
recoding and
lemmatization.
The Changing Tone of Read, Seen or Heard
Responses
−> Trump Advantage −>
6
4
2
Differences in the tone of what respondents
report having “read, seen or heard” about
Donald Trump, versus what they report having
“read, seen or heard” about Hillary Clinton.
REPUBLICAN RESPONDENTS ONLY
0
Sept 2
Sept 26
(debate)
Rolling 3-day (lagged) averages, unweighted.
Tone estimated using the Lexicoder Sentiment Dictionary
The Changing Tone of Read, Seen or Heard
Responses
−> Clinton Advantage −>
6
4
2
Differences in the tone of what respondents
report having “read, seen or heard” about
Donald Trump, versus what they report having
“read, seen or heard” about Hillary Clinton.
DEMOCRATIC RESPONDENTS ONLY
0
Sept 2
Sept 26
(debate)
Rolling 3-day (lagged) averages, unweighted.
Tone estimated using the Lexicoder Sentiment Dictionary
Motivated Reasoning in Recollections of the
First Debate
Based on respondents interviewed on Sept 27 and
28, 61% believe that Clinton won, while 27% believe
that Trump won.* Within partisan groups:
92% of Democrats believe that Clinton won
the debate, and 3% believe that Trump won.
53% of Republicans believe that Trump won
the debate, and 28% believe that Clinton
won.
* Based on Gallup analyses (of the same survey data used here).
Words Distinguishing
Democrats and
Democrat
Republicans’ Read, Seen and Heard about
Clinton, Post-Debate
ella
going great missuniverse
helping
live next sobre debt
night won saw response
really presidential
reaction
plans
bring
taxissues
living done
immunity
opinion
prison
performance good
debate
polls
remember
gets
putting
deleted
money
crook
much
foundation
got
crooked
server
everything
fbi thats
negative media reading
problems health berniesanders
Republican
year supporters
cant
Identifiers
information
involved dishonest unfavorable
criminal truth parents
specifically
one bad
scandal
benghazi
hear
anything
email liar
article
twitter
positive
country
young
saturday
times
talking
poll recent college economy
states
Democratic
prepared women
responses
trail
newyork
Identifiers
better
carolina
president campaigning
strong
statements
children campaign today
education rally
read
election help
Top 100 words that
are most uniquely
associated with
responses about
Clinton or Trump,
font size weighted by
frequency, based on
all responses after
the first debate, using
limited synonym
recoding and
lemmatization.
internet campaign
fat
said
ads
trying
veterans plans
election different
situation recall
country
ago much
results
brought
ive
gonna goingcountries
michigan great
florida
last polls jobs well right really
america
Republican
Identifiers
remember
Words Distinguishing
Republicans and
Republican
Democrats’ Read, Seen and Heard about
Trump, Post-Debate
everyone
foundation
newyork
liar
people
racist
cuba
tax
paid
fact
russia
talks
twitter
night
poor
performanceweight
minorities
made way remarks embargo
many
disrespect
lady
new
debate
women
returns
young pig
called pay
presidential years
Democratic
Identifiers
hasnt
regarding
read
fox
idiot
using
charity
saw deal
good makespeech wants
states email
kind
doesnt
got
plan
issues rally
facebook
makeamericagreat
immigration
whatever
economy media better talking rallies back
little
contest
Top 100 words that
are most uniquely
associated with
responses about
Clinton or Trump,
font size weighted by
frequency, based on
all responses after
the first debate, using
limited synonym
recoding and
lemmatization.
Summary of Findings (thus far)
Republicans’ read, seen or heard responses have
been more systematically positive about Trump
than Democrats have been about Clinton.
Clinton’s read, seen or heard responses were
dominated by ‘email’ until Sept 11. Recalled
information about Clinton has been more positive
since that time.
Summary of Findings (thus far)
A majority of respondents believe that Clinton won
the first debate. But there are real differences in
what partisans report having read, seen or heard
since then.
Clinton: Democrats cite her debate performance,
Republicans continue to cite news on email and
other scandals
Trump: Republicans cite his debate performance,
while Democrats cite his complex record with
women, particularly Alicia Machado.
Trump
debate
rather
refuses
like
called
seeing mismo
mainly year
brought
para
kids
recall put
lost
nafta
read
shared
health
sources muchtelevision
four left
bengazi facebook black
around
change
live
the experience
helping
cartoon situation dishonest
government corruption daughter
saw
crooked
saturday
foundations
womens
Top 100 words
email
0
Differences in the tone of what respondents
report having “read, seen or heard” about
Donald Trump, versus what they report
having “read, seen or heard” about Hillary
Clinton.
INDEPENDENT RESPONDENTS ONLY
−4
anything
back
Clinton
−2
showed
fat fact
paid everyday
even
support
2
veterans
sending
children
foundation
feed
let
send
scandal bad
think
want
smart
tax
missuniverse
returns twitter
federal
honest
towards
morning women
life
4
−> Trump Advantage −>
doesnt
contestant
liar
6
negative racist
people
drink
trying
forth
two
good
last
girl
performance
Amongst Independents, 59%
believe that Clinton won the
debate and 30% believe that
Trump won the debate.
money
What About Independents?
wants
weight loss
mexico
flint complaining
mrs handled hasnt
uniquely associated
with Clinton or Trump,
amongst Independents
only, font size weighted
by frequency, based on
all responses after the
first debate, using
limited synonym
recoding and
lemmatization.
Rolling 3-day (lagged)
averages, unweighted.
Tone estimated using the
Lexicoder Sentiment
Dictionary
−6
Sept 2
Sept 26
(debate)