UNCLASSIFIED
APPROVED FOR PUBLIC RELEASE
DISTRIBUTION UNLIMITED
Investigating Social Biases in Information
Transmission:
Experimental Design and Preliminary
Analyses
Heather Roy
Social Scientist
U.S. Army Research Laboratory
September 6, 2016
UNCLASSIFIED
The Nation’s Premier Laboratory for Land Forces
UNCLASSIFIED
Social Media as the New Frontier
• With the advances and availability of mobileoriented services, the global internet usage
will increase beyond 50% of the world’s
population in 2016
• There are hundreds of social networking sites
available, many of which users can access
directly from their mobile devices
• There are over 2 billion social media users
across the world
• Social media sites are steadily growing in
importance as information sources on political
issues and current events
UNCLASSIFIED
The Nation’s Premier Laboratory for Land Forces
UNCLASSIFIED
Social Media as the New Frontier
• The U.S. Army has
classified Social Media
as a new form of
Signals Intelligence
– Challenges:
• Accuracy
• Reliability
• Case Study:
– Boston Marathon Bombing
UNCLASSIFIED
The Nation’s Premier Laboratory for Land Forces
UNCLASSIFIED
Experimental Objectives
Objectives:
• This study seeks to inform the social media reliability challenge by examining how
differences in reliability impact decisions to share or contradict information.
• Additionally, this experiment explores how social factors such as group bias influence
information transmission.
• An understanding of how human biases effect information transmission behavior will
help us to inform models of human behavior in social media to develop less biased factfinding and credibility ranking algorithms.
Information-Theoretic Model of Human Behavior in Social Media:
UNCLASSIFIED
The Nation’s Premier Laboratory for Land Forces
UNCLASSIFIED
Experimental Design
Mixed Design:
• Between-participants factors: Condition
–
•
Manipulating three variables of interest: Group Bias, Information Source, and Reliability
Within-participant factors: Event Type (Traffic, Natural Disaster, and Conflict)
Condition
Group Bias
Source
Reliability
Participants
Group A
Out
Participant as
Eyewitness
Reliable
143
Group B
In
Participant as
Eyewitness
Reliable
128
Group C
Out
Secondhand
Unreliable
114
Group D
Out
Secondhand
Reliable
127
Group E
In
Secondhand
Unreliable
118
Group F
In
Secondhand
Reliable
129
UNCLASSIFIED
The Nation’s Premier Laboratory for Land Forces
UNCLASSIFIED
Social Biases Study Model
UNCLASSIFIED
The Nation’s Premier Laboratory for Land Forces
UNCLASSIFIED
Experimental Task
•
Participants were asked to:
•
Individually review 15 statements
• Three event types: Traffic (Neutral Control), Natural Disaster, Conflict
•
Identify on a 7pt Likert-scale how likely they were to share or contradict the information
• Avenues of transmission: Twitter, Facebook, other Social Media, Phone
Event Category
Event Text Statement
Traffic
A major car accident is blocking all lanes of Highway 66, the main
Highway of Zalawera.
Natural Disaster After days of intense rain, the river of Zalawera is flooding nearby
neighborhoods.
Conflict
Gunshots are exchanged between the Clays’ family and the Lions’ family,
the two warring Zalawera families.
UNCLASSIFIED
The Nation’s Premier Laboratory for Land Forces
UNCLASSIFIED
Scenario Example
Hosted online via Amazon Mechanical Turk
UNCLASSIFIED
The Nation’s Premier Laboratory for Land Forces
UNCLASSIFIED
Examination of Means
Overall Means:
•
Participants more likely to share than to contradict information
Main Effects:
•
Eyewitness vs. Secondhand:
–
–
•
Reliable (as Eyewitness) vs. Reliable Secondhand Sources:
–
–
•
Eyewitness groups most likely to share and the least likely to contradict
Eyewitness and Secondhand Reliable source groups were most likely to share Natural Disaster, Conflict, and Traffic
information in that order
Reliable vs. Unreliable Sources:
–
–
–
–
•
Eyewitness groups somewhat more likely to share information and somewhat less likely to contradict
Eyewitness groups more likely to share information related to Natural Disaster and least likely to contradict this type
Unreliable source groups somewhat less likely to share and somewhat more likely to contradict
Unreliable source groups were most likely to share information related to Natural Disaster, but equally likely to share
Conflict or Traffic information
Secondhand Reliable source groups were more likely to share regardless of In or Out group orientation
Unreliable source groups were most likely to contradict information across event types
Group Bias:
–
In-groups were both more likely to share and to contradict
UNCLASSIFIED
The Nation’s Premier Laboratory for Land Forces
UNCLASSIFIED
Examination of the Means: Event Type
Event Type:
•
•
•
Most likely to share information related to Natural Disasters
Most likely to contradict information related to Conflicts
Least likely to share or contradict information related to Traffic
Sharing or Contradicting Conflict Information in
Secondhand Unreliable Groups:
•
–
–
•
–
•
More likely to contradict claims that were more
antagonistic (ranging x = 4.27 at top to 2.91 at bottom)
Less likely to share information that was antagonistic
More likely to share information that was sympathetic
Out-Group:
–
You see violent riots and looting erupt,
carried out by the Clays’ family on the Main
Street in Zalawera.
You see unmanned drones hover in the air
above Zalawera, carrying the Clays’ family
logo.
In-Group:
–
Conflict Statements
Roughly equally unlikely to contradict any of the
statements (x = 2-3)
No significant trends observed in sharing behavior
You see that gunshots are exchanged
between the Clays’ family and the Lions’
Family, the two warring Zalawera families.
You see that local Zalawera government
militia shoot at a peaceful Clays’ family
gathering causing several fatalities.
You see that a huge explosion rattles the
Clays’ family place of worship in Zalawera.
Both In and Out Groups most likely to share and
contradict information by phone to friend or family
UNCLASSIFIED
The Nation’s Premier Laboratory for Land Forces
UNCLASSIFIED
Preliminary Results
Chi-Square test for Association
•
Likert-scale response divided at the median and dichotomized
Statistically significant relations were found between likelihood of sharing
information and:
•
•
•
Type of Source X2(1) = 15.963, p < 0.001
– Eyewitness group participants were 1.870 odds greater to have been
likely to share information (OR = 1.870, 95%, CI [1.373, 2.547]).
Source reliability X2(1) = 17.67, p < 0.001
– Reliable source group participants were 2.091 odds greater to have
been likely to share information (OR = 2.091, 95%, CI [1.480, 2.976]).
Group reliability X2(1) = 15.963, p < 0.001
– In-group condition participants were 1.490 odds greater to have been
likely to share information (OR = 1.490, 95%, CI [1.103, 2.012]).
No statistically significant relations observed between likelihood of contradicting
information
UNCLASSIFIED
The Nation’s Premier Laboratory for Land Forces
UNCLASSIFIED
Populated Social Biases Study Model
UNCLASSIFIED
The Nation’s Premier Laboratory for Land Forces
UNCLASSIFIED
Survey: Social Media Use and Access
Social Media Use:
• On average, over 50% of participants used social media 1-2 hours daily
• Over 93% indicated having used social media platforms for two or more years
• 53% specifically endorsed social media use for an average of 5-10 years
Social Media Access:
• 83% mobile devices
• 73.9% laptops
• 56.4% desktops
• 43.2% tablets
• 1.2% other means
• Participants often endorsed using multiple devices
–
–
UNCLASSIFIED
Participants most frequently endorsed using at least three devices (36.2%)
The most frequent (17.4%) combination endorsed included four devices (mobile, desktop, laptop, and tablet)
The Nation’s Premier Laboratory for Land Forces
UNCLASSIFIED
Survey: Reasons for Use
Reasons Endorsed for using Social Media
Facebook
Twitter
Use
708 (93.28%)
450 (59.29%)
Number of Reasons for Use (1-11)
5-7 (35%)
1-2 (37%)
Reasons Endorsed for using Social Media
UNCLASSIFIED
Reasons
Facebook
Twitter
Whiting & Williams (2013)
Social Interaction
622 (87.85%)
265 (58.89%)
80%
Information Seeking
391 (55.22%)
249 (55.33%)
80%
Passing Time
528 (74.58%)
202 (44.89%)
76%
Entertainment
466 (65.82%)
234 (52.00%)
64%
Relaxation
258 (36.44%)
75 (16.67%)
60%
Things to Talk About
266 (37.57%)
147 (32.67%)
56%
Convenience
240 (33.90%)
81 (18.00%)
52%
Expression of Opinion
339 (47.88%)
200 (44.44%)
56%
Information Sharing
429 (60.59%)
190 (42.22%)
40%
Knowing About Others
434 (61.30%)
160 (35.56%)
32%
Professional
124 (17.51%)
82 (18.22%)
N/A
Other
22 (3.10%)
11 (2.44%)
N/A
The Nation’s Premier Laboratory for Land Forces
UNCLASSIFIED
Survey:
Information Transmission Behavior
• Participants appeared likely to share information related to all the categories
• Participants most often indicated refuting or denying information related to politics, news or
current events, or themselves (personal).
Sharing, Refuting, and Denial Information Behavior on Social Media
Share
Refute
Deny
Personal
568 (74.84%)
73 (9.62%)
56 (7.38%)
Political
298 (39.26%)
192 (25.30%)
80 (10.54%)
News or Current Events
441 (58.10%)
174 (22.92%)
55 (7.25%)
Professional
218 (28.72%)
28 (3.69%)
8 (1.05%)
Health
180 (23.72%)
53 (6.98%)
18 (2.37%)
Entertainment
497 (65.48%)
60 (7.91%)
19 (2.50%)
Charity
119 (15.68%)
14 (1.84%)
2 (0.26%)
Religion
107 (14.10%)
84 (11.07%)
38 (5.01%)
Other
31 (4.08%)
9 (1.19%)
8 (1.05%)
UNCLASSIFIED
The Nation’s Premier Laboratory for Land Forces
UNCLASSIFIED
Conclusions
•
Examined the effects of personal bias in relation to a user’s likelihood of sharing or
contradicting information concerning: traffic, natural disasters, and conflict events
•
Preliminary results suggest that the type of information, group bias, and source reliability
influence information transmission decisions:
– Statistically significant Results:
• Participants more likely to share information from a reliable source compared to an
unreliable source
• More likely to share information they witnessed compared to received secondhand
• More likely to share information when part of an In-Group compared to an Out-Group
•
Way Forward: The results of this study will be used to quantify social parameters that will
be incorporated to enhance the performance of fact-finding algorithms under
development by academic research partners at UIUC.
UNCLASSIFIED
The Nation’s Premier Laboratory for Land Forces
UNCLASSIFIED
Thank You!
Heather Roy
[email protected]
UNCLASSIFIED
The Nation’s Premier Laboratory for Land Forces