Automating Content Analysis of
Video Games
T. Bullen and M. Katchabaw
Department of Computer Science
The University of Western Ontario
N. Dyer-Witheford
Faculty of Information and
Media Studies
The University of Western Ontario
Outline
1.
2.
3.
4.
5.
Introduction
Automating Content Analysis
Prototype Implementation
Experiences and Discussion
Concluding Remarks
Introduction

Content analyses of video games involve coding,
enumerating, and statistically analyzing various
elements and characteristics of games
–

This includes violence, offensive language, sexual
content, gender and racial inclusiveness, and so on
While content analysis has its limitations, it is
invaluable in providing a quantitative assessment
of games to go with more qualitative analyses
–
It can be an important tool to many people dealing
with various aspects of games and the games industry
Introduction

Problems arise, however, when one tries to apply
traditional content analysis processes, for
example from film or television, to games
–
–
–
–
Processes are manual and are consequently time
consuming and labour-intensive
This tends to result in significantly reduced play times
or limiting analyses to only a very few games
Traditional analyses also tend not to consider
interactivity and non-linearity that occurs in games
The rapid rate at which games are released and the
industry evolves makes keeping up difficult
Introduction


In the end, with the limited time and resources
often available, it is exceedingly difficult to
perform thorough content analyses on even a
reasonable portion of games
To address these problems, our current work
examines automating the process of content
analysis for video games
–
Through automation, it is hoped that time and
resources can be used more efficiently and
effectively to permit more thorough studies
Automating Content Analysis


To automate content analysis, we take
advantage of the fact that, unlike other forms of
media, video games are software executing on
some kind of computing device
This can permit two forms of automation:
–
–
Partial automation: software executing along side the
game monitors game execution and collects and
reports the data normally collected manually
Full automation: further software elements take the
role of the player and generate gameplay experiences
without the need for a human player
Automating Content Analysis:
Instrumentation
Coordinator
Sensor 1
Game
Object 1
Game
Object 2
Game Application
Code
Game
Object 3
Sensor 2
Game
Object n
Sensor n
Prototype Implementation

As a proof of concept, we have used our
instrumentation framework to instrument
Epic’s Unreal Engine to enable automated
content analyses of Unreal-based games
–

Unreal is a popular engine amongst professional and
amateur developers, providing numerous possible
games for content analysis experiments
Instrumentation was implemented using the
UnrealScript language
–
Source level access to the engine was not available
Prototype Implementation
Game Info
Game Rules
Sensor
Mutators
Coordinator
Game
Object
Prototype Implementation

Sensors have been developed to collect a wide
variety of data useful for content analyses:
–
–

Death of characters, weapon use by characters, use
of offensive language, gender and racial diversity in
characters, and a variety of other game statistics
Data can be reported throughout a game or only as
summaries at the end of games
Sensors can be configured at run-time to tailor
the data collected to the needs of the content
analyses being conducted
Experiences and Discussion

To validate our prototype implementation, we
conducted several content analysis experiments
on Unreal Tournament 2004
–

This game is one of the flagship titles driven
by the Unreal Engine
It is a fairly popular First Person Shooter that
has numerous gameplay options
–
–
–
Several different game types and rule sets
Individual and team-based games
Single player, multiplayer, and spectator modes
Experiences and Discussion:
Deathmatch Game
------------Level Info-----------Level Name: Rrajigar
Game Type: DeathMatch
Total Players: 14
AI Players: 13
Human Players: 1
Spectators: 0
Male Players: 13
Female Players: 1
Level Loaded: 0:26:45
Game Finished: 0:30:29
Gameplay Elapsed (Seconds): 240.88
AI Dialog: 28
Human Dialog: 27
-------------------------------------------All Player Stats-------Total Deaths: 47
Total Suicides: 1
Total Kills: 46
Total AI Deaths: 46
Total Human Deaths: 1
Total Deaths Caused By AIs: 22
Total Deaths Caused By Humans: 25
Total Female Deaths: 12
Total Male Deaths: 35
Total Deaths Caused By Females: 6
Total Deaths Caused By Males: 41
-----------------------------------------Local Player Stats-------Player Deaths: 1
Player Suicides: 0
Player Killed: 1
Deaths Caused By Player: 25
Player Killed By AI: 1
Player Killed By Human: 0
Player Killed By Male: 1
Player Killed By Female: 0
AI Deaths Caused By Player: 25
Human Deaths Caused By Player: 0
Female Deaths Caused By Player: 7
Male Deaths Caused By Player: 18
Deaths Witnessed By Player: 29
----------------------------------------------Expletives----------ass: 2
----------------------------------
Experiences and Discussion:
Onslaught Game
------------Level Info-----------Level Name: Arctic Stronghold
Game Type: Onslaught
Total Players: 12
AI Players: 11
Human Players: 1
Spectators: 0
Male Players: 9
Female Players: 3
Level Loaded: 23:16:43
Game Finished: 23:32:3
Gameplay Elapsed (Seconds): 964.18
AI Dialog: 216
Human Dialog: 31
-------------------------------------------All Player Stats-------Total Deaths: 142
Total Suicides: 5
Total Kills: 137
Total AI Deaths: 138
Total Human Deaths: 4
Total Deaths Caused By AIs: 119
Total Deaths Caused By Humans: 23
Total Female Deaths: 26
Total Male Deaths: 116
Total Deaths Caused By Females: 8
Total Deaths Caused By Males: 134
-----------------------------------------Local Player Stats-------Player Deaths: 4
Player Killed: 4
Deaths Caused By Player: 23
Player Killed By AI: 4
Player Killed By Human: 0
Player Killed By Male: 4
Player Killed By Female: 0
AI Deaths Caused By Player: 23
Human Deaths Caused By Player: 0
Female Deaths Caused By Player: 9
Male Deaths Caused By Player: 14
Deaths Witnessed By Player: 43
---------------------------------------------Team Info------------Female Allies: 1
Male Allies: 4
Friendly Fire Deaths: 5
Allies Killed By Player: 0
Player Killed By Ally: 0
Experiences and Discussion

Quality of data
–

Quantity of data
–

Data collected through automation matched manual
results, and in some cases was better
We found that we could collect massive amounts of
data with no visible impact on gameplay, even when
data was reported throughout a game
Partial versus fully automated analyses
–
–
We found that results could be very different
Which is ultimately better?
Concluding Remarks



Content analysis plays several important roles to
the video games industry, but is unfortunately an
arduous task to complete in a thorough fashion
Our current work addresses this issue by
providing an automated approach to content
analysis based on software instrumentation
Initial experimentation with a prototype
implementation of this approach demonstrates
its usefulness and shows great promise
Concluding Remarks

Directions for future work include the following:
–
–
–
–
–
Conduct further experimentation and more detailed
content analyses of Unreal Tournament 2004, and
combine qualitative analyses with our results
Expand experimentation to other Unreal-based games
Investigate instrumentation of other popular game
engines and conduct further analyses this way
Create sensors for measuring other content metrics
Further explore the issue of partial versus fully
automated content analyses