Murray State's Digital Commons
Honors College Theses
Honors College
Spring 4-28-2017
Review Prohibition Periods as a Signal of Video
Game Quality
Richard Jacob Applin
Murray State University
Follow this and additional works at: http://digitalcommons.murraystate.edu/honorstheses
Part of the Econometrics Commons, and the Other Economics Commons
Recommended Citation
Applin, Richard Jacob, "Review Prohibition Periods as a Signal of Video Game Quality" (2017). Honors College Theses. 13.
http://digitalcommons.murraystate.edu/honorstheses/13
This Dissertation/Thesis is brought to you for free and open access by the Honors College at Murray State's Digital Commons. It has been accepted for
inclusion in Honors College Theses by an authorized administrator of Murray State's Digital Commons. For more information, please contact
[email protected].
Murray State University Honors College
HONORS THESIS
Certificate of Approval
Review Prohibition Periods as a Signal of Video Game Quality
Richard Jacob Applin
April 2017
Approved to fulfill the
requirements of HON 437
Approved to fulfill the
Honors Thesis Requirement
of the Murray State Honors
Diploma
Dr. David Eaton, Professor and Chair
Economics and Finance
Dr. Warren Edminster, Executive Director
Honors College
Examination Approval Page
Author: Richard Jacob Applin
Project Tile: Review Prohibition Periods as a Signal of Video Game Quality
Department: Economics and Finance
Date of Defense: April 28, 2017
Approval by Examining Committee:
(Dr. David Eaton, Advisor)
(Date)
(Dr. Sunayan Acharya, Committee Member)
(Date)
(Dr. James Humphreys, Committee Member)
(Date)
Review Prohibition Periods as a Signal of Video Game Quality
Submitted in partial fulfillment
of the requirements
for the Murray State University Honors Diploma
Richard Jacob Applin
April 2017
Copyright ©2017 by Richard Jacob Applin
All rights reserved
To my late cat, Pepper, and to my dog, Pascal, for always recognizing and appreciating what
others do not.
ii
Contents
List of Tables
iv
Acknowledgements
viii
Abstract
ix
Chapter 1: Introduction and Background
1
1.1 Some Further Background for the Non-Gamer
5
Chapter 2: Literature Review and Theoretical Framework
10
Chapter 3: Economic Theory and Review Prohibition Periods
15
Chapter 4: Data and Methodology
18
4.1 Sources
18
4.2 Variables
21
4.3 Philosophy
30
4.4 Initial Criteria
32
4.5 Creating the Sample Data Set
37
4.6 Analyzing the Data
42
4.7 Potential Issues
44
4.8 Recap
45
iii
Chapter 5: Results
5.1 Review Prohibition Periods as a Signal of Video Game Quality
47
48
5.2 Estimating the Effects of Review Prohibition Periods on Video Game
Sales
62
Chapter 6: Discussion and Conclusions
77
Appendices
80
Appendix A: Sample Set Summary Statistics
80
Appendix B: Full Results from Estimates Presented in the Main Text
85
Appendix C: Some Models/Estimates With Alternative Variables and
Alternative Forms
103
Appendix D: Some Models/Estimates with Interaction Terms
115
Bibliography
119
iv
Tables
Table 5.1: Estimates for Model (1) with Heteroskedasticity-Consistent Errors
49
Table 5.2: Estimates for Model (1) without Heteroskedasticity-Consistent Errors
49
Table 5.3: Estimates for Model (2) with Heteroskedasticity-Consistent Errors
51
Table 5.4: Estimates for Model (2) without Heteroskedasticity-Consistent Errors
52
Table 5.5: Estimates for Model (3) with Heteroskedasticity-Consistent Errors
55
Table 5.6: Estimates for Model (3) without Heteroskedasticity-Consistent Errors
56
Table 5.7: Estimates for Model (4) with Heteroskedasticity-Consistent Errors
59
Table 5.8: Estimates for Model (4) without Heteroskedasticity-Consistent Errors
59
Table 5.9: Estimates for Model (5) with Heteroskedasticity-Consistent Errors
60
Table 5.10: Estimates for Model (5) without Heteroskedasticity-Consistent Errors
60
Table 5.11: Estimates for Model (6) with Heteroskedasticity-Consistent Errors
64
Table 5.12: Estimates for Model (6) without Heteroskedasticity-Consistent Errors
64
Table 5.13: Estimates for Model (7) with Heteroskedasticity-Consistent Errors
67
Table 5.14: Estimates for Model (7) without Heteroskedasticity-Consistent Errors
68
Table 5.15: Level Estimates for Model (7) with Heteroskedasticity-Consistent Errors
69
Table 5.16: Level Estimates for Model (7) without Heteroskedasticity-Consistent Errors 70
Table 5.17: Estimates for Model (8)
71
Table 5.18: Level Estimates for Model (8) with Heteroskedasticity-Consistent Errors
73
Table 5.19: Level Estimates for Model (8) without Heteroskedasticity-Consistent Errors 73
Table 5.20: Estimates for Model (9)
75
Table 5.21: Estimates for Model (10) with Heteroskedasticity-Consistent Errors
75
Table 5.22: Estimates for Model (10) without Heteroskedasticity-Consistent Errors
76
v
Table A.1: Summary Statistics on Metascore Data
80
Table A.2: Counts for Dummy Variables Based on Metascore Data
81
Table A.3: Summary Statistics Variables Related to Review Prohibition Periods
81
Table A.4: Summary Statistics for Sales Data
82
Table A.5: Counts for Dummy Variables Based on Console
82
Table A.6: Counts for Dummy Variables Based on Genre
83
Table A.7: Counts for Dummy Variables Based on Triple-A/Obscure Classification
83
Table A.8: Counts for Dummy Variables Based on Exclusive and Sequel Classification
84
Table B.1: Full Estimates for Model (1) with Heteroskedasticity-Consistent Errors
85
Table B.2: Full Estimates for Model (1) without Heteroskedasticity-consistent Errors
85
Table B.3: Full Estimates for Model (2) with Heteroskedasticity-consistent Errors
86
Table B.4: Full Estimates for Model (2) without Heteroskedasticity-consistent Errors
86
Table B.5: Full Estimates for Model (3) with Heteroskedasticity-Consistent Errors
87
Table B.6: Full Estimates for Model (3) without Heteroskedasticity-consistent Errors
88
Table B.7: Full Estimates for Model (4) with Heteroskedasticity-Consistent Errors
89
Table B.8: Full Estimates for Model (4) without Heteroskedasticity-Consistent Errors
89
Table B.9: Full Estimates for Model (5) with Heteroskedasticity-Consistent Errors
90
Table B.10: Full Estimates for Model (5) with Heteroskedasticity-Consistent Errors
91
Table B.11: Full Fill Estimates for Model (6) with Heteroskedasticity-Consistent Errors 92
Table B.12: Full Estimates for Model (6) without Heteroskedasticity-Consistent Errors
92
Table B.13: Full Estimates for Model (7) with Heteroskedasticity-Consistent Errors
93
Table B.14: Full Estimates for Model (7) without Heteroskedasticity-Consistent Errors
94
vi
Table B.15: Full Level Estimates for Model (7) with Heteroskedasticity-Consistent
Errors
95
Table B.16: Full Level Estimates for Model (7) without Heteroskedasticity-Consistent
Errors
96
Table B.17: Full Estimates for Model (8)
97
Table B.18: Full Level Estimates for Model (8) with Heteroskedasticity-Consistent
Errors
98
Table B.19: Full Level Estimates for Model (8) without Heteroskedasticity-Consistent
Errors
99
Table B.20: Full Estimates for Model (9)
100
Table B .21: Full Estimates for Model (10) with Heteroskedasticity-Consistent Errors
101
Table B.22: Full Estimates for Model (10) without Heteroskedasticity-Consistent Errors 102
Table C.1: Log Model of metascore with Heteroskedasticity-Consistent Errors (before)
103
Table C.2: Log Model of metascore without Heteroskedasticity-Consistent Errors
(before)
104
Table C.3: Model of metascore Using Alternative Definitions of Triple-A and Obscure
without Heteroskedasticity-Consistent Errors (before)
105
Table C.4: Model of metascore Using Alternative Definitions of Triple-A and Obscure
with Heteroskedasticity-Consistent Errors (before)
106
Table C.5: Model of initial Using Alternative Definitions of Triple-A and Obscure
(before)
107
Table C.6: Model of global with Heteroskedasticity-Consistent Errors (before)
108
Table C.7: Model of global without Heteroskedasticity-Consistent Errors (before)
109
Table C.8: Model of initial utilizing metascore with Heteroskedasticity-Consistent
Errors (before)
110
Table C.9: Model of initial utilizing metascore without Heteroskedasticity-Consistent
Errors (before)
111
Table C.10: Model of userscore (before)
112
vii
Table C.11: Model of initial utilizing userscore without Heteroskedasticity
Consistent Errors (before)
113
Table C.12: Model of initial utilizing userscore with Heteroskedasticity
Consistent Errors (before)
114
Table D.1: Model of metascore with interaction between before and triple-a and
obscure classification with Heteroskedasticity-Consistent Errors
115
Table D.2: Model of metascore with interaction between before and triple-a and
obscure classification without Heteroskedasticity-Consistent Errors
116
Table D.3: Model of metascore with interaction between magnitude and triple-a and
obscure classification with Heteroskedasticity-Consistent Errors
116
Table D.4: Model of metascore with interaction between magnitude and triple-a and
obscure classification with Heteroskedasticity-Consistent Errors
117
Table D.5: Model of initial with interaction between before and triple-a and
obscure classification with Heteroskedasticity-Consistent Errors
117
Table D.6: Model of initial with interaction between before and triple-a and
obscure classification without Heteroskedasticity-Consistent Errors
118
Table D.7: Model of initial with interaction between magnitude and triple-a and
obscure classification
118
viii
Acknowledgements
First, the author would like to thank all readers of this paper. Furthermore, the author
welcomes comments and may be reached at [email protected]. Additionally, the author
would like to thank the following individuals and groups for their commendable efforts in
making this thesis a possibility:
To the entire faculty of the Department of History of Murray State University. In
particular, the author would like to offer a special thanks to Dr. Marjorie Hilton, Dr. Aaron Irvin,
and Mr. Ted Belue for always challenging my writing skills.
To the entire faculty of the Economics and Finance Department of Murray State
University. In particular, the author would like to offer a special thanks to Dr. Eran Guse, Mr.
Todd Broker, Dr. David Brasfield, Dr. Steven Lacewell, and Dr. Gilbert Mathis for instilling me
with an excellent sense of economic and financial intuition.
To Nintendo for making such incredible games that inspired this author’s interest in not
only video games but also the video game industry.
To John Mayer for making such great music that made writing much less monotonous.
To my best friends, Jaime Staengel and Patrick Burke, for not only being great friends to
talk economics with, but for being great friends in general.
To my parents, Janet and Todd Applin, not only for bringing me into this world (since it
is hard to write a thesis without being born!) but for being such great, loving parents that have
always supported and encouraged both my academic and personal interests.
To my committee members, Dr. Warren Edminster, Dr. Sunayan Acharya, and Dr. Jim
Humphreys for not only agreeing to read a thesis on video games, but for being such wonderful
professors and teachers that have always encouraged and pushed me to be the best possible
student I can possibly be.
Last, but certainly not least, no thesis would ever be completed without a thesis advisor.
And this author could not have asked for a better one. Thus, this author would like to offer a
particularly special thanks to Dr. David Eaton. Not only for allowing me to write an economics
thesis on video games, but for being such a wonderful advisor, professor, and most of all,
mentor.
ix
Abstract
Video game developers typically prohibit media outlets from publishing reviews of
upcoming video games until a specified date and time. This study hypothesizes that review
prohibition periods function as an economic signal of quality and thus impact sales. Specifically,
this study predicts that permissive review embargo periods, relative to games with restrictive
review embargo periods, are indicative of better quality games and in turn better sales. To test
this conjecture the study collected a random sample of observations (video games) from four
popular video game consoles and examined data regarding each observation’s review embargo
period as well as data on their Metascores and sales. Findings demonstrate strong evidence that
review prohibition periods indicate quality of a game. However, findings for sales are extremely
mixed and suggest that review embargo periods may not significantly impact consumer
purchasing decisions. The paper concludes with discussion of potential directions for future
research on similar topics as well as discussion on applications for the findings in the study.
1
1. Introduction and Background
In December of 2013, Hello Games, the humble games studio behind the modestly
successful Joe Danger series, announced No Man’s Sky1 – a game that promised to be not just a
video game but to be the video game. The hype for the game quickly began to build and
skyrocket following the announcement. By June 2016, just two months away from the game’s
official release, the hype for the game had arguably built up to an unprecedented level in modern
video game history. For any consumer of video game news, it seemed as if every other article
discussed Hello Games’ project. As the cliché goes, though, the higher something goes the
harder it crashes – and in July of 2016, a mere month away from the August release, the hype
began to crash. Although there had been talk over the years since the announcement of the game
that it may not meet expectations, no evidence had been unearthed to debase nor to affirm such
fears. The sentiment changed, however, when it became unclear whether Hello Games would be
releasing a review copy to media organizations that publish reviews. 2 In the video game industry,
typical practice sees publishers send out copies of the game to the video game media. The
industry refers to these copies as review copies. The practice allows video game review outlets to
spend some time with a game before release, thus allowing for reviews to be published prior to a
game’s release. Failure to do so generally stirs up fears that a game may not be as good as its
publisher claims it to be; especially games that receive a considerable amount of hype.
Therefore, the possibility that Hello Games may not follow tradition provided legitimacy to the
Martin Gaston, “Space Adventure No Man’s Sky is One of Next-Gen’s Most Ambitious titles,” Gamespot,
December 7, 2013, accessed November 27, 2016, http://www.gamespot.com/articles/space-adventure-no-man-s-skyis-one-of-next-gen-s-most-ambitious-titles/1100-6416605/.
2
Jason Schreier, “The No Man’s Sky Review Debacle,” Kotaku, August 5, 2016, accessed November 27, 2016,
http://kotaku.com/the-no-mans-sky-review-copy-debacle-1784873415.
1
2
fear that the game may not be what the studio had promised. 3 Eventually the studio made clear
that it would be releasing review copies to review outlets. This did nothing to quell fears, as even
though the studio would be releasing early copies, such copies would be sent out considerably
late relative to release. In fact, some review outlets did not receive an early copy until as late as
the eighth of August,4 a mere day before the game’s release on the PlayStation 4 and only three
days before the game’s release on Microsoft Windows. Sure enough, the fears proved true - the
vast majority of reviews for the game proved to be quite mediocre; especially when considered
in light of the amount of hype the game had received.
The story of No Man’s Sky not only provides a lesson about the importance of managing
hype for publishers and developers, but it also serves as the prototype of a signaling problem that
often arises in the video game industry. Although a number of issues could arguably be attributed
to causing the hype for No Man’s Sky to crash and burn, most of the video game media seem to
overwhelmingly consider what Kotaku dubbed the No Man’s Sky “review debacle”5 to be the
source of the massive downward turn that expectations took. The thinking goes, as alluded to
above, the decision to release review copies seems to serve as a signal that the game will be good
quality, or at the very least, decent. A corollary to this line of thinking that No Man’s Sky also
demonstrates pertains to how far from release reviews can be published. Publishers typically
prohibit review outlets from publishing their reviews until a specified date – a date that tends to
be a few days to a week or so before release. The industry refers to this practice as a “review
embargo,” a prohibition period on reviews. Note that some other commonly used terms include
3
Ibid.
Oli Welsh, “Our No Man’s Sky Review Will be Late, and Here’s Why,” Eurogamer, August 8, 2016, accessed
November 27, 2016, http://www.eurogamer.net/articles/2016-08-08-our-no-mans-sky-review-will-be-late-andheres-why.
5
Schreier, “The No Man’s Sky Review Debacle.”
4
3
“news embargos” and “press embargos.” This paper terms the concept a prohibition period, as
the author feels this term is a bit more intuitive in meaning. In some cases, publishers impose
explicit prohibition periods, meaning they set a specific date for review outlets to follow.
However, these prohibition periods can also be implicit in that the publisher releases review
copies so late that review outlets simply do not have enough time to publish their reviews well in
advance of a game’s actual release (No Man’s Sky took the implicit route). The logic seems to
suggest that the more generous the prohibition period, the more likely that a game will be good.
Whatever the case, the video game industry takes the decisions involving review copies and
review prohibition periods very seriously. As video game journalist Stephen Totilo notes,
“publishers are, of course, interested in maximizing sales” – something which publishers have
evidently found to be best done by “increasing pre-release hype rather than enabling better
access for reviews.”6
As noted above, the industry considers review copies and prohibition periods to be key
decisions in maximizing profit. Perhaps, to state it more correctly, the industry thinks review
copies and prohibition periods play a key role. Of course, just because the industry believes
something to be true does not mean that something will indeed prove to be true when put to the
test – one only need to look at the dozens and dozens of articles fawning over No Man’s Sky
before the review copy debacle. Everyone in the industry thought it would be the greatest game
ever made. Instead, it turned out to be a mediocre video game that happened to have tons of hype
behind it prior to its release. The point here, of course, regards the question of whether video
game consumers actually appear to factor in decisions regarding review copies and review
Stephen Totilo, “You’re Going to Get Fewer Early Game Reviews from Everyone,” Kotaku, October 27, 2016,
accessed November 27, 2016, http://www.kotaku.com.au/2016/10/youre-going-to-get-fewer-early-game-reviewsfrom-everyone/.
6
4
prohibition periods when making video game purchasing decisions. This paper seeks to
determine an answer to this question. As the industry’s logic appears to be arguably reasonable,
the hypothesis going in maintains that review embargo periods function as a signal of quality and
that, in turn, consumer expectations (and thus sales) are affected by them. More specifically, this
study predicts that permissive review prohibition periods (i.e., those that permit reviews to be
published prior to release) signal better quality games relative to restrictive prohibition periods.
Before going further, some final points should be addressed for clarity. The most
important pertains to the use of the terms “publisher” and “developer.” In the context of the
video game industry, the publisher manages the business side of things – i.e., manufacturing,
production, marketing, etc. In contrast, the developer only handles the actual development of the
game. However, the publisher and developer can be the same entity in some cases. Regarding the
topic of this paper, it should be noted that generally the publisher handles decisions regarding
review embargoes. Thus, for the sake of clarity, the remainder of this paper will tend to use the
term “publisher” liberally with the understanding that in some cases a publisher can also be the
developer. Additionally, for brevity, the rest of this paper will refer to decisions pertaining to
review embargoes as “review policy.” Specifically, the term “permissive review policy” should
be taken to refer to review prohibition periods that permit reviews to be published prior to
release. Likewise, the term “restrictive review policy” should be taken to refer to review
prohibition periods that forbid reviews to be published prior to release. For information on how
this paper will go about quantifying review prohibition periods, see section 4.2.2.
5
1.1 Some Further Background for the Non-Gamer
Note that the material in this section primarily serves to familiarize the non-gamer with
some more basics of the video game industry. Any person that considers themselves to be a
gamer will probably want to skip the rest of this section – there will likely be nothing too
enlightening here! Otherwise, this material will be very useful for those who call all video game
consoles “Nintendos!”
To begin, one should first know that discussion of video games generally takes place
under the context of “generations 7,” which typically become associated with the gaming consoles
that dominate them (if the term “console” means nothing, fear not, it will further down below).
For example, the Nintendo Wii, Sony PlayStation 3, and Microsoft Xbox 360 dominated the
seventh generation of video games – which lasted from around 2006 to around 2013. Moreover,
the Microsoft Xbox One, Sony PlayStation 4, Nintendo Wii U, and as of recently, the Nintendo
Switch make up the current (eighth) generation.
Now for some terms of the art (or perhaps more appropriately, terms of the hobby). For
those that know absolutely nothing about video games, it might be useful to start with defining
the term “video game console.” Although it might sound fancy, the term simply denotes a
“specialized desktop computer used to play video games.”8 However, in the context that the term
“console” gets thrown around in, this definition is a bit misleading – the term console typically
refers to a proprietary system, or a system that is “under the control of [its] respective
manufacturers and the software is geared towards to the machine’s capabilities [and] is not
7
Kirk Hamilton, "The Games that Defined the Last Generation." Kotaku. November 15, 2013. Accessed April 3,
2017. http://kotaku.com/the-games-that-defined-the-last-generation-1464704318.
8
PC Magazine, “Definition of: Video Game Console,” PC Magazine Encyclopedia, accessed April 14, 2017,
http://www.pcmag.com/encyclopedia/term/53848/video-game-console.
6
interchangeable with other game consoles.” 9 In layman’s terms, a console is simply a computer
that is designed to only play games (and in the case of modern consoles, also run media
applications such as Netflix). They are not designed to run Microsoft Word to use to write a
thesis on video games. An example of a video game console is the Sony PlayStation 4. That bit
about software (i.e., the games) not being interchangeable is important but it will prove useful to
clarify some other points before discussing this fact.
The video game market can be broken down into three core markets: the
Computer/Personal Computer (PC) gaming market, the Home Console gaming market, and the
Handheld/Mobile Console Market. Because the world of handheld gaming is so different
compared to the PC and console markets, this paper does not examine any trends in this market;
hence, no description of handhelds will be given here. With that said, some readers are
understandably wondering about the difference in PCs and consoles. Rather than get bogged
down in the technical differences, it will prove much more helpful to simply provide elementary
examples and descriptions. The author asks the reader to simply think of the computer they own
or perhaps have at work. If the reader installed a game on their computer, then the reader would
be engaging in PC gaming, not console gaming. Even more simply, if it runs Microsoft Windows
or Apple’s macOS, then it is a candidate for PC gaming (provided the hardware on the computer
used has the processing power to handle games – but that constitutes an unnecessary digression).
If it does not run Microsoft Windows or Apple’s macOS and it plays games, then it is probably a
game console. Now, one might be wondering, “but the definition above said that video game
systems do not run Microsoft Word?” That is true – of video game consoles! Personal
Computers are not video game consoles. While PCs may be used to play video games, they can
9
PC Magazine, “Definition of: Video Game Console.”
7
also perform more general tasks, such as running Microsoft Word in order to write a thesis on
video games.
At this point one might wonder, “why not just use a personal computer to play games?” –
Since they can be used for both gaming and general tasks, would it not make more sense to just
use a personal computer? Perhaps, for some users. However, if one wants to play modern games
with impressive graphics (as opposed to Minesweeper), one needs the hardware to do so – which
comes at a price!10 Many consider gaming on consoles to not only be vastly simpler, but also
much cheaper.11
Now recall that that little detail about software being restricted to consoles. To put this in
elementary terms, suppose going out to the store to purchase both a Microsoft Xbox One and a
Sony PlayStation 4. However, assume one only purchases a game from the Xbox One section of
the store. Putting that Xbox One game disc into the PlayStation 4 will yield failure – as it is an
Xbox One game. Thus, one must play it on the Xbox One! But wait, do some games not come
out on multiple consoles? Yes, yes they do! Such games are termed multiplatform or crossplatform games. Moreover, it is commonplace to see console versions of games and PC versions
of games! However, to play a game on a particular console one must purchase that console’s
specific version (i.e., owning an Xbox One copy of a game does not entitle one to a free copy of
the PlayStation 4 version of that game). To complicate matters, some games are actually
exclusive to some consoles. In other words, if one wants to play an exclusive game, then one
must own the platform that game is exclusive to – perhaps the best example is Nintendo’s Mario
franchise. If one wants to play the latest Mario game, one must own the latest Nintendo console,
Dave Thier, “Why I Play on Consoles Instead Of PC,” Forbes, September 03, 2015, accessed April 14, 2017,
https://www.forbes.com/sites/davidthier/2015/09/02/why-i-play-on-consoles-instead-of-pc/.
11
Ibid.
10
8
as the Mario franchise is exclusive to Nintendo consoles. To make things even more confusing, a
recent trend blurs the definition of “exclusive”: many games will be exclusive to one console, but
these very same games will also appear on the PC!12
With the above facts established, now serves as a good time to walk the reader through
how video games generally get developed and released. The typical video game follows a
relatively predictable production process. First, preliminary development begins. If it is decided
that the game might prove to be good, then production continues. At this point the publisher
might choose to go ahead and announce the title to the public – this could be less than a year
away from the game’s actual projected release date or it could be as far as two or three years
prior to the game’s projected release date. For games announced long before the projected
release date, there may still be a good possibility that the game could ultimately be cancelled.
During the next year or so, the publisher might trickle out a few showcases of the game in its
current state to essentially remind the public of its existence as well as to assure the public that
the project has not been cancelled. Once a game starts nearing its release date, publishers
(especially large ones with the money) will begin to roll out a marketing campaign to boost
awareness. During this time publishers will typically (but not always – the core concern of this
thesis!) send out review copies. As discussed earlier, the publisher will generally specify a
review embargo – i.e., a period of time before a certain date set by the publisher in which media
outlets are prohibited from publishing reviews. As stated earlier, these review embargoes may be
explicit or implicit. Finally, the release date will roll around and gamers either buy the game or
Stephen Totilo, “Another Microsoft-Published Xbox One Launch Game Is Coming To PC.” Kotaku, August 7,
2014, accessed April 14, 2017. http://kotaku.com/divisive-xbox-one-launch-game-ryse-is-coming-to-pc-acc1617573860.
12
9
they do not. Of course, this is an incredibly general description of the entire process – the actual
process can vary from game to game. Even so, most games do follow a path similar to the above.
Before wrapping up this introduction to the video game industry, it will prove helpful to
take some time to delineate the terms “triple-A” games (sometimes stylized as “AAA” games)
and “indie games.” Though no formal definition exists, the term triple-A tends to denote games
with relatively large production costs and budgets. 13 Thus, having a publisher tends to be a
prerequisite to produce a triple-A video game. In contrast, indie games tend to be games that are
produced without a publisher. 14 The distinction can get confusing, though. For instance, take No
Man’s Sky. Hello Games – a very small games studio – technically served as both the publisher
and the developer. Thus, it could be considered indie, but this might be a misnomer, though, as
the game generated so much hype that its scope greatly expanded and Hello Games ended up
partnering with Sony for help with distribution. 15 In general, indie games tend to be smaller,
more obscure titles that appeal to niche audiences whereas triple-A titles tend to be titles
designed to appeal to a much wider audience.
This concludes this brief crash course on video games. Perhaps obviously, the above is
not exhaustive in any sense of the word. Furthermore, some finer details were omitted. While
most of these omitted details are not pertinent to any discussion in the text of this paper, some
will be important. Rather than get bogged down in such details here, though, these details will be
presented and elaborated on as becomes necessary.
Rob Fahey, “Charging More for Games Won't Rescue Triple-A,” GamesIndustry.biz, July 5, 2013, accessed
December 1, 2016, http://www.gamesindustry.biz/articles/2013-07-05-charging-more-for-games-wont-rescue-aaa.
14
Mary Jane Irwin, “Indie Game Developers Rise Up,” Forbes, November 20, 2008, accessed December 01, 2016,
http://www.forbes.com/2008/11/20/games-indie-developers-tech-ebiz-cx_mji_1120indiegames.html.
15
Christopher Dring, “PlayStation's Gara Talks Price Wars, Bloodborne, No Man's Sky and Black Friday,” MCV
UK, May 14, 2015, accessed December 02, 2016, http://www.mcvuk.com/news/read/playstation-s-gara-talks-pricewars-bloodborne-no-man-s-sky-and-black-friday/0149460.
13
10
2. Literature Review and Theoretical Framework
Ever since George Akerlof’s groundbreaking work on information asymmetry in 1970,
the subfield pertaining to asymmetric information and economic signaling has received a
considerable amount of attention in economic research. To summarize all of it would be a
daunting, if not impossible, task. Luckily, the most essential points of the literature have come
from a select few seminal and monumental works. Moreover, knowledge of every single
minutiae concept that has been established in theories pertaining to information asymmetry and
signaling will not be needed to develop a conceptual framework for this paper. Hence, this paper
will strive to review only the literature that will prove imperative when putting findings in a
theoretical light.
As alluded to above, the basic core of the issues underlying review policy involves
asymmetric information. Arguably all work done on asymmetric information stems from George
Akerlof’s famous paper “The Market for ‘Lemons’: Quality Uncertainty and the Market
Mechanism.” Akerlof’s paper established the notion that the presence of underwhelming
information about quality of the goods in a particular market can result in a downward turn in
business – i.e., as Akerlof famously put it himself in the context of the automobile market, “good
cars may be driven out of the market by lemons.”16 However, there exists a mechanism in which
firms can prevent and remedy the problems caused by the presence of information asymmetry:
economic signaling, a technique formally conceptualized by Michael Spence which can be best
described as a way in which parties can reliably transmit information to one another. 17 For
George A. Akerlof, “The Market for “Lemons": Quality Uncertainty and the Market Mechanism,” The Quarterly
Journal of Economics 84, no. 3 (1970): 470, accessed November 26, 2016, doi:10.2307/1879431.
17
Michael Spence, “Job Market Signaling,” The Quarterly Journal of Economics 87, no. 3 (1973): 357 accessed
November 26, 2016, doi:10.2307/1882010.
16
11
example, Spence’s work devotes a considerable amount of attention to the way in which
education functions as a signal of a quality worker to employers. 18 The logic behind this pertains
not to the possibility that education might increase productive capacity, but to the fact that the
relatively large investment required for education signifies a worker who may be more likely to
be a good employee. 19 These basic ideas established by Akerlof and Spence will form the basis
for the conceptual framework through which this paper will operate under.
It may not be surprising to find out that there appears to be a lack of scholarly work on
review policy found in the video game industry. What may be surprising, however, concerns the
fact that there exists a plethora of literature which will be useful in the context of this paper on a
problem in finance that helps expand the framework that the work of Akerlof and Spence forms.
Consider thinking of video game consumers as “investors” in a video game and video game
publishers as “managers.” In keeping this analogy in mind, consider the work done on dividends
– which have posed an apparent paradox for researchers ever since Merton Miller and Franco
Modigliani propositioned that dividends should be irrelevant. 20 Although there have been
countless studies attempting to solve this paradox, Asquith and Mullins’ research provide
considerably useful insights. Particularly, they contend that dividends function as a quality signal
to investors.21 In doing so, Asquith and Mullins address the reasons why managers might opt to
use dividends as a signal rather than “postcards.”22 The findings of their work suggest that
investors consider dividends to encapsulate much more quality information than would a letter
18
Ibid., 364.
Ibid., 357.
20
Merton H. Miller and Franco Modigliani, “Dividend Policy, Growth, and the Valuation of Shares,” The Journal of
Business 34, no. 4 (1961): 414, accessed November 26, 2016, doi:10.1086/294442.
21
Paul Asquith and David W. Mullins, “Signaling with Dividends, Stock Repurchases, and Equity Issues,”
Financial Management 15, no. 3 (1986): 27, accessed November 26, 2016, doi:10.2307/3664842.
22
Ibid., 35.
19
12
from the firm in question reassuring investors that the business continues to do well. 23 Applying
these findings to the signaling issue found in the video game industry forms a nice foundation for
why the problem exists in the first place: video game consumers, like investors, want and require
a quality signal in order to invest in – read, purchase – a video game. If one finds this conclusion
to be trivial it may be helpful to consider how standard marketing (e.g., the “postcard”) arguably
constitutes enough to sway consumers in other product areas – meaning, the fact that video game
firms may need to take additional measures that transcend marketing makes this a unique issue.
One important feature of this paper regards the fact that review policy constitutes a way
in which for video game firms to influence and control a third-party signal: reviews. Third-party
signals differ from standard signals in that the parties they affect do not directly control them.24
The reasoning behind why consumers might place more confidence in third-parties concerns the
fact that first-parties – perhaps by nature – cannot be regarded as being disinterested when
transmitting information. 25 Thus, third-party signals have been found to be crucial in establishing
and maintaining consumer trust in some markets. 26 The idea, of course, being that third-parties
should presumably be less biased, thereby giving them an advantage in facilitating consumer
trust in the market.27 However, this facilitation of trust comes at a cost for the first-party
producers: third-party signals appear to dilute the power of a firm’s own first-party signals.28
Moreover, third-party signals can sometimes actually end up having an negative impact of firm
23
Ibid., 36.
Susan P. Shapiro, “The Social Control of Impersonal Trust,” American Journal of Sociology 93, no. 3 (1987):
641, accessed November 26, 2016, doi:10.1086/228791.
25
Ibid.
26
Paul A. Pavlou and David Gefen, “Building Effective Online Marketplaces with Institution-Based Trust,”
Information Systems Research 15, no. 1 (2004): 38, accessed November 26, 2016, doi:10.1287/isre.1040.0015.
27
Ibid.
28
M. Billur Akdeniz, Roger J. Calantone, and Clay M. Voorhees, “Signaling Quality: An Examination of the Effects
of Marketing- and Nonmarketing-Controlled Signals on Perceptions of Automotive Brand Quality,” Journal of
Product Innovation Management 31, no. 4 (2013): 740, accessed November 27, 2016, doi:10.1111/jpim.12120.
24
13
sales.29 Hence, firms that deal with products in which consumers rely heavily on third-party
information in making their purchasing decisions have an incentive to develop a clear
understanding of the ways they can control and influence such third-party information. Indeed,
firms can potentially stand to earn considerable returns by earning the approval of third-parties.30
One question that remains, however, pertains to ways in which firms can influence the effect of
third-party signals. The research conducted by Akdeniz and Voorhees point to brand reputation
as one of the stronger “moderators” against third-party signals. 31 This makes sense intuitively: if
a firm has built up an exceptional reputation, then consumers would likely have less of a
motivation to pursue third-party information about the firm’s products.
Before moving on, it may be helpful to quickly recap the essential parts of the literature
this paper builds on. The core of the issue in the video game industry goes back to Akerlof’s
concept of asymmetric information, a situation that may put the video game market at risk of
declining32 unless actors in the market utilize signals – ways in which to transmit information. 33
In expanding on this, one can think of the problem in terms of investor-management issues.
Thinking through the issue in this perspective makes it easy to imagine that consumers may be
placing a higher premium on non-marketing signals a firm utilizes – similar to how investors
appear to place a higher premium on dividends than they do on “postcards” from a firm. 34
Finally, the issues pertaining to review policy involve the publisher having to face the presence
Akdeniz, Calantone, Voorhees, “Signaling Quality,” 740.
Guodong (Gordon) Gao, Anandasivam Gopal, and Ritu Agarwal, “Contingent Effects of Quality Signaling:
Evidence from the Indian Offshore IT Services Industry,” Management Science 56, no. 6 (2010): 1012, accessed
November 27, 2016, doi:10.1287/mnsc.1100.1162.
31
M. Billur Akdeniz, Roger J. Calantone, and Clay M. Voorhees, "Effectiveness of Marketing Cues on Consumer
Perceptions of Quality: The Moderating Roles of Brand Reputation and Third-Party Information," Psychology &
Marketing 30, no. 1 (2012): 87, accessed November 27, 2016, doi:10.1002/mar.20590.
32
Akerlof, “The Market for Lemons,” 470.
33
Spence, “Job Market Signaling,” 86.
34
Asquith and Mullins, “Signaling with Dividends,” 27.
29
30
14
of third-party signals which may pose a threat to earnings. 35 However, despite the literature that
does exist on third-party signals, it should be noted this remains a relatively new area of interest
in the broader field of economic signaling. As such, there remain gaps in the knowledge in the
literature pertaining to this subfield. In particular, even with Akdeniz and Voorhees’ research on
brand reputation, there appears to be a glaring lack of literature that observes the way in which
firms interact with third-party signals. Thus, this will constitute a primary area in which this
paper hopes to expand. Recall that this paper will hypothesize that review policy impacts
consumer purchasing decisions in the market for video games. However, note that this actually
constitutes an indirect impact, as the direct impact actually pertains to the producers of video
game reviews – i.e., the third-party signals. Thus, this paper will hopefully provide evidence as
to whether firms may actually be able to impact third-party signals.
35
Akdeniz, Calantone, and Voorhees, "Signaling Quality,” 740.
15
3. Economic Theory and Review Prohibition Periods
Traditional economic theory assumes transactions take place under the presence of
perfect information – i.e., essentially every party in the transaction knows every relevant detail
required about the resource being allocated in the transaction so that a rational decision can be
made by each party. Developments in economic theory over the past few decades have
consistently challenged this notion of perfect information. Indeed, as alluded to in the literature
review, the major underlying economics in this study concerns asymmetric information:
publishers know information – quality of their game – that gamers do not. It should be noted that
this study takes it as given that information asymmetry exists in the market for video games; i.e.,
proving its presence in the video game market will not be a concern.
One unfamiliar with this concept might wonder why this would constitute a problem.
The answer, hinted at earlier, lies in Akerlof’s work. Should information asymmetry go
unresolved, a market will find itself at the risk of adverse selection, which can be defined as the
tendency for there, “to be a reduction in the average quality of goods and also in the size of the
market” due to sellers having an incentive to “market poor quality merchandise.” 36 While it
might be interesting to assess whether adverse selection has permeated the market for video
games – some pundits in the industry would certainly jump at the chance to argue that most
games produced these days have remarkably low standards compared to games of the past – this
does not constitute the core focus of this study. Rather, this study concerns itself with the
methods by which the video game industry attempts to prevent and/or alleviate adverse selection
that stems from the presence of information asymmetry. Akerlof concluded his seminal paper by
36
Akerlof, “The Market for Lemons,” 488.
16
noting that there do potentially exist, “counteracting institutions,” 37 that can help to deafen the
effects of asymmetric information. However, Akerlof did not provide the formal names for such
institutions. Instead, the names of these institutions, “signaling” and “screening,” and their
respective frameworks would be pioneered by Michael Spence in his seminal “Job Market
Signaling” paper (see literature review) and by Joseph Stiglitz 38 respectively. For the purposes of
this study, discussion of screening will be omitted since it will not be pertinent to the issue being
examined.
Signaling resolves the problems of asymmetric information by permitting the party with
more information to transfer information to the party with less information in a credible manner.
In general, signaling tends to be utilized primarily by sellers of a good as opposed to buyers of a
good. These theories lead to the overarching question of this study: do review prohibition periods
serve as a signal of video game quality? This question can be broken down into two core
components. First off, given information on a review prohibition period and its expiration date
for a particular game, can the quality be predicted? Second, if the answer to the first question
turns out to be yes, do gamers actually respond to the signal – i.e., does the signal appear to have
an impact on sales? Although these two questions might appear to be the same, they do indeed
differ albeit in a subtle manner. To see why, suppose it turns out that quality can be predicted
from review prohibition periods, this does not necessarily mean gamers realize this and thus it
does not mean that gamers necessarily respond to the signal.
Akerlof, “The Market for Lemons,” 499.
Joseph E. Stiglitz, and Bruce C. Greenwald, “Externalities in Economies with Imperfect Information and
Incomplete Markets,” The Quarterly Journal of Economics 101 no. 2 (May 1986): 229–64, accessed February 27,
2017, doi: 10.2307/1891114.
37
38
17
Going further, this study will consider some of the more recent work on signaling
concerning the concept of “first party” signals and “third party” signals. As their names suggest,
first-party signals come from the actual seller (for this study, the publishers) while third-party
signals come from parties not directly involved in the transaction (for this study, the reviewers).
Although this might seem trivial and perhaps even obvious, by distinguishing the two one can
get a clearer idea of what exactly gives a signal its power. Furthermore, as noted earlier, the
publisher vs. video game reviewer scenario presents an interesting opportunity to potentially
examine how first parties attempt to deafen and/or influence third parties: the review prohibition
period functions as a restraint on the power reviewers hold as a signal, hence, its effect as a
signal may open up questions concerning the value of the methods by which first-party can
mitigate the effect of third-parties.
To sum up, this study will ask the question of whether review prohibition periods serve as
a signal of video game quality. In turn, this study also ask whether review prohibition periods
affect sales. This study hypothesizes that review prohibition periods should function as signals of
quality and in turn impact sales. More specifically, this study conjectures that permissive review
policy, relative to restrictive policy, functions as a signal of better quality games and in turn
better sales.
18
4. Data and Methodology
The data for this project comes from two main sources: VGChartz (a firm/website that
specializes in producing estimates of video game sales) and Metacritic (a website run by CBS
Interactive that specializes in aggregating review scores for video games and producing a
Metascore – which will be described in more detail below). It should be noted that both of these
sources have their flaws, which will be laid out below. Even so, they very likely represent the
best possible sources for their respective specialties. While other potential sources certainly exist
– particularly in regards to review score aggregation – no other sources likely hold the influence
these two hold given how much one will see each cited in industry news. Hence, the thinking
here goes that if the industry considers these sources to be valid enough to be the standard, then
these sources should be the sources utilized. Before delving into the particulars of constructing a
data set, it will be best to go ahead and describe the sources in a bit more detail.
4.1 Sources
4.1.1 VGChartz
As stated above, VGChartz specializes in producing estimates of video game sales. They
do so by utilizing a variety of methods, including, “passively polling end users [and] retail
partners,” fitting statistical and historical trends, “studying resell prices,” and “consulting
publishers.”39 Additionally, the firm notes that they check their data against manufacturer
shipments as well as against data from other sales tracking firms. 40 As for any further details, the
company (unsurprisingly) does not give much else to go on in terms of how they produce their
numbers. Even so, this information alone tips off a few potential, albeit likely minor, issues. First
39
40
“VGChartz Methodology,” VGChartz, accessed March 14, 2017, http://www.vgchartz.com/methodology.php.
Ibid.
19
off, the data VGChartz produces amount to nothing but estimates – not precise figures. Only
publishers will ever truly know exact sales numbers. This might beg the question of why not go
straight to the source then – some publishers occasionally release sales data, so why not use that?
The answer concerns a number of reasons. For one thing, as a practical matter, it would prove
difficult to hunt down sales data for each individual game in a random sample containing games
made by a variety of publishers! Hence, resorting to a tracking firm such as VGChartz makes a
project like this much more feasible. Second, publishers do not always release sales data. If this
project relied on publisher provided data a massive amount of missing values would likely mount
up. Third, and perhaps most importantly, publishers themselves (including large ones) utilize the
services done by firms like VGChartz. In fact, VGChartz notes that publishers such as Sony
Computer Entertainment and Square Enix (for the uninitiated, these would count as considerably
major/notable clients in the industry) use the professional services the firm offers.41 The logic
goes that if some major publishers choose to utilize the firm’s tracking services, then the data the
firm produces must have considerable integrity and accuracy. Thus, taking these points into
consideration, VGChartz likely stands to be one of the best choices available to get sales data.
4.1.2 Metacritic
Like VGChartz, Metacritic comes with its own unique baggage. Recall from above that
Metacritic specializes in aggregating reviews for a variety of media, including video games, and
producing a single score – the Metascore, which is between 0 and 100 - for each title. One minor
detail to keep in mind is that Metacritic produces a Metascore for only games that Metacritic can
find at least four reviews for. Metacritic produces this single score by scouring publications –
both physical and digital – for reviews and then taking a weighted average of the review scores
41
“Market Intelligence for the Videogame Industry,” VGChartz Pro, VGChartz, Ltd, accessed March 14, 2017.
20
found.42 The fact that the score amounts to a weighted average proves to be a double-edged
sword in a sense. On the one hand, as Metacritic notes, some publications consistently publish
high-quality reviews and thus deserve the added weight when compared to less mature
publications. However, this fact obviously begs the question of how Metacritic goes about
weighting critics and publications. This question presents some trouble, as Metacritic’s answer
amounts to “we are not going to tell and we are never going to tell.” 43 Thus, one must trust that
Metacritic employs a rigorous and fair method in determining the weight each critic and/or
publication receives. Considering Metacritic chooses to be secretive about this arguably
important question, one might ask why not choose another source – for instance, those familiar
with the industry may know of OpenCritic, a new competitor of Metacritic’s that came onto the
scene a few years ago. The reasoning for the decision to utilize Metacritic despite this potential
problem parallels the logic of choosing VGChartz despite its potential issues: Metacritic
represents one of the main industry metrics for video game firms in deciding whether to consider
a game successful or not. Indeed, whether for better or for worse, a game’s Metascore can be
make or break for the team responsible for developing the game; often bonuses and, in extreme
cases, people’s jobs, ride on the Metascore a video game receives. 44 In other words, the industry
places enough trust (again, for better or for worse) in Metascores to make relatively important
business decisions upon them. The same cannot be said for scores produced by competitors such
as OpenCritic. Hence, it would seem to make the most sense to collect data from the de facto
industry standard for reviews.
“How do you compute METASCORES?” Metacritic Frequently Asked Questions, accessed March 27, 2017.
http://www.metacritic.com/faq#item11.
43
Ibid.
44
Jason Schrreier, “Metacritic Matters: How Review Scores Hurt Video Games.” Kotaku, August 8, 2015, accessed
March 27, 2017, http://kotaku.com/metacritic-matters-how-review-scores-hurt-video-games-472462218.
42
21
4.2 Variables
Now that the sources have been established in more detail, it stands as a natural step to
delineate the variables – i.e., the information – that this study actually pulls from them. Some
variables will prove self-explanatory. However, others will likely require a bit more attention.
The following discussion will go over the variables in three phases: first, the information pulled
from VGChartz will be described. Then the discussion will move towards a description of the
variables pulled from Metacritic. Finally, this section will conclude with discussion of some
other variables that this study defined. Note that in the listings that follow, the actual variable
name assigned is listed in italics in parentheses. For the interested reader, summary statistics for
a number of variables are available in Appendix A.
4.2.1 Variables From VGChartz
From VGChartz, this study collected eleven core variables: the title of a game (game), the
platform a game was released on (platform), the year a game was released in (year), the genre of
a game (genre), a publisher of a game (publisher), the sales in millions of units of a game in
North America (NA), the sales in millions of units of a game in Europe (Eur), the sales in
millions of units of a game in Japan (Japan), the sales in millions of units of a game in the rest of
the world (ROW) of the game, and the total global sales in millions of units of a game (global),
and a game’s first-week initial sales in number of units (initial). Most of these do not require
much discussion. However, it may be helpful to comment on the genre variable as well as the
sales variables.
VGChartz assigns to each game one of twelve genres: action, adventure, fighting, “misc,”
platformer, puzzle, racing, role-playing, shooter, simulation, sports, and strategy. Any gamer
reading this is probably already opening their mouth to note that these are very broad genres.
22
Indeed, one could arguably break down the role-playing genre down into countless sub-genres,
such as Western role-playing games, Japanese role-playing games, first person role-playing
games, third-person role-playing games, etc. Sitting down to assign the most appropriate subgenre to each title would be time consuming (and likely not all too enlightening in terms of what
doing so would add to results!) Hence, in the interest of completing this study within this
millennium, it proved natural to stick with the genres VGChartz uses. In order to make use of the
genres in terms of quantitative analysis, this study converted each genre into its own dummy
variable. For example, the “racing” dummy variable means that if VGChartz classified the game
as a racing game, then the variable would take on a value of “1”. Otherwise, it would take on a
value of “0.” The only genre that has a slightly different variable name is role-playing, the
variable for which is denoted as rpg for short. For the most part, most genres are represented
quite well – however, a few dominate and a few are a bit underrepresented in the sample. For
instance, action games dominate with 76 observations being classified as action-games. In
contrast, the sample contains only two puzzle games and four simulation games. For more details
on the makeup, as noted in the beginning of this section, the reader can find a plethora of
summary statistics in Appendix A.
Similar to genres, the platforms for games were converted to dummy variables. These
dummy variables are named accordingly. For example, the dummy variable xboxone simply
denotes whether a game is an Xbox One game or not. If a game is an Xbox One game, this
variable takes on the value “1.” Otherwise, it takes on a value of “0.”
As for the sales data, it will be crucial to interpret the values correctly. First, one needs to
note that the total sales data values - North America, Europe, Japan, rest of the world, and global
– are in terms of millions of units (where units are simply copies of the game). For example, say
23
the North American sales value for a particular game is 0.31. One would interpret this as
meaning 310,000 copies of this game were sold in North America. Furthermore, one should note
that the value for global sales is simply the sum of the North America, Europe, Japan, and rest of
the world sales. In contrast, initial sales (initial) values are interpreted slightly differently – they
are interpreted as their number (i.e, not in millions). Thus, a value of 132,000 means that a game
sold 132,000 copies in its first week of its release. Something to note here concerns the
importance of the variable initial. The core focus of this paper pertains to review prohibition
periods – which essentially determine whether consumers can consume reviews prior to release
or not. However, consider a consumer that is thinking about purchasing a game a year after its
release. Does it make sense that the game’s review prohibition period would factor in to this
consumer’s ultimate decision? Probably not, as this consumer will likely have a plethora of
resources (i.e. reviews) available to them to determine this game’s quality! In other words, it is
logical to think that review prohibition periods should only be a factor for initial sales. Thus, the
models in the main text will utilize the variable initial as opposed to the other sales variables.
4.2.2 Variables From Metacritic
Moving on to Metacritic, this study collected ten basic variables from the review
aggregation site: the total number of reviews Metacritic had listed for a game (NumOfReviews),
the Metascore (metascore) for the game, the Meta-User Score for the game (userscore), the
highest review score Metacritic had listed for the game (high), the lowest review score Metacritic
had listed for the game (low), the number of positive reviews Metacritic listed for the game
(positive), the number of mixed reviews Metacritic listed for the game (mixed), the number of
negative reviews Metacritic listed for the game (negative), the release date Metacritic listed for
the game (releasedate), and the publication date for the very first review for a game (first). Since
24
earlier discussion described metascore, the following discussion will refrain from rehashing it
(simply recall it acts as a weighted average of all the review scores Metacritic lists). With that
said, some variables were created based on the variable metascore. Metacritic considers a
Metascore of 75 and above to be indicative of mostly positive reviews, a Metascore between 50
and 74 to be indicative of mostly mixed reviews, and a Metascore below 50 to be indicative of
mostly negative reviews. Hence, it was possible to define the following variables: msabove75,
ms50to74, and msbelow50. As their names suggest, they simply denote what category a
Metascore falls under. For instance, if a game has a Metascore of 88, msabove75 will take on a
value of “1” while ms50to74 and msbelow50 will take on a value of “0.” As for the other
variables taken from Metacritic, it will be useful to describe some of them in a bit more detail.
To begin, consider the variable userscore. As the name suggests, it is simply the average
score internet users (i.e., not professional critics) of Metacritic give an observation. Like many
review websites, users can chime in and prescribe their own rating to a game (on Metacritic, a
user may rate a game on a scale of 1 to 10 with 1 being the worst and 10 being the best). The
usefulness of this variable is questionable given that internet users rarely tend to be a good metric
for anything (yes, as shocking as it is, internet users tend to not be the most authoritative and
trustworthy officials on a subject). Hence, this variable was more or less recorded for
completeness purposes. This study did use Meta-User score along with Metascore in order to
create a variable that attempts to reflects both the feelings of professional critics as well as actual
consumers: average. As the name implies, average is simply the average of metascore and
userscore.
The next few variables function as a simple means to get an idea of the variance in
review scores: %positive, %mixed, and %negative. These variables are based on the variables
25
NumOfReviews, positive, mixed, and negative. Specifically, %positive denotes the percentage of
positive reviews out of the total number of reviews for a game, %mixed the percentage of mixed
reviews out of the total number of reviews for a game, and %negative the percentage of negative
reviews out of the total number of reviews for a game. For example, suppose a game has 100
reviews listed on Metacritic. Furthermore, suppose 50 of these are positive. Then, the value of
%positive would be 0.50. Again, these variables simply provide an idea of the variance of
reviews for a game.
The final two variables that this study pulls from Metacritic are arguably the most
important in terms of answering this study’s core question: the release date of the game (release)
and the date of the first published review (first). To see why, recall that this study asks whether
or not review prohibition periods serve as a signal of quality and whether consumers respond to
them if they are a signal. Unfortunately, however, there does not appear to be a source from
which to gather hard data on specific review prohibition periods for games. In articles about
review embargoes one might occasionally see a specific embargo date mentioned but such
articles are far and few in between. Even so, it is possible to proxy this variable. Suppose a
particular game’s review prohibition period was such that reviewers would be permitted to
publish a review prior to the game’s release date. Then, it seems logical that the first review
should come prior to the release date. In other words, the date of the first review published
should serve well in determining whether the prohibition period for a game was restrictive or not.
In order to create quantitative values conducive for analysis, this study used the release date and
date of the first review to create three key variables: a variable that proxies the directional
magnitude of the review prohibition period (magnitude), a dummy variable that proxies whether
the prohibition period permitted reviews to be published prior to release (before), and a dummy
26
variable that proxies whether the prohibition period restricted reviews from being published until
after release (after) – note that after is simply the inverse of before.
Because of the importance of magnitude, before, and after, it will be useful to describe
them in a bit more detail. First, consider magnitude. Again, this simply proxies the directional
magnitude of the review prohibition period. To create this variable, this study subtracted the date
of the first review from the release date (in terms of calendar days). For example, suppose a
game released on January 7th, 2017 and its first review was published on January 1 st, 2017. Thus,
one subtracts January 1st from January 7th to get a value of -6 days. One interprets this as
meaning, for this game, that the first review was published six days before the release date. In
other words, the negative direction indicates the first review for a game was published before the
game’s release date. Similarly, a positive direction (or a value of zero) indicates the first review
for a game was published on (or after) the game’s release date. To simplify this, one might think
of this in terms of an analogy to golf: negative values are “good” in a sense, while positive
values are “bad” in a sense.
Once one understands the meaning of the variable magnitude, then the meaning of the
dummy variables before and after become relatively easy to grasp. If the direction of magnitude
for a game is negative, then before takes on a value of “1” for that game (because the negative
direction indicates the first review was published prior to that game’s release date). Otherwise, it
takes on a value of “0.” Similarly, if the direction of magnitude is non-negative, then after takes
on a value of “1” (because the non-negative direction indicates the first review was published on
or after the game’s release date). Otherwise, it takes on a value of “0.” Figure 4.1 below provides
a visual to help underscore the meaning of these variables.
27
Figure 4.1: Visualizing magnitude, before, and after
-
reviews before release
release date = 0
reviews after release
magnitude is negative
before = 1
after = 0
magnitude = 0
before = 0
after = 1
magnitude is positive
before = 0
after = 1
+
magnitude (in days) = date of first review – release date
Figure 4.1 illustrates that if the first review of a game is published before a game’s
release date, then magnitude is negative and before = 1 and after = 0. Similarly, if the first
review of a game is published after a game’s release date, then magnitude is positive and
before = 0 and after = 1.
4.2.3 Other Variables
Using some of the variables outlined above, this study was able to define some other
metrics. Recall from the introduction that one characteristic that may be of interest concerns
whether a game is a triple-a game (i.e., a blockbuster game with a large budget). The best way to
classify a game as “triple-A” or otherwise would likely concern a game’s budget. Unfortunately,
there does not seem to be any source that provides data on video game budgets (and publishers
rarely release such data themselves). However, there do exist means in which to proxy whether a
game is triple-A or not. First, if it has not become obvious to the reader thus far, this author is a
life-long gamer. Given this, the author can provide an opinion of whether a game is triple-A or
not. Taking this author’s opinion into account yielded the dummy variable tripleAO (short for
Triple-A-Author’s-Opinion). For this dummy variable, if the author felt a game was triple-A, the
observation got assigned a value of “1” for this variable. Otherwise, it received a value of “0.”
For the record, using the author’s opinion resulted in there being 81 triple-a titles (out of 269
28
observations in the sample – see the subsection 4.5 “Constructing the Data Set” below for details
on the creation of the actual sample set).
Admittedly, using the author’s opinion as the sole metric for classifying a game is likely
poor. Thus, this study sought a few metrics to do so. Consider the following logic: if a game is a
triple-A game, one might expect it to end up having many reviews on Metacritic (since the game
should have a large amount of interest). Thus, this study used the total number of reviews a game
received as a means to proxy whether a game was triple-A. First, some basic statistics – out of
the 269 observations in the sample set, the average number of total reviews for the sample was
34.72 reviews, the median 31 reviews, the high 102 reviews, the low 4 reviews, the 75th
percentile 48 reviews, and the 25th percentile 17 reviews. Using the above logic, this study
reasoned that it would be likely for triple-A games to yield a total number of reviews on
Metacritic at or above the 75th percentile. Hence, this study based the dummy variable triplea75q
(short for Triple-A-75th-Percentile) on the total number of reviews a game had listed on
Metacritic. If this number is at or above the 75th percentile value (48), then the variable
triplea75q takes on a value of “1.” Otherwise, the variable triplea75q takes on a value of “0.”
Using this metric, there ended up being 65 games being classified as triple-A games in the
sample. Moreover, this metric agreed with the author’s opinion for about 200 observations (out
of 269).
One other way of classifying a game as triple-A might be to look at its publisher – since
some publishers consistently publish more triple-A games than others. However, this proved
ineffective in practice. Without getting into the finer details, defining triple-A games in this way
resulted in more than 110 games being classified as triple-A (note that this differs greatly from
29
the metrics that use the author’s opinion and the 75th percentile of number of reviews.) Hence,
this study felt this ended up being a poor way to define triple-A.
Another interesting characteristic one might consider pertains to whether or not the game
is “obscure.” Note, an alternative term might be “indie,” but the author feels this would
constitute a misnomer. By obscure, this study simply means that a game might have a
considerably niche audience. As a result, obscure games might risk introducing some skewness
into the data. However, if one can come up with a way to classify a game as “obscure,” then that
skewness can perhaps be filtered out. Of course, this means having to come up with a way to
actually define obscureness. To do so, this study simply repeated the process above that was used
in defining triple-A games. First, the author classified each observation as obscure or not using
his own knowledge (this resulted in the obscureAO dummy variable – short for ObscureAuthor’s-Opinion). Second, for a more quantitative based metric, the study looked towards the
25th percentile of total number of reviews for the sample set. The logic here goes that if triple-A
titles generate a high amount of interest (and thus a relatively large number of reviews), obscure
games should generate a low amount of interest (and thus a relatively low number of reviews).
Hence, if a game had a number of reviews less than or equal 25 th percentile of the sample (17
reviews), then the dummy variable obscure25q – short for Obscure-25th-Percentile – would be
assigned the value of “1.” Otherwise, the variable would be assigned the value “0.”
Finally, it might be worth examining whether a game is exclusive to a particular console
and also whether a game is a sequel. Naturally, this study takes care of this by converting these
concepts to dummy variables: exclusive and sequel respectively. For example, if exclusive takes
on a value of 1 for an observation, then that observation is exclusive to only one console.
Although determining whether a game was an exclusive did not provide much of a challenge,
30
determining whether a game should count as a sequel, believe it or not, proved a bit ambiguous.
For instance, many older series occasionally attempt to revive themselves by “rebooting” the
series (similar to the movie industry). Should these games be considered sequels? In answering
this question, it might be useful to think about the following question: Assuming being a sequel
matters, what might explain why being a sequel matters? Although this is more or less a guess,
one answer to this might be that sequels matter because people know the franchise – they know
what to expect. Hence, even though a particular title might be a reboot, it still offers familiarity.
For this reason, this study classified any game that was not the very first entry in its series as a
sequel.
4.3 Philosophy
Prior to detailing the criteria utilized in constructing a sample data set to analyze, it will
prove useful to consider the philosophy that will guide the establishment of such criteria. The
primary guiding principle for constructing a data set for this study concerns what this study
deems the “average gamer:” Ideally, the sample data set will contain observations (i.e., video
games) that reflect the type of games the “average gamer” might consume. Of course, in
employing this philosophy, it will be pertinent to come up with an idea of what defines the
“average gamer.”
Perhaps unsurprisingly, it proves somewhat difficult to define the average gamer due to
the subjective nature of the idea. Luckily, there exist some statistics that can help pin down some
basic characteristics that make up the average gamer. According to the Entertainment Software
Association (a trade association for the U.S. video game industry) 56% of gamers play video
31
games on a PC and 53% of gamers play video games on a dedicated gaming console. 45 This
would suggest that computer gaming slightly dominates console gaming. However, digging a
little deeper reveals that in terms of software sales, console video game sales (in terms of units
sold) have consistently been more than approximately 5000% higher than PC video game sales.
Thus, it would seem that the average gamer consumes most video games on a dedicated console.
To extend the definition of the average gamer, it may prove helpful to look at the
distribution of console video game sales in terms of genre. In 2015, the ESA broke video games
into the following genres and listed each genre’s respective percentage of total console sales:
strategy (3.8%), sports (13.2%), shooter (24.5%), role-playing (11.6%), racing (4.1%), other
(0.8%), action (22.9%), adventure (7.7%), casual (0.9%), family entertainment (3.6%), and
fighting (6.7%).46 These figures demonstrate that there does not appear to be one genre that one
could pin down as defining the average gamer.
Additionally, one might want to consider whether the average gamer plays on a specific
console. At first the obvious solution to determining this would be to look at total units sold of
each console. This, however, would not provide the full story. Consider this: the Sony
PlayStation 2 (which first came out in 2000) takes the cake for the best selling video game
console of all time47. Does this mean that the average gamer in 2017 plays most games on the
PlayStation 2? Probably not. This author would guess that a good number of the PlayStation 2s
out in the wild sit in attics across the world. To get an idea on the consoles that the average
gamer games on, it becomes necessary to get some more specific metrics concerning how people
“2016 Essential Facts About the Computer and Video Game Industry,” Entertainment Software Association
Industry Facts, accessed April 12, 2017, http://www.theesa.com/wp-content/uploads/2016/04/Essential-Facts2016.pdf.
46
Ibid.
47
“Platform Totals”, VGChartz Ltd., accessed April 12, 2017, http://www.vgchartz.com/analysis/platform_totals/.
45
32
play video games. Based off of a 2014 Nielsen report, it appears that people spend most of their
time playing console games on either a seventh generation console (Xbox 360, PlayStation 3,
and Wii) or an eighth generation console (Xbox One, PlayStation 4, and Wii U – note at this time
the recently released Nintendo Switch did not exist yet). 48 Thus, the average gamer arguably
tends to game on either a seventh generation or eighth generation console.
The above has strived to give a rough sketch of the “average gamer” – a person whom
this study strives to reflect in its sample data set. To recap, the average gamer appears to be a
person that spends most of their time gaming on a seventh and/or eighth generation console.
Moreover, the average gamer does not appear to flock to any single particular genre in an
overwhelming fashion. Note that the reason for taking the time to distinguish the concept of the
average gamer concerns the overarching principle this study used in constructing a sample data
set. In the criteria and restrictions that follow, many decisions made during this study resulted in
part from the ideal of reflecting the average gamer.
4.4 Initial Criteria
With the sources, variables, and overarching philosophy established, discussion can now
move towards that of the criteria used in determining what observations to include in any sample.
The very first question in constructing the sample data set asked a very simple question: what
video game consoles should observations be taken from? In other words, how far back should the
dataset go? Although it might be interesting to collect observations that date all the way back to
the first generation of video games (Pong, anyone?), one needs to take into account the infancy
of the industry (as well as the video game review industry) at the time. In regards to the review
Nielson, “Multi-Platform Gaming for the Win!,” Nielsen Newswire: Media and Entertainment, May 27, 2014,
accessed April 12, 2017, http://www.nielsen.com/us/en/insights/news/2014/multi-platform-gaming-for-thewin.html.
48
33
prohibition period concept, a natural question one might ask concerns when the whole process
became a norm in the industry - as this might appear to be the obvious start date for observations.
Unfortunately, this proves somewhat difficult to answer. One of the earliest mentions in a video
game journalistic context dates back to 2006 (the year the seventh generation began). 49 This
implies the practice most certainly existed, at the very least, a few years before that – or in other
words, existed in some form during the fifth generation of video games. With this in mind, this
author stands fairly confident in claiming that the concept had come to be an industry norm by
the time the sixth generation began. Operating under this claim, it makes sense to only consider
video games released during or after the seventh generation (or, more simply, video games
released during or after 2006).
Even with a starting year for observations established, a few more important issues
pertaining to restrictions on data collection remain. The above discussion obviously omitted any
mention of the PC. Some people might argue that PC hardware defines gaming generations – and
not consoles (for those interested, a quick Google search of something akin to “PC vs. Console”
will yield a massive amount of internet debates on the topic). With that said, first recall that the
average gamer seems to play more console games than PC games. Hence, it seems permissible to
ignore the PC in terms of the average gamer ideal. However, there exist further reasons than just
this study’s overarching goal for a data set to exclude PC games than: pulling some observations
from the PC would present unique challenges that may very well skew results. In the world of
console gaming, getting one’s game onto a particular console requires a relatively considerable
amount of effort. Not only do publishers face licensing fees, they also must meet certain quality
Joystiq Staff, “Joystiq video review: LEGO Star Wars II (update 1),” Engadget, September 7, 2006, accessed
April 3, 2017, https://www.engadget.com/2006/09/07/joystiq-video-review-lego-star-wars-ii-update-1/.
49
34
standards set by the respective console manufacturer. 50 In other words, the flash game someone’s
best friend coded last night while intoxicated will likely not be showing up on the Xbox One any
time soon. This also suggests games produced with the arguable intent of solely scamming (i.e.,
extremely low-effort) people out of money should be relatively rare on consoles.
In contrast, PC gaming constitutes a completely different world. Without getting into the
history of PC game distribution, one should know first that Valve’s Steam (a digital storefront
and game management software) has been the primary way in which most PC gamers get their
games over the past decade. One might think that there would be some form of quality control
necessary to make it onto the Steam store. For better or for worse, however, the reality turns out
to be the opposite. Industry consensus tends to be that the majority of games on Steam amount to
“shovelware” – i.e., incredibly low-effort games produced with the sole attempt of scamming a
few gamers out of money. 51 The following statistic serves to underscore this industry conjecture:
“Nearly 40% of all steam games were released in 2016 [alone].” 52 Considering this with the fact
that Steam first came onto the scene in 2003, this statistic suggests the store may have indeed
have some quality control issues. Although it may be possible that each and every one of the
games released in 2016 were developed with love and care, this likely proves a difficult pill to
swallow. This digression serves as the justification for the following decision: this study will not
collect observations from the PC. However, do note, this does not mean that a game that gets
released on both a console and PC will be excluded – i.e., only console versions of games will be
considered for inclusion in the sample data set. Again, do note that this does decision does not
Ralph Edwards,”The Economics of Game Publishing.” IGN, May 5, 2006, accessed April 5, 2017,
http://www.ign.com/articles/2006/05/06/the-economics-of-game-publishing.
51
Kyle Orland, "Valve Discusses User-Centric Changes to Steam’s Game Discovery Problem,” Ars Technica, April
04, 2017, accessed April 05, 2017, https://arstechnica.com/gaming/2017/04/valve-taps-wisdom-of-crowds-to-helpfilter-steam-games/.
52
Luke, Plunkett, “Nearly 40% Of All Steam Games Were Released In 2016,” Kotaku, November 30, 2016,
Accessed April 5, 2017, http://kotaku.com/nearly-40-of-all-steam-games-were-released-in-2016-1789535450.
50
35
stray from this study’s goal of constructing a data set that reflects the average gamer given that
the average gamer appears to play more on console anyways.
Before moving from console criteria, one final point remains. One of the big three
console manufacturers (Microsoft, Sony, and Nintendo) exhibits particularly unique
characteristics in regards to its consoles. A person with familiarity with the industry may already
know, of course, that this comment hints at Nintendo. A common claim one might hear in any
discussion that compares Nintendo’s console offerings to Microsoft’s and/or Sony’s systems
notes that people buy Nintendo’s consoles so that they can play first-party (i.e., Nintendo’s own)
games.53 Indeed, this author purchased a Nintendo Switch not with the hopes of being able to
play the next Call of Duty game – an incredibly popular multiplatform third party title – but to
play Nintendo’s own The Legend of Zelda: Breath of the Wild. In other words, Nintendo’s recent
consoles have not received as much third-party support as Microsoft’s and Sony’s respective
consoles.
One might be asking why it matters that Nintendo’s consoles do not receive as many
third-party games. The answer to this question contains two parts. First, in general, the average
Nintendo game tends to be very good at worst – Metacritic ranked Nintendo as the top third
publisher of 2016 in terms of average review score. 54 Second, Nintendo’s recent consoles have
arguably been much more vulnerable to attracting shovelware (even despite quality standards). 55
In other words, the first-party games tend to be incredibly well received while a good amount of
Ben Gilbert, “Nintendo's New Console Still Doesn't Solve Nintendo's Biggest Problem,” Business Insider,
February 8, 2017, accessed April 5, 2017, http://www.businessinsider.com/nintendo-switch-third-party-gamessupport-2017-2.
54
Jason Dietz, "Metacritic's 7th Annual Game Publisher Rankings," Metacritic, Feburary 13, 2017, accessed April
05, 2017, http://www.metacritic.com/feature/game-publisher-rankings-for-2016-releases.
55
Matthew Kato, “Nintendo's Software Problem - Is the Wii Buried Under Shovelware?” Game Informer,
September 27, 2009, accessed April 5, 2017, http://www.gameinformer.com/b/news/archive/2009/09/27/feature-wiisoftware-problem-shovelware.aspx.
53
36
third party games constitute low-quality games that aim to make money and not be good in any
sense of the word. Now suppose to solve this potential problem, one only permitted first-party
Nintendo games. While that might be fine, most Nintendo games tend to be extremely well
received. Thus, such a dataset would be skewed towards the positive side of things. Now
consider doing the opposite – permitting only third-party releases for Nintendo console data. The
problem gets flipped: with there possibly being a relatively large amount of shovelware on
Nintendo consoles, such a dataset stands to be at risk of being skewed towards the negative side
of things. One can now see the problem in collecting observations from Nintendo consoles.
Although it would be possible to define a rigorous set of criteria for collecting such observations,
it proves much simpler to not collect observations from Nintendo consoles.
In terms of the decision to exclude Nintendo still permitting the construction of a data set
that reflects the average gamer, unfortunately there did not seem to be hard data that would
suggest gamers play Microsoft and Sony consoles more than they do Nintendo consoles. With
that said, this author can offer up the following anecdotal observation: many people tend to own
two primary game consoles, one Nintendo Console and either a Microsoft console or a Sony
console. The reason for this tends to be that gamers want the third-party software that typically
only gets released on Microsoft’s and Sony’s respective consoles but they also want to be able to
play Nintendo’s first party titles. Hence, the norm seems to be many gamers own and play
Nintendo consoles strictly for the purpose of playing Nintendo’s own first-party software (and
recall that because Nintendo titles tend to be so good, including them could risk skewing the
data). Moreover, the industry does appear to recognize this as a fact. For example, Paul Tassi of
Forbes notes that, “Nintendo [is] getting by almost solely on its first-party hits” and that, “it’s
37
clear almost all of the best games [for Nintendo consoles] are made by Nintendo directly.” 56 The
claim that most gamers own Nintendo consoles mostly to only play Nintendo software stands as
anecdotal observation. Even so, if this can be taken as given, then it can be said that the
excluding Nintendo will still allow for the creation of a data set that will reflect the average
gamer.
Before continuing, it will be useful to recap the above. In constructing a data set, this
study will collect observations (i.e., games) that first got released during or after 2006 – in other
words, during or after the seventh generation of video games. Furthermore, due to the unique
challenges the platform introduces, the study excludes collecting data from the PC. Additionally,
due to the risk of introducing a considerable amount of skewness, this study does not collect data
from Nintendo consoles. To sum up, all of this simply means that this study will collect
observations from Microsoft’s Xbox 360 and Xbox One and Sony’s PlayStation 3 and
PlayStation 4. As delineated above, restricting observations to these consoles should allow for
the creation of a data set that reflects the average gamer.
4.5 Creating the Sample Data Set
With all of the criteria established above, discussion finally moves on to how this study
constructed a sample data set to analyze in answering the question of whether review prohibition
periods function as a signal of quality.
To start, this study first pulled all of the games and their respective sales data associated
with them listed on VGChartz for each of the four consoles under consideration (Xbox 360,
Xbox One, PlayStation 3, and PlayStation 4) and placed them into four separate spreadsheets:
Paul, Tassi, “150 Games 'For Everything But Wii U' Highlights Nintendo's Third Party Problem,” Forbes,
September 25, 2014, accessed April 07, 2017, https://www.forbes.com/sites/insertcoin/2014/09/25/150-games-foreverything-but-wii-u-highlights-nintendos-third-party-problem/.
56
38
one containing only the Xbox 360 games, one containing only the Xbox One games, one
containing only the PlayStation 3 games, and one containing only the PlayStation 4 games). In
essence, each of these initial spreadsheets constitutes the “population” of games for each
respective console – of course the term population is used loosely here given that it cannot be for
certain that VGChartz lists every single game known to man for each console. In total, there
were 3,672 Xbox 360 games, 514 Xbox One games, 3,311 PlayStation 3 games, and 864
PlayStation 4 games in these proxy populations.
The next step in this process concerned scrubbing these proxy populations to eliminate
observations likely to be poor. To do so, this study removed all observations in which the total
global sales (in millions of units) was listed as 0.00. The reason for cleaning up the proxy
populations this way pertains to the fact that looking into any observation with sales of 0.00
tended to feature problems. For instance, the Xbox One proxy population contained Halo 6 – a
game that, as far as this author knows, has not even been officially announced yet! Thus, not only
does the game have zero sales, but this game also very obviously has zero reviews – it is hard to
review a game that does not exist! Hence, removing all entries with zero sales likely proves to be
a very efficient method in cleaning up these proxy populations. For the record, some other
common problems with such observations included such observations not having a Metascore
(due to there not being enough reviews; recall that Metacritic requires four reviews to produce a
Metascore) and in some cases, such observations not even being listed on Metacritic! Cleaning
up the proxy populations in this manner resulted in there being 1,298 Xbox 360 games, 271
Xbox One games, 449 PlayStation 4 games, and 1,363 PlayStation 3 games (for a total of 3,381
members in the overall proxy population).
39
Note that due to pulling data from two different sources, this study required “matching”
up the data for each observation. Obviously this proved to be a very tedious task (more on this
below). This fact stands as the primary reason for the next step in this process: for each “cleaned
up” proxy population, this study assigned a random integer between 1 and 1000 to each
observation. Then, using Microsoft Excel, the observations in each cleaned up proxy population
were sorted from least to greatest based on these random numbers. From this list, the first onehundred observations were taken and placed into a respective sample set for each console. In
other words, one-hundred observations for each console were randomly selected for that
console’s respective sample. In total, this meant that the overall sample set at this point (i.e., the
set combining these four console sample sets with each other) contained four-hundred
observations. At this point, the sub-sample sets for each console were combined into one large
sample with the four-hundred observations.
With a preliminary sample set finally established, the last (and most tedious!) step
concerned simply matching up this data (which recall, up to this point, has strictly been from
VGChartz) with Metacritic data – see the “Variables” section from earlier for details of the
specific information gathered). For the most part, this matching process went smoothly.
However, having to handle each observation individually resulted in some observations
displaying troubling characteristics that did not get addressed in the process of cleaning up the
proxy populations. Particularly, the most common problem that tended to pop up concerned the
observation having no Metascore due to there being less than four reviews on Metacritic.
Obviously keeping such observations would introduce problems given that one of the key
variables in this study concerns the quality of the game. Hence, such games were removed from
the sample set.
40
One other problem that cropped up concerned Metacritic listing some games’ release
dates incorrectly. Such flawed information here could prove to problematic for results. In order
to assure the correctness of each release date for each release date, this study opted to check
Metacritic’s listed release dates against the release dates listed by IGN (a major video game
reviewer/website). For the most part, errors were far and few in between. If there was indeed a
mismatch between Metacritic and IGN, this study then consulted GameSpot (another venerable
video game reviewer/website.) In such cases (approximately twenty games or so in total), there
were no mismatches between IGN and GameSpot. For the games in which a game’s release date
was listed incorrectly, the study simply corrected Metacritic’s error.
Another issue that popped up concerned Metacritic sometimes listing the date of the first
review incorrectly. This, however, proved simple to correct for: the study simply went straight to
source (i.e., the reviewer) to verify such dates for each observation. If the corrected information
resulted in another review actually being the first, the study verified that review’s publication
date as well to assure that the date of the first review is correct as possible.
Because nothing can ever be simple, this study also ran into the following problem: it
appears that Metacritic only started listing publication dates for each review sometime after
2010. In other words, finding out the date of the first review for observations before 2010
required even more legwork (or, perhaps more appropriately, more handwork given all of this
was done electronically!) With this realization, the study essentially had two choices on its
hands: cut all observations that were released prior to 2010, or dig deeper to attempt to find out
the actual date the first review was published on. The first choice, while much easier, would have
dramatically decreased the sample size. Thus, this study chose the latter. For observations that
Metacritic did not list publication dates of reviews for, this study turned to a lesser known review
41
aggregation site: GameRankings – which does list the publication dates of all reviews the site has
for every game – including those released prior to 2010.
Again, nothing can be simple. While GameRankings proved to be a useful solution for
determining the date the first review was published for a game for the most part, there were still
a few observations in which there was seemingly a problem. Particularly, for a few observations
in which review publication dates were pulled from GameRankings, there would be a few
reviews GameRankings listed the first of the release month as the publication date while the rest
of the reviews were all published much later on in the month. This is a little suspicious because it
suggests one of the two sets is incorrect, as many reviews tend to all be published near or around
the same date. Thus, such dramatic differences would suggest the presence of an error.
Unfortunately, because of the age of many of these observations (released before 2010), often
times going straight to the source proved out of the question due to many reviews not being
available online (note that the years prior to 2010 were indeed a different time – the journalism
industry was still transitioning to an all-digital world!) With no other practical means to verify
some of the crucial data for the few observations in which the range of review publication dates
from GameRankings proved incredibly large, the study found it best to cut these observations
from the sample. In total, about twenty or so observations got removed due to these issues.
Taking all of the above into account produced this study’s finalized sample set. In total,
due to the cuts described in the previous paragraphs, the final sample set ended up featuring 269
observations. In particular, it contains 71 Xbox One Games, 62 Xbox 360 Games, 74 PlayStation
4 games, and 62 PlayStation 3 games. One concern one might have will pertain to the seemingly
small size of this sample set. Recall that the overall proxy population contained 3,381 members.
Thus, this sample constitutes roughly 8% of the population. However, the actual percentage
42
likely stands to be higher. Note that for each console about thirty to forty games got cut out of
the initial sample due to issues that the initial cleaning up of the population did not reveal –
meaning, about 30% to 40% got cut for each console. Assuming this stayed consistent, had the
matching process been applied to the entire proxy population, the final set would reflect a higher
percentage of the population of interest. Even so, it would admittedly prove useful to increase the
sample size in any future studies on this issue.
One final comment about the above pertains to any concern over human error. As should
be evident from the above discussion, this study made every effort possible to minimize any
error in the data. Of course, any process like this can be subject to human error. In order to
combat this, this study placed a great emphasis on going through the dataset a number of times to
assure that a value was not entered incorrectly throughout the matching process. Admittedly
though, it would likely be misspeaking to say that the data contain no errors whatsoever. Given
that this study went over each observation with a fine toothed comb in the matching process a
number of times, however, any errors that did sneak in should be very minor and should be
relatively rare throughout the dataset.
4.6 Analyzing the Data
Prior to detailing the statistical analysis applied to the data, it may be beneficial to recall
the two broad questions this study seeks to answer: First, do review prohibition periods appear to
function as a signal of video game quality? Second, do review prohibition periods appear to
impact sales?
In analyzing the sample data, this study employs the standard Ordinary Least Squares
regression techniques. In particular, this study considers at a few variables (see next paragraph)
as being functions (i.e., being dependent on) of a number of independent variables. Moreover,
43
this study utilizes the White Test (without inclusion of the cross products due to the relatively
small sample size) in order to check the data for symptoms consistent with heteroskedasticity. In
the case of heteroskedasticity likely being present, this study implements the Huber-White
Method for the production of results with heteroskedasticity-consistent standard errors.
The first step in the analysis process concerns determining what variables to use as lefthand side (i.e., dependent) variables. Broadly speaking, this study’s two overarching questions
pertain to game quality and consumer response. The most natural left-hand side variables, thus,
are likely Metascore (metascore) and sales variables (particularly, initial). A subtle detail to note
here concerns the fact that a game’s Metascore is implicitly being used as a proxy for video
game quality.
Next it must be considered what variables should be included as independent variables.
This, however, constitutes a much broader decision than the decision of what to consider as a
dependent variable. To account for this, this study ran multiple regressions with various
combinations of right-hand side variables. The variables used most consistently, however, were
the variables that pertain to review prohibition periods: the variable magnitude (a variable that
measures the difference between the date of the first review published and the release date of a
game), before (a dummy variable which indicates whether the first review was published prior to
a game’s release date), and after (a dummy variable which indicates whether the first review was
published on or after a game’s release date). This, of course, should come as no surprise since
this study’s core questions concern the effects review prohibition periods have. In the interest of
time, this paper reports and discusses the estimates of only a few models in the “Results” section.
However, for the interested reader, the appendices of this paper feature a number of alternative
models along with their respective estimates.
44
4.7 Potential Issues
The author feels that the data utilized should do exceptionally well in helping to answer
the questions in this study. Even so, despite all the controls in place, there are some potential
issues that may still exist. In particular, there are two main issues that could very well still be
present: reviewer bias and the problem of pre-orders.
An implicit assumption this paper operates under concerns the unbiased reviewer. That
is, this study assumes that reviewers do not let their emotions get in the way of judging a game.
Of course, reviewers are human and this likely is not the case. Hence, it is worth addressing the
risk that an overly grumpy or overly optimistic reviewer might present. For the most part, the
author believes the impact a biased reviewer could have on the dataset is considerably low.
Recall that a Metascore is a weighted average. Furthermore, recall that Metacritic bases weights
on reputation – sites that have a history of consistently producing thoughtful reviews are given a
large weight in the averaging process. Assuming Metacritic’s weights actually reflect this ideal,
then there should arguably be little bias present in Metascores. Even so, there exist other factors
at play that may help to eliminate reviewer bias. In particular, one might note the mean and
median number of reviews on Metacritic for observations in the dataset: 35 and 31 respectively.
With that in mind, one should consider the following: would the presence of one or two
considerably biased reviews impact the Metascore all that much for a game receiving more than
30 reviews? The author would wager probably not. Yes, bias in reviews (and thus in Metascores)
is certainly a possibility, but, the impact that bias has should be relatively minor.
The second issue to consider pertains to preorders. Though VGChartz does not state one
way or the other, the author would guess that preorders are factored into initial sales. One might
easily guess why this might be an issue: Since many preorders may be made far in advance
45
before a game’s release, review prohibition periods likely do not factor in to them! Thus,
preorders could skew initial sales numbers! Unfortunately, there is little one can do about this
problem as data on preorders is incredibly difficult to find. Although VGChartz does offer some
data on preorders, the format of this data is not useful for filtering preorders out of initial sales
numbers. With that said, however, preorders may not be as big of a deal as they might appear to
be at first. First, one should note that a consumer can cancel a preorder at any time. Hence,
prohibition periods could possibly affect preorder decisions (and hence, initial sales). Suppose a
consumer preorders a game months in advance. However, suppose that near release news comes
out that the publisher is imposing a restrictive review policy. This consumer could easily cancel
their preorder in light of this news. In fact, the author has some anecdotal evidence to give
credence to this possibility. During No Man’s Sky infamous review debacle, the author recalls
reading comments on a number of internet sites of people claiming to cancel their preorders of
the game. Although anecdotal, at the very least such internet comments suggest some people are
aware of the implications of review policy.
4.8 Recap
This section has been long. However, it has been necessary – results are only as good as
the data they are based off. In reading the results in the section that follows, it will be helpful to
recall a few key facts from this section. First and foremost, remember that this study seeks to
determine whether review prohibition periods appear to function as a signal of video game
quality as well as determine whether consumers appear to respond to review prohibition periods
if this is the case. Additionally, do recall that in regards to review prohibition periods, this study
uses proxy variables due to the lack of hard data on review embargoes for games. Specifically,
remember there are three key variables that proxy review prohibition periods: magnitude, before,
46
and after. Magnitude is simply the difference in days between the date of publication for the first
review and the release date of a game. A negative difference indicates the first review was
published prior to a game’s release date. In turn, a positive difference indicates the first review
was published on or after a game’s release date. The variables before and after simplify the
concept into dummy variables. For games that have a negative magnitude value, before takes on
a value of “1” while after takes on a value of “0.” Likewise, for games that have a non-negative
magnitude value, before takes on a value of “0” while after takes on a value of “1.” The logic,
recall, pertains to the fact that if a review prohibition period permitted reviews to be published
prior to a game’s release date, then it is likely that the first review would have a publication date
that is before the release date. Finally, the reader should remember that a plethora of summary
statistics are available in Appendix A.
47
5. Results
Before jumping into the results of this study, it will prove helpful to once again recall a
few key points that have been made up to this point. Perhaps most crucial, it is essential to keep
this study’s overarching questions in mind: do review prohibition periods appear to function as a
signal of quality and, if so, do consumers appear to respond to them? The underlying economic
framework here places these questions under the context of an asymmetric information problem
– video game publishers know something about their product that consumers do not (the quality
of video games). Hence, both publishers and consumers seek out ways to resolve this problem.
This study predicts that games with review prohibition periods that permit consumers to read
reviews prior to a game’s release date should be of better quality and see higher sales relative to
games with restrictive prohibition periods. The logic goes that if publishers are fine with
consumers being able to read reviews about their games prior to release, then publishers must
feel confident that they have quality games on their hands. Similarly, publishers that seek to
prohibit consumers from being able to read reviews about their games prior to release must feel
less confident that they have quality games on their hands.
The following discussion of results will be broken down into two core sections: First,
discussion will address the results in regards to review prohibition periods being signals of
quality. Second, discussion will then move on to address the results in regards to consumers’
response to review prohibition periods. Do note that this discussion is not exhaustive: for the
interested reader, there are a plethora of regression results that are omitted in this section but do
appear in the appendix.
48
5.1 Review Prohibition Periods as Signal of Video Game Quality
To kick off discussion of results, this paper begins by simply taking the variable
metascore (recall this is the study’s core proxy for video game quality) as a function of the
dummy variable before (which recall takes on a value of “1” if a game’s first review was
published prior to a game’s release date and takes on a value of “0” otherwise). In other words,
consider the following model:
𝑚𝑒𝑡𝑎𝑠𝑐𝑜𝑟𝑒 = 𝑐 + 𝛽1 (𝑏𝑒𝑓𝑜𝑟𝑒) + 𝑢
(1)
Here, c simply represents the constant term , 𝛽1 the coefficient on before, and u the error term.
Estimates for this model using Ordinary Least Squares (OLS) follow on the next page in Tables
5.1 and 5.2.
Testing for heteroskedasticity using the White test (without cross terms) yielded evidence
of heteroskedasticity being present. Hence, the estimates for the same model are presented twice
to demonstrate estimates with White’s heteroskedasticity-consistent standard errors and
estimates without White’s heteroskedasticity-consistent standard errors (note that qualitatively
little is actually changed). As can be seen from the Tables 5.11 and 5.2, before is statistically
significant at the 99% significance level when taking metascore to be a simple function of
before. Perhaps just as important is the coefficient on before: which is nearly 12. Thus, the above
results suggest that, in general, games that have their first review published prior to their release
dates score nearly 12 points higher in terms of quality. This means review prohibition periods
may in fact tell the consumer something about video game quality. However, this is merely a
49
simple regression and serves as only a starting point. Moreover, one should note the relatively
low R-squared. Hence, at this stage it is crucial to take the results with a grain of salt.
Table 5.1: Estimates for Model (1) with Heteroskedasticity-Consistent Standard Errors
Dependent Variable: metascore
Variable
Coefficient
Standard Error
P-Value
constant
before
65.77778
11.87894
1.305941
1.520721
0.0000
0.0000
R-Squared
0.185482
N
269
F-Statistic
60.80315
F P-Value
0.0000
Source: Data from Metacritic, 2006—2017.
Table 5.2: Estimates for Model (1) without Heteroskedasticity-consistent Standard Errors
Dependent Variable: metascore
Variable
Coefficient
Standard Error
P-Value
constant
before
65.77778
11.87894
1.075220
1.523425
0.0000
0.0000
R-Squared
0.185482
N
269
F-Statistic
60.80135
F P-Value
0.0000
Source: Data from Metacritic, 2006—2017.
It cannot be stressed enough that the above model (1) simply serves as a starting point.
To truly get a grip on the potential effects review prohibition periods may have, expanding the
above model will be necessary. The following discussion presents some potential regression
models that this study feels best represent what it is likely happening. However, as stated earlier,
50
the appendix to this paper contains a plethora of alternative models and estimates for the
interested reader – indeed, the author encourages readers to look at these alternative models.
The first alternative expanded model takes on the following form:
𝑚𝑒𝑡𝑎𝑠𝑐𝑜𝑟𝑒 = 𝑐 + 𝛽1 (𝑏𝑒𝑓𝑜𝑟𝑒) + 𝛽2 (𝑡𝑟𝑖𝑝𝑙𝑒𝑎75𝑞) + 𝛽3 (𝑜𝑏𝑠𝑐𝑢𝑟𝑒25𝑞) + 𝛽4 (𝑥𝑏𝑜𝑥𝑜𝑛𝑒) +
𝛽5 (𝑥𝑏𝑜𝑥360) + 𝛽6 (𝑝𝑠3) + 𝛽7 (𝑒𝑥𝑐𝑙𝑢𝑠𝑖𝑣𝑒) + 𝛽8 (𝑠𝑒𝑞𝑢𝑒𝑙) + 𝑢
(2)
Before going any further, one should take a moment to recall what some of the above variables
represent. The variables triplea75q and obscure25q are dummy variables that proxy whether a
game can be classified as triple-A (i.e., a major blockbuster video game) or as obscure (meaning
the game might appeal to only a niche audience). Remember that these proxies are based on the
number of reviews Metacritic lists for a game. If a game has 48 (the 75th percentile for number of
reviews in the sample) or more reviews on Metacritic, then triplea75q takes on a value of “1,”
otherwise it takes on a value of “0.” Similarly, if a game has 17 (the 25th percentile for number of
reviews in the sample) or less reviews on Metacritic, then obscure25q takes on a value of “1,”
otherwise it takes on a value of “0.” Like triplea75q and obscure25q, xboxone, xbox360, and ps3
are dummy variables that account for the platform the observation was released on. Hence, if an
observation was an Xbox One video game, then xboxone takes on a value of “1.” Note that the
coefficients on the console variables should be interpreted as being relative to PlayStation 4
releases (since ps4 is the dummy variable left out). Finally, remember that exclusive and sequel
are dummy variables that denote whether an observation was an exclusive game (i.e., was only
released on one platform) or a sequel repsectivly. The estimates for model (2) appear on the
following pages.
51
Again, it is worth noting that the White test (without cross terms) indicated the presence
of heteroskedasticity (hence, two sets of results appear again for the same model – one with
heteroskedasticity-consistent errors and one with normal standard errors). Furthermore, one
should notice that the these estimates do exhibit a somewhat higher R-squared, meaning it is a
theoretically more useful than the simple model estimated from earlier. Like the estimates for the
simple model, before remains statistically significant – and still at the 99% significance level to
boot. The only real change from the simple model is a slightly smaller coefficient on before.
Still, the coefficient here still stands to be practically significant. These estimates suggest that
holding the other variables constant, a game that has its first review published before release
should have a Metascore about 9.5 points higher than a game that has its first review published
on or after release. Given that permissive review prohibition periods should go along with first
reviews being published prior to release, this model continues to provide some evidence that
these embargo periods may be indicative of quality.
Table 5.3: Estimates for Model (2) with Heteroskedasticity-consistent Standard Errors
Dependent Variable: metascore
Variable*
Coefficient
Standard Error
P-Value
constant
before
triplea75q
obscure25q
xbox360
sequel
68.78259
9.395494
4.097533
-5.549972
-4.689126
6.079752
2.061884
1.635120
1.691340
2.245982
2.056773
1.517975
0.0000
0.0000
0.0161
0.0141
0.0234
0.0001
R-Squared
0.304576
N
269
F-Statistic
14.23410
F P-Value
0.0000
*Non-significant variables omitted here. See Appendix for full results.
Source: Data from Metacritic, 2006—2017.
52
Table 5.4: Estimates for Model (2) without Heteroskedasticity-consistent Standard Errors
Dependent Variable: metascore
Variable*
Coefficient
Standard Error
P-Value
constant
before
triplea75q
obscure25q
xbox360
sequel
63.78259
9.395494
4.097533
-5.549972
-4.689126
6.079752
2.071745
1.577733
1.939374
1.889737
2.104945
1.540365
0.0000
0.0000
0.0356
0.0036
0.0268
0.0001
R-Squared
0.304576
N
269
F-Statistic
14.23410
F P-Value
0.0000
*Non-significant variables omitted here. See Appendix B for full results.
Source: Data from Metacritic and VGChartz, 2006—2017.
Other comments on these estimates pertain to the other significant variables. The
significance of triplea75q and obscure25q are not too surprising. Nor are the signs of their
coefficients all that surprising. Given the massive budgets triple-A games have, one might expect
such games to be of higher quality (exactly what the coefficient on triplea75q suggests).
Likewise, the niche nature of obscure games may likely result in there being few reviewers
having actual interest in them. Thus, the lower interest of the reviewers that end up reviewing
them anyways could very well result in obscure games receiving consistently lower review
scores. Even so, whereas the coefficient on before equates to almost an entire “letter grade,” the
coefficient on triplea75q and the absolute value of the coefficient on obscure25q are relatively
much smaller – amounting to only a slight increase in Metascore in the case of triplea75q and
only a slight decrease in Metascore in the case of obscure25q.
The real surprise (at least for this author) in the estimates for model (2) concerns the
significance – both statistical and practical – of the coefficient on xbox360. To be honest, this
53
author only recorded these dummy variables for the sake of completion – a console being
significant in regards to quality was something that made no sense: Why should the particular
console a game is released on matter that much in terms quality? Whereas one might feasibly
imagine why the case might be different for sales (e.g., consoles with higher market shares might
be expected to have higher sales for any given game), the result is odd for quality. Thus, the
estimate for xbox360 is admittedly surprising. Before offering up an explanation for why this
peculiar result might occur, it will prove helpful to explicitly delineate how to interpret the
coefficient. Recall that this variable is interpreted relative to PlayStation 4 games. Hence, the
coefficient on xbox360 suggests that games released on the Xbox 360, as opposed to the
PlayStation 4, have Metascores that are lower by around 6 points.
The best explanation this study can offer up for this peculiarly significant result concerns
the raw power of the Xbox 360 compared to the other three consoles in the data set. First recall
some basic facts: the Xbox 360 and PlayStation 3 are seventh-generation home consoles while
the Xbox One and PlayStation 4 are eighth-generation home consoles. Hence, the Xbox 360 and
PlayStation 3 are both inferior pieces of hardware compared to the Xbox One and PlayStation 4
in terms of raw computing specifications. Next, during the seventh-generation of video games,
the PlayStation 3 tended to be considered the more powerful console relative to the Xbox 360. 57
In other words, the Xbox 360 constitutes the least powerful console in the data set. Furthermore,
the industry tends to consider the PlayStation 4 to be more powerful than the Xbox One. 58 Thus,
the coefficient on xbox360 might be argued to be the result of a basic systematic difference: not
Jesse Schedeen, “Xbox 360 vs. PlayStation 3: The Hardware Throwdown,” IGN, August 26, 2010, Accessed
April 10, 2017, http://www.ign.com/articles/2010/08/26/xbox-360-vs-playstation-3-the-hardwarethrowdown?page=2.
58
Eddie Makuch, “PS4 More Powerful Than Xbox One, But Microsoft Always Improving -- Metro Dev,”
GameSpot, August 27, 2014, accessed April 10, 2017. https://www.gamespot.com/articles/ps4-more-powerful-thanxbox-one-but-microsoft-alwa/1100-6421940/.
57
54
only is Xbox 360 the weakest console out of the bunch (which perhaps might mean games on it
are slightly lower quality than more powerful consoles), but when compared to the PlayStation 4,
the system is especially weak.
Another model to consider has the following form:
𝑚𝑒𝑡𝑎𝑠𝑐𝑜𝑟𝑒 = 𝑐 + 𝛽1 (𝑏𝑒𝑓𝑜𝑟𝑒) + 𝛽2 (𝑡𝑟𝑖𝑝𝑙𝑒𝑎75𝑞) + 𝛽3 (𝑜𝑏𝑠𝑐𝑢𝑟𝑒25 + 𝛽4 (𝑥𝑏𝑜𝑥𝑜𝑛𝑒)
+ 𝛽5 (𝑥𝑏𝑜𝑥360) + 𝛽6 (𝑝𝑠3) + 𝛽7 (𝑒𝑥𝑐𝑙𝑢𝑠𝑖𝑣𝑒) + 𝛽8 (𝑠𝑒𝑞𝑢𝑒𝑙) + 𝛽9 (𝑎𝑐𝑡𝑖𝑜𝑛)
+ 𝛽10 (𝑎𝑑𝑣𝑒𝑛𝑡𝑢𝑟𝑒) + 𝛽11 (𝑓𝑖𝑔ℎ𝑡𝑖𝑛𝑔) + 𝛽12 (𝑝𝑙𝑎𝑡𝑓𝑜𝑟𝑚𝑒𝑟) + 𝛽13 (𝑝𝑢𝑧𝑧𝑙𝑒)
+ 𝛽14 (𝑟𝑎𝑐𝑖𝑛𝑔) + 𝛽15 (𝑟𝑝𝑔) + 𝛽16 (𝑠ℎ𝑜𝑜𝑡𝑒𝑟) + 𝛽17 (𝑠𝑖𝑚𝑢𝑙𝑎𝑡𝑖𝑜𝑛) + 𝛽18 (𝑠𝑝𝑜𝑟𝑡𝑠)
+ 𝛽19 (𝑠𝑡𝑟𝑎𝑡𝑒𝑔𝑦) + 𝑢
(3)
This model simply serves as an expansion of the previous one. As can be seen, this model allows
genre of a game to be a factor of quality as well. Recall that there were twelve genre
classifications. The one left out in the model is the “misc” genre. Hence, one should interpret
coefficients on genre terms as being relative to the “misc” genre. Estimates for this model are
reported in the typical fashion on the following page in Tables 5.5 and 5.6.
One might find the significance (both statistical and practical) of the coefficient for
puzzle to be puzzling. Before solving that puzzle, however, one should note the consistency of
the other results. Being classified as a triple-A title or an obscure title continues to be significant.
Moreover, the respective magnitudes of the coefficients are virtually unchanged from the
previous regression model. Additionally, xbox360 as well as sequel continue to exhibit statistical
significance – but ever so slightly less practical significance in terms of their coefficients this
time around. Perhaps most importantly, before remains very statistically significant – the
coefficient also continues to maintain considerable practical significance. One other feature to
55
note is R-squared, which has gone up by a marginal amount. Even so, admittedly it does remain
small.
Now, on to the puzzle presented in these estimates: puzzle games. Although there might
be something interesting in this result, one might want to recall there are very few puzzle games
in the sample set: out of the 269 observations in the sample, only two are puzzle games (see Data
and Methodology section as well as Appendix A for more summary statistics). This fact also
likely accounts for the relatively high standard error for the variable puzzle. Thus, attempting to
argue that puzzle games are consistently much higher quality than games that fall under the
“misc” genre would probably be a mistake. Hence, it may very well prove interesting in future
studies on similar topics to intentionally include more puzzle games in a sample in order to see if
this result can actually hold up.
Table 5.5: Estimates for Model (3) with Heteroskedasticity-consistent standard errors
Dependent Variable: metascore
Variable*
Coefficient
Standard Error
P-Value
constant
before
triplea75q
obscure25q
xbox360
sequel
puzzle
62.29037
9.138273
3.954544
-5.648930
-4.246354
6.211801
14.43632
4.028484
1.644265
1.859947
2.420594
2.121587
1.599045
4.651117
0.0000
0.0000
0.0345
0.0204
0.0464
0.0001
0.0021
R-Squared
0.326425
N
269
F-Statistic
6.351004
F P-Value
0.0000
*Non-significant variables omitted here. See Appendix B for full results.
Source: Data from Metacritic and VGChartz, 2006—2017.
56
Table 5.6: Estimates for Model (3) without Heteroskedasticity-consistent standard errors
Dependent Variable: metascore
Variable*
Coefficient
Standard Error
P-Value
constant
before
triplea75q
obscure25q
xbox360
sequel
puzzle
62.29037
9.138273
3.954544
-5.648930
-4.246354
6.211801
14.43632
3.533709
1.605414
2.063333
1.995068
2.210267
1.705820
8.967067
0.0000
0.0000
0.0564
0.0050
0.0558
0.0003
0.1087
R-Squared
0.326425
N
269
F-Statistic
6.351004
F P-Value
0.0000
*Non-significant variables omitted here. See Appendix B for full results.
Source: Data from Metacritic and VGChartz, 2006—2017.
Up to this point, all models have utilized the dummy variable before. In other words,
these models have only indicated whether a first review being published before release matters –
the actual number of days before (or after) release has not been a factor. However, recall that
before is simply based of the variable magnitude, which does indicate the number of days
between the first review publication date and release date of a game. Thus, it may be interesting
to look at a model that makes use of magnitude. Before doing so, however, it will be important to
remember precisely how magnitude is defined – doing so will make interpretation much easier.
As delineated in section 4.2, magnitude is the difference (in days) between the publication date
of a game’s first review and a game’s release date. Negative numbers indicate that the first
review preceded a game’s release while non-negative values indicate that the first review was
published on or after a game’s release. For convenience, Figure 4.1 is reprinted on the following
page to reinforce this concept.
57
Figure 5.1: Reprint of Figure 4.1: Visualizing magnitude, before, and after
-
reviews before release
release date = 0
reviews after release
magnitude is negative
before = 1
after = 0
magnitude = 0
before = 0
after = 1
magnitude is positive
before = 0
after = 1
+
magnitude (in days) = date of first review – release date
To demonstrate magnitude’s potential as a signal, consider models (4) and (5), which are
simply replications of models (2) and (3) except for before being swapped out with magnitude:
𝑚𝑒𝑡𝑎𝑠𝑐𝑜𝑟𝑒 = 𝑐 + 𝛽1 (𝑚𝑎𝑔𝑛𝑖𝑡𝑢𝑑𝑒) + 𝛽2 (𝑡𝑟𝑖𝑝𝑙𝑒𝑎75𝑞) + 𝛽3 (𝑜𝑏𝑠𝑐𝑢𝑟𝑒25𝑞) + 𝛽4 (𝑥𝑏𝑜𝑥𝑜𝑛𝑒)
+ 𝛽5 (𝑥𝑏𝑜𝑥360) + 𝛽6 (𝑝𝑠3) + 𝛽7 (𝑒𝑥𝑐𝑙𝑢𝑠𝑖𝑣𝑒) + 𝛽8 (𝑠𝑒𝑞𝑢𝑒𝑙) + 𝑢
(4)
𝑚𝑒𝑡𝑎𝑠𝑐𝑜𝑟𝑒 = 𝑐 + 𝛽1 (𝑚𝑎𝑔𝑛𝑡𝑖𝑢𝑑𝑒) + 𝛽2 (𝑡𝑟𝑖𝑝𝑙𝑒𝑎75𝑞) + 𝛽3 (𝑜𝑏𝑠𝑐𝑢𝑟𝑒25 + 𝛽4 (𝑥𝑏𝑜𝑥𝑜𝑛𝑒)
+ 𝛽5 (𝑥𝑏𝑜𝑥360) + 𝛽6 (𝑝𝑠3) + 𝛽7 (𝑒𝑥𝑐𝑙𝑢𝑠𝑖𝑣𝑒) + 𝛽8 (𝑠𝑒𝑞𝑢𝑒𝑙) + 𝛽9 (𝑎𝑐𝑡𝑖𝑜𝑛)
+ 𝛽10 (𝑎𝑑𝑣𝑒𝑛𝑡𝑢𝑟𝑒) + 𝛽11 (𝑓𝑖𝑔ℎ𝑡𝑖𝑛𝑔) + 𝛽12 (𝑝𝑙𝑎𝑡𝑓𝑜𝑟𝑚𝑒𝑟) + 𝛽13 (𝑝𝑢𝑧𝑧𝑙𝑒)
+ 𝛽14 (𝑟𝑎𝑐𝑖𝑛𝑔) + 𝛽15 (𝑟𝑝𝑔) + 𝛽16 (𝑠ℎ𝑜𝑜𝑡𝑒𝑟) + 𝛽17 (𝑠𝑖𝑚𝑢𝑙𝑎𝑡𝑖𝑜𝑛) + 𝛽18 (𝑠𝑝𝑜𝑟𝑡𝑠)
+ 𝛽19 (𝑠𝑡𝑟𝑎𝑡𝑒𝑔𝑦) + 𝑢
(5)
As one will see in tables 5.7, 5.8, 5.9, and 5.10 (which appear on the following pages),
results are qualitatively similar to the models that utilize before for the most part. One detail that
might strike the reader at first concerns the fact that magnitude has a negative coefficient. This,
however, is precisely what one should expect: a negative coefficient indicates that the further
away in the future that a game’s first review is published relative to a game’s release the lower a
58
game’s Metascore. As an example, suppose a game’s first review comes out 10 days before its
release. The variable magnitude would be equal to -10. Multiplying by the negative coefficient
gives a positive value – which indicates a boost to metascore. Hence, this remains very much in
line with what the models that use before have indicated.
In terms of practical significance, results for magnitude are arguably a bit less practically
significant than the results for before: a one day increase in magnitude results (when rounding
up) to about a point decrease in Metascore; of course, this adds up! A first review being
published ten days after release would nearly lose an entire letter grade according to these
results. That said, the decreased practical significance should not come as too much of a surprise.
The important issue in regards to review prohibition periods being a signal of quality likely
pertains to whether they permit reviewers to publish reviews prior to release or not. It makes
sense that there is not much difference in what a review embargo that permits reviews to be
published three days before release and a review embargo that permits reviews to be published
two days before release in terms of what they might suggest about quality. However, there is
arguably a considerable difference in what a review embargo that permits reviews to be
published one day before release and a review embargo that forbid reviews from being published
until one day after release. This is because the former would be classified as permissive while the
latter would be classified as restrictive.
59
Table 5.7: Estimates for Model (4) with Heteroskedasticity-Consistent Errors
Dependent Variable: metascore
Variable*
Coefficient
Standard Error
P-Value
constant
magnitude
triplea75q
obscure25q
xbox360
sequel
67.92521
-0.880109
6.054874
-3.927962
-5.923882
5.099176
1.802949
0.164074
1.645082
2.421950
2.047601
1.527226
0.0000
0.0000
0.0003
0.1061
0.0050
0.0010
R-Squared
0.296082
N
269
F-Statistic
13.67015
F P-Value
0.0000
*Non-significant variables omitted here. See Appendix B for full results.
Source: Data from Metacritic, 2006—2017.
Table 5.8: Estimates for Model (4) without Heteroskedasticity-Consistent Errors
Dependent Variable: metascore
Variable*
Coefficient
Standard Error
P-Value
constant
magnitude
triplea75q
obscure25q
xbox360
sequel
67.92521
-0.880109
6.054874
-3.927962
-5.923882
5.099176
1.961187
0.155833
1.882233
1.991764
2.109822
1.561730
0.0000
0.0000
0.0015
0.0497
0.0054
0.0012
R-Squared
0.296082
N
269
F-Statistic
13.67015
F P-Value
0.0000
*Non-significant variables omitted here. See Appendix B for full results.
Source: Data from Metacritic, 2006—2017.
60
Table 5.9: Estimates for Model (5) with Heteroskedasticity-Consistent Errors
Dependent Variable: metascore
Variable*
Coefficient
Standard Error
P-Value
constant
magnitude
triplea75q
obscure25q
xbox360
sequel
puzzle
67.10209
-0.844586
5.967323
-4.153478
-5.406716
5.333358
11.67371
3.676498
0.169591
1.847546
2.183269
2.109822
1.606197
5.606253
0.0000
0.0000
0.0014
0.0139
0.0054
0.0010
0.0383
R-Squared
0.313723
N
269
F-Statistic
5.990910
F P-Value
0.0000
*Non-significant variables omitted here. See Appendix B for full results.
Source: Data from Metacritic, 2006—2017.
Table 5.10: Estimates for Model (5) without Heteroskedasticity-Consistent Errors
Dependent Variable: metascore
Variable*
Coefficient
Standard Error
P-Value
constant
magnitude
triplea75q
obscure25q
xbox360
sequel
puzzle
67.10209
-0.844586
5.967323
-4.153478
-5.406716
5.333358
11.67371
3.513289
0.161964
2.020059
2.111611
2.226964
1.730628
9.074952
0.0000
0.0000
0.0034
0.0503
0.0159
0.0023
0.1995
R-Squared
0.313723
N
269
F-Statistic
5.990910
F P-Value
0.0000
*Non-significant variables omitted here. See Appendix B for full results.
Source: Data from Metacritic, 2006—2017.
61
One final comment to make about the estimates for models (4) and (5) concerns
heteroskedasticity. White’s Test (without cross products) produced results on the margin – i.e.,
the question of whether heteroskedasticity is actually present may be debatable. To remain on the
cautious side of things, this paper decided it best to present two sets of results in typical fashion.
Discussion of estimating quality as a function of review prohibition periods and other
variables ends at this point in favor of moving on towards the estimating the effects prohibition
periods have on sales. However, before moving on, there are a few key takeaways from the
above models and their respective estimates. First and foremost, one should not only note the
statistical and practical significance of the variable before, but one should also note the
consistency of its statistical and practical significance. Furthermore, one might also want to take
note of magnitude’s statistical significance – even despite its slightly lower practical
significance. Although not proof in any sense of the word, the above results do suggest review
prohibition periods may indeed be indicative of quality. Of course, the above models are not
exhaustive: the author implores the reader to take a look at the numerous regression models and
their respective results that are located in the appendix. Not only do some of these models
display somewhat interesting results, many of them continue to underscore the consistent
significance of the variables before and magnitude.
62
5.2 Estimating the Effects of Review Prohibition Periods on Video Game Sales
Prior to delineating some of the empirical results that relate to sales, it will be necessary
to take a moment to recall some of the key variables that represent sales. Remember that this
study breaks down sales into two broad categories: initial sales and total sales. Initial sales,
denoted with the variable initial, are the number of units a game sold within its first week of
release (or, in the case of a few observations, the number of units a game sold within the first
week reported by VGChartz). In turn, overall sales are the total number of units (in millions) that
a game has sold to date.
Additionally, remember that overall sales can be broken down by region: North
American overall sales (variable NA), European overall sales (variable Eur), Japanese overall
sales (variable Japan), and overall sales in other regions of the world (variable ROW). The
variable global is simply the sum of NA, Eur, Japan, and ROW. Note that this study reasoned
earlier that the sales variable of interest here is likely initial. Recall that the logic goes that initial
sales are the ones that should be responsive to review prohibition policies: whereas initial
potential buyers have relatively limited information available to them to discern quality, potential
consumers that look into purchasing a game much later on after its release date have a plethora
of resources available to them to discern quality. Thus, this section of the paper will present only
regression results that have initial as the dependent variable – more specifically, this section will
actually make use of log models since percentage changes in sales are likely more of interest
than just changes in unit sales. As stressed earlier, however, the appendix to this paper presents
several alternative models – some which do present estimates using these alternative sales
variables.
63
Like before, discussion begins with examining a very simple model:
log(𝑖𝑛𝑖𝑡𝑖𝑎𝑙) = 𝑐 + 𝛽1 (𝑏𝑒𝑓𝑜𝑟𝑒) + 𝑢
(6)
Note that this is a log-level model. Hence, one should interpret the coefficient 𝛽1 on before as
indicating the percentage change in initial sales that occurs if a game’s first review is published
prior to a game’s release date. Estimates for this model appear in typical fashion on the following
page in Tables 5.11 and 5.12.
As one can see from the tables, estimates for this model indicate that before is very
significant both statistically and practically. Indeed, the coefficient on before indicates that a
game having its first review published prior to release results in an over 100% increase in initial
sales! That is quite the boost! Of course, no one should get too excited here: this is an estimate
for a very elementary model. Moreover, the R-squared is dismally low – thus, one needs a lot of
salt for this result.
Also worth noting is that heteroskedasticity may be present. However, in contrast to the
estimates of models of video game quality, the White test (without cross terms) yielded results
on the margin (i.e., one could feasibly make a case for either side.) Thus, to be on the safe side, it
made sense to present two sets of results as is typical of this paper at this point.
64
Table 5.11: Estimates for Model (6) with Heteroskedasticity-Consistent Errors
Dependent Variable: log(initial)
Variable*
Coefficient
Standard Error *
P-Value
before
1.198899
0.221372
0.0000
R-Squared
0.100420
N
264
F-Statistic
29.24697
F P-Value
0.0000
*Constant term omitted here. See Appendix B for full results.
Source: Data from Metacritic and VGChartz, 2006—2017.
Table 5.12: Estimates for Model (6) without Heteroskedasticity-Consistent Errors
Dependent Variable: log(initial)
Variable*
Coefficient
Standard Error *
P-Value
before
1.198899
0.221688
0.0000
R-Squared
0.100420
N
264
F-Statistic
29.24697
F P-Value
0.0000
*Constant term omitted here. See Appendix B for full results.
Source: Data from VGChartz, 2006—2017.
65
Similar to discussion of modeling video game quality, discussion will now pertain to
expanding the simple model in order to examine the robustness of before. To begin, first
consider model (7) below:
log(𝑖𝑛𝑖𝑡𝑖𝑎𝑙) = 𝑐 + 𝛽1 (𝑏𝑒𝑓𝑜𝑟𝑒) + 𝛽2 (𝑚𝑠𝑎𝑏𝑜𝑣𝑒75) + 𝛽3 (𝑚𝑠50𝑡𝑜74) + 𝛽4 (𝑡𝑟𝑖𝑝𝑙𝑒𝑎75𝑞)
+ 𝛽5 (𝑜𝑏𝑠𝑐𝑢𝑟𝑒25𝑞) + 𝛽6 (𝑥𝑏𝑜𝑥𝑜𝑛𝑒) + 𝛽7 (𝑥𝑏𝑜𝑥360) + 𝛽8 (𝑝𝑠3)
+ 𝛽9 (𝑒𝑥𝑐𝑙𝑢𝑠𝑖𝑣𝑒) + 𝛽10 (𝑠𝑒𝑞𝑢𝑒𝑙) + 𝑢
(7)
Model (7), as one might see, is extremely similar to the first alternative model of video
game quality. There are only two crucial differences: first is obviously the dependent variable.
Second, and perhaps more importantly, is the inclusion of msabove75 and ms50to74, two dummy
variables that reflect whether a game’s Metascore is indicative of mostly positive reviews or
mixed reviews respectively (note that the coefficients for these will be relative to a game having
a Metascore indicative of mostly negative reviews – i.e., msbelow50). In other words, this model
implies that quality of a game factors into its initial sales. However, one might wonder why not
just include the variable metascore, as opposed to these dummy variables that indicate only
certain ranges of Metascores. The logic for this goes that gamers likely do not make purchasing
decisions on whether a game’s Metascore is 68 or 69. In contrast, what likely matters is whether
the Metascore of a game falls into a certain range (e.g., it makes sense to think that a Metascore
of 75 or above, which is indicative of mostly positive reviews, might have a positive impact on
sales). Estimates for this model appear in Tables 5.13 and 5.14 on pages 65 and 66 respectively.
Obviously, the point of interest here concerns the insignificance of before – which,
assuming before functions as an appropriate proxy of review prohibition periods, indicates
66
prohibition periods do not impact initial sales in a significant manner. However, it is worth
discussing the coefficient itself. The coefficient is in the positive direction (which is what this
study would expect). Furthermore, if before was statistically significant, the magnitude of the
coefficient would not be anything to sneeze at, as it would suggest a game having its first review
published prior to release would result in initial sales increasing by about 12 or so percent. Of
course, before is not statistically significant, so this coefficient is arguably not that reliable.
As for the other results, there is not anything necessarily surprising here. It should not
come as much of a shock that having a Metascore of 75 or above (i.e., a Metascore indicating
mostly positive reviews) would give a nice boost to initial sales – a humble 79% increase
according to these estimates! Being classified as a triple-A game (at least based on the metric for
the dummy variable triplea75q) amounts to a whopping 163% increase in initial sales according
to the estimates of the above model! Take heart executives of triple-A publishers, throwing
millions of dollars at games appears to be worth it in end! As for other variables, the author was
a bit surprised to see the direction of the coefficients for xboxone and xbox360.
Given that the Xbox 360 has the largest market share in terms of units sold between the
PlayStation 4, Xbox 360, and Xbox One, 59 while the Xbox One has the lowest market share
among these three consoles 60, the author would expect the coefficients for xbox360 and xboxone
to be swapped. That is, it would make sense to the author to see Xbox 360 games receiving a
percentage boost to initial sales and Xbox One games receiving a percentage decrease to sales
relative to the PlayStation 4. The estimates above, of course, suggest the opposite. Hence, this
may constitute something that would be worth looking into in future studies as this result
contradicts basic intuition.
59
60
VGChartz, Ltd, “Platform Totals”, accessed April 12, 2017, http://www.vgchartz.com/analysis/platform_totals/.
Ibid.
67
Finally, there’s the coefficient for sequel, which suggests being a sequel results in about a
43% increase in initial sales. This is simultaneously surprising and not surprising to the author.
On the one hand, from the author’s own experience as a gamer, it is common to see the first
entry in a series be extremely popular while later releases in a series see enthusiasm (and in turn,
sales) decrease. On the other hand, this author also knows plenty of cases where successful first
entries in a series leave gamers wanting more – thus resulting in increased anticipation and
enthusiasm for future entries. The estimate for sequel perhaps can be taken as implying the latter
anecdotal experience may be more common. Again, this may be something to look into in future
studies.
Table 5.13: Estimates for Model (7) with Heteroskedasticity-Consistent Standard Errors
Dependent Variable: log(initial)
Variable*
Coefficient
Standard Error
P-Value
before
msabove75
triplea75q
obscure25q
xboxone
xbox360
sequel
0.125851
0.790636
1.637736
-1.327773
0.782112
-0.499814
0.437046
0.221250
0.344579
0.248322
0.262546
0.283635
0.289084
0.208315
0.5700
0.0226
0.0000
0.0000
0.0062
0.0850
0.0369
R-Squared
0.390636
N
264
F-Statistic
16.21872
F P-Value
0.0000
*Most insignificant variables omitted here. See Appendix B for full results.
Source: Data from Metacritic and VGChartz, 2006—2017.
68
Table 5.14: Estimates for Model (7) without Heteroskedasticity-Consistent Standard Errors
Dependent Variable: log(initial)
Variable*
Coefficient
Standard Error
P-Value
before
msabove75
triplea75q
obscure25q
xboxone
xbox360
sequel
0.125851
0.790636
1.637736
-1.327773
0.782112
-0.499814
0.437046
0.217459
0.405527
0.254384
0.252649
0.270061
0.282586
0.205581
0.5633
0.0523
0.0000
0.0000
0.0041
0.0781
0.0345
R-Squared
0.390636
N
264
F-Statistic
16.21872
F P-Value
0.0000
*Most insignificant variables omitted here. See Appendix B for full results.
Source: Data from Metacritic and VGChartz, 2006—2017
.
Before moving on to present a few more models, the author would like to take some more
time to comment about the significance of before. Specifically, consider whether its
insignificance is really all that surprising in the context of initial sales. Though the author was a
bit surprised (especially given that its insignificance goes against what the study predicted),
when thought about more, it really is not all that surprising of a result. Although review
prohibition periods may function as a signal of quality (especially in light of previous results),
they are arguably a very subtle signal. In other words, this is likely not something publishers
make that big of a deal of when attempting to communicate information about their games to
consumers. Furthermore, it is likely only the most hardcore and informed gamers (such as the
author) that actually pay much attention to review embargos. Hence, in some ways it makes
sense that consumers are not all that responsive to review prohibition periods. What one might
69
want to think about is whether consumers should start to be more responsive to them in light of
the results presented in this paper. With all that said, if one actually considers a level model of
initial sales, results appear to be marginally significant. Indeed, one may want to consider the
estimates presented in tables 5.15 and 5.16.
If looking at only estimates for the level version of model (7), it appears that before may
be statistically significant. Furthermore, it arguably demonstrates practical significance as well:
the estimate indicates a game receiving its first review prior to release results in about a 57,500
unit boost to initial sales.
The estimates for the level model also contains a surprising result: a game having a
Metascore of 75 and above (i.e., indicative of mostly positive reviews) turned out not to be
significant! Hence, the estimates for the level and log versions of model (7) suggest somewhat
contradictory results – meaning it is certainly worth looking into a few more models.
Table 5.15: Level Estimates for Model (7) with Heteroskedasticity-Consistent Errors
Dependent Variable: initial
Variable*
Coefficient
Standard Error
P-Value
before
triplea75q
xboxone
sequel
57,472.09
198,061.0
85,423.77
45,758.22
25,335.99
29,465.28
50,011.32
26,737.28
0.0241
0.0002
0.0888
0.0882
R-Squared
0.390636
N
264
F-Statistic
16.21872
F P-Value
0.0000
*Most insignificant variables omitted here. See Appendix B for full results.
Source: Data from Metacritic and VGChartz, 2006—2017.
70
Table 5.16: Level Estimates for Model (7) without Heteroskedasticity-Consistent Errors
Dependent Variable: initial
Variable*
Coefficient
Standard Error
P-Value
before
triplea75q
xboxone
sequel
57,472.09
198,061.0
85,423.77
45,758.22
31,980.49
37,410.92
39,716.48
30,233.72
0.0735
0.0000
0.0324
0.1314
R-Squared
0.254343
N
264
F-Statistic
8.629810
F P-Value
0.0000
*Most insignificant variables omitted here. See Appendix B for full results.
Source: Data from Metacritic and VGChartz, 2006—2017.
Consider model (8) below:
log(𝑖𝑛𝑖𝑡𝑖𝑎𝑙) = 𝑐 + 𝛽1 (𝑏𝑒𝑓𝑜𝑟𝑒) + 𝛽2 (𝑚𝑠𝑎𝑏𝑜𝑣𝑒75) + 𝛽3 (𝑚𝑠50𝑡𝑜74) + 𝛽4 (𝑡𝑟𝑖𝑝𝑙𝑒𝑎75𝑞)
+ 𝛽5 (𝑜𝑏𝑠𝑐𝑢𝑟𝑒25𝑞) + 𝛽6 (𝑥𝑏𝑜𝑥𝑜𝑛𝑒) + 𝛽7 (𝑥𝑏𝑜𝑥360) + 𝛽8 (𝑝𝑠3)
+ 𝛽9 (𝑒𝑥𝑐𝑙𝑢𝑠𝑖𝑣𝑒) + 𝛽10 (𝑠𝑒𝑞𝑢𝑒𝑙) + 𝛽11 (𝑎𝑐𝑡𝑖𝑜𝑛) + 𝛽12 (𝑎𝑑𝑣𝑒𝑛𝑡𝑢𝑟𝑒)
+ 𝛽13 (𝑓𝑖𝑔ℎ𝑡𝑖𝑛𝑔) + 𝛽14 (𝑝𝑙𝑎𝑡𝑓𝑜𝑟𝑚) + 𝛽15 (𝑝𝑢𝑧𝑧𝑙𝑒) + 𝛽16 (𝑟𝑎𝑐𝑖𝑛𝑔) + 𝛽17 (𝑟𝑝𝑔)
+ 𝛽18 (𝑠ℎ𝑜𝑜𝑡𝑒𝑟) + 𝛽19 (𝑠𝑖𝑚𝑢𝑙𝑎𝑡𝑖𝑜𝑛) + 𝛽20 (𝑠𝑝𝑜𝑟𝑡𝑠) + 𝛽21 (𝑠𝑡𝑟𝑎𝑡𝑒𝑔𝑦) + 𝑢 (8)
The change here, of course, pertains to the addition of the dummy terms to denote genre.
Estimates appear on the following page in Table 5.17 (note that heteroskedasticity did not appear
to be present this time around – hence, only one set of estimates appear).
As one can see, the results stay fairly the same, with little new to comment on. The
variable before remains statistically insignificant. Furthermore, triplea75q and obscure25q
71
remain quite significant – both in statistical terms and practical terms. The only new significant
variables here are the dummy variables that represent the genres puzzle, role-playing, games, and
games. The direction of the coefficients on the dummy variables for these genres is not all that
surprising: whereas role-playing games, shooter games, and sports games are among the most
popular genres (hence explaining the positive coefficients), puzzle games rank lower in
popularity (which accounts for its negative coefficient). 61
Table 5.17: Estimates for Model (8)
Dependent Variable: log(initial)
Variable*
Coefficient
Standard Error
P-Value
before
msabove75
triplea75q
obscure25q
xboxone
xbox360
puzzle
rpg
shooter
sports
0.160257
0.813981
1.478414
-1.115239
0.550392
-0.674697
-3.493740
1.352195
1.511831
1.362912
0.205339
0.386122
0.252953
0.252953
0.260290
0.277427
1.080035
0.452193
0.411324
0.420003
0.4359
0.0361
0.0000
0.0000
0.0355
0.0157
0.0014
0.0031
0.0055
0.0013
R-Squared
0.489213
N
264
F-Statistic
11.03707
F P-Value
0.0000
*Most insignificant variables omitted here. See Appendix B for full results.
Source: Data from Metacritic and VGChartz, 2006—2017.
Niall, McCarthy, “America's Favorite Video Game Genres [Infographic],” Forbes, June 26, 2015, accessed April
12, 2017, https://www.forbes.com/sites/niallmccarthy/2015/06/26/americas-favorite-video-game-genresinfographic/.
61
72
One other comment about the above estimates concerns the R-squared, which is much
higher than any previous R-squared – in fact, this stands as one of the highest R-squareds in the
entire text. Hence, these estimates stand among the most theoretically useful.
Regarding the insignificance of before (which suggests review prohibition periods may
not impact consumer purchasing decisions), the reader might still be interested to know that
changing the model to a level model once again results in before appearing to be significant.
Estimates for such a level model appear in Tables 5.18 and 5.19 on page 71 (two sets appear
again as heteroskedasticity appears to be present in the level model).
Like the level version of model (7), transforming model (8) to a level model sees before
become significant while making the variables related to Metascore insignificant! This trend of
seeing variables related to review prohibition periods and variables related to Metascore flipflopping in terms of significance in estimates of models of initial sales continues in some of the
alternative models/estimates presented in the appendix depending on model specifications.
Hence, it will certainly be worth looking into this phenomenon in future studies to see if this
contradictory trend holds.
Like the analysis of quality (e.g., metascore), the reader might be interested to know how
these sales models change if before is swapped out with the variable magnitude. To answer this
question, this study simply modifies model (8) – both its log-level and level-level versions – by
swapping out before with the variable magnitude. Models (9) and (10), which appear on page 73,
illustrate the change. Note that heteroskedasticity only appeared to be present in Model (10).
73
Table 5.18: Level Estimates for Model (8) with Heteroskedasticity-Consistent Errors
Dependent Variable: initial
Variable*
Coefficient
Standard Error
P-Value
before
triplea75q
xboxone
platform
rpg
shooter
sports
59,860.71
184,271.9
75,291.98
-11,712.89
116,095.0
113,655.0
98,240.34
24,739.76
49,039.75
47,591.05
44,151.23
64,693.67
41,516.52
37,416.07
0.0163
0.0002
0.1149
0.0723
0.0740
0.0066
0.0092
R-Squared
0.299519
N
264
F-Statistic
4.927476
F P-Value
0.0000
*Most insignificant variables omitted here. See Appendix B for full results.
Source: Data from Metacritic and VGChartz, 2006—2017.
Table 5.19: Level Estimates for Model (8) without Heteroskedasticity-Consistent Errors
Dependent Variable: initial
Variable*
Coefficient
Standard Error
P-Value
before
triplea75q
xboxone
platform
rpg
shooter
sports
59,860.71
184,271.9
75,291.98
-11,712.89
116,095.0
113,655.0
98,240.34
31,968.74
39,381.81
40,523.98
88,478.99
70,400.98
64,039.19
65,389.34
0.0623
0.0000
0.0644
0.3701
0.1004
0.0772
0.1343
R-Squared
0.299519
N
264
F-Statistic
4.927476
F P-Value
0.0000
*Most insignificant variables omitted here. See Appendix B for full results.
Source: Data from Metacritic and VGChartz, 2006—2017.
74
As one will see in the results (which appear on the following pages), the log-level model
is quite similar to the log-level model that utilized before – model (8). However, results - at least
in regards to the variable related to review prohibition periods - vary greatly from the level –level
version of model (8): whereas before appeared to be significant in level terms, magnitude is
incredibly insignificant in level terms! Again, this may not be all that surprising – as what
matters likely pertains to simply whether a particular embargo period is permissive or restrictive;
not necessarily the extent to which an embargo is permissive or restrictive.
log(𝑖𝑛𝑖𝑡𝑖𝑎𝑙) = 𝑐 + 𝛽1 (𝑚𝑎𝑔𝑛𝑖𝑡𝑢𝑑𝑒) + 𝛽2 (𝑚𝑠𝑎𝑏𝑜𝑣𝑒75) + 𝛽3 (𝑚𝑠50𝑡𝑜74) + 𝛽4 (𝑡𝑟𝑖𝑝𝑙𝑒𝑎75𝑞)
+ 𝛽5 (𝑜𝑏𝑠𝑐𝑢𝑟𝑒25𝑞) + 𝛽6 (𝑥𝑏𝑜𝑥𝑜𝑛𝑒) + 𝛽7 (𝑥𝑏𝑜𝑥360) + 𝛽8 (𝑝𝑠3)
+ 𝛽9 (𝑒𝑥𝑐𝑙𝑢𝑠𝑖𝑣𝑒) + 𝛽10 (𝑠𝑒𝑞𝑢𝑒𝑙) + 𝛽11 (𝑎𝑐𝑡𝑖𝑜𝑛) + 𝛽12 (𝑎𝑑𝑣𝑒𝑛𝑡𝑢𝑟𝑒)
+ 𝛽13 (𝑓𝑖𝑔ℎ𝑡𝑖𝑛𝑔) + 𝛽14 (𝑝𝑙𝑎𝑡𝑓𝑜𝑟𝑚) + 𝛽15 (𝑝𝑢𝑧𝑧𝑙𝑒) + 𝛽16 (𝑟𝑎𝑐𝑖𝑛𝑔) + 𝛽17 (𝑟𝑝𝑔)
+ 𝛽18 (𝑠ℎ𝑜𝑜𝑡𝑒𝑟) + 𝛽19 (𝑠𝑖𝑚𝑢𝑙𝑎𝑡𝑖𝑜𝑛) + 𝛽20 (𝑠𝑝𝑜𝑟𝑡𝑠) + 𝛽21 (𝑠𝑡𝑟𝑎𝑡𝑒𝑔𝑦) + 𝑢
(9)
initial = 𝑐 + 𝛽1 (𝑚𝑎𝑔𝑛𝑖𝑡𝑢𝑑𝑒) + 𝛽2 (𝑚𝑠𝑎𝑏𝑜𝑣𝑒75) + 𝛽3 (𝑚𝑠50𝑡𝑜74) + 𝛽4 (𝑡𝑟𝑖𝑝𝑙𝑒𝑎75𝑞)
+ 𝛽5 (𝑜𝑏𝑠𝑐𝑢𝑟𝑒25𝑞) + 𝛽6 (𝑥𝑏𝑜𝑥𝑜𝑛𝑒) + 𝛽7 (𝑥𝑏𝑜𝑥360) + 𝛽8 (𝑝𝑠3)
+ 𝛽9 (𝑒𝑥𝑐𝑙𝑢𝑠𝑖𝑣𝑒) + 𝛽10 (𝑠𝑒𝑞𝑢𝑒𝑙) + 𝛽11 (𝑎𝑐𝑡𝑖𝑜𝑛) + 𝛽12 (𝑎𝑑𝑣𝑒𝑛𝑡𝑢𝑟𝑒)
+ 𝛽13 (𝑓𝑖𝑔ℎ𝑡𝑖𝑛𝑔) + 𝛽14 (𝑝𝑙𝑎𝑡𝑓𝑜𝑟𝑚) + 𝛽15 (𝑝𝑢𝑧𝑧𝑙𝑒) + 𝛽16 (𝑟𝑎𝑐𝑖𝑛𝑔) + 𝛽17 (𝑟𝑝𝑔)
+ 𝛽18 (𝑠ℎ𝑜𝑜𝑡𝑒𝑟) + 𝛽19 (𝑠𝑖𝑚𝑢𝑙𝑎𝑡𝑖𝑜𝑛 ) + 𝛽20 (𝑠𝑝𝑜𝑟𝑡𝑠) + 𝛽21 (𝑠𝑡𝑟𝑎𝑡𝑒𝑔𝑦)
+𝑢
(10)
75
Table 5.20: Estimates for Model (9)
Dependent Variable: log(initial)
Variable*
Coefficient
Standard Error
P-Value
constant
magnitude
msabove75
triplea75q
obscure25q
xboxone
xbox360
puzzle
rpg
shooter
sports
9.018809
-0.023962
0.783454
1.505243
-1.050830
0.554812
-0.700282
-3.574739
1.299847
1.121079
1.226727
0.532023
0.020180
0.383598
0.246190
0.258029
0.258609
0.277206
1.080240
0.454301
0.411705
0.683101
0.0000
0.2362
0.0422
0.0000
0.0001
0.0329
0.0122
0.0011
0.0046
0.0069
0.7402
R-Squared
0.490893
N
264
F-Statistic
11.03707
F P-Value
0.0000
*Most insignificant variables omitted here. See Appendix B for full results.
Source: Data from Metacritic and VGChartz, 2006—2017.
Table 5.21: Estimates for Model (10) without Heteroskedasticity-Consistent Errors
Dependent Variable: initial (in units sold)
Variable*
Coefficient
Standard Error
P-Value
magnitude
triplea75q
msabove75
xboxone
rpg
shooter
sports
-107.4133
201,259.0
68,982.51
84,014.49
119,488.0
115,412.6
98,182.04
3,330.549
50,523.70
33,959.28
48,020.38
64,232.81
42,574.88
36,736.33
0.9746
0.0001
0.0433
0.0815
0.0641
0.0072
0.0080
R-Squared
0.289374
N
264
F-Statistic
4.692606
F P-Value
0.0000
*Most insignificant variables omitted here. See Appendix B for
Source: Data from Metacritic and VGChartz, 2006—2017.
76
Table 5.22: Estimates for Model (10) without Heteroskedasticity-Consistent Errors
Dependent Variable: initial (in units sold)
Variable*
Coefficient
Standard Error
P-Value
magnitude
triplea75q
msabove75
xboxone
rpg
shooter
sports
-107.4133
201,259.0
68,982.51
84,014.49
119,488.0
115,412.6
98,182.04
3,169.683
38,669.06
60251.74
40,619.66
71,357.07
64,666.50
65,964.52
0.9730
0.0000
0.2534
0.0397
0.0953
0.0756
0.1379
R-Squared
0. 289374
N
264
F-Statistic
4.692606
F P-Value
0.0000
*Most insignificant variables omitted here. See Appendix B for full results.
Source: Data from Metacritic and VGChartz, 2006—2017.
77
6. Discussion and Conclusions
Before making concluding remarks, it will be helpful to recall one final time the two
overarching questions considered by this study: First, do review prohibition periods appear
function as a signal of video game quality? Second, do review prohibition periods appear to
impact consumer purchasing decisions? Note that this study hypothesized that the answer to both
of these questions is yes – the logic went that restrictive review policies might be indicative of
low confidence on the publisher’s part, which in turn might be indicative of a low-quality game.
Consequently, gamers should be deterred from purchasing the game, hence, harming sales. In
fact, the results found by this study suggest the actual answer may be a bit mixed! In terms of
functioning as a signal of quality, the results demonstrate strong evidence that review prohibition
periods are indicative of video game quality. However, in terms of impacting sales, the answer is
very uncertain. In terms of percentage changes to sales, review prohibition periods appear to be
quite insignificant. Strangely enough, though, when considering actual unit changes to sales,
review prohibition periods appear to have some significance. If this is indicative of anything, it is
likely indicating that this is certainly what future studies on this topic might want to focus on.
Though the author stands confident in claiming that review prohibition periods signal quality, the
author feels it would be misleading to make any definite conclusions for the impact on sales with
these results.
Although not the core focus of this paper, another interesting result from this study
concerns the classification of games as either triple-A or as obscure. Similar to getting
recognition in life, whereas the blockbuster triple-A games that always enter the market with an
exorbitant amount of marketing and fanfare see arguably superb returns (in terms of sales), the
more obscure and humble games – although some may be just as if not more outstanding –
78
appear to be at a clear disadvantage. Of course, this is likely not all that surprising of a result.
Indeed, this is precisely the result the author expected.
Going back to the mixed results concerning review embargo periods, one might recall the
discussion that took place in the literature review as well as the section tying embargoes to
economic theory. The results in this paper indicate review prohibition periods function as a
signal of quality, however, they do not necessarily impact sales. A question to consider might be
as follows: Is this what publishers would like to see? Although publishers might jump at the
chance to dismiss the conjecture that review embargoes say something about quality, they would
likely love the latter result (perhaps in secret though!) The reason, of course, pertains to the fact
that the seemingly contradictory results suggest review restrictive prohibition periods are a
“safe” way to somewhat mitigate the third-party influence – they prevent consumers from
reading bad reviews before release but do not appear to negatively impact sales. Future studies
would do well to consider examining the mitigating effects review prohibition periods may have
on third-parties in much more detail.
In going forward, there exist a number of ways in which this study could be expanded
upon and improved. Not surprisingly, the author feels one of the best ways to improve upon this
study would be to simply increase the sample size. Of course, this is much more difficult (or
perhaps more appropriately, more tedious) in practice than it sounds on paper. The development
of a robust database made up of a plethora of data on video games in a format useful for
researchers in order to streamline the process would be a wonderful development for the field.
Furthermore, as discussed, many results included relatively low R-squared. Thus, future studies
might want to consider additional variables that might result in more explanatory power. Finally,
any individual that considers furthering work on this topic might want to consider current trends
79
in the industry. Indeed, the video game industry is ever-changing. The traditional media outlet is
arguably being supplanted by “YouTubers” and “Streamers” who provide consumers with a
much more direct and intimate look at games than do traditional reviewers. Something else
future studies might want to consider regards applying the logic of this study to other forms of
media. For instance, the author knows that movies have review embargo periods as well
(however, the author considers his comparative advantage to be in writing about the video game
industry rather than the movie industry!) It would be very interesting to see if review prohibition
periods in other industries have similar consequences and results.
To wrap up, the author would like to address the reader as a gamer. As noted above,
review prohibition periods certainly do appear to be indicative of quality. Does this necessarily
mean that review embargos are bad? Or does this mean that if a game has a restrictive review
embargo that one should avoid purchasing it? No and no. Review prohibition periods constitute
only one part of the picture. Yes, they certainly do appear to be indicative of quality. However,
there are likely a plethora of other factors that are also indicative of quality. For any gamer
reading this paper, these other qualities should be considered as well. If anything, a restrictive
review embargo simply indicates that one might want to heighten their guard a bit rather than to
deem it “game over.”
80
Appendices
Appendix A: Sample Set Summary Statistics
The following tables provide a variety of summary/descriptive statistics on a number of
variables examined in this study. Any reader that would like more information/and or access to
the full data is invited to contact the author directly.
Table A.1: Summary Statistics on Metascore Data
Variable
N
Mean
Median
Max
Min
Range
25thQ 75thQ
metascore
userscore
average1
269
269
71.79
66.46
69.17
75
69
71.5
95
92
92.5
19
12
15.5
76
80
77
64
63
63
81
78
78
msRange2
269
37.77
38
76
6
70
30
48
numPositive3
numMixed3
numNegative3
numreviews4
21.83
11.11
2.03
35.04
15
7
0
31
99
54
43
102
0
0
0
4
99
54
43
98
5
3
0
17
31
15
1
48
%positive5
%Mixed5
%Negative5
54.93
35.12
9.18
63.15
30.77
0
1
1
1
0
0
0
1
1
1
22
11
0
86
56
7.5
1. The average of an observation’s Metascore and Userscore.
2. The difference between an observation’s highest Metascore and lowest Metascore.
3. numPositive, numMixed, and numNegative indicate the number of positive reviews (Metascore >74), mixed
reviews (49 < Metascore < 75), and negative reviews (Metascore < 50) an observation received respectively.
4. The total number of reviews an observation had on Metacritic.
5. %Positive, %Mixed,and %Negative are calculated by dividing the number of positive reviews by the total number
of reviews for an observation, dividing the number mixed reviews by the total number of reviews for an observation,
and dividing the number of negative reviews for an observation by the total number of reviews for an observation
respectively. All values should be read as percentages with the exception of range.
Source: Data from Metacritic, 2006—2017.
81
Table A.2: Counts for Dummy Variables based on Metascore Data
Dummy Variable
Number = 11
Number = 01
Total
msabove752
ms50to742
msbelow502
135
112
22
134
157
247
269
269
269
Total
269
1. The values for Number = 1 and Number = 0 indicate the number of observations that took on a value of 1 and 0
respectively for the dummy variables.
2. msabove75, ms50to74, and msbelow50 are dummy variables that indicate whether a game’s Metascore was 75
and above, between 50 and 74, or below 50 respectively. For example, if a game’s Metascore is below 50, then
msbelow50 takes on a value of 1 and ms50to74 and msabove75 take on values of 0.
Source: Data from Metacritic, 2006—2017.
Table A.3: Variables Related to Review Prohibition Periods
Variable
N
Mean
Median
Max
Min
Range
25thQ 75thQ
magnitude1
269
-0.74
0
21
-21
42
-4
Dummy Variable
Number = 1
Number = 0
before2
after2
134
135
135
134
1
1. magnitude is calculated by taking the difference (in days) between the first review publication date and the release
date of an observation. A negative value indicates the first review was published prior to release. Likewise a nonnegative value indicates the first review was published on or after release.
2. before and after are dummy variables based off of magnitude. If magnitude is negative for an observation, before
takes on a value of 1 and after a value of 0. Likewise, if magnitude is non-negative for an observation, before takes
on a value of 0 and after a value of 1.
Source:Data from Metacritic, 2006—2017.
82
Table A.4: Summary Statistics for Sales Data
Variable
N
Mean
Median
Max
Min
Range
25thQ 75thQ
Total Sales (In Millions of Units Sold)
NA1
Europe 1
Japan1
ROW1
269
269
269
269
0.41
0.26
0.03
0.09
0.19
0.12
0.00
0.05
3.92
2.61
0.89
1.00
0.00
0.00
0.00
0.00
3.92
2.61
0.89
1.00
0.07
0.02
0.00
0.01
0.48
0.31
0.02
0.12
global2
269
0.80
0.41
6.42
0.01
6.41
0.14
1.00
1822
0.08
1821.92
9.09
117.7
Initial Sales (In Thousands of Units Sold)
initial3
264
131.6
36.03
1. NA, Europe, Japan, and ROW represent sales in millions of units for North America, Europe, Japan, and the rest
of the world respectively.
2. global corresponds to the total worldwide sales in millions of units.
3. initial indicates the first week sales in thousands of units (or, for a few observations, the first week of sales in
which weekly sales data was available).
Source: Data from VGChartz, 2006—2017.
Table A.5: Counts for Dummy Variables Based on Console1
Dummy Variable
Number = 1
Number = 0
Total
xboxone
ps42
xbox360
ps32
71
74
62
62
198
195
207
207
269
269
269
269
Total
269
1. Variables indicate what console an observation was for. For example, if an observation was an Xbox One version
of a game, then xboxone takes on a value of 1 and the other variables take on a value of 0 for that observation.
2. ps4 and ps3 short for PlayStation 4 and PlayStation 3 respectively.
Source: Data from VGChartz, 2006—2017.
83
Table A.6: Counts for Dummy Variables Based on Genre1
Dummy Variable
Number = 1
Number = 0
Total
action
adventure
fighting
misc
platform
puzzle
racing
rpg2
shooter
simulation
sports
strategy
76
7
10
18
11
2
18
26
49
4
41
7
193
262
259
251
258
267
251
243
220
265
228
262
269
269
269
269
269
269
269
269
269
269
269
269
Total
269
1. All these variables represent genre. For instance, if an observation is classified as action, then action takes on a
value of 1 and all other genre dummy variables take on a value of 0 for that observation.
2. Abbreviation for role-playing.
Source: Based on VGChartz genre classifications, 2006—2017.
Table A.7: Counts for Dummy Variables Based on Triple-A/Obscure Classification1
Dummy Variable
Number = 1
Number = 0
Total
tripleaAO2
obscureAO2
81
59
188
210
269
269
triplea75q3
obscure25q3
65
64
204
205
269
269
1. Variables indicate whether an observation might be triple-A or obscure.
2. tripleAO and obscureAO based on author’s opinion/knowledge. For example, if the author considered a game to
be triple-A, then tripleAO takes on a value of 1.
3. triplea75q and obscure25q based on 75th percentile and 25th percentile of sample data on number of number of
reviews. For instance, if the number of reviews for an observation equaled or exceeded the 75 th percentile of the
number of reviews in the sample, triplea75q takes on a value of 1. See Table A.1 numreview for percentile data.
Source: Data from Metacritic, 2006—2017.
84
Table A.8: Counts for Dummy Variables Based on Exclusive and Sequel Classification1
Dummy Variable
Number = 1
Number = 0
Total
exclusive
sequel
59
175
210
94
269
269
1. These variables indicate whether a game was exclusive and whether a game was a sequel. For example, in the
case that a game was both exclusive to a particular console and a sequel, then both exclusive and sequel would take
on a value of 1.
Source: Data from VGChartz, 2006—2017.
End of Appendix A.
85
Appendix B: Full Results from Estimates Presented in the Main Text
The following presents the full estimates for the models presented in the main text. Like
in the main text, models with two sets of estimates indicate that heteroskedasticity was present
based on the White Test (without cross-products).
Table B.1: Full Estimates for Model (1) with Heteroskedasticity-Consistent Errors
Dependent Variable: metascore
Variable
Coefficient
Standard Error
P-Value
constant
before
65.77778
11.87894
1.305941
1.520721
0.0000
0.0000
R-Squared
0.185482
N
269
F-Statistic
60.80315
F P-Value
0.0000
Source: Data from Metacritic, 2006—2017.
Table B.2: Full Estimates for Model (1) without Heteroskedasticity-consistent Errors
Dependent Variable: metascore
Variable
Coefficient
Standard Error
P-Value
constant
before
65.77778
11.87894
1.075220
1.523425
0.0000
0.0000
R-Squared
0.185482
N
269
F-Statistic
60.80135
F P-Value
0.0000
Source: Data from Metacritic, 2006—2017.
86
Table B.3: Full Estimates for Model (2) with Heteroskedasticity-consistent Errors
Dependent Variable: metascore
Variable
Coefficient
Standard Error
P-Value
constant
before
triplea75q
obscure25q
xboxone
xbox360
ps3
exclusive
sequel
68.78259
9.395494
4.097533
-5.549972
1.589195
-4.689126
2.526614
-1.429962
6.079752
2.061884
1.635120
1.691340
2.245982
2.119143
2.056773
1.841270
1.659209
1.517975
0.0000
0.0000
0.0161
0.0141
0.4540
0.0234
0.1792
0.3896
0.0001
R-Squared
0.304576
N
269
F-Statistic
14.23410
F P-Value
0.0000
Source: Data from Metacritic, 2006—2017.
Table B.4: Full Estimates for Model (2) without Heteroskedasticity-consistent Errors
Dependent Variable: metascore
Variable
Coefficient
Standard Error
P-Value
constant
before
triplea75q
obscure25q
xbox360
sequel
63.78259
9.395494
4.097533
-5.549972
-4.689126
6.079752
2.071745
1.577733
1.939374
1.889737
2.104945
1.540365
0.0000
0.0000
0.0356
0.0036
0.0268
0.0001
R-Squared
0.304576
N
269
F-Statistic
14.23410
F P-Value
0.0000
Source: Data from Metacritic and VGChartz, 2006—2017.
Appendix B continues on the following page.
87
Table B.5: Full Estimates for Model (3) with Heteroskedasticity-Consistent Errors
Dependent Variable: metascore
Variable
Coefficient
Standard Error
P-Value
constant
before
triplea75q
obscure25q
xboxone
xbox360
ps3
exclusive
sequel
action
adventure
fighting
platform
puzzle
racing
rpg
shooter
simulation
sports
strategy
62.29037
9.138273
3.954544
-5.648930
2.029514
-4.246354
2.947162
-1.591656
6.211801
0.552956
0.688828
4.880932
-0.818811
14.43632
0.910933
5.389384
1.304019
0.876005
-0.149731
2.834298
4.028484
1.644265
1.859947
2.420594
2.142488
2.121587
1.921309
1.677428
1.599045
3.633245
4.766307
4.833579
5.062686
4.651117
4.752512
3.643236
3.443399
6.169078
3.675868
4.265090
0.0000
0.0000
0.0345
0.0204
0.3444
0.0464
0.1263
0.3436
0.0001
0.8792
0.8852
0.3136
0.8716
0.0021
0.8482
0.1403
0.7139
0.8872
0.9683
0.5070
R-Squared
0.326425
N
269
F-Statistic
6.351004
F P-Value
0.0000
Source: Data from Metacritic and VGChartz, 2006—2017.
Appendix B continues on the following page.
88
Table B.6: Full Estimates for Model (3) without Heteroskedasticity-consistent Errors
Dependent Variable: metascore
Variable
Coefficient
Standard Error
P-Value
constant
before
triplea75q
obscure25q
xboxone
xbox360
ps3
exclusive
sequel
action
adventure
fighting
platform
puzzle
racing
rpg
shooter
simulation
sports
strategy
62.29037
9.138273
3.954544
-5.648930
2.029514
-4.246354
2.947162
-1.591656
6.211801
0.552956
0.688828
4.880932
-0.818811
14.43632
0.910933
5.389384
1.304019
0.876005
-0.149731
2.834298
3.533709
1.605414
2.063333
1.995068
2.117445
2.210267
2.100298
1.909268
1.705820
3.168070
5.384340
4.672575
4.674241
8.967067
3.975365
3.705109
3.364415
6.590645
3.423354
5.278899
0.0000
0.0000
0.0564
0.0050
0.3388
0.0558
0.1618
0.4053
0.0003
0.8616
0.8983
0.2972
0.8611
0.1087
0.8189
0.1470
0.6986
0.8944
0.9651
0.5918
R-Squared
0.326425
N
269
F-Statistic
6.351004
F P-Value
0.0000
Source: Data from Metacritic and VGChartz, 2006—2017.
Appendix B continues on the following page.
89
Table B.7: Full Estimates for Model (4) with Heteroskedasticity-Consistent Errors
Dependent Variable: metascore
Variable
Coefficient
Standard Error
P-Value
constant
magnitude
triplea75q
obscure25q
xboxone
xbox360
ps3
exclusive
sequel
67.92521
-0.880109
6.054874
-3.927962
2.344464
-5.923882
-2.340662
-2.385304
5.099176
1.802949
0.164074
1.645082
2.421950
2.074601
2.047601
1.873529
1.736130
1.527226
0.0000
0.0000
0.0003
0.1061
0.2495
0.0050
0.2127
0.1706
0.0010
R-Squared
0.296082
N
269
F-Statistic
13.67015
F P-Value
0.0000
Source: Data from Metacritic, 2006—2017.
Table B.8: Full Estimates for Model (4) without Heteroskedasticity-Consistent Errors
Dependent Variable: metascore
Variable
Coefficient
Standard Error
P-Value
constant
magnitude
triplea75q
obscure25q
xboxone
xbox360
ps3
exclusive
sequel
67.92521
-0.880109
6.054874
-3.927962
2.344464
-5.923882
-2.340662
-2.385304
5.099176
1.961187
0.155833
1.882233
1.991764
2.059868
2.109822
2.041566
1.878825
1.561730
0.0000
0.0000
0.0015
0.0497
0.2561
0.0054
0.2526
0.2054
0.0012
R-Squared
0.296082
N
269
F-Statistic
13.67015
F P-Value
0.0000
Source: Data from Metacritic, 2006—2017.
90
Table B.9: Full Estimates for Model (5) with Heteroskedasticity-Consistent Errors
Dependent Variable: metascore
Variable
Coefficient
Standard Error
P-Value
constant
magnitude
triplea75q
obscure25q
xboxone
xbox360
ps3
exclusive
sequel
action
adventure
fighting
platform
puzzle
racing
rpg
shooter
simulation
sports
strategy
67.10209
-0.844586
5.967323
-4.153478
2.756027
-5.406716
2.904918
-2.463704
5.333358
-0.006462
-0.131638
3.524671
-1.373320
11.67371
-1.216068
3.916582
0.411995
-1.105134
-1.086558
3.255194
3.676498
0.169591
1.847546
2.183269
2.585620
2.109822
1.952310
3.547619
1.606197
3.547619
4.523634
4.671515
5.199386
5.606253
4.675275
3.550179
3.475227
6.602431
3.645944
3.989449
0.0000
0.0000
0.0014
0.0139
0.1952
0.0054
0.1380
0.1668
0.0010
0.9985
0.9768
0.9768
0.7919
0.0383
0.7950
0.2710
0.9057
0.8672
0.7659
0.4153
R-Squared
0.313723
N
269
F-Statistic
5.990910
F P-Value
0.0000
Source: Data from Metacritic, 2006—2017.
Appendix B continues on the following page.
91
Table B.10: Full Estimates for Model (5) with Heteroskedasticity-Consistent Errors
Dependent Variable: metascore
Variable
Coefficient
Standard Error
P-Value
constant
magnitude
triplea75q
obscure25q
xboxone
xbox360
ps3
exclusive
sequel
action
adventure
fighting
platform
puzzle
racing
rpg
shooter
simulation
sports
strategy
67.10209
-0.844586
5.967323
-4.153478
2.756027
-5.406716
2.904918
-2.463704
5.333358
-0.006462
-0.131638
3.524671
-1.373320
11.67371
-1.216068
3.916582
0.411995
-1.105134
-1.086558
3.255194
3.513289
0.161964
2.020059
2.111611
2.123985
2.226964
2.119919
1.940033
1.730628
3.200361
5.453688
4.727470
4.724617
9.074952
4.051042
3.916582
3.406718
6.639093
3.460453
3.255194
0.0000
0.0000
0.0034
0.0503
0.1956
0.0159
0.1718
0.2053
0.0023
0.9984
0.9808
0.4566
0.7715
0.1995
0.7643
0.3000
0.9038
0.8679
0.7538
0.5418
R-Squared
0.313723
N
269
F-Statistic
5.990910
F P-Value
0.0000
Source: Data from Metacritic, 2006—2017.
Appendix B continues on the following page.
92
Table B.11: Full Fill Estimates for Model (6) with Heteroskedasticity-Consistent Errors
Dependent Variable: log(initial)
Variable
Coefficient
Standard Error
P-Value
constant
before
9.766235
1.198899
0.141644
0.221372
0.0000
0.0000
R-Squared
0.100420
N
264
F-Statistic
29.24697
F P-Value
0.0000
Source: Data from Metacritic and VGChartz, 2006—2017.
Table B.12: Full Estimates for Model (6) without Heteroskedasticity-Consistent Errors
Dependent Variable: log(initial)
Variable
Coefficient
Standard Error
P-Value
constant
before
9.766235
1.198899
0.157350
0.221688
0.0000
0.0000
R-Squared
0.100420
N
264
F-Statistic
29.24697
F P-Value
0.0000
Source: Data from Metacritic and VGChartz, 2006—2017.
Appendix B continues on the following page.
93
Table B.13: Full Estimates for Model (7) with Heteroskedasticity-Consistent Errors
Dependent Variable: log(initial)
Variable
Coefficient
Standard Error
P-Value
constant
before
msabove75
ms50to74
triplea75q
obscure25q
xboxone
xbox360
ps3
exclusive
sequel
9.412195
0.125851
0.790636
-0.007694
1.637736
-1.327773
0.782112
-0.499814
0.249651
-0.095237
0.437046
0.429193
0.221250
0.344579
0.320762
0.248322
0.262546
0.283635
0.289084
0.251680
0.256956
0.208315
0.0000
0.5700
0.0226
0.9809
0.0000
0.0000
0.0062
0.0850
0.3222
0.7108
0.0369
R-Squared
0.390636
N
264
F-Statistic
16.21872
F P-Value
0.0000
Source: Data from Metacritic and VGChartz, 2006—2017.
Appendix B continues on the following page.
94
Table B.14: Full Estimates for Model (7) without Heteroskedasticity-Consistent Errors
Dependent Variable: log(initial)
Variable
Coefficient
Standard Error
P-Value
constant
before
msabove75
ms50to74
triplea75q
obscure25q
xboxone
xbox360
ps3
exclusive
sequel
9.412195
0.125851
0.790636
-0.007694
1.637736
-1.327773
0.782112
-0.499814
0.249651
-0.095237
0.437046
0.429193
0.217459
0.405527
0.387092
0.254384
0.252649
0.270061
0.282586
0.267509
0.247470
0.205581
0.0000
0.5633
0.0523
0.9842
0.0000
0.0000
0.0041
0.0781
0.3516
0.7007
0.0345
R-Squared
0.390636
N
264
F-Statistic
16.21872
F P-Value
0.0000
Source: Data from Metacritic and VGChartz, 2006—2017.
Appendix B continues on the following page.
95
Table B.15: Full Level Estimates for Model (7) with Heteroskedasticity-Consistent Errors
Dependent Variable: initial (in units sold)
Variable
Coefficient
Standard Error
P-Value
constant
before
msabove75
ms50to74
triplea75q
obscure25q
xboxone
xbox360
ps3
exclusive
sequel
24,729.73
57,472.09
50,660.90
-42,691.51
198,061.0
-50,062.17
85,423.77
-58,356.37
-24,247.82
3,239.382
45,758.22
43,484.63
25,335.99
32625.29
51,510.56
29,465.28
32,905.10
50,011.32
41,359.65
35,354.60
39,019.33
26,737.28
0.5701
0.0241
0.1217
0.0002
0.0002
0.1294
0.0888
0.1595
0.4935
0.9339
0.0882
R-Squared
0.254343
N
264
F-Statistic
16.21872
F P-Value
0.0000
Source: Data from Metacritic and VGChartz, 2006—2017.
Appendix B continues on the following page.
96
Table B.16: Full Level Estimates for Model (7) without Heteroskedasticity-Consistent
Errors
Dependent Variable: initial
Variable
constant
before
msabove75
ms50to74
triplea75q
obscure25q
xboxone
xbox360
ps3
exclusive
sequel
Coefficient
24,729.73
57,472.09
50,660.90
-42,691.51
198,061.0
-50,062.17
85,423.77
-58,356.37
-24,247.82
3,239.382
45,758.22
Standard Error
P-Value
63,119.17
31,980.49
59,683.70
56,927.48
37,410.92
37,155.79
39,716.48
41,558.37
39,341.12
36,394.02
30,233.72
0.6955
0.0735
0.3964
0.4540
0.0000
0.1791
0.0324
0.1615
0.5382
0.9291
0.1314
R-Squared
0.254343
N
264
F-Statistic
8.629810
F P-Value
0.0000
Source: Data from Metacritic and VGChartz, 2006—2017.
Appendix B continues on the following page.
97
Table B.17: Full Estimates for Model (8)
Dependent Variable: log(initial)
Variable
Coefficient
Standard Error
P-Value
constant
before
msabove75
ms50to74
triplea75q
obscure25q
xboxone
xbox360
ps3
exclusive
sequel
action
adventure
fighting
platform
puzzle
rpg
racing
shooter
simulation
sports
strategy
8.905157
0.160257
0.813981
0.036775
1.478414
-1.115239
0.550392
-0.674697
0.085240
0.006020
0.175429
0.492000
-0.695070
0.609441
0.259257
-3.493740
1.352195
0.436524
1.511831
0.940585
1.362912
0.226551
0.527986
0.205339
0.386122
0.367187
0.252953
0.252953
0.260290
0.277427
0.258965
0.235936
0.212148
0.387212
0.688717
0.586780
0.568310
1.080035
0.452193
0.483483
0.411324
0.800900
0.420003
0.684227
0.0000
0.4359
0.0361
0.9203
0.0000
0.0000
0.0355
0.0157
0.7423
0.9797
0.4091
0.2051
0.3139
0.3000
0.6487
0.0014
0.0031
0.3675
0.0055
0.2414
0.0013
0.7409
R-Squared
0.489213
N
264
F-Statistic
11.03707
F P-Value
0.0000
Source: Data from Metacritic and VGChartz, 2006—2017.
Appendix B continues on the following page.
98
Table B.18: Full Level Estimates for Model (8) with Heteroskedasticity-Consistent Errors
Dependent Variable: initial
Variable
Coefficient
Standard Error
P-Value
constant
before
msabove75
ms50to74
triplea75q
obscure25q
xboxone
xbox360
ps3
exclusive
sequel
action
adventure
fighting
platform
puzzle
racing
rpg
shooter
simulation
sports
strategy
-24,251.24
59,860.71
40,886.85
-45,909.27
184,271.9
-34,383.20
75,291.98
-61,382.31
-21,165.57
18,681.98
42,812.21
36,709.87
-51,399.66
22,841.31
-79,451.32
-11,712.89
28,871.15
116,095.0
113,655.0
-29,096.43
98,240.34
-34,198.19
53,795.95
24,739.76
35,186.77
32,783.60
49,039.75
32,378.32
47,591.05
43,434.90
35,274.79
39,192.95
28,828.49
29,398.11
47,444.97
35,941.05
44,151.23
73,397.72
41,405.45
64,693.67
41,516.52
61,920.29
37,416.07
40,184.86
0.6512
0.0163
0.2464
0.1627
0.0002
0.2893
0.1149
0.1589
0.5491
0.6340
0.1390
0.2130
0.2797
0.5257
0.0723
0.8735
0.4763
0.0740
0.0066
0.6388
0.0092
0.3956
R-Squared
0.299519
N
264
F-Statistic
4.927476
F P-Value
0.0000
Source: Data from Metacritic and VGChartz, 2006—2017.
Appendix B continues on the following page.
99
Table B.19: Full Level Estimates for Model (8) without Heteroskedasticity-Consistent
Errors
Dependent Variable: initial
Variable
Coefficient
Standard Error
P-Value
constant
before
msabove75
ms50to74
triplea75q
obscure25q
xboxone
xbox360
ps3
exclusive
sequel
fighting
platform
puzzle
racing
rpg
shooter
simulation
sports
strategy
-24,351.24
59,860.71
40,886.85
-45,909.27
184,271.9
-34,383.20
75,291.98
-61,382.31
-21,165.57
18,681.98
42,812.21
22,841.31
-11,712.89
-11,712.89
28,871.15
116,095.0
113,655.0
-29,096.43
98,240.34
-34,198.19
82,201.08
31,968.74
60,114.53
57,166.63
39,381.81
38,727.32
40,523.98
43,191.98
40,317.67
36,732.42
33,028.89
91,354.53
88,478.99
168,148.4
75,272.55
70,400.98
64,039.19
124,690.4
65,389.34
106,526.0
0.7673
0.0623
0.4971
0.4227
0.0000
0.3755
0.0644
0.1566
0.6001
0.6115
0.1961
0.8028
0.3701
0.9445
0.7016
0.1004
0.0772
0.8157
0.1343
0.7485
R-Squared
0.299519
N
264
F-Statistic
4.927476
F P-Value
0.0000
Source: Data from Metacritic and VGChartz, 2006—2017.
Appendix B continues on the following page.
100
Table B.20: Full Estimates for Model (9)
Dependent Variable: log(initial)
Variable
Coefficient
Standard Error
P-Value
constant
magnitude
msabove75
ms50to74
triplea75q
obscure25q
xboxone
xbox360
ps3
exclusive
sequel
action
adventure
fighting
platform
puzzle
racing
rpg
shooter
simulation
sports
strategy
9.018809
-0.023962
0.783454
0.021681
1.505243
-1.050830
0.554812
-0.700282
0.091593
-0.021141
0.154725
0.476206
-0.733091
0.565159
0.228547
-3.574739
0.367563
1.299847
1.121079
0.907621
1.226727
0.226727
0.532023
0.020180
0.383598
0.366290
0.246190
0.258029
0.258609
0.277206
0.258512
0.237335
0.211714
0.386844
0.688818
0.586845
0.568411
1.080240
0.487020
0.454301
0.411705
0.797984
0.683101
0.683101
0.0000
0.2362
0.0422
0.9528
0.0000
0.0001
0.0329
0.0122
0.7234
0.9291
0.4656
0.2195
0.2883
0.3365
0.6880
0.0011
0.4512
0.0046
0.0069
0.2565
0.7402
0.7402
R-Squared
0.490893
N
264
F-Statistic
11.11154
F P-Value
0.0000
Source: Data from Metacritic and VGChartz, 2006—2017.
Appendix B continues on the following page.
101
Table B .21: Full Estimates for Model (10) with Heteroskedasticity-Consistent Errors
Dependent Variable: initial (in units sold)
Variable
Coefficient
Standard Error
P-Value
constant
magnitude
msabove75
msto50to74
triplea75q
obscure25q
xboxone
xbox360
ps3
exclusive
sequel
action
adventure
fighting
platform
puzzle
racing
rpg
shooter
simulation
sports
strategy
-15,778.70
-107.4133
68,982.51
-33,146.00
201,259.0
-45,781.63
84,014.49
-64,332.54
-25,933.35
22,846.67
39,393.92
37,185.05
-36,816.95
21,620.52
-71,527.72
-13,242.84
31,988.97
119,488.0
115,412.6
-43,907.36
98,182.04
-34,379.56
54,505.42
3,330.549
33,959.28
31,698.86
50,523.70
36,722.82
48,020.38
43,492.10
35,657.44
40,890.17
29,267.18
29,094.45
45,237.25
32,225.62
42,572.78
63,027.10
43,463.26
64,232.81
42,574.88
67,614.22
36,736.33
38,111.51
0.7725
0.9746
0.0433
0.2968
0.0001
0.2137
0.0815
0.1404
0.4667
0.5769
0.1796
0.2024
0.4165
0.5029
0.0942
0.8338
0.4624
0.0641
0.0072
0.5167
0.0080
0.3679
R-Squared
0.289374
N
264
F-Statistic
4.692606
F P-Value
0.0000
Source: Data from Metacritic and VGChartz, 2006—2017.
Appendix B continues on the following page.
102
Table B.22: Full Estimates for Model (10) without Heteroskedasticity-Consistent Errors
Dependent Variable: initial (in units sold)
Variable
Coefficient
Standard Error
P-Value
constant
magnitude
msabove75
msto50to74
triplea75q
obscure25q
xboxone
xbox360
ps3
exclusive
sequel
action
adventure
fighting
platform
puzzle
racing
rpg
shooter
simulation
sports
strategy
-15,778.70
-107.4133
68,982.51
-33,146.00
201,259.0
-45,781.63
84,014.49
-64,332.54
-25,933.35
22,846.67
39,393.92
37,185.05
-36,816.95
21,620.52
-71,527.72
-13,242.84
31,988.97
119,488.0
115,412.6
-43,907.36
98,182.04
-34,379.56
83,564.86
3,169.683
57,533.12
57,533.12
38,669.06
40,528.58
40,619.66
43,540.68
40,604.54
37,278.27
33,253.91
60,761.52
108,192.6
92,175.76
89,280.35
169,673.3
76,496.23
71,357.07
64,666.50
125,339.4
65,964.52
107,294.6
0.8504
0.9730
0.2534
0.2534
0.0000
0.2598
0.0397
0.1408
0.5227
0.5405
0.2373
0.5411
0.7339
0.8148
0.4238
0.9379
0.6762
0.0953
0.0756
0.7264
0.1379
0.7489
R-Squared
0. 289374
N
264
F-Statistic
4.692606
F P-Value
0.0000
Source: Data from Metacritic and VGChartz, 2006—2017.
End of Appendix B.
103
Appendix C: Some Models/Estimates with Alternative Variables and Alternative Forms
Here one can find a variety of additional models – including some variations of models
from the main text. For instance, a log model of metascore is presented here. Additionally, some
of these models use alternative variables, such as variables that classify games based on the
author’s own knowledge as opposed to the variables utilized in the main text which were based
off the number of reviews a game had received. Note that for models in which two sets of results
appear, one can assume heteroskedasticity was detected. Tables begin on the following page. In
parentheses at the end of each table heading is the key variable the reader should look at (i.e.,
before or magnitude). Furthermore, each table presents full results – equations are not explicitly
written.
Table C.1: Log Model of metascore with Heteroskedasticity-Consistent Errors (before)
Dependent Variable: log(metascore)
Variable
Coefficient
Standard Error
P-Value
constant
before
triplea75q
obscure25q
xboxone
xbox360
ps3
exclusive
sequel
4.129147
0.148445
0.059320
-0.106895
0.025018
-0.091651
0.085240
0.019378
0.103454
0.033807
0.026358
0.026347
0.041185
0.034426
0.036702
0.027744
0.025903
0.026272
0.0000
0.0000
0.0252
0.0100
0.4681
0.0131
0.1873
0.4551
0.0001
R-Squared
0.300014
N
269
F-Statistic
13.92951
F P-Value
0.0000
Source: Data from Metacritic and VGChartz, 2006—2017.
104
Table C.2: Log Model of metascore without Heteroskedasticity-Consistent Errors (before)
Dependent Variable: log(metascore)
Variable
Coefficient
Standard Error
P-Value
constant
before
triplea75q
obscure25q
xboxone
xbox360
ps3
exclusive
sequel
4.129147
0.148445
0.059320
-0.106895
0.025018
-0.091651
0.085240
0.019378
0.103454
0.034985
0.026643
0.032750
0.031911
0.034785
0.035546
0.258965
0.031313
0.026012
0.0000
0.0000
0.0712
0.0009
0.4727
0.0105
0.7423
0.5366
0.0001
R-Squared
0.300014
N
269
F-Statistic
13.92951
F P-Value
0.0000
Source: Data from Metacritic and VGChartz, 2006—2017.
Appendix C continues on the following page.
105
Table C.3: Model of metascore Using Alternative Definitions of Triple-A and Obscure
without Heteroskedasticity-Consistent Errors (before)
Dependent Variable: metascore
Variable
Coefficient
Standard Error
P-Value
constant
before
tripleaAO1
ObscureAO1
xboxone
xbox360
ps3
sequel
65.25942
9.164256
7.232953
-5.906423
-2.769913
-4.090513
1.602191
3.524314
1.996227
1.443375
1.700845
1.903980
1.903980
1.963016
1.945992
1.513673
0.0000
0.0000
0.0000
0.0012
0.1469
0.0382
0.4111
0.0207
R-Squared
N
F-Statistic
F P-Value
0.358648
269
20.85036
0.0000
1. Games classified as triple-A or obscure using author’s expertise/opinion.
Source: Data from Metacritic and VGChartz, 2006—2017.
Appendix C continues on the following page.
106
Table C.4: Model of metascore Using Alternative Definitions of Triple-A and Obscure with
Heteroskedasticity-Consistent Errors (before)
Dependent Variable: metascore
Variable
Coefficient
Standard Error
P-Value
constant
before
tripleaAO1
ObscureAO1
xboxone
xbox360
ps3
sequel
65.25942
9.164256
7.232953
-5.906423
-2.769913
-4.090513
1.602191
3.524314
2.295102
1.465298
1.556083
1.964733
1.911742
2.055134
1.829700
1.574023
0.0000
0.0000
0.0000
0.0029
0.1486
0.0476
0.3820
0.0260
R-Squared
N
F-Statistic
F P-Value
0.358648
269
20.85036
0.0000
1. Games classified as triple-A or obscure using author’s expertise/opinion.
Source: Data from Metacritic and VGChartz, 2006—2017.
Appendix C continues on the following page.
107
Table C.5: Model of initial Using Alternative Definitions of Triple-A and Obscure (before)
Dependent Variable: log(initial)
Variable
Coefficient
Standard Error
P-Value
constant
before1
msabove75
ms50_74
tripleaAO
ObscureAO
xboxone
xbox360
ps3
sequel
9.408936
0.418923
0.366449
0.379286
1.592569
-1.154371
-0.240484
-0.144707
0.126865
-0.063173
0.434432
0.297393
0.401999
0.379286
0.233697
0.253654
0.262856
0.265428
0.262249
0.207706
0.0000
0.0444
0.0636
0.3349
0.0000
0.0000
0.3611
0.5992
0.0804
0.7613
R-Squared
N
F-Statistic
F P-Value
0.402579
264
17.04873
0.0000
1. The author feels a bit compelled to note the significance of before here (as opposed to its insignificance in a similar
model that appeared in the main text). Yes, it appears significant here. But the fact that slightly changing the definition
of other variables (the variables for triple-a and obscure games) boosts its significance suggests it is not all that
significant.
Source: Data from Metacritic and VGChartz, 2006—2017.
Appendix C continues on the following page.
108
Table C.6: Model of global with Heteroskedasticity-Consistent Errors (before)
Dependent Variable: log(global)
Variable
Coefficient
Standard Error
P-Value
constant
before1
msabove75
ms50_74
triplea25q
obscure25q
xboxone
xbox360
ps3
exclusive1
sequel
-2.14807
0.385419
0.778741
0.061371
1.028378
-0.727809
0.189161
0.175645
0.709694
-0.433833
0.421933
0.248183
0.160628
0.209731
0.182580
0.152700
0.199668
0.218590
0.187110
0.189757
0.177234
0.144076
0.0000
0.0171
0.0003
0.7370
0.0000
0.0003
0.3876
0.3492
0.0002
0.0150
0.0037
R-Squared
N
F-Statistic
F P-Value
0.441548
269
20.39912
0.0000
1. Mildly interesting result: one of very few models in which exclusive was significant.
Source: Data from Metacritic and VGChartz, 2006—2017.
Appendix C continues on the following page.
109
Table C.7: Model of global without Heteroskedasticity-Consistent Errors (before)
Dependent Variable: log(global)
Variable
Coefficient
Standard Error
P-Value
constant
before
msabove75
ms50_74
triplea25q
obscure25q
xboxone
xbox360
ps3
exclusive1
sequel
-2.148070
0.385419
0.778741
0.061371
1.028378
-0.727809
0.189161
0.175645
0.709694
-0.433833
0.421933
0.297162
0.155470
0.284436
0.179845
0.179845
0.178079
0.218590
0.190819
0.189930
0.172208
0.145532
0.0000
0.0138
0.0066
0.8203
0.0000
0.0001
0.3876
0.3225
0.0002
0.0124
0.0031
R-Squared
N
F-Statistic
F P-Value
0.441548
269
20.39912
0.0000
1. Mildly interesting result: one of very few models in which exclusive was significant.
Source: Data from Metacritic and VGChartz, 2006—2017.
Appendix C continues on the following page.
110
Table C.8: Model of initial utilizing metascore with Heteroskedasticity-Consistent Errors
(before)
Dependent Variable: log(initial)
Variable
Coefficient
Standard Error
P-Value
constant
before
metascore
triplea75q
obscure25q
xboxone
xbox360
ps3
exclusive
sequel
7.000274
0.025385
0.040109
1.485575
-1.159485
0.746102
-0.297612
0.308565
-0.099746
0.325320
0.640790
0.215918
0.008362
0.251045
0.253754
0.278285
0.288943
0.244626
0.250802
0.211468
0.0000
0.9065
0.0000
0.0000
0.0000
0.0078
0.3040
0.2083
0.6912
0.1252
R-Squared
0.413008
N
264
F-Statistic
19.85716
F P-Value
0.0000
Source: Data from Metacritic and VGChartz, 2006—2017.
Appendix C continues on the following page.
111
Table C.9: Model of initial utilizing metascore without Heteroskedasticity-Consistent Errors
(before)
Dependent Variable: log(initial)
Variable
Coefficient
Standard Error
P-Value
constant
before
metascore
triplea75q
obscure25q
xboxone
xbox360
ps3
exclusive
sequel
7.000274
0.025385
0.040109
1.485575
-1.159485
0.746102
-0.297612
0.308565
-0.099746
0.325320
0.576148
0.213365
0.007951
0.251419
0.246621
0.264737
0.274398
0.259886
0.242258
0.202972
0.0000
0.9054
0.0000
0.0000
0.0000
0.0052
0.2791
0.2362
0.6809
0.1102
R-Squared
0.413008
N
264
F-Statistic
19.85716
F P-Value
0.0000
Source: Data from Metacritic and VGChartz, 2006—2017.
Appendix C continues on the following page.
112
Table C.10: Model of userscore (before)
Dependent Variable: userscore1
Variable
Coefficient
Standard Error
P-Value
constant
before
triplea75q
obscure25q
xboxone
xbox360
ps3
exclusive
sequel
61.56976
3.823607
2.302040
1.882771
-3.194308
-1.843593
3.837963
4.884397
2.031991
2.612828
1.970695
2.424779
2.387231
2.583656
2.636276
2.559785
2.335566
1.941815
0.0000
0.0534
0.3433
0.4310
0.2175
0.4850
0.1350
0.0375
0.2963
R-Squared
N
F-Statistic
F P-Value
0.082263
266
2.879600
0.004361
1. userscore is the average score users of Metacritic give a game – between 0 and 10. For easy comparison with models
of metascore, userscore was scaled up by a factor of 10.
Source: Data from Metacritic and VGChartz, 2006—2017.
Appendix C continues on the following page.
113
Table C.11: Model of initial utilizing userscore without Heteroskedasticity
Consistent Errors (before)
Dependent Variable: log(initial)
Variable
Coefficient
Standard Error
P-Value
constant
before
userscore
triplea75q
obscure25q
xboxone
xbox360
ps3
exclusive
sequel
9.171831
0.381331
0.006169
1.637173
-1.388673
0.835059
-0.479600
0.425409
-0.138371
0.553350
0.528057
0.233079
0.006952
0.245782
0.272447
0.295251
0.283369
0.265876
0.255833
2.216067
0.0000
0.1031
0.3757
0.0000
0.0000
0.0051
0.0918
0.1108
0.5891
0.0110
R-Squared
N
F-Statistic
F P-Value
0.354837
262
15.39985
0.000000
1. userscore is the average score users of Metacritic give a game – between 0 and 10. For easy comparison with models
of metascore, userscore was scaled up by a factor of 10.
Source: Data from Metacritic and VGChartz, 2006—2017.
Appendix C continues on the following page.
114
Table C.12: Model of initial utilizing userscore without Heteroskedasticity
Consistent Errors (before)
Dependent Variable: log(initial)
Variable
Coefficient
Standard Error
P-Value
constant
before
userscore
triplea75q
obscure25q
xboxone
xbox360
ps3
exclusive
sequel
9.171831
0.381331
0.006169
1.637173
-1.388673
0.835059
-0.479600
0.425409
-0.138371
0.553350
0.504993
0.212407
0.006742
0.262328
0.256610
0.272863
0.285161
0.275618
0.259140
2.648256
0.0000
0.0736
0.3611
0.0000
0.0000
0.0030
0.0938
0.1240
0.5938
0.0086
R-Squared
N
F-Statistic
F P-Value
0.354837
262
15.39985
0.000000
1. userscore is the average score users of Metacritic give a game – between 0 and 10. For easy comparison with models
of metascore, userscore was scaled up by a factor of 10.
Source: Data from Metacritic and VGChartz, 2006—2017.
End of Appendix C.
115
Appendix D: Some Models/Estimates with Interaction Terms
To end the appendices, a few models with interaction terms are presented here. Due to
the relatively small sample size, these models are much more concise than other models in the
text. Again, the presence of two sets of results indicates heteroskedasticity was detected using
White’s Test.
Table D.1: Model of metascore with interaction between before and triple-a and obscure
classification with Heteroskedasticity-Consistent Errors
Dependent Variable: metascore
Variable
Coefficient
Standard Error
P-Value
constant
before
triplea75q
obscure25q
before*triplea75q
before*obscure25q
67.73333
9.220513
1.112821
-5.924823
1.241025
4.265094
1.566107
1.859060
4.620135
2.917915
4.867784
4.426677
0.0000
0.0000
0.8098
0.0433
0.7990
0.3362
R-Squared
0.262150
N
269
F-Statistic
11.54697
F P-Value
0.0000
Source: Data from Metacritic and VGChartz, 2006—2017.
116
Table D.2: Model of metascore with interaction between before and triple-a and obscure
classification without Heteroskedasticity-Consistent Errors
Dependent Variable: metascore
Variable
Coefficient
Standard Error
P-Value
constant
before
triplea75q
obscure25q
before*triplea75q
before*obscure25q
67.73333
9.220513
1.112821
-5.924823
1.241025
4.265094
1.428536
2.095618
3.716729
2.301559
4.371737
4.081054
0.0000
0.0000
0.7649
0.0106
0.7767
0.2969
R-Squared
0.262150
N
269
F-Statistic
11.54697
F P-Value
0.0000
Source: Data from Metacritic and VGChartz, 2006—2017.
Table D.3: Model of metascore with interaction between magnitude and triple-a and
obscure classification without Heteroskedasticity-Consistent Errors
Dependent Variable: metascore
Variable
Coefficient
Standard Error
P-Value
constant
magnitude
triplea75q
obscure25q
before*triplea75q
before*obscure25q
70.63706
-1.036618
3.985332
-2.426190
0.222395
0.101406
1.102688
0.239860
2.163694
2.588788
4.368792
0.427973
0.0000
0.0000
0.0666
0.3495
0.5470
0.8129
R-Squared
0.195058
N
269
F-Statistic
12.74636
F P-Value
0.0000
Source: Data from Metacritic and VGChartz, 2006—2017
117
Table D.4: Model of metascore with interaction between magnitude and triple-a and
obscure classification without Heteroskedasticity-Consistent Errors
Dependent Variable: metascore
Variable
Coefficient
Standard Error
P-Value
constant
magnitude
triplea75q
obscure25q
before*triplea75q
before*obscure25q
70.63706
-1.036618
3.985332
-2.426190
0.222395
0.101406
1.108117
0.249031
2.263656
2.071536
0.456793
0.360321
0.0000
0.0000
0.0795
0.2426
0.6268
0.8129
R-Squared
0.195058
N
269
F-Statistic
12.74636
F P-Value
0.0000
Source: Data from Metacritic and VGChartz, 2006—2017.
Table D.5: Model of initial with interaction between before and triple-a and
obscure classification without Heteroskedasticity-Consistent Errors
Dependent Variable: log(initial)
Variable
Coefficient
Standard Error
P-Value
constant
before
triplea75q
obscure25q
before*triplea75q
before*obscure25q
10.05825
0.495858
0.833326
-1.072307
0.633867
-0.113588
0.167275
0.293167
0.322090
0.308019
0.441430
0.360321
0.0000
0.0920
0.0102
0.0006
0.1522
0.8597
R-Squared
0.269934
N
264
F-Statistic
19.07853
F P-Value
0.0000
Source: Data from Metacritic and VGChartz, 2006—2017.
118
Table D.6: Model of initial with interaction between before and triple-a and
obscure classification with Heteroskedasticity-Consistent Errors
Dependent Variable: log(initial)
Variable
Coefficient
Standard Error
P-Value
constant
before
triplea75q
obscure25q
before*triplea75q
before*obscure25q
10.05825
0.495858
0.833326
-1.072307
0.633867
-0.113588
0.190059
0.277932
0.508799
0.309069
0.593644
0.542112
0.0000
0.0756
0.1027
0.0006
0.2866
0.8342
R-Squared
0.269934
N
264
F-Statistic
19.07853
F P-Value
0.0000
Source: Data from Metacritic and VGChartz, 2006—2017.
Table D.7: Model of initial with interaction between magnitude and triple-a and
obscure classification
Dependent Variable: log(initial)
Variable
Coefficient
Standard Error
P-Value
constant
magnitude
triplea75q
obscure25q
10.19488
-0.070800
1.603324
-0.911943
0.145476
0.032582
0.299821
0.059935
0.0000
0.0307
0.0000
0.0010
magnitude
*triplea75q
0.068344
0.059935
0.2552
magnitude
*obscure25q
-0.005334
0.047406
0.9105
R-Squared
0.269934
N
264
F-Statistic
19.07853
F P-Value
0.0000
Source: Data from Metacritic and VGChartz, 2006—2017
119
Bibliography
Akdeniz, M. Billur, Roger J. Calantone, and Clay M. Voorhees. “Effectiveness of Marketing
Cues on Consumer Perceptions of Quality: The Moderating Roles of Brand Reputation
and Third-Party Information.” Psychology & Marketing 30, no. 1 (2012): 76-89.
Accessed November 27, 2016. doi:10.1002/mar.20590.
———. “Signaling Quality: An Examination of the Effects of Marketing and NonmarketingControlled Signals on Perceptions of Automotive Brand Quality.” Journal of Product
Innovation Management 31, no. 4 (2013): 728-743. Accessed November 27, 2016.
doi:10.1111/jpim.12120.
Akerlof, George A. “The Market for ‘Lemons:’ Quality Uncertainty and Market Mechanism.”
The Quarterly Journal of Economics 84, no. 3 (1970): 488-500. Accessed November 26,
2016. doi:10.2307/1879431.
Asquith, Paul, and David W. Mullins. “Signaling with Dividends, Stock Repurchases, and Equity
Issues.” Financial Management 15, no. 3 (1986): 27-44. Accessed November 26, 2016.
doi:10.2307/3664842.
Dietz, Jason. “Metacritic’s 7th Annual Game Publisher Rankings.” Metacritic. February 13,
2017. Accessed April 5, 2017. http://www.metacritic.com/feature/game-publisherrankings-for-2016-releases.
Dring, Christopher. “PlayStation’s Gara Talks Price Wars, Bloodborne, No Man’s Sky, and
Black Friday.” MCV UK. May 14, 2015. Accessed December 2, 2016.
http://www.mcvuk.com/news/read/playstation-s-gara-talks-price-wars-bloodborne-noman-s-sky-and-black-friday/0149460.
Edwards, Ralph. “The Economics of Game Publishing.” IGN. May 5, 2006. Accessed April 5,
2017. http://www.ign.com/articles/2006/05/06/the-economics-of-game-publishing.
Entertainment Software Association. “2016 Essential Facts About the Computer and Video
Game Industry.” Accessed April 12, 2017. http://www.theesa.com/wpcontent/uploads/2016/04/Essential-Facts-2016.pdf.
Fahey, Rob. “Charging More for Games Won’t Rescue Triple-A.” GamesIndustry.biz. July 5,
2013. Accessed December 1, 2016. http://www.gamesindustry.biz/articles/2013-07-05charging-more-for-games-wont-rescue-aaa.
Gao, Guodong (Gordon), Anandasivam Gopal, and Ritu Agarwal. “Contingent Effects of Quality
Signaling: Evidence from the Indian Offshore IT Services Industry.” Management
Science 56, no. 6 (2010): 1012-1029. Accessed November 27, 2016.
doi:10.1287/mnsc.1100.1162.
120
Gaston, Martin. “Space Adventure No Man’s Sky is One of Next-Gen’s Most Ambitious Titles.”
Gamespot. December 7, 2013. Accessed November 27, 2016.
http://www.gamespot.com/articles/space-adventure-no-man-s-sky-is-one-of-next-gen-smost-ambitious-titles/1100-6416605/.
Gilbert, Ben. “Nintendo’s New Console Still Doesn’t Solve Nintendo’s Biggest Problem.”
Business Insider. February 8, 2017. Accessed April 5, 2017.
http://www.businessinsider.com/nintendo-switch-third-party-games-support-2017-2.
Hamilton, Kirk. “The Games that Defined the Last Generation.” Kotaku. November 15, 2013.
Accessed April 3, 2017. http://kotaku.com/the-games-that-defined-the-last-generation1464704318.
Irwin, Mary Jane. “Indie Game Developers Rise Up.” Forbes. November 20, 2008. Accessed
December 1, 2016. http://www.forbes.com/2008/11/20/games-indie-developers-techebiz-cx_mji_1120indiegames.html.
Joystiq Staff. “Joystiq video review: LEGO Star Wars II (Update 1).” Engadget. September 7,
2006. Accessed April 3, 2017. https://www.engadget.com/2006/09/07/joystiq-videoreview-lego-star-wars-ii-update-1/.
Kato, Matthew. “Nintendo’s Software Problem – Is the Wii Buried Under Shovelware?” Game
Informer. September 27, 2009. Accessed April 5, 2017.
http://www.gameinformer.com/b/news/archive/2009/09/27/feature-wii-softwareproblem-shovelware.aspx.
Makuch, Eddie. “PS$ More Powerful Than Xbox One, But Microsoft Always Improving -Metro Dev.” Gamespot. August 27, 2014. Accessed April 10, 2017.
https://www.gamespot.com/articles/ps4-more-powerful-than-xbox-one-but-microsoftalwa/1100-6421940/.
Metacritic. CBS Interactive. http://www.metacritic.com.
McCarthy, Niall. “America’s Favorite Video Game Genres [Infographic].” Forbes. June 26,
2015. Accessed April 12, 2017.
https://www.forbes.com/sites/niallmccarthy/2015/06/26/americas-favorite-videogame-genres-infographic/.
Miller, Merton H., and Franco Modigliani. “Dividend Policy, Growth, and the Valuation of
Shares.” The Journal of Business 34, no. 4 (1961): 411-433. Accessed November 26,
2016. doi:10.1086/294442.
121
Nielson. “Multi-Platform Gaming for the Win!” May 27, 2014. Accessed April 12, 2017.
http://www.nielsen.com/us/en/insights/news/2014/multi-platform-gaming-for-thewin.html.
Orland, Kyle. “Valve Discusses User-Centric Changes to Steam’s Game Discovery Problem.”
Ars Technica. April 4, 2017. Accessed April 5, 2017.
https://arstechnica.com/gaming/2017/04/valve-taps-wisdom-of-crowds-to- help-filtersteam-games/.
Paylou, Paul A., and David Gefen. “Building Effective Online Marketplaces with InstitutionBased Trust.” Information Systems Research 15, no. 1 (2004): 37-59. Accessed
November 26, 2016. doi:10.1287/isre.1040.0015.
PC Magazine. “Definition of: Video Game Console.” PC Magazine Encyclopedia. Accessed
April 14, 2017. http://www.pcmag.com/encyclopedia/term/53848/video-game-console.
Plunkett, Luke. “Nearly 40% of All Steam Games Were Released In 2016.” Kotaku. November
30, 2016. Accessed April 5, 2017. http://kotaku.com/nearly-40-of-all-steam-games-werereleased-in-2016-1789535450.
Schedeen, Jesse. “Xbox 360 vs. PlayStation 3: The Hardware Throwdown.” IGN. August 26,
2010. Accessed April 10, 2017. http://www.ign.com/articles/2010/08/26/xbox-360-vsplaystation-3-the-hardware-throwdown?page=2.
Schreier, Jason. “The No Man’s Sky Review Debacle.” Kotaku. August 5, 2016. Accessed
November 27, 2016. http://kotaku.com/the-no-mans-sky-review-copy-debacle1784873415.
———. “Metacritic Matters: How Review Scores Hurt Video Games.” Kotaku. August 8, 2015.
Accessed March 27, 2017. http://kotaku.com/metacritic-matters-how-review-scores-hurtvideo-games-472462218.
Shapiro, Susan P. “The Social Control of Impersonal Trust.” American Journal of Sociology 93,
no. 3 (1987): 623-658. Accessed November 26, 2016. oi:10.1086/228791.
Spence, Michael. “Job Market Signaling.” The Quarterly Journal of Economics 87, no. 3 (1973):
355-374. Accessed November 26, 2016. doi:10.2307/1882010.
Stiglitz, Joseph E., and Bruce C. Greenwald. “Externalities in Economies with Imperfect
Information and Incomplete Markets.” The Quarterly Journal of Economics 101, no. 2
(May 1986): 229-64. Accessed February 27, 2017. doi: 10.2307/1891114.
Tassi, Paul. “150 Games ‘For Everything But Wii U’ Highlights Nintendo’s Third Party
Problem.” Forbes. September 25, 2014. Accessed April 7, 2017.
https://www.forbes.com/sites/insertcoin/2014/09/25/150-games-for-everything-but-wii-uhighlights-nintendos-third-party-problem/.
122
Thier, Dave. “Why I Play On Consoles Instead of PC.” Forbes. September 3, 2015. Accessed
April 14, 2017. https://www.forbes.com/sites/davidthier/2015/09/02/why-i-play-onconsoles-instead-of-pc/.
Totilo, Stephen. “Another Microsoft-Published Xbox One Launch Game Is Coming to PC.”
Kotaku. August 7, 2014. Accessed April 14, 2017. http://kotaku.com/divisive-xbox-onelaunch-game-ryse-is-coming-to-pc-acc-1617573860.
———. “You’re Going to Get Fewer Early Game Reviews from Everyone.” Kotaku.
October 27, 2016. Accessed November 27, 2016.
http://www.kotaku.com.au/2016/10/youre-going-to-get-fewer-early-game- reviews-fromeveryone/.
VGChartz. VGChartz, Ltd. http://www.vgchartz.com.
Welsh, Oli. “Our No Man’s Sky Review Will be Late, and Here’s Why.” Eurogamer. August 8,
2016. Accessed November 27, 2016. http://www.eurogamer.net/articles/2016-08-08-ourno-mans-sky-review-will-be-late-and-heres-why.