The Collaborative Filtering Effect of Netflix Ratings for Indie Films versus Blockbusters
and Heavy Users versus Casual Users
May 2014
Henry Zhu Tang
Department of Economics
Stanford University
Stanford, CA 94305
[email protected]
Under the direction of
Professor Timothy Bresnahan
ABSTRACT
Collaborative filtering algorithms, whose adoption by online recommendation engines has
markedly increased in recent years, serve to match users with items based on what they have
consumed in the past or the tastes of similar users. Meanwhile, Internet economists and
marketing experts have cited a new phenomenon driven by online platforms called the "Long
Tail," which is a distributional shift towards lesser-known niche products. In this paper, I test
whether an online platform that uses a collaborative filtering algorithm can help match non-mass
market goods with previously un-informed demanders and how this can affect user heterogeneity.
I choose to examine movie ratings made publicly available in the Netflix Prize, an open source
competition to improve the existing algorithm used for Netflix recommendations. Looking at
demand across different movie categories, I find a stronger responsiveness of demand to early
user ratings for indie films, relative to blockbusters. This effect is further magnified for "heavier"
users with greater rating histories. As movie "buffs" with more variety in tastes, these users not
only are greater influencers of demand, but are also more inclined to be influenced by others in
their own demand, in contrast to what one observes on other online platforms.
Keywords: Long Tail, indie, collaborative filtering, recommendations, heavy users, Netflix
Acknowledgments: I would like to thank my advisor, Professor Bresnahan, for being the
inspiration behind my topic, as well as for his constant guidance throughout the thesis research
and writing process. I would also like to thank Professor Marcelo Clerici-Arias for his invaluable
advice on my preliminary research, his input on structuring my thesis, and his recommendations
on statistical resources. This thesis is dedicated to my close family and friends.
Table of Contents
1. Introduction ………………………………………………………………………………page 1
2. Literature Review ………………………………………………………………………...page 5
2.1 The Film Industry …………………………………………………………….....page 5
2.2 The Long Tail …………………………………………………………………...page 8
2.3 Empirical Studies on Films and the Long Tail .……………………………......page 10
2.4 A Previous Netflix Study ………………………………………………….…..page 12
3. Data & Methodology ……………………………………………………………………page 14
3.1 Description of Data Set ………………………………………………………..page 14
3.2 Study Design …………………………………………………………………..page 20
4. Results …………………………………………………………………………………..page 22
4.1 Regression Models …………………………………………………………….page 22
4.2 User Heterogeneity …………………………………………………………….page 24
5. Conclusion ………………………………………………………………………………page 28
6. Appendix ………………………………………………………………………………..page 31
6.1 Stata Code ………………………………………………………………….......page 31
6.2 Tables .………………………………………………………………………....page 33
6.3 Figures …………………………………………………………………………page 39
7. References ………………………………………………………………………………page 42
Tang 1
1 Introduction
The film industry, a key driver of the global multibillion-dollar entertainment market, has
underdone many changes over the past century, thanks to constant innovations in technology.
Films, like other taste-based goods, can be widely accessed by consumers today thanks to a
revolutionary development: the Internet. Before the turn of the new century, films that had
already gone through their theatrical run were available almost exclusively as video cassettes in
physical stores such as a Blockbuster or Hollywood Video. However, in 1997, Reed Hastings
and Marc Randolph founded Netflix, an online DVD-by-mail retailer that usurped the traditional
brick-and-mortar model. At once, a wider library of titles had become available to consumers
than ever before. Netflix introduced a proprietary recommendation system, powered by a
collaborative filtering algorithm, to select movies to watch for its customers, a feature it
continues to use for its global video streaming service today. This collaborative filtering
algorithm would further highlight indie or niche films that could not be found (or were
prohibitively difficult to find) in stores.
Introduced in 2004 by editor-in-chief of Wired Magazine Chris Anderson, the Long Tail
is an evolution of the original Pareto principle in marketing1. The conventional Long Tail theory
argues that niche products (i.e. those in the tail of the sales distribution) gain increasingly greater
market share thanks to online innovations such as search queries and collaborative filtering, so
that the top-selling items make up less and less of the sales distribution. With the shift to online
(versus offline) sales, more efficient supply chain management has reduced the need to carry instore inventories for retailers, and search costs have been greatly reduced for consumers.
1
According to the Pareto principle, 20 percent of all the products generate 80 percent of all the revenues/sales.
Tang 2
Collaborative filtering is a method invented and developed over the past two decades to
recommend items to consumers online, matching their preferences based on their past browsing
or purchase histories and their similarities with other consumers or users. It is used for the
recommendation engines of many notable Internet services today, including Pandora, Amazon,
YouTube, Reddit, Spotify, and iTunes.
There has been much literature in economics and marketing analyzing both the Long Tail
and the box office performance of films, but little research has been done specifically on the
Long Tail distribution for films in online markets. A frequently cited example of the Long Tail,
Netflix has become widely known for using a collaborative filtering algorithm for the explicit
recommendations it provides users, based on each user's ratings and how similar a user is to
others in the network, among other factors. Netflix has helped revolutionize the movie industry
by offering a platform for niche movies to thrive. Given that product variety2 on Netflix has
always been increasing (with some of its products naturally being lesser-known), overall demand,
as measured by the number of ratings given, for niche titles should be increasing as well.
Taking a step from the past definitions of the Long Tail, I choose to classify the Long
Tail of the Netflix library as "indie" films with limited budgets (implying a constraint on
marketing). These movies were marginalized previously by the theatrical market's high fixed
costs, unable to afford the advertising or screens to meet consumers who would presumably be
interested if they had been informed about the movies. Thanks to the new capacity to distribute
films via the Internet, the rules of the old system no longer apply; movies do not face these same
constraints. I assert that the indies in the Long Tail can benefit due to one of two reasons:
2
The number of distinct movies.
Tang 3
1) There had previously been fragmented demand; indies were for fragments of consumers
with odd tastes but went unfound. Collaborative filters instead have helped connect
supply to demand in many different "communities."
2) There exists a group of "heavy" users or frequent reviewers (more closely representative
of movie "critics" or "buffs" who have a large variety of tastes) that could not find all
these movies until now. Collaborative filters offer recommendations to this group of
users who find them to be valuable.
After my analysis, I find the second theory to be more compelling.
In analyzing the demand distribution and Long Tail in particular, I determine what a
rating on Netflix actually represents. The ratings users have given are for movies they have
presumably seen; even if the score is not favorable, the rating itself can show Netflix the types of
movies this user watches. Since users are generally aware that their ratings help connect movies
with other users (and with themselves), ratings can serve as implicit endorsements or
recommendations. At the very least, ratings serve as signals of demand and perceived quality or
enjoyment.
I hypothesize that there would be an "indie" effect: a movie rating for a niche movie
would be considered more informative and valuable, as Netflix serves to match niche movies to
users who otherwise could not discover them. The rating scores, especially early on, would be a
better leading indicator (or perhaps even influencer) of the overall demand experienced for niche
movies than for blockbusters. I confirm this effect, quantitatively measuring the difference in
responsiveness of demand to ratings for blockbusters versus indies.
Tang 4
As users are matched to lesser-known Long Tail movies that satisfy their tastes, I also
hypothesize that user heterogeneity can increase over time. In other words, user tastes become
more diverse, leading to the formation of sub-communities or different fragments of demanders.
However, I have already said that after breaking down users by "heaviness" or frequency of
rating activity, I find that rather than leading to increased heterogeneity in tastes, Netflix
recommendations serve to especially benefit these heavy users who were previously
underinformed and simply have a greater taste for variety to begin with. Heavy users, not only
are powerful influencers of demand (with their propensity to give more ratings, especially early
after a movie appears on Netflix), but they are also markedly influenced by the ratings of others.
In all, there are three main dichotomies I seek to examine:
1) Blockbusters versus Indies (Hits versus Niches)
2) Frequent Raters versus Infrequent Raters (Influencers of Demand)
3) Movie "Buff" versus Casual Movie Watcher (Likelihood to be Influenced)
To test these hypotheses, I use the training data set of movie ratings made publicly
available during the Netflix Prize, an open competition held in 2006 to improve Netflix's existing
Cinematch collaborative filtering algorithm3. Using additional descriptors such as production
budget from The Numbers movie database, I categorize specific movies within the Netflix data
set into blockbusters, indies, and an intermediate category. I then build linear regression models
of different definitions of demand as the dependent variable and independent variables that
include how well a film is received early on during its run on Netflix. If niche movie ratings are
actually more powerful, I would expect the demand for niche titles to be more responsive to
3
A successful entrant had to improve on the RMSE of the Cinematch algorithm by 10%. Further prizes would be
given for progress beyond this.
Tang 5
ratings than the demand for blockbusters (i.e. the indie effect to hold). I also specify which users
are heavy, offering an absolute as well as a time-varying definition of heaviness.
The remainder of this thesis is organized as follows. Section 2 provides a literature
review of the economics of the film industry, an overview of the Long Tail as defined in the past
and past empirical studies, and a particular study by Tan & Netessine (2009) using the Netflix
Prize data set. Section 3 describes my data sets as well as the methodology I take to test my
hypotheses about the indie effect and increasing user heterogeneity. Section 4 presents an
analysis of my findings from the regression models and a further discussion on the role of heavy
users over time. Section 5 offers conclusions and further thoughts on movies, the Netflix
platform going forward, collaborative filtering, and additional research.
2 Literature Review
2.1 Film Industry
Much economics literature has been published about the film industry, as it continues to
innovate and the dynamics of competition evolve. Eliashberg et al. (2006) note that each of the
three main stages of the film value chain -- production, distribution, and exhibition -- have
greatly changed over time. Before a movie is produced, it must be "green-lit" or go through an
approval process. To better maximize the success rate and minimize risk of failures, marketing
researchers have made much progress in developing box office forecasting models to assess
possible demand. One important consideration is that moviegoers (or movie-watchers) are
heavily influenced by others' opinions and choices. Studios have also tried to cope with risk by
pursuing established movie franchises, making sequels more prevalent and a "safer bet." In turn,
Tang 6
distributors are also investing more than ever into advertising and development, focusing on a
small number of "blockbusters" that nowadays capture an increasingly larger share of attention
and revenues. This can make or break a distributor's overall box-office performance and has led
towards a "winner-take-all" market (e.g., flops are bigger, and the average production cost for a
movie has dramatically increased). While the box office market for big-name movies is still
booming, it has become increasingly more difficult for niche movies with limited budgets to
make it onto theater screens. Thus, movie producers and viewers alike have to turn more towards
ancillary markets such as home video or digital distribution (on-demand streaming), where there
are potentially much greater revenues and profits to be had.
Moul (2005) discusses the issues of when to release a film and how to capitalize on (or
minimize the effect of) word-of-mouth, essential questions in the supply chain of the film
industry. He also looks at the impact of critics as well as the starring cast on movie demand.
Movies can present a unique problem in marketing because they possess traits of both durable
products (i.e. sales come mostly from first-time purchases) and nondurables (i.e. sales mostly
come from repeat purchases). Since branding is conspicuously absent from production and
distribution4, the star power of the cast, and to a lesser extent, the director, play the primary role
in establishing intent-to-view for movies. The presence of a star associated with specific types of
movies can provide a signal on what one can expect or that the studio has put a notable
investment into the movie, and serve as promotional effort or free publicity.
Karniouchina (2011) and Berger, Sorensen, and Rasmussen (2010) discuss whether
movie stars can increase movie demand even through negative consumer buzz, both separately
4
There are a few exceptions, perhaps most prominently of which is Pixar.
Tang 7
finding that this indeed is the case. Using a combination of empirical studies and more rigorous
econometric analysis, Berger, Sorensen, and Rasmussen demonstrate that negative publicity is
more likely to hurt products already with broad awareness, but help products that are relatively
unknown (e.g., those in the Long Tail or considered niche). Meanwhile, Karniouchina finds that
the early buzz generated by stars ultimately has a net positive impact for box office sales, even
for poorly received films, because of the initial boost in revenues.
Eliashberg and Shugan (1997) discuss the role that movie critics play for a movie's box
office performance: whether they act as influencers who are regarded as thought leaders and can
sway consumer sentiment, or as predictors who are merely leading indicators. Ultimately, the
authors find that critics, perhaps contrary to what one would expect, appear to act more like
leading indicators. On Netflix, the closest role to a movie critic is that of "heavy users," who take
the time to rate more movies than the vast majority of the user community. They likely watch
more movies than the average user5, but they also provide more ratings likely because they
believe recommendations to be useful to themselves and others. Since ratings are anonymous,
one would imagine that heavy users are also predictors rather than influencers of demand.
As one can see though, the vast majority of movie economics literature still focuses on
box office performance, rather than newer ancillary markets such as DVDs or video-on-demand.
This is due in large part to the availability or reliability of data, especially given possible
concerns on privacy.
5
One can watch a movie and not rate it on Netflix. One can also rate movies without having watched them through
Netflix.
Tang 8
2.2 The Long Tail
Regarding online platforms, much research has been done on the phenomenon of the
Long Tail. The term, coined by Wired editor-in-chief Chris Anderson (2006), refers to the shift
away from a small number of hits ("mainstream" products) at what he calls the "head of the
demand curve" to a larger number of niches in the "tail." According to Anderson, there are a
number of factors that explain this: the far greater number of niche goods available, the decrease
in distribution and search costs, and "filters" or recommendations that sort through the increased
product variety and drive demand toward the tail. The demand curve thus flattens; there are still
hits and niches, but the hits are relatively less popular and the niches relatively more so
(Appendix: Figure 6). With the costs of reaching consumers lowered, niches can suddenly reach
as huge a potential market as hits can. Thanks further to recommendation systems, supply and
demand are connected more closely than before. Finally, consumers are even able to guide
themselves to what they are looking for by posting reviews or ratings of past products, shaping
recommendations that drive demand down into the niches further. As their interests and tastes
narrow, consumers can form sub-communities of affinity.
Brynjolfsson, Hu, and Smith (2010) go through a number of supply-side and demand-side
drivers that contribute to the increase in product variety and decrease in sales concentration for
the Long Tail. On the supply-side, the cost of stocking products greatly decreases; Internet
retailers only need to add the inventory of a listed product to their centralized warehouses, as
opposed to a brick-and-mortar model where the product must be carried in-stores. On the
demand-side, new technologies such as Google queries have greatly reduced search costs.
Collaborative filters used for personalization and recommendation technologies can
disproportionately help consumers find obscure products they would otherwise not have known,
Tang 9
although they can also lead to disproportionate gains in sales of popular products relative to
niche products due to the amount of existing information available (Fleder and Hosanagar, 2009).
This Superstar or "blockbuster" effect, the opposite of the Long Tail, can also be furthered by
online communities and social networks, which attach themselves to the latest fads and
blockbusters. Brynjolfsson et al. (2011) use a data set for an online and catalog retailer that
allows them to isolate the demand-side and individual-level consumer search behavior.
Ultimately, they find that purchases made through the Internet channel by consumers with prior
Internet experience (i.e. more experienced users) are more skewed towards obscure products, and
the Internet channel exhibits a significantly less concentrated sales distribution than the catalog
channel.
Anderson offers a fairly limited and vague definition of the Long Tail, primarily viewing
it as a principle for the marketing realm. These Long Tail items, or "niches," are characterized by
constraints in either appeal or production, failing to become "mainstream." Anderson argues that
niches can be as economically attractive as hits though, thanks to the Internet, if one is able to
find the requisite demanders. He does not take the step to describe why exactly these niches have
been fixed on the tail of the demand distribution (e.g., separating why they are not mainstream or
commercial between demand and supply sides, and how one would characterize the potential
demanders for these products). Brynjolfsson et al. do offer additional insights on both demand
and supply-side factors as well as the characteristics of demanders for items in the tail; however,
they describe online catalogs or stores where it is likely easier to quantify demand, rather than
subscription-based services such as a Netflix.
There have been detractors as well as proponents of the Long Tail. Elberse and
Oberholzer-Gee (2008) investigate the actual profitability of the Long Tail by looking at Nielsen
Tang 10
sales data for videos/music and data from Quickflix (an Australian equivalent of Netflix). They
find that the tail becomes longer, but it represents a rapidly increasing number of titles that sell
very rarely if ever; success is becoming concentrated in fewer best-sellers at the head of the
distribution. Elberse (2008) directly challenges Anderson's view of the long tail by citing
sociologist William McPhee's theory of exposure, which describes two phenomena of
distribution: natural monopoly and double jeopardy. Natural monopoly refers to the "monopoly"
that hit products have on "light" consumers or those with otherwise marginal participation in the
market. McPhee also notes that the larger the proportion of consumers unfamiliar with a given
alternative, the less likely those who are familiar with it to like it especially. Niche products
therefore have a double disadvantage ("double jeopardy"): first, they are not well-known; second,
even if they are known, it is by people who "know better" and still prefer the popular products.
Because of these reasons, Elberse suggests that retailers should market the most popular products
but broaden their selection with more niche products if they want to cater to "heavier" users.
Lastly, Elberse finds that niche or obscure titles receive, on average, lower ratings (which, as I
found, was the case in Netflix) - this could suggest that niche titles similarly rated as popular
titles are actually of higher quality (assuming that niche titles get lower ratings simply due to a
lack of popularity or misaligned demand).
2.3 Empirical Studies on Movies and the Long Tail
There have been a number of recent empirical studies pertaining to the weight of online
reviews or recommendations. Tucker and Zhang (2011) conduct a field experiment to test
whether popularity information benefits niche products with narrow "appeal" disproportionately,
because the same level of popularity (a possible measure of demand) implies higher quality for
narrow-appeal products than for broad-appeal products. They experiment with shifting a
Tang 11
wedding service website from a "yellow pages" listing with no popularity information to a "bestseller list" style ranked by number of clicks for each vendor. The definition of "appeal" is based
on the population of the vendor's town. Using a differences-in-differences (DID) method as their
primary approach, they find that narrow-appeal vendors receive more visits than equally popular
broad-appeal vendors after the "treatment" of popularity information, consistent with their
hypothesis.
Dellarocas, Gao, and Narayan (2010) examine the postconsumption propensity to give
movies reviews. They describe two opposing forces in effect: consumers prefer posting reviews
for products less available and successful in the market (Dichter's theory of self-involvement);
however, at the same time, they are also more likely to contribute reviews for products that many
other people have already commented on online. The tension between these forces leads to a Ushaped relationship between the average propensity to review a movie postconsumption and that
movie's box office revenues: moviegoers are more likely to contribute reviews for very obscure
movies but also for huge blockbusters. Movies falling in the average-performing range would be
least likely then to get consumer reviews. Based on this, niche and blockbuster movies could
both be benefitting from Netflix (while the "middle-of-the-pack" movies see the least returns).
Finally, Zhou and Duan (2012) consider the interaction of a demand-side factor (online
reviews) and a supply-side factor (product variety) for the Long Tail in the context of online
software downloads on CNET. Using a quantile regression technique, they find that the impact
of both positive and negative reviews is weakened as product variety increases. The increase in
product variety also reduces the impact of reviews on popular products more than on niche
products, suggesting that a review or rating for a niche product carries more weight,
strengthening the Long Tail. On Netflix, over time each new rating for a movie naturally carries
Tang 12
less weight; however, this further confirms that niche ratings should be viewed as more
informative of a movie's quality or "likeability."
2.4 Netflix Study
Tan and Netessine (2009) use the Netflix Prize data set in its entirety6 to look at
aggregate as well as individual-level demand on Netflix. They assert that the pre-existing
definition of the Long Tail is too static, and that it implicitly excludes the impact of increasing
product variety. An increase in product variety would likely create demand diversification, but it
could also lead to a conflicting definition of "hits" and "niches." When product variety is large,
the demand for any one product would almost certainly be smaller than when product variety is
small. Furthermore, when the consumer base is large, learning about new products is faster than
when the consumer base is small. The authors address the empirical question of whether
consumers can (and actually do) keep up with discovering "obscure" products as they appear on
the marketplace. They offer a more dynamic definition of the Long Tail that takes into account
increasing product variety as a measure for relative popularity. Demand in their case is the
number of ratings for a movie.
The authors in their analysis test two separate hypotheses:
Hypothesis 1: If popularity is measured in absolute terms, over time, demand for hits will
decrease, while demand for niches will increase.
Hypothesis 2: If popularity is measured in relative terms, over time, demand for hits will
increase, while demand for niches will decrease.
6
Over 100 million ratings submitted by approximately 480,000 users on Netflix from 2000 to 2005.
Tang 13
They analyze the yearly distribution of cumulative demand using a Popularity variable (defined
as the ranking for a particular movie by number of ratings during the time period). Relative
demand and popularity are measured by including a Product Variety variable (defined as the total
number of different movies rated during a period). Tan and Netessine also control for product
rating to ensure that the Long Tail is not simply a manifestation of hit movies deteriorating in
quality over time.
Since movie demand is increasing exponentially, Tan and Netessine use a logarithmically
transformed time series model to examine the dynamics of demand across the distribution.
Equation 1. Logarithmic model of demand on a trend variable. (Tan & Netessine, 2009)
They eventually make the following findings:
1) The top 20 percent of movies constitute approximately 85 percent of total demand,
significantly more than 80 percent, leading them to reject the Long Tail effect.
2) Consumers over time indeed watch more niche movies in absolute terms (Hypothesis 1),
but the rate at which they shift demand from the hits to the niches is considerably lower
than the growth rate of product variety.
3) Normalizing for product variety and measuring popularity in relative terms, consumers
watch more and more hits over time (Hypothesis 2).
4) Consumers who do watch niches tend to be heavy users, who constitute only a small part
of the entire user base. Therefore, hits continue to drive the market.
While Tan and Netessine make interesting conclusions, their results by and large are from
a high-level point of view and do not incorporate more meaningful differences among movies
Tang 14
and users. They view hits and niches according to the existing demand distribution and how it
changes over time, given specific cutoffs (Figure 1).
Figure 1. Distribution of demand or ratings for movies on Netflix over time. (Tan & Netessine, 2009)
Meanwhile, I use a different approach in considering which movies are in the Long Tail, offering
an ex-ante definition based on individual movie characteristics, and how movies benefit directly
from their ratings. I also further observe behavior of subgroups of users, specifically those users
with "heavier" rating activity.
3 Data & Methodology
3.1 Description of Data Set
Like Tan & Netessine, I use the Netflix Prize training data set7, made publicly available
in the company's open competition in 2006 for an improved collaborative filtering algorithm to
7
Netflix provided what they refer to as a training data set and qualifying data set. Participating teams in the Netflix
Prize were to use the qualifying set in the actual competition by predicting rating scores for the entire set while only
being informed of the scores for half of the data. The training set was larger and used only for teams to practice.
Tang 15
predict user ratings for movies in the Netflix library. My data consists of a collection of 17,770
text files (one for each Netflix movie title), with approximately 100 million ratings in total8.
Each observation in the Netflix Prize data set contains several fields: movie ID, user ID, rating
(from 1 to 5, with 5 being the best), and date of rating. Additionally, the Netflix Prize data
provides a movie index file that lists the movie IDs, their corresponding movie titles, and the
years of theatrical release. Since the algorithm is used to give recommendations based only on a
user's past ratings, no other information about the users or movies is made available.
Using OpusData Explorer to search a movie database called The Numbers, I filter out all
its movies by total budget and release date. I then add additional fields to these movies, such as
production budget, genre, box office (domestic & international) figures, MPAA rating,
distributor, date of movie release, and whether the film is a sequel. By further differentiating my
selection of movies by specific characteristics such as budget, I create three categories
(blockbusters, indies, and intermediate). For simplicity, I choose to look only at movies that had
been released in theaters, and all movies chosen were released domestically between 1999 and
2005 (or the approximate time period in which all the Netflix ratings were given), limiting the
Netflix ratings I would actually observe to a much smaller subset. I classify movies with
production budgets9 of over $100 million as "blockbusters," while movies with under $10
million budgets were considered "indie films." For an intermediate category, I select all the
movies that the 20 highest-grossing actors of all-time (according to Box Office Mojo) appeared
in between 1999 and 2005, excluding any that were already in the blockbuster or indie categories.
8
This was still only a fraction of the over 1 billion ratings Netflix already had at this point.
9
Marketing budgets would have been a better measure, but proved more difficult to find.
Tang 16
These intermediate films are an attempt to isolate the word-of-mouth "marketing" associated
with star power, in contrast to the actual marketing campaigns that blockbusters would run.
To analyze the data I had downloaded in raw text format, I selectively import the Netflix
Prize text files for my movies of interest (as identified by their movie IDs), as well as the movie
index file, into Stata ultimately as a single data set. I then download the additional movie data
fields from The Numbers into Excel spreadsheets and import into Stata, merging on movie names
with my aggregate Netflix Prize data set.
For my analysis, I generate the following variables, defined in Table 1 and described
further in my Study Design.
Table 1. Variable Descriptions
Early response
The average rating score for a movie within 180 days of its first rating.
Heavy user
A user who has given at least 100 ratings.
Nonheavy user
A user who has given between 1 and 99 ratings.
Blockbuster
Movie with production budget of over $100 million.
Indie
Movie with production budget of under $10 million.
Intermediate
Movie in which one of the 20 highest-grossing actors of all time has
appeared, excluding any already labeled as blockbuster or indie.
Movie age
Difference between date of movie rating for observation and date of first
movie rating.
Popular genre
Action/adventure and comedy (including romantic comedy, excluding
black comedy).
Other genre
Black comedy, concert/performance, documentary, drama, horror,
musical, thriller/suspense, and western.
Demand (1)
Total ratings for movie.
Tang 17
Demand (2)
Total ratings for movie in particular month, divided by total users who
rated anything that month. (Measured after first six months for a movie.)
Heavy demand
Total ratings by heavy users for a movie.
Heavy early
response
The average rating score by heavy users for a movie within 180 days of
its first rating.
As reference, Table 2 lists the averages for the key variables in my models, which become
important to my results by allowing me to compare the relative impact of regressors in
regressions across movie categories.
Table 2. Summary of Means
Variable
Overall
Blockbuster
Indie
Intermediate
Popular
Genre
Other
Genre
Rating
3.570
3.672
3.508
3.553
3.542
3.602
Early response
3.356
3.726
3.248
3.411
3.422
3.311
Heavy user rating score
3.424
3.580
3.356
3.403
3.404
3.444
Heavy user number of
ratings
126.0
24.0
34.8
67.2
63.9
62.1
Nonheavy user
- rating score
3.605
3.689
3.551
3.590
3.573
3.642
26.1
6.5
6.1
13.5
13.9
12.2
1180.1
1133.8
1076.6
1344.8
1213.1
1157.6
Movie-level
Demand (1)
37450.1
88196.7
16985.1
53458.1
48751.9
29735.3
Movie-month
Demand (2)
0.00501
0.00982
0.00202
0.00744
0.00582
0.00444
Heavy demand
7235.1
13625.0
3739.8
10586.2
9066.9
5990.0
Heavy early
response
3.235
3.626
3.084
3.346
3.300
3.190
Nonheavy user
- number of
ratings
Movie age (in
number of
days)
Tang 18
For the blockbuster category, there are over 3.4 million ratings, 39 films rated, and
approximately 420,000 unique users who have given ratings for these films. For the indie
category, there are over 3.5 million ratings, 207 films rated, and approximately 385,000 unique
users who have rated these films. For the intermediate category, there are over 7.5 million ratings,
141 films rated, and over 450,000 unique users who have rated these films. In total, the aggregate
data set has approximately 14.5 million observations, over 470,000 unique users, and 387 films
rated. Blockbusters receive higher rating scores on average, likely because of their mass appeal.
Early response, which only includes the ratings received for a film within the first six months of
its first rating, has an even greater disparity in rating score for blockbusters versus indies.
Though indies are less well-received at first, their average rating greatly increases over time,
confirming that recommendations become better at matching users with these films.
There is a very different breakdown by genre between these three categories of movies
(blockbusters, indies, and intermediate), as illustrated by Table 3. Blockbusters have a much
higher percentage of action/adventure films, which one would expect based on the typically
higher budgets given to these special effects-laden films. On the other hand, comedies tend to
make up much more of the Indie and Intermediate categories because they typically do not
require a large budget. Based on what is most frequently "green-lit" in Hollywood, I categorize
action/adventures and comedies (including romantic comedies or "rom-coms") as the "popular"
genres. Many other genres, including horror, documentary, and thriller/suspense, are not even
present among the Blockbusters.
Tang 19
Table 3. Breakdown of genres for movies across categories
Action
Adventure
Black Comedy
Comedy
Concert/Performance
Documentary
Drama
Horror
Musical
Not reported
Romantic Comedy
Thriller/Suspense
Western
(missing)
Total
Blockbuster
Number
Percent
16 Percent 41.03
15
38.46
2
5.13
5
12.82
1
2.56
39
100.00
Indie
Number Percent
3
1.45
2
0.97
2
0.97
43
20.77
2
0.97
11
5.31
86
41.55
10
4.83
2
0.97
28
13.53
8
3.86
10
4.83
207
100.00
Intermediate
Number Percent
15
10.64
12
8.51
4
2.84
32
22.70
2
1.42
41
29.08
4
2.84
8
5.67
19
13.48
1
0.71
3
2.13
141
100.00
I also divide users by their user activity (i.e. how many ratings they have given). I
consider the most active users as "heavy users," or those likely to be "movie buffs" as opposed to
casual movie-watchers/raters. Heavy users, despite making up only 4.7 percent of the total users,
are responsible for 19.25 percent of the total ratings. Aside from being more critical in their
ratings than non-heavy users, they also watch different types of movies. Approximately 26.5
percent of all ratings by heavy users are for action/adventure movies, compared to 31 percent for
other users. Heavy users instead watch more comedies (approximately 26.5 percent of their
ratings versus approximately 24.5 percent of non-heavy users' ratings) as well as more movies in
the smaller genres, such as horror, western, musical, and concert/performance, which are
associated with indies rather than blockbusters.
Tang 20
3.2 Study Design
Using my aggregate Netflix data set, I test the following hypotheses:
Hypothesis 1: Collaborative filtering helps match non-mass market movies to demanders.
Hypothesis 2: Collaborative filtering permits more heterogeneity in users or demanders.
Hypothesis 1 is connected to the indie effect, or how movies on the Long Tail of Netflix are
affected. Hypothesis 2 examines the heterogeneity of tastes over time, also taking into account
the dichotomy of users between movie buffs (with a greater variety in tastes) and casual moviewatchers.
For Hypothesis 1, I build a linear regression model (1) of movie-level "demand" (the total
number of ratings a movie receives10) on early response and a time-correction variable (i.e. when
the movie was first rated). I test whether the early response for an indie movie is more important
in predicting its total demand than the early response for a blockbuster, given the substantial
marketing that big-budget movies would already enjoy. I also build a linear regression model (2)
with early response and movie age as the independent variables and demand on a movie-month
level as the dependent variable. To measure only the subsequent impact from the early ratings, I
exclude the first 180 days of ratings in this definition of demand. This definition of demand also
normalizes the number of ratings for a movie by the total number of users who have given any
ratings on Netflix that month. In other words, this "demand" considers how popular a movie is
relative to all its peers at a specific time.
10
The number of viewings would likely be a more appropriate measure of demand, but data on this is unavailable. I
use the number of ratings as a proxy for viewings, assuming that users are equally likely to give ratings for movies
they view across movie categories.
Tang 21
Movie-level Model:
Demand   0  1earlyresponse   2 date _ firstrating  
Movie-month Model: Demand   0  1earlyresponse   2 movie _ age  
(1)
(2)
For Hypothesis 2, I consider whether there is greater user heterogeneity over time or
whether users who do exhibit a greater variety in tastes (the heavier users) are simply benefitting
more. By splitting up users by the extent of their rating activity and classifying them as either
heavy or non-heavy, I look specifically at heavy demand on a movie-level, a slight modification
of Model 1. I also look specifically at heavy early response (the average of only the ratings given
by heavy users within six months of a movie's first rating) to see if heavy early response has a
different effect than early response in general. To see if heavy users are becoming more or less
common, I observe how the breakdown between heavy and non-heavy users evolves over the
time period of the data set, proceeding with the current definition of heavy users and then
offering a more dynamic definition that better reflects which users are heavy year over year.
By introducing a new definition of heavy users as those who give over double the
average number of ratings that all users give in a year, I more accurately identify whether heavier
users have become more or less common in relative terms. This definition allows the possibility
for users to be considered either heavy or not year-to-year. As Table 4 shows, the average rating
score by heavy users goes up markedly (especially for indies), indicating they are either
becoming more lenient or they are getting better matches.
Heavy user
A user who has given more than double the number of ratings that users gave
on average in a year.
Tang 22
Table 4. Summary of means for ratings by heavy users overall, and in 2000 versus 2005
Overall
Rating
Blockbuster
3.464
Number of ratings per
user
3.617
95.98
Rating (in the year 2000)
Number of ratings per
user (in the year 2000)
3.421
18.28
3.315
27.95
Rating (in the year 2005)
Indie
3.516
24.93
3.626
3.024
2.67
3.13
3.635
3.468
Intermediate
3.431
52.76
3.319
22.15
3.494
Number of ratings per
user (in the year 2005)
104.67
21.34
27.14
56.19
Heavy demand
3925.52
7388.33
1922.58
5892.87
Heavy early response
3.206
3.648
3.035
3.332
Finally, using my second definition of heaviness, I reconstruct heavy demand and heavy early
response, rerunning my regressions with these variables.
4 Results
4.1 Regression Models
Table 5 presents the results from regressing movie-level demand (1) on early response
and the date of a movie's first rating (a time-correction variable), per each category of movie.
Table 5. Movie-level Demand
(1)
Blockbuster
EARLY_RESPONSE
DATE_FIRSTRATING
(Intercept)
77,393
(15,419)
-22.57
(13.95)
153,180
(202,846)
(2)
Indie
16,770
(2,449)
0.0618
(3.129)
-38,455
(48,699)
(3)
Intermediate
58,974
(6,078)
-3.199
(5.232)
-98,359
(77,893)
(4)
Popular Genre
50,360
(5,804)
1.560
(5.989)
-147,808
(88,732)
(5)
Other Genre
30,263
(3,594)
-10.13
(4.057)
87,361
(63,405)
Tang 23
Table 6 shows the results from regressing movie-month demand (2) on early response and movie
age, per each category of movie.
Table 6. Movie-month Demand
EARLY_RESPONSE
MOVIE_AGE
(Intercept)
(1)
Blockbuster
(2)
Indie
(3)
Intermediate
0.00721
(0.000512)
-6.72e-06
(6.06e-07)
-0.0106
(0.00201)
0.00196
(6.92e-05)
-2.18e-06
(1.04e-07)
-0.00247
(0.000246)
0.00521
(0.000337)
-8.28e-06
(3.37e-07)
-0.00223
(0.00121)
(4)
Popular Genre
0.00397
(0.000281)
-6.13e-06
(3.33e-07)
-0.00202
(0.00103)
(5)
Other Genre
0.00416
(0.000140)
-4.23e-06
(1.86e-07)
-0.00558
(0.000499)
The movie-level and movie-month demand are, as what one would expect, most responsive to
the early response for blockbusters in absolute terms. However, once I take into account the
average level of demand (movie-level or movie-month level) for each category that I
summarized in Table 2, I see that demand responds relatively more to early response for indies
than for blockbusters, confirming an indie effect. For instance, an increase in early response by a
rating score of one results in a 98.73 percent increase in total movie demand for the average indie,
compared to a 87.75 percent increase for the average blockbuster.
There are several caveats I wish to address about the results for these models. Although
ratings are more likely to suffer from reviewer bias11 with a smaller sample of ratings, the
reviewers who do rate these niche titles are more likely to be heavy users. One would presume
that these heavy users are more likely than non-heavy users to give their own ratings, free of
external biases. I am also not measuring the score of subsequent ratings, but simply the number
of ratings (in total, or per month). The results for the Intermediate category also prove to be too
inconclusive for me to make any meaningful comments. One issue with the selection of the films
11
The first or average rating tends to dictate what the following ratings will be.
Tang 24
for this category is likely that the films are bounded both above and below (by budget). There is
also the possibility that some of these highest-grossing actors I looked at do not actually have the
star power to draw in audiences, but instead have had extended careers absent much fanfare (or
minimal roles in a series of blockbusters). Big-name directors or producers (or even studios) can
also play a part in users' intent-to-view, which I do not account for.
Finally, there is also not as significant a difference between the relative changes in
demand for movies of "popular" genres versus other genres as there is for blockbusters versus
indies, so this effect is not nearly as interesting. The difficulty with this could be that comedies,
which are clearly a popular genre of films for studios to produce, are technically indies for the
most part because of their inexpensive budgets. To produce a more accurate hits versus niches
dichotomy using genre, I could have factored in marketing budget12 and whether the movie is
produced by a major studio.
4.2 User Heterogeneity
Tables 7 and 8 presents the results from regressing heavy demand on early response and
regressing heavy demand on heavy early response, respectively.
Table 7. Heavy Demand (1)
(1)
Blockbuster
EARLY_RESPONSE
(Intercept)
12
7,500
(1,369)
-14,317
(5,150)
(2)
Indie
2,943
(401.3)
-5,751
(1,329)
(3)
Intermediate
7,142
(760.1)
-13,844
(2,634)
(4)
Popular Genre
(5)
Other Genre
6,556
(676.1)
-13,366
(2,353)
4,649
(563.0)
-9,363
(1,899)
As mentioned prior, marketing budget numbers tend to be far more difficult to find than overall budgets, however.
Tang 25
Table 8. Heavy Demand (1)
(1)
Blockbuster
EARLY_RESPONSE_HEAVY
(Intercept)
7,996
(1,280)
-15,372
(4,689)
(2)
Indie
3,114
(355.2)
-5,825
(1,127)
(3)
Intermediate
(4)
Popular Genre
(5)
Other Genre
6,966
(699.4)
-12,677
(2,371)
6,328
(634.5)
-11,867
(2,140)
4,958
(487.7)
-9,751
(1,589)
I find that heavy demand for indies is more responsive to early ratings (i.e. the relative changes
are greater) than it is for blockbusters, and that the difference in responsiveness is greater than it
was for overall demand for indies and blockbusters. An increase in early response by a rating
score of one implies 2943 more ratings by heavy users for an indie, compared to the 3739 total
ratings by heavy users (on average) per indie, or a 78.7 percent increase in heavy demand.
Meanwhile, the same increase in early response implies only a 55.0 percent increase in heavy
demand for blockbusters.
This disparity in responsiveness of demand between indies and blockbusters is clearly
greater when I look only at heavy users than before when I included all users. Therefore, heavy
users (who, as we have seen, watch a large number of indies and films of less popular genres)
actually seem to be influenced more by how others have rated indies than blockbusters. A higher
score, even if the ratings are from non-heavy users, is more valuable for these "long tail" films in
the decision-making process of heavy users on what to watch. Contrary to the theory that casual
users would benefit most from the ratings associated with recommendations, the "movie buff"type users actually find the ratings more meaningful. Thus, they not only give ratings the most
frequently, thereby influencing others, they also use ratings the most in deciding whether to
"demand" an unwatched movie. Casual users more likely are influenced by outside sources such
as advertising, so they naturally gravitate more to blockbusters regardless of Netflix rating. The
Tang 26
effect remains very similar if early response is restricted to only ratings given in the first six
months by heavy users, as one would expect. Ratings are given anonymously, though heavy
users are likely to make up many of the first reviewers for any new movies.
When looking at user heterogeneity over time, it is difficult to isolate different fragments
of tastes and whether the changes in user tastes can be attributed to the Netflix algorithm itself or
outside forces instead (e.g., changes in content). For example, the composition of ratings by
genre does not change much over time. Instead, I have just seen how heavy users with a greater
variety in tastes are benefitting, which I consider a more plausible phenomenon than movies
being reached to more fragments or "communities" of demanders.
In 2000, there are 6845 total users who gave ratings, 1285 of which are "heavy users"
according to the absolute definition I previously offered. In 2005, there are 424,737 total users,
22,084 of which are heavy users. At first glance, it appears more casual users had been joining
Netflix, and early adopters of the Netflix service are much more likely to become heavy users
because they keep getting recommended good matches and using the service. However, this
definition does not lend a fair comparison over time, since 2005 would likely have newer users
who simply have not had enough time to give that many ratings while the users in 2000 very
likely had not given over 100 ratings until sometime after 2000.
Using my year-over-year relative definition of heavy users instead, I find that in 2000,
only 131 out of the 6845 total users or 1.91 percent are considered heavy (overall users giving on
average 11.99 ratings that year). In 2005, users give on average 41.21 ratings; 8405 of the
424,737 total users or 1.98 percent are heavy. Thus, even as there is an influx of total users and
users are giving many more ratings on average, the proportion of heavy users stays relatively the
Tang 27
same. Heavy users continue to make up only a small proportion of the total users, but naturally
their ratings take up a much larger proportion of total ratings.
I run through the same models as before, with heavy demand and heavy early response
altered to only include ratings by this year-over-year group of heavy users. Tables 9 and 10
present the results from regressing heavy demand (2) on early response and regressing heavy
demand (2) on heavy early response, respectively.
Table 9. Heavy Demand (2)
(1)
Blockbuster
EARLY_RESPONSE
(Intercept)
4,077
(944.0)
-7,853
(3,552)
(2)
Indie
1,515
(229.3)
-2,991
(759.4)
(3)
Intermediate
(4)
Popular Genre
(5)
Other Genre
3,754
(500.8)
-6,962
(1,735)
3,391
(414.1)
-6,711
(1,441)
2,533
(343.2)
-5,142
(1,158)
Table 10. Heavy Demand (2)
EARLY_RESPONSE_HEAVY
(Intercept)
(1)
Blockbuster
(2)
Indie
4,200
(886.6)
-7,985
(3,269)
1,397
(185.4)
-2,316
(582.2)
(3)
(4)
(5)
Intermediate Popular Genre Other Genre
3,325
(417.9)
-5,187
(1,416)
3,287
(361.5)
-5,970
(1,220)
2,264
(268.5)
-3,851
(868.0)
Based on these results, heavy users are still reacting much more strongly to early response for
indies than for blockbusters, even under my new definition for heaviness. For indies, there is a
78.8 percent increase in heavy demand associated with an increase in rating score of one for
early response; for blockbusters, there is only a 55.2 percent increase.
Tang 28
5 Conclusion
Companies and marketplaces residing on the Internet preserve large banks of private or
proprietary information on inventory and customers; Netflix is no exception. There is no
generally available industry-wide data on the performance of movies via online distributors,
explaining why economic experiments on this market are much less frequent than for box office
performance. However, this thesis uses one of the rare publicly available data sets on moviewatching behavior through online channels, in an attempt to verify beliefs about the types of
movies that benefit from electronic commerce and search/recommendation tools such as
collaborative filtering. Netflix, while having released only a fraction of its total data, still is an
excellent example of an Internet service that uses collaborative filtering and a testing ground for
which types of items belong to the Long Tail.
I ultimately find that positive early feedback benefits to a greater degree "indie" films that
fall under my definition of the Long Tail, as compared to blockbusters with presumably much
larger marketing budgets. This has direct implications for Netflix or other online movie
platforms, as well as content providers (such as movie studios and television networks). Given
the increasing difficulty in getting movies exhibited in theaters, more studios outside of the Big
Six (Disney, Warner Bros., Universal, Columbia, Paramount, and 20th Century Fox) will be able
to green-light and produce films with the thought of ancillary markets such as Netflix in mind.
Meanwhile, an ongoing challenge for Netflix will be to continually license diverse and lesserknown content that caters to the tastes of its subscribers. By filtering these programs correctly to
the proper demanders, Netflix can even turn niches ("long tail" films) into hits on its platform,
fueled by positive buzz or word-of-mouth.
Tang 29
Perhaps the most intriguing inference from my results is that heavy users are apparently
more influenced by ratings than casual users are. Since all of us as consumers have been
preconditioned by our experiences with other online services such as Yelp or Angie's List, this
finding seems counterintuitive at first. On Yelp, for example, the casual user is almost certainly
the one benefiting the most, thanks to all the reviews provided by heavy users or "foodies." On
Netflix though, the heavy user benefits the most in using other users' ratings as information for
his or her movie-watching decision process. Of course, Netflix also differs from the above
mentioned platforms because its ratings are purely anonymous and absent of written reviews.
One cannot filter ratings by "heavy" or "elite" users, though my models indicated that heavy
early response did not affect heavy demand much differently than aggregate early response did.
By the first definition of heavy users (with an absolute cutoff), there is a relatively small
group of users who have contributed to a relatively large proportion of total ratings. Under the
stricter second definition (that measures heaviness relative to a moving yearly average of user
activity), even as users give more ratings on average over time, a similar fraction of users
remains heavy year-over-year. I would claim that these users remain dedicated to giving such a
great quantity of ratings (and users in general are giving more ratings each year) because they
find existing ratings for movies valuable and informative. By proceeding to use these ratings as a
basis to watch more movies, they then give more ratings as well to improve their matches and so
that other users would be informed. This positive feedback loop could help explain why Netflix
retention rates have been so high, even as global subscribers are increasing at a rapid rate.
With the advent of big data, analysts have speculated that Netflix could use the data it
holds on user behavior to strategically change its content and pricing in the future. Since indies
almost always cost less to license than blockbusters (especially if using my definitions), it would
Tang 30
be beneficial to Netflix to license and recommend more "Long Tail"-type content for its users to
watch. As Benjamin Shiller (2013) shows, Netflix data scientists can also use the rating
behaviors of its subscribers (in additional to other information such as web browsing and
demographics) for first-degree price discrimination, leading to higher economic profits. Heavy
users ("movie buffs") not only watch more movies, but also find the Netflix ratings themselves
more useful in guiding them to appropriate movies to watch, as we have seen. For now though,
Netflix remains adamant that its pricing policy will remain the same.
Further studies of interest using the Netflix Prize data set in combination with additional
movie databases such as The Numbers could include box office numbers (an entirely different
market) and how strongly correlated they are with viewership in ancillary markets such as DVDs
or video streaming, how sequels or remakes perform (e.g., whether they have a return on
investment comparable to the original film), and how the timing of a movie release on Netflix
affects its demand (e.g., any seasonality patterns). One could further break down the films by
MPAA rating, Rottentomatoes aggregate rating, or specific genres to see which "types" of films
benefit more or become more prevalent over time, and whether heavy users behave differently
across these different categories.
For further additional research, it would be interesting to look at another popular service
(e.g., Amazon) that uses a collaborative filter for its recommendations but with one key
difference: ratings are not anonymous. This could have an impact on several variables, including
the number of ratings per user or item (especially in comparison to views or "purchases"), the
impact of early ratings relative to more recent ratings, and the relative impact or influence of a
heavy user's rating versus a casual user's rating.
Tang 31
6 Appendix
6.1 Stata Code
Importing Netflix Prize data from text files:
Generating variables:
Tang 32
Constructing models:
Tang 33
6.2 Tables
Table 1. Variable Descriptions
Early response
The average rating score for a movie within 180 days of its first rating.
Heavy user
A user who has given at least 100 ratings.
Nonheavy user
A user who has given between 1 and 99 ratings.
Blockbuster
Movie with production budget of over $100 million.
Indie
Movie with production budget of under $10 million.
Intermediate
Movie in which one of the 20 highest-grossing actors of all time has
appeared, excluding any already labeled as blockbuster or indie.
Movie age
Difference between date of movie rating for observation and date of first
movie rating.
Popular genre
Action/adventure and comedy (including romantic comedy, excluding
black comedy).
Other genre
Black comedy, concert/performance, documentary, drama, horror,
musical, thriller/suspense, and western.
Demand (1)
Total ratings for movie.
Demand (2)
Total ratings for movie in particular month, divided by total users who
rated anything that month. (Measured after first six months for a movie.)
Heavy demand
Total ratings by heavy users for a movie.
Heavy early
response
The average rating score by heavy users for a movie within 180 days of
its first rating.
Tang 34
Table 2. Summary of Means
Variable
Overall
Blockbuster
Indie
Intermediate
Popular
Genre
Other
Genre
Rating
3.570
3.672
3.508
3.553
3.542
3.602
Early response
3.356
3.726
3.248
3.411
3.422
3.311
Heavy user rating score
3.424
3.580
3.356
3.403
3.404
3.444
Heavy user number of
ratings
126.0
24.0
34.8
67.2
63.9
62.1
Nonheavy user
- rating score
3.605
3.689
3.551
3.590
3.573
3.642
26.1
6.5
6.1
13.5
13.9
12.2
1180.1
1133.8
1076.6
1344.8
1213.1
1157.6
Movie-level
Demand (1)
37450.1
88196.7
16985.1
53458.1
48751.9
29735.3
Movie-month
Demand (2)
0.00501
0.00982
0.00202
0.00744
0.00582
0.00444
Heavy demand
7235.1
13625.0
3739.8
10586.2
9066.9
5990.0
Heavy early
response
3.235
3.626
3.084
3.346
3.300
3.190
Nonheavy user
- number of
ratings
Movie age (in
number of
days)
Tang 35
Table 3. Breakdown of genres for movies across categories
Action
Adventure
Black Comedy
Comedy
Concert/Performance
Documentary
Drama
Horror
Musical
Not reported
Romantic Comedy
Thriller/Suspense
Western
(missing)
Total
Blockbuster
Number
Percent
16 Percent 41.03
15
38.46
2
5.13
5
12.82
1
2.56
39
100.00
Indie
Number Percent
3
1.45
2
0.97
2
0.97
43
20.77
2
0.97
11
5.31
86
41.55
10
4.83
2
0.97
28
13.53
8
3.86
10
4.83
207
100.00
Intermediate
Number Percent
15
10.64
12
8.51
4
2.84
32
22.70
2
1.42
41
29.08
4
2.84
8
5.67
19
13.48
1
0.71
3
2.13
141
100.00
Table 4. Summary of means for ratings by heavy users overall, and in 2000 versus 2005
Overall
Rating
Number of ratings per
user
Rating (in the year 2000)
Number of ratings per
user (in the year 2000)
Rating (in the year 2005)
3.464
95.98
3.315
27.95
3.516
Blockbuster
3.617
18.28
Indie
3.421
24.93
3.626
3.024
2.67
3.13
3.635
3.468
Intermediate
3.431
52.76
3.319
22.15
3.494
Number of ratings per
user (in the year 2005)
104.67
21.34
27.14
56.19
Heavy demand
3925.52
7388.33
1922.58
5892.87
Heavy early response
3.206
3.648
3.035
3.332
Tang 36
Table 5. Movie-level Demand
(1)
Blockbuster
EARLY_RESPONSE
DATE_FIRSTRATING
(Intercept)
(2)
Indie
77,393
(15,419)
-22.57
(13.95)
153,180
(202,846)
16,770
(2,449)
0.0618
(3.129)
-38,455
(48,699)
(3)
Intermediate
(4)
Popular Genre
58,974
(6,078)
-3.199
(5.232)
-98,359
(77,893)
(5)
Other Genre
50,360
(5,804)
1.560
(5.989)
-147,808
(88,732)
30,263
(3,594)
-10.13
(4.057)
87,361
(63,405)
(4)
Popular Genre
(5)
Other Genre
Table 6. Movie-month Demand
EARLY_RESPONSE
MOVIE_AGE
(Intercept)
(1)
Blockbuster
(2)
Indie
(3)
Intermediate
0.00721
(0.000512)
-6.72e-06
(6.06e-07)
-0.0106
(0.00201)
0.00196
(6.92e-05)
-2.18e-06
(1.04e-07)
-0.00247
(0.000246)
0.00521
(0.000337)
-8.28e-06
(3.37e-07)
-0.00223
(0.00121)
0.00397
(0.000281)
-6.13e-06
(3.33e-07)
-0.00202
(0.00103)
0.00416
(0.000140)
-4.23e-06
(1.86e-07)
-0.00558
(0.000499)
Table 7. Heavy Demand (1)
(1)
Blockbuster
EARLY_RESPONSE
(Intercept)
7,500
(1,369)
-14,317
(5,150)
(2)
Indie
2,943
(401.3)
-5,751
(1,329)
(3)
Intermediate
7,142
(760.1)
-13,844
(2,634)
(4)
Popular Genre
(5)
Other Genre
6,556
(676.1)
-13,366
(2,353)
4,649
(563.0)
-9,363
(1,899)
Table 8. Heavy Demand (1)
(1)
Blockbuster
EARLY_RESPONSE_HEAVY
(Intercept)
7,996
(1,280)
-15,372
(4,689)
(2)
Indie
3,114
(355.2)
-5,825
(1,127)
(3)
Intermediate
(4)
Popular Genre
(5)
Other Genre
6,966
(699.4)
-12,677
(2,371)
6,328
(634.5)
-11,867
(2,140)
4,958
(487.7)
-9,751
(1,589)
Tang 37
Table 9. Heavy Demand (2)
(1)
Blockbuster
EARLY_RESPONSE
(Intercept)
(2)
Indie
4,077
(944.0)
-7,853
(3,552)
(3)
Intermediate
(4)
Popular Genre
(5)
Other Genre
3,754
(500.8)
-6,962
(1,735)
3,391
(414.1)
-6,711
(1,441)
2,533
(343.2)
-5,142
(1,158)
1,515
(229.3)
-2,991
(759.4)
Table 10. Heavy Demand (2)
EARLY_RESPONSE_HEAVY
(Intercept)
(1)
Blockbuster
(2)
Indie
4,200
(886.6)
-7,985
(3,269)
1,397
(185.4)
-2,316
(582.2)
(3)
(4)
(5)
Intermediate Popular Genre Other Genre
3,325
(417.9)
-5,187
(1,416)
3,287
(361.5)
-5,970
(1,220)
2,264
(268.5)
-3,851
(868.0)
Table 11. Breakdown of Netflix ratings by genre for blockbusters, indies, and
intermediate
Blockbusters (by ratings)
Tang 38
Indies (by ratings)
Intermediate (by ratings)
Tang 39
6.3 Figures
Figure 1. Distribution of demand or ratings for movies on Netflix over time. (Tan & Netessine, 2009)
Figure 2. Sample search query on OpusData.
Tang 40
Figure 3. Descriptive statistics for aggregate Netflix Prize data set. (Tan & Netessine, 2009)
Figure 4. Histogram of Netflix ratings from 2000 to 2005. (Tan & Netessine, 2009)
Figure 5. The exponential rise in movie ratings, and linear increase in number of movies being rated. (Tan &
Netessine, 2009)
Tang 41
Figure 6. Three forces of the Long Tail. (Anderson, 2006)
Tang 42
7 References
Anderson, Chris. 2006. The Long Tail: Why the Future of Business Is Selling Less of More.
New York: Hyperion.
Berger, Jonah, Alan Sorensen, and Scott J. Rasmussen. 2010. "Positive Effects of Negative
Publicity: When Negative Reviews Increase Sales," Marketing Science, September 29(5):
pp. 815-827.
Brynjolfsson, Erik, Yu (Jeffrey) Hu, and Duncan Simester. 2011. "Goodbye Pareto Principle,
Hello Long Tail: The Effect of Search Costs on the Concentration of Product Sales,"
Management Science, August 57(8): pp. 1373-1386.
Brynjolfsson, Erik, Yu (Jeffrey) Hu, and Michael D. Smith. 2006. "From Niches to Riches: The
Anatomy of the Long Tail," Sloan Management Review, July 47(4): pp. 67–71.
Brynjolfsson, Erik, Yu (Jeffrey) Hu, and Michael D. Smith. 2010. "Long Tails vs. Superstars:
The Effect of Information Technology on Product Variety and Sales Concentration
Patterns," Information Systems Research, December 21(4): pp. 736-747.
Cieply, Michael. 2013. "Hollywood Wants Numbers on the Digital Box Office." Last updated
09/2013. http://www.nytimes.com/2013/09/16/business/media/movie-industry-wants-toget-a-handle-on-the-digital-box-office.html, accessed September 20, 2013.
Dellarocas, Chrysanthos, Guodong (Gordon) Gao, and Ritu Narayan. 2010. "Are Consumers
More Likely to Contribute Online Reviews for Hit or Niche Products?", Journal of
Management Information Systems, September 27(2): pp. 127-157.
Elberse, Anita. 2008. "Should You Invest in the Long Tail?", Harvard Business Review, July
(86): pp. 88-96.
Elberse, Anita and Oberholzer-Gee, Felix. 2008. "Superstars and Underdogs: An Examination of
the Long Tail Phenomenon in Video Sales," MSI Reports: Working Paper Series, July (4):
pp. 49-72.
Eliashberg, Jehoshua, Anita Elberse, and Mark A.A.M. Leenders. 2006. "The Motion Picture
Industry: Critical Issues in Practice, Current Research, and New Research Directions,"
Marketing Science, November 25(6): pp. 638-661.
Eliashberg, Jehoshua and Shugan, Steven. 1997. "Film Critics: Influencers or Predictors?",
Journal of Marketing, April 61(2): pp. 68-78.
Fleder, Daniel and Hosanagar, Kartik. 2009. "Culture's Next Rise or Fall: The Impact of
Recommender Systems on Sales Diversity," Management Science 55(5): pp. 697-712.
Tang 43
Karniouchina, Ekaterina. 2011. "Impact of Star and Movie Buzz on Motion Picture Distribution
and Box Office Revenue," Intern J. of Research in Marketing, (28): pp. 62-74.
Moul, Charles. 2005. A Concise Handbook of Movie Industry Economics. New York: Cambridge
University Press.
Netflix, Inc. 2009. "Netflix Prize." Last updated 2009. http://www.netflixprize.com, accessed
September 10, 2014.
Shih, Willy, Stephen Kaufman, and David Spinola. 2009. "Netflix," Harvard Business School.
Case No. 607-138, April.
Shiller, Benjamin. 2013. "First Degree Price Discrimination Using Big Data," Brandeis
University. Working Paper No. 58, August.
Tan, Tom and Netessine, Serguei. 2009. "Is Tom Cruise Threatened? Using Netflix Prize Data to
Examine the Long Tail of Electronic Commerce,” Wharton Business School. Working
Paper No. 1361, September.
Tucker, Catherine and Zhang, Juanjuan. 2011. "How Does Popularity Information Affect
Choices? A Field Experiment," Management Science, May 57(5): pp. 828-842.
Vanderbilt, Tom. 2013. "The Science Behind the Netflix Algorithms That Decide What You'll
Watch Next." Last updated 08/2013.
http://www.wired.com/underwire/2013/08/qq_netflix-algorithm/, accessed September 10,
2013.
Zhou, Wenqi and Duan, Wenjing. 2012. "Online User Reviews, Product Variety, and the Long
Tail: An Empirical Investigation on Online Software Downloads," Electronic Commerce
and Research Applications, May 11(3): pp. 275-289.