Winning Crowdsourcing Contests: an Analysis of the Micro-Structure of
Multi-Relational Networks
Jiahui Mo, Zhiqiang Zheng, Xianjun Geng
School of Management, The University of Texas at Dallas, Richardson, TX 75080
{ jxm083020, ericz, geng }@utdallas.edu
Abstract
Do multi-relational and micro-structural social interactions affect a solver's chance of winning
crowdsourcing contests? If yes, how? This research investigates the impact of two fundamental
types of social interactions in crowdsourcing contests - rivalry and friendship - on a solver‟s
chance of winning. We further propose triad embeddedness as the conceptual framework for our
analysis, in which we highlight multi-relational social interactions within a triplet of three
neighboring solvers. We find that a solver's structural importance in friend and rival networks is
positively associated with her chance of winning. Furthermore, in any given triad, we find that
the relationship between two neighboring solves (inter-neighbor) significantly affect a focal
solver's chance of winning, after controlling for all direct relationships between the focal solver
and neighbors. Specifically, in a triad, inter-neighbor friendship (as compared to inter-neighbor
rivalry) benefits a focal solver when she befriends both neighbors; it however hurts a focal
solver's chance of winning when the focal solver rivals both neighbors; its impact on solver
winning chance is insignificant when the focal solver befriends one neighbor yet rivals the other.
Keywords: Crowdsourcing contest, solver, seeker, multi-relational social network, triad
embeddedness.
1
1. Introduction
Crowdsourcing contests refer to a form of contests in which a firm (seeker) outsources a business
task to an open crowd of competing solvers. The winner(s) of the contest collect a pre-specified
bounty offered by the seeker upon completion of the task. The majority of crowdsourcing
contests are conducted online nowadays and are often facilitated by online marketplaces that
specialize in hosting such contests. For instance, InnoCentive, Inc., a crowdsourcing leader in
U.S., established its online marketplace for open innovation in 2001, on which firms can post
research and development (R&D) challenges to a large pool of solvers.1 In China, two leading
crowdsourcing marketplaces, taskcn.com and zhubajie.com, have more than 900,000 registered
solvers each on entering 2010.
On marketplaces with large numbers of solvers, competition among the solvers is often
intense and winning is anything but sure. A recent study by Yang et al. (2008b) reveals that 96.8%
of solvers never won any contest on taskcn.com. Our analysis of zhubajie.com reveals a similar
pattern that 95.5% of solvers have yet to win a contest. While the literature on auctions and
contests offers abundant explanations on the economic factors (e.g. prize) that might affect
bidder behavior and chance of winning (e.g. Archak and Sundararajan 2009), there is limited
understanding on what social factors within a crowdsourcing marketplace contribute to (or
hinder) a solver's chance of winning. 2,3 Given the extremely low winning likelihood for an
average solver in many crowdsourcing marketplaces, the aforementioned question is of apparent
and paramount significance to solvers in practice.
1
As of September 2010, InnoCentive has more than 200,000 registered solvers.
Examples of social factors include solver interactions as well as the characteristics of the network a solver is
embedded in. Later in the paper we will discuss what specific social factors we are interested in.
3
Online marketplaces for crowdsourcing is a recent phenomenon enabled by the Internet. Prior to that, social
interactions among solvers are constrained to physical venues only and, understandably, collecting large-scale data
on such interactions is hard if not impossible.
2
2
Social interactions in the context of crowdsourcing marketplaces also pose a number of
research challenges. First and foremost, social interactions among solvers in a crowdsourcing
marketplace are often multi-relational in nature. Multi-relational network (also known as
multiplex network) is defined as a network where a node may link/tie to another node with more
than one relation (Wasserman and Faust 1994). For example, Powell et al. (2005) studies four
types of linkages that coexist in a community within the biotechnology industry: finance,
licensing intellectual property, sales and marketing ties. Some crowdsourcing marketplaces, such
as zhubajie.com and innovationspring.com, allow solvers to explicitly designate friends or
contacts. Friends on zhubajie.com are then able to use communication channels including
electronic mails and instant messenger (usually the popular messenger called QQ 4 ) that are
exclusive to themselves. 5 Crowdsourcing marketplaces, nevertheless, differ prominently from
typical social networking websites in that, by design, there is rivalry relationship among solvers
who compete in a same contest. The co-existence of both friendship and rivalry relationships is a
distinct characteristic of social activities in crowdsourcing marketplaces that few other social
networking websites share.
It is important to note that analyzing the structural properties of the network (e.g. various
centrality measures) alone may not be adequate in understanding multi-relational networks. For
example, it may not be enough to focus exclusively on how many actors one interacts with (e.g.
degree) without taking into account what kind of relation is involved (e.g. friends or rivals) or
how well solvers know each other. The compounding impact of these heterogeneous social ties
on a solver's chance of winning is thus not immediately clear and requires a thorough
understanding on the multiplex nature of the networks.
4
Alexa ranks QQ the 11th communication tools globally in September 2010.
In addition, friends are allowed to log onto each other's „space‟ – a personal web page a solver maintains on
zhubajie – to communicate with the solver or track her activities that can be invisible to non-friends.
5
3
Second, a subtle but interesting feature of crowdsourcing contests is that competition as a
form of social interaction not only directly affects a solver's chance of winning in a current
contest, it may indirectly affect her chance of winning future contests because of experiences or
knowledge accrued through competition. Evidence exists for both. For example, a study by Yang,
Chen and Pavlou (2009) indicates that the higher the prize offered by the seeker the more solvers
compete in the contest, which implies that high prize leads to high competition intensity and thus
lower winning probability for each solver. However, experiencing strong competition in the past
may improve a solver's chance of future wins as a result of learning from social ties (e.g.
Brabham 2008). Another indirect but strong empirical support is the different level of social
interactions for winners and non-winners: at zhubajie.com, on average each winner befriends
three times more solvers and competes in twice as many as contests compared to yet-to-win
solvers. Taken together, all these call for a more comprehensive study to analyze how dynamic
multi-relational networks influence crowdsourcing contests.
Third, in a crowdsourcing marketplace, social connections - both friendship and rivalry are often sparse and clustered according to solver interest and skills. For example, logo-design
tasks on zhubajie.com (the most popular type of contest hosted there) demand artistic skills on
color, pattern and aesthetics. As a result, solvers tend to focus on niches of the marketplaces
where their skills are best suited for, and social interactions are often clustered around these
niches. In contrast to macro-structural analysis which focuses on understanding the
characteristics of an entire network structure, we are interested in the micro-structure of the
network, which refers to the immediate neighborhoods a solver is embedded within. The
importance of analyzing social interactions at the micro-level is also echoed by several recent
studies (Rogers 2003, Hanneman et al. 2005, Forman et al. 2008, Singh et al. 2010). For example,
4
in their study of Amazon‟s online review community, Forman et al. (2008) point out that the
relevant community to a reviewer is likely to be described by a core group of people with a
common set of interests in a particular book category, who are likely to have repeated encounters
with one another. Singh et al. (2010) also show that social capital is not equally accessible to or
appropriated by all projects. Most of the time, most workers interact with a fairly small set of
others, most of whom know one another well. From a theoretical note, Hanneman et al. (2005)
argue that it is the immediate neighbors that most directly constrain actors, and provide them
with access to opportunities. The ways in which individuals are attached to macro-structures is
often by way of their immediate neighbors. All these suggest that the more relevant unit of
analysis of the crowdsourcing network is the micro-structure of a solver‟s immediate
neighborhood.
While solver winning can be affected by a plural of economic and social factors, our
interest in this research is to investigate and highlight the possible linkages between multirelational, dynamic social interactions among solvers at the micro-structural level and a solver's
chance of winning. To facilitate our analysis, we further propose triad embeddedness as a central
concept for this research. We define triad embeddedness as a solver being embedded in a threeactor neighborhood, social interactions among which are multi-relational and dynamic. This
micro-structural view of the network that consists of the focal solver and two immediate
neighbors forms the building block of our analysis.6 Within the context of this research, for any
given solver, there are six basic forms of triads as shown in Figure 1.7 They capture the multirelational nature of solver interactions. For ease of exposition, for a given solver such as A in
6
Immediacy means the neighbor is a friend and/or a rival of the focal solver in at least one past or ongoing contest.
One might argue that the lack of a relationship (i.e. two solvers being neither friends or rivals) also embodies a
type of relationship. But as we will explain later, that it has no material effect on our measure because we take a
solver‟s perspective and convert the relationships into three groups of ratios.
7
5
Figure 1, we refer to the interaction between A and any immediate neighbor the "solver-neighbor
interaction," and the interaction between the two neighbors the "inter-neighbor interaction."
A
+
+
B
-
+
-
-
C
+
+
-
+
+
+
-
+
-
-
-
-
Figure 1: The six types of triad embeddedness
Note: + stands for friendship, - stands for rivalry, we use “A” to denote the focal solver on the
pinnacle of the pyramid, and “B” for the left bottom solver and “C” for the right bottom solver.
Our research framework of triad embeddedness is motivated by previous research that
shows dyads -- pair-wise interactions between two solvers without considering their social
context -- is often insufficient in capturing micro-structural social interactions (Simmel 1917,
Hanneman et al. 2005). Georg Simmel (1917) observes that three actors in a triad may allow
qualitatively different social dynamics that cannot be reduced to individuals or dyads. Moreover,
Hanneman et al. (2005) finds that all fundamental forms of social relationship can be observed in
triads, yet not in dyads.
Our research extends prior theory on triadic social interactions in two important
directions. First, our triadic analysis centers on the notion of a "focal actor" ( a focal solver in our
context), and we examine the micro-structural impact of a triad with respect to this focal actor. In
comparison, the literature primarily uses triads for macro-structural analysis where what relevant
6
is the number of different types of triads within the whole social network (Hanneman et al. 2005).
The introduction of focal actor is of practical relevance. For example, the micro-structural
analysis of a triad helps answer how this triad affects the focal solver's winning, the paramount
question for a solver. Second, we examine two particular contrasting yet simultaneous social
interactions -- friendship and rivalry – in the triadic social analysis. The second keyword in "triad
embeddedness" emphasizes our approach of embedding triadic analysis into a multi-relational
and dynamic context.
Our research interest can now be succinctly re-phrased as to investigate the possible
linkages between a solver's triad embeddedness and her chance of winning. In particular, this
paper answers two research questions. First, for any given focal solver, how do focal solverneighbor interactions directly affect the focal solver's chance of winning a crowdsourcing
contest? For example, if we take A as the focal solver in Figure 1, this first research question
focuses on A-B and A-C interactions. We analyze how friendship and rivalry between a solver
and her immediate neighbors affect her winning, respectively. We also analyze the potential
moderating effects of solver capabilities among the multiple relationships.
Our second research question centers around the third link of a triad: within a triadic
context, for any given focal solver, how do inter-neighbor interactions indirectly affect the focal
solver's chance of winning? For example, in Figure 1 and for focal solver A, this second research
question is on the relationship between A's chance of winning and B-C interactions.
Fundamental to this second research question is the insight in the theory on triadic social
networks (Hanneman et al. 2005) that complex triadic social dynamics (that affect a focal actor)
cannot be delineated into simple dyadic ones. After controlling for focal solver-neighbor
interactions, we are particularly interested in whether inter-neighbor friendship will have a
7
significantly different impact on a focal solver's chance of winning than inter-neighbor rivalry,
and furthermore whether this difference is moderated by the three types of focal solver-neighbor
ties (Figure 1) and by solver capabilities.
Using longitudinal data in zhubajie.com from 2007 to 2010, we derive the following
interesting results. First, focal solver-neighbor interactions as measured by degrees -- be it
friendship, rivalry or their interactions -- have significantly positive impacts on the focal solver's
chance of winning a crowdsourcing contest. Second, after controlling focal solver-neighbor
interactions, solver efforts and ability, and crowdsourcing task characteristics, we find the
striking result that, within a triad, the multi-relational inter-neighbor interactions may have
significant impact on the focal solver's performance. Furthermore, this impact is moderated by
focal solver-neighbor relationships. Specifically, in a triad where the focal solver befriends both
neighbors, a friend relationship between the two neighbors is positively associated with the focal
solver's change of winning (as compared to a rival relationship between the two neighbors). In a
sharp contrast, a friend relationship between the two neighbors is negatively associated with the
focal solver's change of winning if the focal solver is a rival of both neighbors. In a triad where
the focal solver is befriending one neighbor yet competing with the other neighbor, the interneighbor interaction has insignificant impact on the focal solver's chance of winning.
The rest of the paper proceeds as follows. We first review related research on
crowdsourcing contest in research tournament, sociology, as well as in organizational literatures.
We then present in order our theory and hypotheses, research method, findings, and concluding
discussions.
8
2. Literature review
We review three related streams of research: crowdsourcing contest; multi-relational network;
and triads in a network.
2.1 Crowdsourcing contest
Crowdsourcing contests (also referred to as open innovation8) have been found to be an effective
platform to tap into the wisdom of crowds for a variety of tasks (Yang et al. 2009, Terwiesch and
Xu 2008). Recently, many firms start using crowdsourcing contests routinely for micro-tasks,
which are low prize, frequently recurring and less-innovative tasks such as multimedia
processing, business document preparation, software coding and debugging. 9 For example, there
are now more than 100 online crowdsourcing marketplaces in China that cater to micro-tasks,
attracting more than 6 million solvers in total.10
A small but rapidly growing literature has studied crowdsourcing contests using theories
in auction (including contest), computer science, social networks, co-opetition and organizational
learning. Terwiesch and Xu (2008) are among the first to apply economic models of contests to
studying R&D crowdsourcing. They find that solver strategy and contest outcome are contingent
on the type of tasks (expertise-based project, ideation project or trial-and-error project).
Furthermore, using a solver performance function that is an additive function of solver expertise
and effort, they find that a seeker can benefit from a large solver population because of the
diverse expertise the crowd brings in - thus providing one theoretical support to the openness of
crowdsourcing. Archak and Sundararajan (2009) use the all-pay-auction approach with a
deterministic performance function, and find that optimal prize structure is dependent on solver
8
We would like to note that the notion of open innovation is not equivalent to crowdsourcing contest, however most
of related researches use the same examples under wither term and do not differentiate them.
9
For instance, in Zhubajie.com, the average prize per contest is 347 Chinese Yuan, or roughly $50. Over a span of 3
years, 76% of winners (solvers who won at least one contest) made less than 1000 Chinese Yuan in total.
10
Source: http://www.citnews.com.cn/news/201003/1057.html (an IT news site in Chinese).
9
risk attitude. DiPalantino and Vojnovic (2009) also adopt the all-pay-auction framework but
consider multiple tasks (yet do not introduce strategic seekers). They find that rewards have
diminishing returns on inducing participation. Yang, Chen and Pavlou (2009) explore how to
identify the optimal number of solvers for the tasks. A common feature of the above theoretical
papers is that they take a seeker‟s perspective, and try to design mechanism for the seekers to
attract more solvers. However, as pointed by Nikolay Archak (2010), designing efficient
crowdsourcing mechanisms is not possible without deep understanding of incentives and
strategic choices of all participants. So, the problems related to the solvers, such as which factors
drives solvers‟ winning in a contest, are equally important.
Yang et al. (2008a) provide a descriptive analysis of the solvers‟ strategies. They find that
experienced solvers tend to avoid tasks with too many competitors, and that a very small core
group of multiple-task-winning solvers are able to improve their win-to-submission ratio over
time (yet the vast majority of solvers cannot). Nikolay Archak(2010) throughan empirical
analysis show how the reputation systems in the crowdsourcing contests influences the solvers‟
performance. They also find that solvers behave strategically (e.g. using cheap talks) to soften
the competition. Their research focuses on the current contest, and does not consider how the
past contest orthe social interactions driving the winning probability. An exception is Bayus
(2008) which examines whether individuals in the crowd continue to propose creative ideas over
time. Their results show that the past success cannot promise the creativity over time. A notable
distinction of his work with ours is that its analysis does not build on social network analysis, as
we do in this paper. Yang et al. (2009 b) are among the first to apply social network analysis to
studying solver strategy in crowdsourcing contests. They find that a higher prize attracts more
and higher-than-average-prestige solver participation. However the focus of this research is on
10
the macro-structural factors that impact winning the current contest. A we pointed out earlier,
solvers‟ winning probability will also be influenced by the activities in the past contests they
participated. For example competing with strong solvers or making friends with highly-skilled
solves can foster the learning process and thus improve the wining probability in the future. As
pointed out by Brabham (2008) in a study on iStockphoto, the opportunity to develop one‟s
creative skills and the desire to network with friends and other creative people are among the
main motivations of solver participation, along with the opportunity to make money. Similarly,
Lakhani et al. (2007) found that intrinsic motivators (e.g., “enjoying problem solving and
cracking a tough problem”), as well as financial reward, are significantly and positively
correlated to success as a solver on InnoCentive. Research that focuses on current contests tend
to overlook the rich information embedded in a solver‟s historical behaviors such as how one‟s
capability was developed over time. In contrast, we explicitly study the relationship in the
dynamic, historical network to explore how the social interactions influence the winning.
2.2 Multi-relational network
The importance of multiple networks in understanding social structures was emphasized by
White, Boorman and Breiger (1976) and Boorman and White (1976) in their seminal work. They
argued that many different types of ties are needed to completely portray social structures
becasue the patterning and interweaving of different types of ties are characteristic of social
positions and social structure. Multi relational network has increasingly been recognized as a
fundamental aspect of social relations as human relationships are complex and multi-faceted
(Lewicki et al. 1998). A relationship between two actors can take many forms and serve many
different purposes. The relationships among actors inevitably will develop new features across
an extended time period (Robins etc. 2006)
11
Several recent social network studies began to investigate the multi-relational nature of
certain social networks. Lomi and Pattison (2006) provide an example of inter-organizational
dependencies across multiplex networks including supply, technology transfer, and equity
networks in the transportation manufacture industry in Southern Italy. Ingram and Roberts (2000)
documented that the multiplex characteristics of relationships, particularly in friendship ties
among managers of competing hotels increased their performance. Lee and Monge (2009) focus
on finding the patterns of association between multiplex network domains and the determinants
behind the formation of multiplex ties.
Another related research area in computer science is signed network, which classifies
social ties into positive and negative ones. For example, using massive multiplayer online game
data, Szell et al.(2010) extract networks of six different types of one-to-one interactions between
the players: three positive connotation (friendship, communication, trade), three negative
ties(enmity, armed aggression, punishment). However, they find that negative interactions differ
from positive interactions by their lower reciprocity, weaker clustering and fatter-tail degree
distribution. Leskovec, Huttenlocher, Kleinberg etc. (2010) develop a status method to predict
the positive and negative ties. However, these researches analyze each network separately,
implicitly asuming that each type of network operates independently, though all these different
type of relations are of the same actor. According to Lee (2009), in multi-relational networks,
there is often mutual dependence among different type of networks. The analysis appropriate to
single networks and may not be applicable to multiple networks. Therefore, it is necessary to
analyze the multiple networks simultaneously ( Robins and Pattison 2006.)
2.3 Triads in social networks
12
We adopt triad as the foundation of our micro-structural analysis for several reasons. First,
as Georg Simmel (1917) argued, triads are a fundamental unit of sociological analysis. Three
actors in a triad may allow qualitatively different social dynamics that cannot be reduced to
individuals or dyads. Similarly, Serrour et al. (2010) perceive triangle as the basic unit of
transitivity and redundancy in a graph and it is the simplest but most fundamental unit in
analyzing a complex network. Triadic relationships can be generalized to larger networks,
offering predictions for the level of cooperation in groups as a function of network structure
(Monge and Contractor 2003). A number of scholars have further argued that all fundamental
forms of social relationship can be readily observed in triads (Robins, Pattison and Woolcock
2005, Hanneman et al. 2005). For example, among three parties A, B and C, party A may have a
dyadic relation to C but also may have an indirect relation to C through B. Party B may then
serve to alter the strength or the nature of the relation between A and C, such as solidifying an
alliance or mediating a conflict.
Second, the analysis of a triad could lead to effective explanation of the multi-relational
ties between the neighbors, because triads accommodate a much wider range of possible
relations (as compared to dyads). If there are two kinds of ties (friendship and rivals), then
different from the dyad which only allows for two kinds of relationship (friendship or rivals), the
analysis of triad will allow for six types of relationship combinations in shown in Figure 1.
Batjargal (2007) points out that ties initiated, formed, and maintained between two actors in
triads may have various contents such as friendship, information sharing, scientific collaboration,
and learning. Coleman (2010) argued that closure in triad leads to interpersonal trust, greater
cooperation, and enforcement of norms. Because of the multi-relationship combinations in the
13
triads, small group theorists argue that many of the most interesting and basic questions of social
structure arise with regard to triads.
Triadic analysis has been used in the literature, yet overwhelmingly at the macrostructural level on issues such as the network-wide transitivity (Batjargal 2007). At the interorganizational level, Uzzi and Gillespie (2002) found that small firms learn from embedded
relationships with their banks, and leverage that financial knowledge in relationships with their
trade creditors. They argue that knowledge transfers in triads improve firms‟ debt performance.
Batjargal (2007) uses the postulate of transitivity of social network to examine the interpersonal
trust on the referrals and investment decisions of venture capitalists in Chinese. Furthermore,
most existing research on triads in social networks do not consider the effects of different
relationships in a triad.
3 Theory and hypotheses development
We organize our theories into two groups: those with respect to direct ties (of a focal
solver) and those of indirect ties within a triad.
In crowdsourcing contests, solvers are situated in two types of network, the friend
network and the rival network. Thus a triplet of actors embodies six possible types of
relationship as shown in Figure 1. Given the relationships between the focal solvers and her
neighbors, the inter-neighbor relationship can potentially have different influences on the focal
solver‟s performance.
Direct ties generally offer individuals the opportunity to share and learn skills that will
improve their personal lives (Kehler 2010). From the cohesion perspectives on networks, direct
cohesive ties are a mechanism for gaining fine-grained information. Actors who share direct
14
connections with each other are likely to possess more common information and knowledge of
each other (Coleman, Katz, and Menzel, 1966). Cohesion can also be viewed as the capacity for
social ties to carry information that reduces uncertainty and promotes trust between actors
(Granovetter, 1973; Podolny, 1994; Gulati, 1995a; Burt and Knez, 1995).
Actors in a relationship also bring with them the knowledge and experience from their
interactions with other actors (Gulati and Gargiulo 1999). Therefore, indirect ties may also play a
role.
3.1 The effects of Direct Ties
We first examine the direct effects of the triad in the multi-relational networks.
3.1.1 Direct ties in the friend network
Friend network is among an individual‟s most important social capital. Friendship ties are found
to have higher interaction frequency than other types of ties (Granovetter 1973). Learning, for
example, frequently occurs within a friend network (Burt 1995.) Friend network usually implies
mutual trust, shared interest or common beliefs between each other. Rogers (1995) asserts that
homophily among between actors in a network would increase the likelihood of the diffusion of
shared ideas. Friendship usually leads to strong ties due to frequent interactions. According to
Hansen (1999), strong ties promote the transfer of complex knowledge.
Direct reciprocity often arises when individuals have repeated one-to-one interactions
(Gu et al. 2007.) Friendship usually entails reciprocal relationships, leads to conduit information
transmission. Having more friends enables a solver to share more valuable information, access
more resources and have a better chance to enhance her skills through mutual learning and
ultimately lead to higher chance of winning. Therefore, we postulate that:
15
Hypothesis 1a: The more friends a solver has, the higher the winning chance for the
solver.
3.1.2 Direct ties in the rival network
In the case of the rival network, the rivals may not be willing to directly share valuable
information (e.g. how to use Photoshop) to a focal solver. But the focal solver can observe what
the rivals do and indirectly learn from them.
For example, after a contest, the solver can
download the winners‟ solution (which is public information in zhubajie.com) and observe the
color, the style and other knowledge from the rivals. As such, we assert that
Hypothesis 1b: The more past rivals a solver competed with, the higher the winning
chance for the solver in future tasks.
3.1.3 Direct ties in the co-opetitor network
We refer to the intersection of the rival and friend networks the co-opetitor network. A link in the
co-opetitor network implies that two solvers engage in both collaborative and competitive
actions. In industrial competition, Garcia and Velasco (2002) and Loebbecke et al. (1999) find
that a firm benefits the most from other competing firms with which it forms strategic alliance at
the same time. Similar phenomena are also observed within multi-unit organizations (Tsai 2002).
We extend the theory to consumer level (as solvers competing for micro-tasks are usually
individuals) and state the following hypotheses:
Hypotheses 1c: A solver’s structural importance in her co-opetitor network is positively
associated with her chance of winning.
3.2 Effects of Indirect Ties in Triads
3.2.1 Two friends
16
According to Gulati (1998), an actor in a social network can derive control advantages by being
the tertius gaudens, or one who is situated between two other actors (also known as structural
hole or brokerage.) This can occur when two or more actors are after the same relationship with a
focal actor, as is the case when multiple firms want to form an alliance with a certain firm; or
when an actor is the tertius in separate relationships with two actors having conflicting demands.
In both instances, firms in the tertius role can gain advantages by playing one off against the
other and brokering tension between the other players. Further according to the balance theory,
the micro-structure tends to become a balance state, two guys with a common friend tends to
become friends. Preferential attachment theory also suggests that when two actors share a
common friend, these two actors tends to become friends. Therefore, in the triad with three
positive ties, the solver tends to trust each other, leading to mutual benefit. Therefore, we
propose that
Hypothesis 2: Consider a triad in which the focal solver befriends both neighbors. A
friend (rival) relationship between the two neighbors is positively (negatively) associated with
the focal solver's change of winning.
3.2.2 Two rivals
When a solver competes with two other solvers who are friends between themselves, the solver
may be in a serious disadvantage due to the collaboration and trust of them. Similar phenomenon
is observed in the case of firm competition where the firms usually adopt alliance strategy to gain
the advantage to compete with other firms (Kanter 1994). We propose that
Hypothesis 3: Consider a triad in which the focal solver is a rival of both neighbors. A
friend (rival) relationship between the two neighbors is negatively (positively) associated with
the focal solver's change of winning.
17
3.2.3 One rival and one friend
Reciprocity is especially important in social dilemmas where private interests conflict with
collective interests (Rabin 1993). For example, in the crowdsourcing contest, suppose solver A
(the focal solver) and B are friends; A and C are rivals; B and C are friends. Then A and C are
not in the directly reciprocal relationship. In such a case, B and C need to contemplate on
whether and how much information to share with each other. Actor C in particular, may not be
willing to transfer all the information to B, because she worries that B might transfer the
information to her rival A. Therefore, we propose that
Hypothesis 4: Consider a triad in which the focal solver has one neighboring friend and
one neighboring rival. A friend (rival) relationship between the two neighbors is negatively
(positively) associated with the focal solver's change of winning.
4. Data, Measures and Method
4.1 Data
Our data source is zhubajie.com, a leading online crowdsourcing marketplace in China.
Launched in October 2006,11 zhubajie.com has experienced tremendous growth in recent years:
as of March 2010,12 it has hosted 108,058 contests and attracted more than 2.9 million solver
submissions. It offers crowdsourcing contests in a few dozen task categories. The top three
categories in terms of popularity are design, promotion and website, with 21816, 14550 and 9666
contests have been hosted respectively. Under the design category, 11,584 (or 53% of) tasks
belong to the logo design subcategory, which is the focal subcategory on which we collect data.
11
Zhubajie fully rolled out the crowdsourcing service in March 2007 and this is the starting point of our data
collection. Zhubajie was operating in an experimental mode in the first six months (October 2006-March 2007) and
in total only 500 contests were run during the experimental period.
12
All the statistics on Zhuibajie.com reported in this section are as of March 2010, unless otherwise noted
18
We select logo design for two reasons. First, good logo design requires both artistic & design
skills and significant efforts from a solver, making it an appealing candidate task for
investigating crowdsourcing contests (Terwiesch and Yi 2008, Yang et al. 2009). Second, logo
design is the most popular subcategory as it represents 10.7% of all contests hosted on the site.
We downloaded all 11,584 logo design tasks in zhubajie.com. In terms of prize structure,
the vast majority of tasks (94%) use winner-take-all policy. Only 693 (or 6%) tasks selected
multiple winners eventually. Fraud is a major problem plaguing multi-winner tasks on Zhubajie:
91.9% of these tasks started as winner-take-all contests yet were forced to switch to multiple
winners due to disputes and possible fraud between seekers and solvers. In these cases, Zhubajie
normally acts as an arbitrator and typically select multiple winners to split the award. Since
multi-winner tasks only represent a small portion of total projects and we cannot identify which
of them are fraudulent, we include only the winner-take-all tasks in this research. In the end, we
retained 10,891 single-winner logo-design tasks spanning from March 2007 to March 2010.
These tasks have attracted 66,859 unique solvers over the three-year period. Table 4.1 provides
some descriptive statistics over the sample.
Table 4.1 Descriptive Statistics
Avg. duration of tasks
Avg. prize of tasks
Avg. # of solvers per task
Avg. # of solutions per task
Avg. # of tasks joined per solver
Avg. # of solution submitted per solver
Avg. active time per solver
12 days
341 RMB (≈$50)
42
58 submissions
6
9
107 days
4.2 Network Construction
We first extracted all the friends listed by the 66,859 solvers, based on which the undirected
friend network is built. There is one tie when two solvers designate each other as friends.
19
Because we do not observe which solver initiates the friendship, we only construct an undirected
friend network.
We then constructed the undirected historical rival network based on solver-task pairs.
Any two solvers who participate in the same contest at least once are considered to be rivals. We
simplify the network into an un-weighted graph though solvers can participate in the same tasks
multiple times.
Note that both the friend and rival networks are constructed in a dynamic manner. For
each task of interest, we identify past friends and rivals up to the month of this task. We
consider all friends of a solver starting from the onset of Zhubajie.com to the month of the task
under examination. We include all friends because friendship is explicit, physical and typically
long-lasting. In contrast the rivalry relationship in crowdsourcing is implicit, virtual and
ephemeral. Typically the relationship only lasts through the task co-participated by two solvers.
We thus only consider the rivals during the previous month of the task under examination to
construct the rival network. 13 The descriptive statistics are presented in Table 4.2. Lastly we
identify the pairs of solvers who are both friends and rivals. These solvers pairs form the coopetitor network. The multi-relational network can be seen in the feature 4.1.
Table 4.2 Summary Statistics for the three Overall Networks (March 2007-March 2010)
Descriptive Statistics
Mean
Friend network
6.97
Rival network
309.29
Co-opetitor Network
4.80
St.Dev
24.08
863.15
14.36
Sum
Min
Max
140,244
1
1,569
20,678,984
0
39,085
39,942
1
397
Network centralization
0.03%
4.63%
0.06%
Total number of solvers
20,109
66,859
8,325
13
We also conducted a robust check by constructing rival networks using two and three-month windows. We found
that the rivals that a solver encountered two or three months before do not play a significant role.
20
4.3 Measures
4.3.1 Dependent variables
The dependent variable we consider is a binary measure Yi,j indicating whether solver i wins
contest j, where 1 means the solver is a winner and 0 otherwise.
4.3.2 Independent variables
Relational Embededdness
In order to measure the relational embeddedness, we first count the closed triangles for each
individual and then distinguish the triangles according to the different relationships. For example,
there are six types of triangles. (+++,++-,+-+,+--,--+,---) Given the first two relations (with the
focal solver), the relationship in the third (indirect) tie may have completely different effects. To
simplify the analysis, without loss of generality, we propose a ratio metric. For examples, we
measure the ratio using N+++/(N++- + N+++) at time t to stand for the interaction with the two
networks where N represents the count of each scenario. The ratio measures that given the two
direct relationships, how the degree of the friend (rivalry) relationship in the third tie affect the
focal solvers. In total, three ratios are sufficient to capture all the six scenarios delineated in
Figure 1. They are N+-+/(N+-- + N+-+ ) , N--+/(N--- + N--+ ) and the N+++/(N++- + N+++ ).
Degree centrality
We adopt the most popular centrality measure - degree - to capture the structural importance of a
solver in the three networks.
Jaccard coefficient
Further to measure the co-opetitor network, we adopt the Jaccard coefficient J αβ, defined as the
degree of overlap between two different sets of links α and β ( Szell et al. 2010), i.e., the friend
and competitor relationship in this study. Jaccard coefficient is a similarity score between two
21
sets of elements and is defined as the size of the intersection of the sets divided by the size of
their union as
αβ
.
4.3.3 Control variables
Task-specific
To control heterogeneity and task specific effects, we include the prize and the duration of each
task. Prizes are the awards the seeker designated for a task measured in Chinese Yuan. Duration
measures the number of days between the starting time and deadline of a task.
Solver-specific
We control several solver specific effects including capability and effort the solver exerts. A
solver's capability often affects her chance of winning in crowdsourcing contests. For instance in
a logo-design contest, a solver's aesthetic skill and knowledge of firm preferences in logos (e.g.
more or less color? simple or complex patterns?) play important roles for her to win. Zhubajie
records a „capability‟ score for each solver based on the total prize she has won.
Improving one‟s skills through social interactions has been noted as one of the main motivations
for entities to participate in various social networks such as in open source project development
(Tan et al. 2010, Rajiv Jayant 2010), Wikipedia editing (Michael Zhang 2010). Effort is
measured by the total number of solutions a solver submitted to the contests she participated.
Competitor-specific
A rich literature on all-pay-auction has stated that a solver‟s winning chance is also affected by
her competitors‟ capabilities (Archak et al 2009, Yang et al 2009) and efforts (Moldovanu and
Sela 2001). We measure competitor effects using average capability and effort across all
competitors of a given solver.
22
4.4 Model
We model the probability of winning of a solver i in task j as a probit function of the variables
we described in section above, specifically:
(
)
(
)
is a binary dependent variable indicating whether solver i wins task j. Φ is the normal link
for the probit model. The independent variables
refer to the three
ratios described. Also, the relations among the triads will be dynamically updated to the month
of tasks j-1. The independent variables associated with the friend network is grouped under
, which includes the degree of the historical friend network for solver i up to the
month of task j-1. Similar variables are used for the rival and co-opetitor networks. The control
variables include all variables described in section 4.3.3 for solver i, rivals in the current task j as
well as the task specific variables (prize and duration).
5 . Results
The results are summarized in Table 5.1. Model 1 examines only the effects of control variables.
Model 2 examines how the centrality of the multi-relational network influences the solvers‟
performance. The results support our hypotheses 1a-1c: degrees have significantly positive
influence on a solver‟s winning probability in all three networks: the friend network (0.00041,
P<0.001), the rival network (.0.00002, P<.001) and the co-opetitor network (10.5057, P<0.001)
respectively.
23
Table5.1 Network effects on a solver's chance of winning
Independent variables
Variable type
Variable name
Intercept
Triad
Ratio of +++
Ratio of +-+
Ratio of --+
Structure
Friend degree
Rival degree
Co-optitor degree ratio
Task-specific
logPrize
Logduration
CompetitorsCompetitors’ average efforts
specific
Competitiors’ average capability
SolverSolver’s efforts
specific
Solver’s capability
AIC
Degree of freedom (DF)
sample size
Log-likelihood(LL)
Dependent variable (Probability(win=1))
Model1
-0.8689
Model2
-0.95
-0.2365***
0.0677***
-0.2018***
-0.000234***
0.1683***
0.0004***
93151.230
0.00041***
0.00002***
10.5057***
-0.2364***
0.0662***
-0.1982***
-0.000228***
0.1688***
0.00034***
92282.933
Model3
-0.0392
0.0749 ***
0.00403
-30.5882***
0.00049***
-0.000094***
-0.3476***
-0.2963***
0.0705***
-0.2465***
-0.000518***
0.2419***
0.00028***
77336.865
456485
456485
223583
-46568.615
-46131.4665
-38655.4325
-2 (LLi-1 – LL i), for column i .
χ2(DFi-DFi-1,0.05)
Note: We log transformed the following variables: Rival-Degree, Friend-Degree, Co-opetitor-Degree , Prize and Duration. The significances are
marked as *** for p< 0.001 and ** for p<0.01 * for p<0.05.
Model 3 examines how the inter-neighbor relations in a triad influence a focal solver‟s
performance. We find that if all the three solvers are friends, then this structure tends to have a
very positive effect on the focal solver‟s performance -- thus hypothesis 2 is supported. However,
when the focal solver is a rival with both neighbors, inter-neighbor friendship (as compared to
inter-neighbor rivalry) tends to hurt the focal solver's chance of winning -- thus hypothesis 3 is
supported. We do not find significant influence from the triad with one rival and one friend
(hypothesis 4).
6. Concluding Remarks
To date, we are not aware of any research that studies multi-rational social interactions from a
triadic micro-structural perspective, and how these social interactions influence a solver's chance
of winning crowdsourcing contests. This study makes several contributions. First, our research
24
fills the theory gap on multi-relationships in networks, in which most existing researches focus
on the importance of the macro-network structure and overlook how the different relationships in
the micro-structural network impact actor performance. We find that both friendship and rivalry
(and their interactions) can benefit a solver in future contests. Second, our research differentiates
past rivalry from current rivalry. While the latter is expected to hurt a solver's performance, our
results show the former can benefit a solver's performance in future contests. Third and most
strikingly, we show that even after controlling for direct impacts of neighbors on any given focal
solver, the inter-neighbor relationship in a triad can have significant impacts on the focal solver's
performance.
References
Archak, N., Sundararajan, A. 2009.Optimal design of crowdsourcing contests. Proceedings of the
13th International Conference on Information Systems.
Archak, N., M. Glory. Cheap Talk: Analyzing Strategic Behavior of Contestants in Simultaneous
Crowdsourcing Contests on TopCoder.com. 2010 International World Wide Web Conference
Committee (IW3C2).
Baum, J. A. C., A. V. Shipilov, T. J. Rowley. 2003. Where do small worlds come from? Indust.
Corporate Change. 12 697–725.
Borgatti, S. P., R. Cross. 2003. A relational view of information seeking and learning in social
networks. Management Science. 49(4) 432–445.
Brabham, D. C. 2008. Crowdsourcing as a model for problem solving: an introduction and cases.
The International Journal of Research into New Media Technologies. 14(1) 75–90.
Brabham, D.C. 2010. Moving the crowd at Threadless: Motivations for participation in a
crowdsourcing application. Forthcoming in Information, Communication & Society.
Burt, R. S. 1995. Structural Holes: The social structure of competition. Cambridge, MA: Harvard
University Press.
Cowan, R., N. Jonard. 2003. The dynamics of collective invention.J. Econom. Behav. Organ. 52
513–532.
Cowan, R., N. Jonard. 2004. Network structure and the diffusion of knowledge. J. Econom.
Dynam. Control.28 1557–1575.
25
Cui, Z., Loch, C., Grossmann, B., He, R. 2009. Matching innovation outsourcing motivations
and innovation providers: an empirical study at Siemens. Proceedings of 16th EurOMA
(European Operation Management Association) International Conference.
Dasgupta, P., Stiglitz, J.1980. Industrial-Structure and the Nature of Innovative Activity.
Economic Journal, 90 (358) 266–93.
Liu,D.,Geng,X.,Whinston,A.B.2007. Optimal design of consumer contests. Journal of
Marketing,71 140-155.
DiPalantino, D., Vojnovi´c, M.2009. Crowdsourcing and all-pay auctions. Proceedings of the
10th ACM International Conference on Electronic Commerce.119-128.
Dra. CRISTINA QUINTANA GARCÍA, CARLOS A. BENAVIDES VELASCO. II. 2002. Coopetition and performance: evidence from European biotechnology industry. Conference of
Euram on Innovate research in management. Page.
Enkel, E., Gassmann, O., Chesbrough, H. 2009. Open R&D and open innovation: exploring the
phenomenon. R&D Management.39(4) 311 – 316.
Fleming, L., King, C., and Juda, A. 2007. Small-worlds and innovation. Organization Science.
18 938-954.
Fullerton, R. L., McAfee R. P. 1999. Auctioning Entry into Tournaments. Journal of Political
Economy, 107(3), 573–605.
Gnyawali, D.R.,
. He, J., Madhavan, R. 2006. Impact of co-opetition on firm competitive
behavior:An empirical examination. Journal of Management. 32. 507.
Granovetter, M. 1973. The strength of weak ties. American journal of sociology. 6 1360-1380.
Hanneman, Robert A. and Mark Riddle. 2005. Introduction to social network methods. Riverside,
CA: University of California, Riverside
Howe, J. 2006. The Rise of Crowdsourcing.Wired.14.
Jayanth, R. Varghese S. Jacob, Radhakrishnan, S. 2010.Vendor and Client Interaction for
Requirements Assessment in Software Development: Implications for Feedback
Process.Information systems research.
Kleinberg, J. , Suri, S. , Tardos, E., Wexler, T. 2008. Strategic Network Formation with
Structural Holes. Proc. 9th ACM Conference on Electronic Commerce.
Lakhani, K.R., Panetta, J.A. 2007. The Principles of Distributed Innovation. Innovations:
Technology, Governance, Globalization. 2 (3).
Lee and Monge (2009) The coevolution of multiplex networks in organizational communities.
Annual conference of the international communication association (ICA).
26
Lin, M., Prabhala,N.R., Viswanathan,S. 2009. Can Social Networks Mitigate Information
Asymmetry in Online Markets?. Proceedings of the 30th International Conference on
Information Systems.
Loebbecke C., van Fenema, P.C., Powell, F.P.1999. Co-opetition and knowledge transfers.
Special issue on infomration systems: current issues and future changes. 30 (2) 14 - 25
Mizruchi, M. S. 1996. What Do Interlocks Do? An Analysis, Critique, and Assessment of
Research on Interlocking Directorates. Annual Review of Sociology. 22(1) 271- 298.
Moldovanu, B. Sela, A. 2001. The Optimal Allocation of Prizes in Contests. American Economic
Review, 91 (3),542–58.
Nagler, J. 1994.Scobit: An Alternative Estimator to Logit and Probit. American Journal of
Political Science. 38 (1) 230-255.
M. E. J. Newman, S. H. Strogatz, and D. J. Watts. 2001. Phys. Rev. E 64.
Oh, J., Susarla, A., Tan, Y. 2010. Examing the diffusion of user-generated content in online
social networks. Accepted by Information systems research.
Peng, MW. 2004. Outside Directors and Firm Performance During Institutional Transitions.
Strategic Management Journal. 25(5) 453-471.
Schilling, M.A., Phelps, C.C.. 2007. Interfirm collaboration networks: The impact of large-scale
network structure on firm innovation. Management Science. 53(7) 1113–1126.
Singh, P.V., Youn, N. , Tan, Y. 2010.A Hidden Markov Model of Developer Learning Dynamics
in Open Source Software Projects. Forthcoming at Information Systems Research.
Singh, P.V., Tan, Y.,Mookerjee,V.S. 2010. Network effects: The influence of structural social
capital on open source project success. Under review.
Surowiecki, J. 2004. The Wisdom of Crowds: Why the Many Are Smarter Than the Few and
How Collective Wisdom Shapes Business, Economies, Societies and Nations. US: Little, Brown.
Szell,M., Lambiotte, R., Thurnerz, S.Trade.2010.Conflict and Sentiments:Multi-relational
Organization of Large-scale Social Networks. arXiv:1003.5137.
Taylor, C. R. 1995.Digging for golden carrots: An analysis of research tournaments. American
Economic Review 85(4), 872–90.
Terwiesch ,C., Xu,Y. 2008. Innovation contests, open innovation, and multiagent problem
solving. Management Science. 54(9) 1529-1543.
Tsai, W. 2002. Social structure of "coopetition" within a multiunit organization: Coordination,
competition, and intraorganizational knowledge sharing. Organization science, 13(2) 179-190.
Uzzi, B., J. Spiro. 2006. Collaboration and creativity: The small world problem. Amer. J. Sociol.
111(2) 447–504.
27
Uzzi, B., Amaral, L., and Reed-Tsochas, F. 2007. Small-world networks and management
science research: A review. European Management Review. 4 77-91.
Van Den-Bulte, Wuyts, S.2007.Social Networks and Marketing. Cambridge, MA: Marketing
Varian, H.R. 1980.A Model of Sales. American Economic Review. 70 (4) 651–59.
van de Vrande, V., JeroenP.J.deJongb,Vanhaverbekec, W., Maurice deRochemontd. 2009.Open
innovation in SMEs: Trends, motives and management challenges. Technovation. 29 423–437.
Verspagen, B., Duysters, G. 2004.The small worlds of strategic technology alliances.
Technovation. 24(7) 2004 563-571.
Wasserman,S., Faust,K.1994.Social Network Analysis: Methods and Applications. Cambridge
University Presss.
Watts, D. 1999.Networks, dynamics, and the small-world phenomenon. American Journal of
Sociology. 105 493-527.
Whitla,P.2009. Crowdsourcing and its application in marketing activities. Contemporary
Management Research. 5(1) 15-28.
Yang, J., Adamic, L., Ackerman, M.2008 a. Competing to Share Expertise: the Taskcn
Knowledge Sharing Community. International Conference on Weblogs and Social Media.
Yang, J., Adamic, L., and Ackerman, M. 2008 b. Crowdsourcing and knowledge sharing:
strategic user behavior on Taskcn. Proceedings of the 9th ACM International Conference on
Electronic Commerce.246-255.
Yang Y.,Chen, P., Pavlou, P. 2009. Open innovation: An empirical study of online contest.
Proceedings of the 13th International Conference on Information Systems.
Zhang,X. ,Zhu, F. 2010. Group size and incentives to contribute: A natural experiment at
Chinese Wikipedia. Forthcoming in American Economic Review.
28