Undergraduates
Subject
Effect:
An
Experimental
Investigation Based on Network Exchange Theory
Hui Liu
Jun Liu
(Hui Liu. Associate Prof. Dept. of Soc. Harbin Engineering University）
(Jun Liu，P.D., Prof. Dept. of Soc. Harbin Engineering University；Email:[email protected]）
Abstract：College students are often used as subjects in experimental studies, especially in social
psychology, which brings about the external validity problems of research findings. Based on
three types of connections in Network Exchange Theory (NET), duplicate experiments between
college and non-college students are conducted. Research findings indicate that both college
students and non-college students get significantly more exchanged resources when located in
high-power positions across all three connection types, which justifies the basic propositions of
NET. But among those subjects located in high-power positions, college students show stronger
exclusive dispositions by getting significantly more resources than non-college students do, which
suggests the existence of subject effects in resource exchange experiments. College students have
different value orientations and life experiences from non-college adults, which could account for
the resource exchange results. Therefore, drawbacks exist when using college students as subjects
in some experiments. This paper hopes to provide some methodological references for
experimental designs.
Keywords：Network Exchange Theory；Subjects Effect；Experiment；Resource Exchange
Undergraduate Subject Effect and Investigation Strategy
In social psychology and behavioral sciences, undergraduates are often used as experimental
subjects. Back in 1946, statistician McNemar pointed out that the existing science of human
behavior was largely science of sophomores. Dipboye (1990) reported that, among 54
experimental studies published in Organizational Behavior, Human Decision Process and
Personnel Psychology, 81.5% used undergraduates as research subjects, only 7.4 % used
professionals, managers or technical staff. In social psychology, laboratory studies had become
over-dependent on college students (Sears 1986). Gordon, et al. (1986) reviewed that approximate
75 percent of published studies in social psychology involved college students. Using college
students has never decreased in current studies of social psychology (Henry, 2008). In psychology,
aside from age and gender, commonly, there is no demographic information about subjects in
published thesis. In top journals such as Nature and Science, experimenters frequently extend their
findings from undergraduates to the species, declare generality of their findings (Herich, Heine &
Norenzayan, 2010).
The problem of using college student subjects is, their many demographic variables (such as
educational level, age, etc.) and psychological characteristics are different from adult population.
1
Sears (1986) noted that, compared with non-college students (adults), college students were likely
to have less-crystallized attitudes, less-formulated senses of self, stronger cognitive skills, stronger
tendencies to comply with authority, and more unstable peer group relationships. College
sophomores are also not representative of the general adult population on all 5 factors of
personality, but the relationship between personality and political opinions are virtually identical
for college students and a comparison group of adults (Cooper et al., 2011). According to Carlson
(1971), college students have “unfinished” personalities in a relatively early adult life stage. As
such, they may systematically differ from nonstudents, especially those individuals who are older
and possess more life experiences on a number of specific psychological and behavioral
dimensions. Studies of behavioral economics indicate that ， college students could have
experimental effects because they are lack of decision-making experiences（Carpenter et al., 2008）;
Trust Game, Ultimatum Game and Public Goods Game show that undergraduates provide
significantly less offers than non-college students do, they ride the lower bound on prosociality
measures（Henrich et al., 2010；Bellemare et al., 2007）. In the experiments which subjects are
college students, some preferences are also not constant, in Ultimatum Game offers
undergraduates provide increase with their age and grade, freshmen provide lower offers than
seniors do. In Trust Game，the level of trust increases with the age until age 30（Sutter & Kocher,
2007. An overview by Herich, etc. (2010) showed, compared with diverse and sometimes
representative adult samples, undergraduate subjects consistently set the lower bound for
prosociality in experimental measures of trust, fairness, cooperation, differences between college
students and fully-fledgling adults are robust and significant. Sears（1986）concluded that if
college students were different from general adult population in many important aspects, making a
generation from such a narrow database of college students as research subjects might risk
systematic biases in its portrait of human nature. We could use undergraduate subjects in
experiments, but we must be cautious while drawing a conclusion from experiments. He
encouraged researchers to use a wide range of subjects. Many followers support him, for example,
Reynolds (2010) suggested that data produced from college student subjects need to be tested by
other subpopulations, unless researchers limited the results only to the phenomena of college
students.
In short, using undergraduate may bring about subject effects. So-called undergraduate
subject effect is defined as the external validity of experimental results while using undergraduates
as subjects in experimental research, namely, the extent to which experimental research results can
be generalized to other populations, variable conditions, other time and background , in other
words, it is the universality or applicability of research results and experimental effects (Wang,
2001). If a universal experimental result is expected, subjects need to have similar characteristics
with adult populations, such as key personality, intelligence, emotion, attitude, demographics and
other factors (Campbell & Stanley, 1963). Just from this perspective, Sears (1986) inferred that
psychological research might have produced systematic biases by relying heavily on the narrow
database of college student subjects.
However, the problem of undergraduate subjects is not an absolute one. Cooper et al (2011)
pointed out that all five factors of personalities of college students are significantly different from
general adult population, but the relationship between personality and political opinions is
identical for undergraduates and adults. In behavioral sciences, Dinah, etc. (2009) concluded that
in the bidding experiment, the values for a good that students are willing to pay are not
2
significantly different than non-college students are. In experiments of negotiation, although some
scholars noted that experimental results of college students are not representative of the
negotiating performance of trained professionals, but some studies shows that bargaining
performance of undergraduate subjects has no difference from professionals (e.g. Moore et al ,
1999), therefore, research findings using college student subject might have external validity in
certain conditions. In many cases, the so-called sophomore subject problem is not a real problem.
Kam etc al. (2007) pointed out that it was impossible to obtain a random sample from the
population, what we could do is to select a sample from a subpopulation, compared with a random
sample drawn from the population, any sample frame like this was not convincing. Generally,
college students were similar to the population in many ways, they were more so-called "regular
people ". Kam’s problem is that, if what college student subjects differ from the population is just
that variable that affects the experimental results, they are no more those "regular people".
The key to estimate whether undergraduate subject effects exist, is whether their "special"
psychological characteristics are confounding or moderator variables of experimental results. If
these special psychological characteristics involve dependent variables or independent variables of
the experiment, they may affect the external validity of the experiment, which will make
experimental results using undergraduate subjects not universal. Demographic variables may also
be confounding variables of experiments, researchers often assume some characteristics such as
age, education level will affect the result of experiments. In general, these variables have no
significant variability for college student sample, it is impossible for us to observe their
experimental effects by just using college students (Murray et al., 2013), then we are not sure of
the universality of the experiment. If education level and experience in a study are controlled
variables, the variations of these two variables cannot be ignored (Chan et al., 2008).
Hence，only when we disclose the meanings of psychological characteristics of subjects in an
experiment, can we estimate the mechanisms of differences in an experiment between
undergraduates and non-undergraduates (Gordon, Slade & Schmitt, 1986). Therefore,
undergraduate subject effects need to be evaluated. In general, there are two approaches.
An approach is to consider known differences between college students and the general adult
population , we can estimate the generality of the experimental results by such an approach
（Henry，2008）. A meta-analysis study conducted by Peterson (2001) showed that differences
between college and non-college students did affect some relationships of important findings both
directionally and in magnitude. Undergraduate subjects have their special characteristics, as for
whether these characteristics are confounding variables or moderator variables of the experimental
results, we can judge it from both theoretical and past empirical studies. Therefore, some criteria
should be proposed on the basis of theories, in order to predict which are universal and which are
local variables, so as to focus on how and why that variability produces (Barrett 2006). By
reviewing the literatures, Herich et al.（2010）discussed some possible criteria that might be
considered in the absence of comparative empirical research. First, perhaps there were some
domains in which researchers could expect phenomena to be more universal than they were in
other domains. Second, it might be reasonable to assume that some phenomena were more
fundamental to the extent that they were measured at a physiological or genetic level. Third, there
might be criteria by which one could confidently make generalizations from one well-studied
universal phenomenon to another similar phenomenon. Fourth, it would seem that demonstrating a
process or effect in other species would indicate human universality. Fifth, phenomena which
3
were evident among infants might be reasonably assumed to be more universal than phenomena
identified in older children or adults. Finally, perhaps particular brain regions were less responsive
to experience, such that if a given phenomenon was localized to those regions one could anticipate
more universality. Herich pointed out that all these criteria had the opposite cases, hence, any set
of criteria by which universality could be successfully predicted must be grounded in substantial
empirical data.
Another approach is replication research. While using undergraduate as experimental
subjects, at least we should contrast the experimental results with some other groups , assess the
external validity of experimental results using undergraduate subjects on the basis of the test
results of all subject types, this is the best way to investigate undergraduate subject effects（Herich
et al., 2010. Henry (2008) proposed that, a comparative analysis between college students versus
non-college student sample within the same study were needed, which put all respondents within
the same time frame and were asked the same questions following a similar methodology, so
differences between these two samples could be attributed to subject effects, not other factors.
Another approach to test the undergraduate subject effects was to find whether systematical
replications of such studies occur. Any research results based on college students need to be
replicated with nonstudent subjects prior to the generation of universal principles (Peterson，
2001). Early in 1986, Gordon, etc. proposed such an idea. He argued that generalizability is
ultimately an empirical problem; the most persuasive evidence on the external validity of
undergraduate subjects should be found in empirical studies. Namely, undergraduates and
non-undergraduate data were collected under identical experimental conditions, and we could
compare the responses between two groups. Incorporating two subject types in a particular
investigation as a check on the reproducibility of research results would seem to be an appropriate
and productive methodological strategy (Peterson ， 2001). Replication experiment,
（e.g.,Willer&Szmatka,1993), meta-analysis or comparative analysis of existing empirical studies
(e.g., Peterson 2001; Gordon,et.al.,1986; Henry,2008)）can all be used for such an approach.
As for the purposes of replication experimental test, Willer and Szmatka (1993) in a
cross-cultural replication experimental study argued that what replication experiments test aimed
at was universality of theories, namely, to use test - retest reliability to discover whether a theory
could be applied independent of time and space. However, Henry (2008:50) proposed the other
side of replication research, after he reviewed studies on prejudice, he pointed out that, some
studies had shown college students sometimes replicated the findings of the adult sample,
However, often there could be important differences in a wide variety of prejudice or
prejudice-related domains as well, Problematic to this approach was that interpretations of what
counted as a replication versus an important difference was often something in the eye of the
beholder. He argued that conclusions drawn from student samples could mostly be applied to
general adult population, the problem was that at this point we could not be sure which parts, or
how much of it, could be applied to. These questions were empirical ones that we as a science
should not lose sight of.
If a replication experiment is intended to test the universality of experimental results, it follows
an inductive logic, namely, the objective of the experiment is to improve the confirmation level of
experimental results; if the research objective is to challenge a general conclusion, is intended to
discover variability in replication research, that is the logic of falsification. Popper (1992)
suggested that in experimental studies the power of falsification was greater than that of
4
confirmation. According to Popper, in order to falsify the proposition "all swans are white", we
just need to observe the fact that one swan is black. As long as the experimental result of a
subgroup is significantly different from those of undergraduate subjects, we can basically falsify
the universality of the experimental results based on undergraduate subjects, that is , testify the
existence of undergraduate subject effects. Of course, things are not so simple. The power of
replication studies are needed to further estimated. Sophisticated falsificationism believes that
(Lakatos 1978), any theories can neither be fully confirmed, nor completely empirically falsified
as Popper puts. What is falsified is a research programme that is constituted by a series of theories,
the research programme includes a hard core which is the most basic theory of the programme,
and some auxiliary hypotheses which constitute protective belt around the hard core. The hard
core cannot be falsified by evidences, otherwise the entire research programme will be falsified,
but the protective belt can be adjusted and modified so that the hard core can be protected from
falsification. Therefore, the falsification of replication research should include two levels. First,
experimental results go against the hard core of the theory. However，according to Lakatos
（1978），this result could bring about problems, but not a victory. Because research programmes
are characterized by their positive heuristic, which will set a limit to anomalies, construct a
protective belt that consists of auxiliary hypotheses, predict counter-evidence and turn them
successfully into evidence, therefore, any statements already accepted can be justified in a long
time, empirical falsification does not equal a real rejection. Second, experimental results only
falsify some parts of protective belt of the theory, what this falsification brings about is just an
adjustment of the protective belt, in order to protect the predictive power of the programme, it is
not a threat to the whole theory. Lakatos pointed out that we must turn counter-evidence into
auxiliary hypotheses, protective belt was the first to be adjusted anytime, even to be replaced , if
this adjustment made the programme progressive, it mean that this programme would be a
successful one. Any changes that abnormalities could bring about were just protective belt.
In short, we can infer from the theories whether the particularity of undergraduates can bring
about subject effects, but whether the effects exist or not eventually needs an empirical test , only
on the basis of empirical test ,can we clarify the theoretical meanings of an experiment. We will
take a social psychological theory, that is, network exchange theory as an example, exploring how
to use both theoretical and empirical approach to analyze and verify the existence of
undergraduate subject effects, how to understand and explain such subject effects. We start with
the theoretical approach, based on existing empirical studies, theories and criteria, whether
particularity of undergraduates may affect results of resource exchanges under a given condition
should be logically judged, on the basis of this theoretical approach, we propose hypotheses, and
conduct a replication study using both undergraduates and non-undergraduates under identical
conditions by experimental approach, by examining the similarities and differences of resource
exchanges between two types of subjects, we can give a final empirical explanation and discussion
about subject effects .
Undergraduate Subject Effect in Network Exchange Theory
Network exchange theory is a formal theory that can explain and predict power
distribution ,this distribution embeds in the social relationship structure (Willer，1981a). The
theory includes a modeling and drawing procedure, two principles and two laws. The key to the
5
theory is a modeling procedure, namely, constructing a model that includes both internal
characteristics of actors such as preference, belief, decision-making and external characteristics
of actors such as social relationships and structural conditions, the modeling procedure can
accurately show us a interpretative geometrics about abstract image of social world (Willer ＆
Anderson, 1981). The laws and principles of network exchange theory are based on the basic
assumptions that actors avoid disadvantages, its first law is, the value of the preference alteration
of actor A is, Pa=vx. where preference refers to an action sequence system that actors prefer, the
various sanction flows of each actor is denoted by x, the evaluation value of one single unit
sanction is denoted by v, sanctions here refer to social actions that are received by actor B as well
as alter the preference of actor B while transmitted by actor A.The first principle of network
exchange theory is，all social actors act to maximize their expected preference state alteration .
Each actor in the economic exchange has a "optimal" state of preference, i.e. the maximal
interest, denoted by Pamax; there is another state of preference, it is the interst that a compromise
is not reached or the interest while confrontation occurs, denoted by Pacon. Based on Pamax and
Pbmax , a negotiation set can be constructed between A and B. Actor A has two types of interests
in exchanges: one is the incentive interest Iba =Pamax－Pa, the other is the minimal interest Ica =
Pa－Pacon. For A, the smaller the incentive interest, the greater the minimal interest, then the
smaller the resistance to exchange, the more possible he is willing to exchange. On the basis of
these, we can construct a resistance index, which represents the extent to which an actor could
refuse to exchange, denoted by R. The second law of network exchange theory is, the extent to
which an actor A refuses to exchange, i.e. resistance index is
R a = I b a /I c a =
P a m a x -P a
P a -P a c o n
The second principle is derived from the second law ---- Compromise occurs at equal
resistance for two undifferentiated actors in a full information system (Liu, Willer & Emanuelson
2011).
R
a
=
Pa m a x -Pa
P m a x -Pb
 R
= b
Pa -Pa c o n
Pb -Pb c o n
b
The principles and laws of network exchange theory have been supported by experimental
studies more or less since the theory has been put forward (Willer 1999)，however, subjects in
these experimental studies are mainly college students in western cultural background. Based on
such subjects, the experimental results may have the following two problems on external validity.
One is the subject effect that cultural difference brings about, namely, whether an experimental
research using subjects in the western cultural context is universal across the cultures? the other
is the subject effect that is generated by the particularity of college students, that is, whether the
experimental results using college students can be applied to the general adult population? Just as
Herbst et al. (2011）pointed out，the external validity of experimental studies involved both the
settings of experiments and the characteristics of the experimental subjects. Willer and Szmatka
(1993) explored the problems of settings, they argued that differences in cultural contexts would
affect the generality of experimental results, this issue needed to be resolved by empirical research,
they conduct a replication experiment using Poland and American college student subjects, the
findings verify that experimental results using American college students as subjects are universal
6
across different cultures. However, whether undergraduate subject effects exist in the same
cultural background, Willer et al., have not discussed it yet. In the experiments of network
exchange theory, it is possible for undergraduate subject effects to occur, which has been
mentioned above.
First, college students have different psychological characteristics from the general population,
which may affect the experimental results of network exchange theory. In network exchange
theory, social actor is an important element. Social actors avoid disadvantages with a meaning
system that consists of three basic elements, which are preference, belief and decision-making.
Under the condition of limited exchange resources, actors will consider the preferences and belief
system of other actors in terms of their own preference and belief system, they evaluate actions
other actors might take, negotiate with other actors to allocate resources, all of these are the initial
conditions of network exchanges (Willer 1981b). Special psychological characteristics of college
students will directly affect their meaning systems, further, affect the final results of negotiations
and resource exchanges. For example, for college students, the level of their trust is low, which
will affect actors to join in or leave the existing network ,to build up a new relationship or not,
trust level and trust structure will affect the shift of the network (Liu, Willer & Emanuelson,
2011); prosociality of college students is also low, how social value orientations moderate the
resource exchanges in a network has been focused on by some scholars , who are trying to conduct
experiments to test it ; lack of decision-making experience and life experience could also make
strategy rationality and resource exchange results of college students different from those of adults;
especially, freshmen and sophomores mostly are between ages of 18-20, their socialization has not
been completed, their psychological developments are unstable (Erikson,2000), if using them as
subjects, their immature mental states may also make it impossible for their experimental results
to be extended to the older adult population. Second, based on criteria Herich et. al. (2010) have discussed, undergraduate subject effects
in experimental studies of network exchange theory are possible. Actions of game players follow
strategic rationality in game theory, while the decisions of actors in network exchange theory
follow the same strategic rationality, network exchanges are also games, according to the criterion
three discussed by Herich et.al., we can make an analogy between game theory and network
exchange theory as for the subject effects. Because the resources undergraduates offer in
experiments of Ultimatum Game, Dictator Game and Trust Game are less than those
non-undergraduates offer（e.g.,Henrich et al., 2010；Bellemare et al., 2007），we can infer that the
same subject effects should exist in experiments of network exchange theory. Again, in network
exchange theory, its rational assumption of avoiding disadvantages, its two principles and two
laws derived from this assumption, all involve the biological nature of human (Liu, Willer &
Emanuelson, 2011), according to the criterion one, experimental results related to these should be
universal. However, strategy rationality in resource exchanges involves social nature and
socializations, which are difficult to be reduced to the physiological or genetic level, according to
the criteria two and four, college students may exhibit different characteristics from the general
population in their performances of exchange strategies, which will significantly affect the results
of resource exchanges.
Investigations of Undergraduate Subject Effect in Network
7
Exchange Theory
In network exchange theory, the structural condition is referred to a nature in connected
structures , which will facilitate some positions but go against other positions , it is usually set as
an initially experimental setting, the power strength of the actors just depend on these structural
conditions and the positions in the structure (Willer 1999). Considering these, we conduct a
replication experiment on the basis of three basic connected structures in network exchanges,
assigning non-college and college student subjects under the identical conditions of these three
structures, by comparing the differences and similarities between two types of subjects, we will
estimate whether undergraduate subject effects exist , and if exist, why.
Experiment Condition and Experimental Task
The experiment uses the following three basic connected structures as conditions and tasks
(Liu, Willer & Emanuelson 2011).
1. Inclusively connected structure. In B-A-C network，if A wants to gain, he must exchange
resources with both B and C，we call this network inclusively Connected structure at A. In this
connected structure, the person located in the central position has the lower power than the person
located in a marginal position, the last person who exchange with A is most powerful. The
structure can be seen in figure 1(1).
2. Exclusively connected structure. In B-A-C network，if A can only exchange resources with
any one of B or C，not both of them, namely, one of B or C must be excluded from the exchanges,
we call this network exclusively connected structure at A. This connected structure is a
high-power structure, A has the highest power. The structure can be seen in figure 1(2).
3. Null connected structure. In B-A-C network，if A can exchange resources with any one of
B or C, or with both of them, we call this network null connected structure. In this connected
structure, all exchanges are equal, namely, whether the person located in central position or not, he
always has the equal power. The structure can be seen in figure 1(3).
A
B
A
C
B
A
C
B
C
（1）Inclusively connected structure（2）Exclusively connected structure （3）null connected structure
Figure1 Three Basic Exchange Structures
Resources allocated are set under three network structures respectively. In each period of the
experiment, subjects located in position A, B or C all need to gain their resources through
negotiations in the requirement of given resources and exchange principles. The resources given
and exchange principles required are designed as follows.
1. In inclusively connected structure we assume that twenty-four resources are assigned
between A and C ,and the same number of resources are for A and B. A should exchange
twenty-four resources with anyone of B or C through negotiation, if reaching a compromise, A has
8
an opportunity to exchange another twenty-four resources with another of B or C through
negotiation. A can gain resources only when his negotiation with B and C are all successful, but B
and C can gain their own resources as long as they compromise with A .
2. In exclusively connected structure, we assume twenty-four resources among A,B and C. A
can share these twenty-four resources with B or C through negotiation, but he cannot share
resources with both B and C. B or C can gain resources only when he tries a successful
negotiation with A, if B or C cannot compromise with A, he will gain nothing.
3. In null connected structure, we assure that twenty-four resources are assigned between A and
C, and the same number of resources are for A and B. A should exchange twenty-four resources
with any or both of B and C through negotiation, the person who exchanges successful will gain
their own resources, the loser in the negotiation win nothing.
The experiment requires that A, B or C are strangers under the condition of perfect information ,
private communication is forbidden.
Research Hypothesis
According to Herich (2010), propositions and principles involving biological statements are
likely to be universal across different subpopulations, and vice versa. Avoiding disadvantages is
the most basic assumption in network exchange theory (Willer, 1999), it is a biological human
nature, such a nature in network exchange theory can be operationalized as pursuit of maximum
gain (the first principle and law) and the consensus that are reached at maximum gain point (the
second principle and law) (Willer, 1999). Therefore, if subjects are rational people that are of
avoiding disadvantages, they will have the following common behaviors in resource exchanges,
namely, no matter what types of subjects , and under what kinds of network structures, when
people in high-power positions exchange with people in other positions, they will gain more
resources, and vice versa. Considering all these theoretical propositions, we suggest the following
hypotheses.
H1：Whether subjects are undergraduates or not, provided located in high-power positions,
subjects will gain more resources than those who are in the weak-power positions. This hypothesis
can be operationalized into following three sub-hypotheses.
H1-1：In inclusionary structure，B or C（high-power position）gain more resources than A
（weak-power position）.
H1-2: In Exclusionary structure， A（high-power position）gains more resources than B or C
（weak-power position）.
H1-3: In null connected structure，exchanged resources between A and B（equal-power
position）, A and C（equal-power position）are equal.
Under the condition of identical structure, when subjects adhere to the principle of rational man,
they are affected by the meaning system that consists of belief, preference and decision-making
due to their social nature. Because undergraduates have a lower average level of trust, prosociality,
altruistic value orientation than non-undergraduates do （see Sears，1986；Bellemare et al., 2007），
all these factors can increase the possibilities of selfish and exclusive thinking and actions, hence,
undergraduates can take advantage of their position power to gain more resources while they are
located in any of powerful positions under three structural conditions. Considering all these
theoretical propositions, we propose the following hypotheses.
H2：Compared with non-undergraduates, undergraduates in high-power positions will gain more
9
resources; this hypothesis can be operationalized into following five hypotheses.
H2-1: In inclusionary structure，compared with undergraduates, non-undergraduates located in
position A（weak-power position）exchange more resources with B and C.
H2-2: In inclusionary structure，compared with undergraduates, non-undergraduates located in
position A（weak-power position）gain more resources in their first exchange with B or C.
H2-3：In inclusionary structure，compared with undergraduates, non-undergraduates located in
position A（weak-power position）gain more resources in their second exchange with B or C.
H2-4: In Exclusionary structure，compared with undergraduates, non-undergraduates located in
position A（high- power position）exchanges less resources with B or C（weak-power position）.
H2-5：In null connected structure，exchanged resources in position A（equal-power position）
are not significantly different between two subject types, the same with position B or C
（equal-power position）.
If all sub-hypotheses of H1 are confirmed, the experiment will support the hard core of the network
exchange theory. If hypothesis 2 is verified, the universality of the undergraduate experimental results
will be falsified, which indicates existing undergraduate subject effects. The effects do not falsify basic
assumptions, principles and laws of the theory, because they all indicate that high-power positions
exercise the power over the weak-power position, the only significantly difference between two
subject types is how power is exercised, hence,this falsification just needs to adjust the auxiliary
hypotheses, not hard core.
Undergraduate subject effects threaten the second principle and law in a sense, namely, no matter
what kind of subjects, under the condition of perfect information, the compromised exchange
resources between two actors should be close to the predicted resources in the resistance model.
For example, this model predicts that the second exchange resources in inclusionary structure are,
PA = 9. 72，PC = 13. 97; the final exchange resources in exclusive structure are PA = 23，PB = 1.
However, Willer (1981b :124-125) also argued that there was some theoretical and empirical
limitations while explaining the second principle of network exchange theory, because the exchanges
involved belief systems of actors, negotiating costs, etc., the prediction values of power would change
with different conditions, based on the second principle. Obviously, Willer has justified the hardcore of
network exchange theory, the only problem of his justification is that in this protective belt of network
exchange theory, Willer doesn’t declare that using college students as subjects to conduct experiments
is limited.
Subject Recruition
The experiment has two parts. The first part is an experiment using undergraduates subject, the
second part is a replication study based on the first part, using non-undergraduates (adult) subjects.
The experimental subjects of the first part are from three schools in a university in Harbin, mainly
through recommendation by teachers and voluntary participation by students. 141 college students are
recruited, of which 45 people are for inclusionary structure, 48 people for the other two structures. All
subjects are freshmen and sophomores; 53% and 47% for art (including social science) and nature
science (including engineering) respectively.
The experimental subjects of the second part are from four work units in Harbin, mainly through the
recommendation by the leaders or the contact people, combined with voluntary participation by staff.
30 people are recruited for each structure, a total of 90 people participated in all experiments, whose
age is within 23-40 with a mode of 26 and 27, work year is from 1 to 16 with a mode of 2 and 5, 54%
10
and 46% of them are of art (including social science) and nature science (including engineering) major
respectively with a degree of bachelor or above.
Experimental subjects are paid by their performances in the experiment, the more resources they
gain, the higher the reward they are paid. average rewards are ￥30 and ￥40 for undergraduate and
non-undergraduate subjects respectively.
Experimental Process
1.Experimental environment. The experiment for undergraduate subjects are conducted in a lab
environment through QQ group online, the replication experiment part for non- undergraduates is
conducted under the remote condition through QQ group online.
2. Assignment and rotation of subjects. In the experiment, every group has three subjects, there
are 47 groups for undergraduates and 30 groups for non-undergraduates altogether, subjects in
each group are randomly assigned. Each group is required to participate in three periods of the
experiment, and each subject is required to move to a new position (A,B or C) at the end of each
period under a structural condition. By calculating the mean resources of each position after three
periods, a 10-case data is generated under the condition of each structure for non-undergraduates,
as for college students, a 15-case data is generated under the condition of inclusionary structure,
and a 16-case data is generated under the other two conditions. The purpose of the rotation is
systematic control for the subject differences (Willer et al., 1993), by calculating the mean
resources of three periods of exchanges, the average resources in any position calculated will not
be affected by the characteristics of subjects（Willer,1999）.
3. Experimental process. （ 1 ） Pre-experiment. Two groups of subjects are selected for
non-college students and college students each, some adjustments in experimental procedures are
made in terms of the results of preliminary experiments ; (2) Formal experiment. Experiments are
conducted by trained staff. In each period, we start with a paper instruction or an instruction over
QQ group online after assigning each subject to a specific position, only when we are sure that
every subject has understood the requirements of the experiment as well as his position, does the
experiment begin. Each preiod of the experiment lasts no more than 10 minutes, experimenters
monitor the whole process of the online experiments.
4. Research Findings
Experimental Results of Undergraduate Subjects
1. Experimental results of the inclusionary structure---the higher the power, the more the gain
Independent sample t-test indicates that, the average resources B or C（high-power position）
gains are significantly more than those A (weak-power position) gains when A exchanges with
anyone of B or C in the first (t = 5.739, p <0.01) or second exchange（t=9.937，p<0.01）, the mean
difference of the gained resources between two sides is 3.96 and 7.68 respectively( see table 1).
Table 1
Exchange results for inclusionary structure using undergraduates as subjects（N=15）
The exchanges between
Mean
A and one of B or C
gained by A
The first exchange
11
10.02
resources Mean resources gained
by B or C
Mean difference
13.98
-3.96 ***
The exchanges between
Mean
A and one of B or C
gained by A
resources Mean resources gained
by B or C
Mean difference
The first exchange
10.02
13.98
-3.96 ***
The second exchange
8.16
15.84
-7.68 ***
***
significance at 0.01 level
2. Experimental results of the exclusionary structures--- the higher the power, the more the gain
Independent samples t test indicates that，the average resources A（high-power position）gains
are significantly more than those B or C (weak-power position) gains（t=11.979,p<0.01）, the mean
difference of the gained resources between two sides is 10.54 ( see table 2).
Table2 Exchange results for exclusionary structure using undergraduates as subjects（N=16）
Mean
resources Mean resources gained
gained by A
A exchanges with one of B or C
***
17.27
by B or C
Mean difference
6.73
10.54 ***
significance at 0.01 level
3. Experimental results of the null connected structures--- equal gain with equal-power
Independent samples t test indicates that mean difference of gained resources between A and B
as well as between A and C is 1.04（t=0.807,p=0.426）and 1.38(t=0.997，p=0.327) respectively ,
both of them are not significant( see table 3).
Table3
Exchange results for null connected structure using undergraduates as subjects（N=16）
Mean resources gained by A
Mean resources gained by B or C
Mean difference
A exchanges with B
11.48
12.52
-1.04
A exchanges with C
11.32
12.68
-1.38
The experimental results above confirm the conjectures of all sub-hypotheses in hypothesis 1,
these conclusions drawn from the experiment using college student subjects justify the basic
assumptions and principles of network exchange theory.
Experimental Results of Non-undergraduates
1．Experimental results of the inclusionary structure—the higher the power, basically the more
the gain
Independent samples t test indicates that , when A exchanges with anyone of B or C, the mean
difference of the gained resources between two sides in the first exchange is not
significant(t=0.159，p=0.875); in the second exchange , the average resources that B or C
(high-power position) gains is significantly more than those A (weak-power position) gains
（t=4.013，p<0.01）,the mean difference is 3.06. Combined with the experimental results of
undergraduates under the inclusionary structure, the experimental results basically confirm
hypothesis 1-1 (see table 4).
12
Table 4
Exchange results for inclusionary structure using non-undergraduates as subjects（N=10）
The exchanges between
A and one of B or C
Mean resources gained by A
Mean resources gained by B or C
Mean difference
0.06
The first exchange
12.03
11.97
The second exchange
10.47
13.53
***
-3.06 ***
significance at 0.01 level
2. Experimental results of the exclusionary structures--- the higher the power, the more the gain
Independent samples t test indicates that the average resources A（high-power position）gains
are significantly more than those one of B or C (weak-power position) gains（t=4.613 ,p<0.01）,
the mean difference of gained resources between two sides is 6.34. Combined with the results of
undergraduates under the exclusionary structure, the experimental results confirm hypothesis 1-2
( see table 5).
Table 5
Exchange results for exclusionary structure using non-undergraduates as subjects（N=10）
Mean resources gained
by A
A exchanges with one of B or C
***
Mean resources gained
by B or C
15.17
8.83
Mean difference
6.34 ***
significance at 0.01 level
3. Experimental results of the null connected structures--- equal gains with equal-power
Independent samples t test indicates that mean difference of gained resources between A and B
as well as between A and C is 0.66（t=0.673,p=0,510）and 1.07 (t=1.439，p=0.167) respectively ,
both of them are not significant. Combined with the results of undergraduates under the null
connected structure, the experimental results confirm hypothesis 1-3 ( see table 6).
Table6 Exchange results for null connected structure using non-undergraduates as subjects（N=10）
Mean resources gained by A
Mean resources gained by B or C
Mean difference
A exchanges with B
11.67
12.33
-0.66
A exchanges with C
11.46
12.53
-1.07
In general, the replication experiments above basically confirmed all sub-hypotheses in
hypothesis1，testify the universality of basic principles and assumptions of network exchange
theory.
Test for Undergraduate Subject Effects
1. Experimental results of the inclusionary structure—non-undergraduate subjects in the
high-power positions relatively not gain more
Descriptive statistical analysis and Independent samples t test indicate that，for A(weak-power
position)，the average resources non-undergraduates gain are significantly more than those
undergraduates gain（t=3.837,p<0.01）, the mean difference is 5.61；the average resources
non-undergraduates gain are significantly more than those undergraduates gain in the first and
13
second exchange, the mean difference is 2.01（t=3.527,p<0.01）and 2.31（t=3.837,p<0.01）
respectively. The experimental results confirms hypothesis2-1, hypothesis2-2 and hypothesis2-3
（see table 7）.
Table7 Comparison of resources gained by A between two subject types in the inclusionary structure
Mean
Resources Mean Resources gained by Mean Resources gained by
gained by A
A for the first exchange
A for the second exchange
College students
10
22.50
12.03
10.47
Non-college students
15
16.89
10.02
8.16
Difference
—
5.61***
***
2.01 ***
2.31***
significance at 0.01 level
2. Experimental results of the exclusionary structure — non-undergraduate subjects in
high-power positions relatively not gain more
Independent samples t test indicates that，For A（high-power position），the average resources
non-undergraduates gain are significantly less than those undergraduates gain （ t=1.917,
p=0.067<0.1），mean difference is 2.10. The experimental results confirm hypothesis2-4（see table
8）.
Table8 Comparison of resources gained by A between two subject types in the exclusionary structure
N
Mean resources gained by A
College students
16
17.27
Non-college students
10
15.17
Difference between two subject types
—
2.10*
*
significance at 0.1
level
3. Experimental results of the null connected structure—equal gains with equal-power
Independent samples t test indicates that，though the average resources non-undergraduates gain
are not different from those undergraduates gain at position A、B and C. The experimental results
confirm hypothesis 2-5 (see table 9).
Table9 Comparison of mean resources between two subject types in the null connected structure
N
Mean
Resources Mean Resources gained Mean Resources gained
gained by A
by B
by C
College students
16
22.80
12.52
12.68
Non-college students
10
23.13
12.33
12.53
Difference
—
-0.33
0.19
0.15
t（sig.）
0.145（0.886)
0.146(0.885)
0.111(0.912)
The experimental results above shows that under the condition of three networks, compared
with the undergraduate subjects ， non-undergraduate subjects in high-power positions gain
14
significantly more resources than other positions, confirms the exchanged resources that network
exchange theory declares directionally , testifies the basic assumptions and conclusions of network
exchange theory . However, the resources gained by two types of subjects in high-power positions
are significantly different, which does show a systematical pattern, this indicates an existence of
undergraduate subject effect, but this effect is about the quantities of the exchanged resources, not
the direction of them.
Conclusion and Discussion
Network exchange theory suggests，exclusions that structure brings about can generate power
(Willer，1999), namely, in a connected structure, when some positions have more exclusive
powers than other positions, people who are located in such positions will gain more resources.
The results of this replication experiment indicate that subjects located in high-power positions all
gain significantly more resources ,confirm the conclusion that exclusive powers originate from the
structure bring about powers and comparative advantages, and verify the core propositions of
network exchange theory .
However, replication studies also indicate that non-undergraduate subjects in high-power
positions do not get as many resources as undergraduates, this significant difference under
different structural conditions is robust. This suggests that undergraduate subject effect does exist,
although it is not against the core theories of network exchange theory, it reveals a more complex
relationship between the variables in network exchanges. The reasons for it may be the
distinctions between undergraduates and non- undergraduates in their degree of maturity and their
value orientations, which may affect their meaning systems of resource exchanges. Compared
with undergraduate subjects, non-undergraduate subjects are more likely to be altruistic
individualists and realistic rational men. They will consider the interests of all parties and the
realistic possibilities of the interests they can get, they are more mature than undergraduates for
their more reasonable meaning systems and their more experienced negotiation. From the records
of negotiation in this experiment, non-undergraduate subjects in high-power positions use some
words as follows :“It is not necessary to press the price too hard”（non-undergraduate subject C
in group two under the inclusionary structure）, “I fear that if I want too much that ,I will lose the
opportunity to reach a compromise with A.”（ non-undergraduate subject C in group eight under
the inclusionary structure）, “Only you earn, can I earn”, （non-undergraduate subject A in group
nine under the exclusionary structure ） , “ it is not right to let others earn nothing ！ ”
（non-undergraduate subject B in group one under the inclusionary structures）、
“so ,it is good for
all of us.”（ non-undergraduate subject A in group2 under the null connected structure）.All these
indicate that adult subjects have more mature meaning systems, more realistic ways of thinking
with a Chinese philosophy of leaving survival opportunities for others. However, ways of
rational thinking undergraduate subjects use are more self-centered and away from realities, which
seems due to their lack of experience and their uncompleted socialization. The following script is
excerpted from the negotiation record of the second exchange in the third period in group 4 under
an inclusionary structure, where we can see all the discussions above.
C：“don’t you want to gain more？”
A：“I don’t care.”
15
A：“I don’t expect to get more, but at least I should get what I want. I want 14.
……
C：“you get 10, I get 14.” “Is it right？”
A：“No way.”
C ：“What about 12-12 ?”
A ：“No way.”
C ：“What should we do now? time is up.”
This negotiation ends with a failure, both sides gain nothing. The reason for the failure is not
that subjects are not clear what conditions of the experiments they are, for in every period of
experiment, we start the experiments just after subjects have understood them. From this script,
we can see that ,under the condition of inclusive structure, the person located in the weak-power
position A certainly is a rational person, because he wants to gain more resources, but this rational
person has no negotiating skills , is too rational and self-centered to recognize the reasonable
maximal resources that his position can gain (Based on the theory, the maximal resources position
A in inclusionary structure can gain is 9.72, see Liu, Willer & Emanuelson 2011), undergraduate A
cannot reasonably estimate the preferences and beliefs of the other sides , whose meaning system
lacks of an ability to make a realistic decision. In contrast, in all periods of experiments, under the
condition of inclusionary structure, non-undergraduate subject A all gain more or less resources,
while 11.1 percent of undergraduates gain nothing because of their failures in the second
exchange ,this is due to an unrealistic effort undergraduates have made and a tough negotiation
strategy they use, which indicates an exchange characteristics of ideal egoism, and a strong
exclusivity that brings about a significant difference from relatively mature adult subjects .
This replication experiment basically confirms all assumptions derived from the theory, testify
two approaches to identify undergraduate subject effects. First, using the logic of theories and
existing empirical studies to make inferences, theories cannot eliminate subject effects, but can
help us understand and analyze subject effects. Theories can predict what processes can be robust
across different contexts, what processes may be particular, and why these universalities or
differences occur. Second, conducting empirical investigations, the main problem of this approach
is to test which parts are replicated, which parts are of important differences (Henry, 2008). This
approach can also help us understand in what meaning and which level we use undergraduate
subjects (Peterson, 2001). While interpreting the findings of undergraduate subjects, researchers
always need to consider at three levels. First, whether the experimental results of college students
can be extended to “human”; Second, whether the experimental results of college students can be
generalized to other sub-groups such as professional managers; Third, whether the experimental
results of college students can be applied to undergraduate population in general sense. Answers to
the first two levels involve the problem of external validity; answers to the last level involve the
problem of internal validity. Replication research can answer the problems of the first level. In this
case, there are two objectives. The first is whether the tested theory is universal, for which the
replication experiments aim at confirmation; The second is whether the variability of the
experimental results exists, for which the replication experiment is intended to do is falsification,
and answers what falsifies and the nature of this falsification.
Replication Research can also solve the problems of the second level in some degree, namely,
how undergraduates can be used in a study? Which sub-groups the findings of undergraduates
16
can be generalized to? Chan et. al. (2008) argued that, even we were uncertain whether
undergraduate subject effects existed, we could still use them to do the research, which was
inevitable, but what importance was, the problem of validity. The best way to deal with it was, to
choose a wide variety of possible types of alternative subjects such as undergraduates or graduates
with the subjects in the "real world", and do an exploratory research before a formal experiment.
For example, if managers are the group we want to study, which are obviously very difficult to
recruit to conduct a laboratory research, can we use undergraduate as surrogates for managers to
do a study? Chan et al (2008) reviewed all experimental studies published in some top
accounting journals in 1994-2007, confirmed this approach. Their review indicated that, in most
cases, the experimental results based on graduates of business schools were similar to those of
managers, if the experimental task was of low-contextual knowledge, the experimental results of
business students could be extended to managers. College students with trainings and experiences
in negotiation are also more suitable for a study on negotiation behaviors of professionals (Herbst
& Schwarz, 2011).
In other cases, we may ignore undergraduate subject effects while conducting a research. First,
it involves hypothesis test. Hypothesis needs various types of experimental data to confirm it, in
order to improve its confirmation level, just as the test of hypothesis 1 in this experiment; it is the
logic of justification. Calder, et.al.(1981) suggested that if an experimental study was intended to
confirm some theory, researchers could sacrifice the representativeness of the sample in order to
reduce the possible threats to the internal validity of the test. In a research aiming at justification,
it is legal to use undergraduate subjects (Gordon et al., 1986). Second, it involves exploratory
study. The objective of exploratory research is to discover realities, instead of searching for
universalities. Using a sample by convenience can not only test hypotheses, but also direct basic
insight into what we are thinking about (Murray et al., 2013). While exploring new fields,
universality is not our interest; hence, a special subject group can be used for such research.
Studies aim at discovering social facts can use any subgroups as subjects, which is reasonable
(Herich et al., 2010). Related to this, it is also legal to use any special groups in a study that aims
at falsification, just as shown in this study. Third, it involves a comparative study concerning
contextual variables. Such comparison experiments need estimate the effects of contextual
variables while controlling other variables. Undergraduate subjects have many demographic and
psychological similarities across different contexts, a meta-analysis done by Peterson (2001)
shows that college students do have greater homogeneity than non-college students do, hence,
they are more suitable subjects in a comparative study. A cross-cultural replication study
conducted by Willer and Szmatka (1993) just uses both Polish and American college students as
subjects. Of course, if the results of an experiment are only applied to college students, what we
need to consider , is just their representativeness of undergraduate population, there is no problem
of external validity while using undergraduate student subjects.
In short, using undergraduate subjects has problems, but this does not mean that we cannot use
college student subjects in a research, what we should do is to understand that in what levels we
can use them? and how to explain the experimental findings using undergraduate subjects? If
undergraduate subject effects are found to exist from the perspective of theories and replication
studies, we can reinterpret the research findings and redefine theoretical boundaries. It is also
necessary for us to report as much information as possible about age, educational level, gender
and other personal characteristics of subjects after the experiment, which will help us to explain
17
the experimental results under the condition of we know little about the external validity of the
experiment (Schulz, 1999). In addition, to determine what kind of college students are more
externally valid for our studies or our objective groups is also worth exploring. As in this study,
undergraduate subjects are freshmen and sophomores, who are in lower grades, do research
findings of high-grade college students or graduates are of higher external validity than
lower-grade undergraduate subjects? College students are convenient subjects who are easily to
obtain and whose costs are low, they are always available experimental resources. Hence, it is a
meaningful work for network exchange theory and other experimental research to conduct more
replication experiments between college and non-college students.
Reference
Barrett, H. C. (2006). Modularity and design reincarnation. in the innate mind: Culture and cognition (edited by P.
Carruthers, S. Laurence & S. Stich). 199–217. Oxford University Press.
Bellemare, C. & Kroger, S. (2007). On representative social capital. European Economic Review 51, 183–202.
Carlson, R. (1971). Where is the person in personality research? Psychological Bulletin 75(3), 203–219.
Calder, B.J., Phillips, L.W. (1981). Designing research for application. Journal of Consumer Research 8, 197-250.
Campbell, D.T. & Stanley, J.C. (1963). Experimental and quasi-experimental designs for research. Rand McNally.
Carpenter, J., Burks, S. & Verhoogen, E. (2005). Comparing students to workers: The effects of social framing on
behavior in distribution games. In Field experiments in economics. eds. J. Carpenter, G.W. Harrison & J.A.
List, 261–90. JAI Press/Elsevier.
Carter, J.R. & Irons, M.D. (1991). Are economists different, and if so, why? Journal of Economic Perspectives 5
(Spring), 171–77.
Chan, C., Landry, S.P. & Troy, C. (2008). Examining External Validity Criticisms in the Choice of Students as
Subjects in Accounting Experiment Studies. Accountng, Behavior and Organizations Research Conference.
53-78.
Cooper, C. A & McCord, D.M. & Socha,A. (2011). Evaluating the College Sophomore Problem The Case of
Personality and Politics. The Journal of Psychology 145(1), 23–37.
Dinah, P.T., Depositario, R.M., Nayga Jr., Ximing W., Tiffany P.L. (2009). Should students be used as subjects in
experimental auctions. Economics Letters 102, 122–124.
Dipboye, R. L. (1990). Laboratory vs. field research in industrial and organizational psychology. International
Review of Industrial and organization Psychology (edited by L.Cooper, and I.T. Robertson) 5, 1-34.
Chichester. N.Y.: John Wiley & Sons Ltd.
Erikson, E.H. (2000). Identity: Youth and Crisis. Nanjing: Zhejiang Education Publishing House.
Gordon, M.E., Slade, L. A., & Schmitt, N. (1986). The “science” of the sophomore revisited: from conjecture to
empiricism. Academy of Management Review 11(1), 191–207.
Herich, J. Heine, S.J. & Norenzayan, A. (2010). The weirdest people in the world？Behavioral and Brain Sciences
33, 61–135.
Henry，P.J. (2008). College Sophomores in the Laboratory Redux: Influences of a Narrow Data Base on Social
Psychology's View of the Nature of Prejudice. Psychological Inquiry 19, 49–71.
Herbst, U. & Schwarz, S. (2011). How Valid Is Negotiation Research Based on Student Sample Groups.
Negotiation Journal 4, 147-167.
Kam, C.D., Wilking, J. R., Zechmeister, E.J. (2007). Beyond the “narrow data base”: Another convenience sample
for experimental research. Political Behavior 29(4), 415–440.
18
Lakatos, I. (1978). The Methodology of Scientific Research Programmes. Cambridge: Cambridge University Press.
Liu.J., Willer.D. & Emanuelson, P.(2011). Network Structure and Power Distribution: Explanation from
elementary Theory. Sociological Studies 2，134-166.
McNemar, Q. (1946). Opinion attitude methodology. Psychological Bulletin 43(4), 289–374.
Moore, D.A., J.K. Murnighan. (1999). Alternative models of the future of negotiation research. Negotiation
Journal 15(4), 347–353.
Murray G.R., Cynthia R.R., Dona-Gene M., Jeffery J. M. (2013).Convenient yet not a convenience sample: Jury
pools as experimental subject pools. Social Science Research 42, 246–253.
Peterson, R.A. (2001). On the use of college students in social science research: Insights from a second-order
meta-analysis. Journal of Consumer Research 28(3), 450–461.
Popper, K. (1992). The Logic of Scientific Discovery. N.Y.: Routledge.
Reynolds, C.R. (2010). Measurement and assessment: an editorial view. Psychological Assessment 22(1), 1–4.
Sears, D. O. (1986). College sophomores in the laboratory: Influences of a narrow database on social psychology’s
view of human nature. Journal of Personality and Social Psychology 51(3), 515–530.
Schulz,A.K-D.（1999）Experimental research method in a management accounting context. Accounting and
Finance 39, 29-51.
Sutter, M. & Kocher, M. (2007). Age and the development of trust and reciprocity. Games and Economic Behavior
59, 364–82.
Wang, C. M. (2001). Psychological Research methods. Beijing: People’s Education Press.
Willer, D. (eds.) (1999). Network Exchange Theory. Westport, CT: Praeger.
Willer,D. & Szmatka, J. (1993). Cross-national experimental Investigations of Elementary Theory: Implications for
the Generality of The Theory and the Autonomy of Social Structure. Advances in Group Process 37-81.
Willer, D. (1981a). The Basic Concepts of the Elementary Theory. In Willer，David ＆ Bo An- derson (eds),
Networks, Exchange and Coercion. New York: Elsevier /Greenwood.
Willer, D. (1981b). Quantity and Network Structure. In Willer, Daivd & Bo Anderson (eds.), Networks, Exchange
and Coercion. New York: Elsevier /Greenwood.
19