INDIVIDUAL VALUE CHOICES:
HIERARCHICAL STRUCTURE
VERSUS AMBIVALENCE
AND INDIFFERENCE
William G. Jacoby
Michigan State University
David J. Ciuk
Franklin and Marshall College
January 2015
We would like to thank Cheryl Boudreau, Kim Hill, Jeffrey Koch, David Peterson,
Paul Quirk, George Rabinowitz, James Rogers, Saundra Schneider, Laura Stoker,
and the members of the Spring 2009 New York Area Political Psychology
Seminar for their comments on earlier versions of this paper.
ABSTRACT
Longstanding psychological theories have stressed that individual value preferences are very
stable and arranged into clear preference orders. But, some recent research suggests that
value choices may not be structured in a fully hierarchical manner for many people, due
to ambivalence or indifference. In order to address the important issues raised by these
potentially conflicting perspectives, this paper lays out an explicit model of individual value
choice. Some unique data obtained from an internet survey are used to test predictions
derived from this model. The empirical results show that most people do make consistent
pairwise choices among values. Furthermore, most individuals’ sets of pairwise choices are
fully transitive, meaning that they can be assembled into hierarchical structures. Those
inconsistencies and intransitivities which do occur in the data appear to be due more to
measurement error and indifference, rather than widespread ambivalence.
Political scientists have long believed that values comprise a fundamental element of
public opinion. Despite their acknowledged theoretical importance, however, there is little
scholarly consensus about the ways that ordinary people actually think about basic values.
On the one hand, longstanding psychological theories have stressed that individual value
preferences are very stable and highly structured. On the other hand, recent research suggests
a quite different interpretation: The potential for value ambivalence and indifference raises
questions about citizens’ abilities to differentiate among values in the first place. That,
in turn, would limit the degree to which people can organize their value choices into the
orderly arrangements that provide the basis for modern theories of human values. The
existing empirical evidence is indeterminate. Therefore, it is not clear which of these two
general perspectives provides the most accurate representation of core values.
This paper develops an explicit model of the processes through which individuals choose
between different values, and combine their choices into overall value structures. This model
is advantageous because it generates hypotheses that can be used to test for the existence of
ambivalence and indifference in people’s feelings about values. Building upon this theoretical
foundation, data for the empirical analysis are obtained from some unique items included on
an internet survey.
The results show that people do tend to make consistent pairwise choices among values,
suggesting that the the latter represent meaningful judgments about the relative importance
of different values. Furthermore, most individuals’ value choices are fully transitive, meaning that the pairwise choices can be assembled into rank-ordered structures. Finally, the
inconsistencies and intransitivities which do occur in the data appear to be due more to
measurement error and indifference, rather than to ambivalence.
BACKGROUND
For present purposes, “values” can be defined as an individual’s abstract, general conceptions about the desirable and undesirable end-states of human life (Rokeach 1973). As such,
values provide criteria for evaluating external stimuli and interacting with other elements of
the social environment. They effectively define what is “good” and “bad” in the world.
Human values have been the focus of an enormous amount of research across a variety of disciplines, including political science (Kinder 1983; Kuklinski 2001; Feldman 2003),
sociology (Hitlin and Piliavin 2004), economics (Katona 1975), marketing (Kamakura and
Mazzon 1991), and philosophy (Hansson 2001). However, the modern, empirical study of
values really began in psychology, particularly with the pioneering work of Milton Rokeach
(1973; 1979). Subsequent research has proceeded in a wide variety of directions (Seligman,
Olson, Zanna 1996). But, there is one general idea drawn from this research tradition that
deserves particular emphasis: People are rarely affected by single values, in isolation from
other values (Sniderman, Fletcher, Russell, Tetlock 1996; Davis and Silver 2004). Instead,
rank-ordered value structures are the key to understanding human behavior (e.g., Schwartz
and Bilsky 1987; Schwartz 1992; 1996; Verplanken and Holland 2002).
There is a clear consensus in the psychological literature regarding the hierarchically
structured nature of individuals’ value choices. However, recent research— particularly, in
political science— has taken very different, and potentially conflicting, positions with respect to the characteristics of human values. For example, theories of value ambivalence
(Feldman and Zaller 1992; Alvarez and Brehm 2003) hold that some individuals experience
conflicting feelings about personally salient, but substantively contradictory, core principles.
Alternatively, some people may experience value indifference, meaning that they either fail
to recognize, or simply do not care about, substantive contradictions between various fundamental ideas (Maio and Olson 1998; Bernard, Maio, Olson 2003; Goren 2006).
Although the underlying psychological processes are different, both ambivalence and indifference imply that those affected by these feelings would have trouble making or expressing
reliable and consistent choices between values in the first place. Ambivalent choosers would
experience difficulty selecting one value over another because they believe both values are
2
important. Indifferent choosers do not care about the two values under consideration so they
exercise the mental equivalent of “flipping a coin” to determine their selection.
I assume that an individual’s value structure is constructed by “aggregating” his or her
choices across the separate values. If ambivalence or indifference have a detrimental impact
on the constituent value choices, then they also necessarily compromise the validity of the
resultant value structures. In order to see the effects of ambivalence and indifference, it is
useful to lay out an explicit theory of value choices.
A Model of Value Choice
Let us begin with a set of p values under consideration, designated V al1 , V al2 , . . . V alp .
For any individual, each value has some level of subjective utility, or degree of personal
importance. For value i, this would be shown as U(V ali ).1 Note that this utility is assumed
to be a “global” parameter (for any given individual) which is not dependent upon the
particular context within which the value is invoked. This assumption follows from the
theoretical position that values are nonspecific beliefs that “. . . transcend specific situations
(Schwartz and Bilsky 1987, p. 551).” The “transsituational” nature of human values is a
central characteristic that differentiates this concept from other psychological orientations
and decision-making heuristics (Schwartz 1996; Jacoby 2002).
Feelings about personal value importance represent internal states that are not directly
observable. So, there will be a certain amount of measurement error associated with estimates
of each value’s degree of utility. This error is designated as ei for value i. It could represent
an individual’s lack of self-awareness or uncertainty about his or her own feelings. Or, it
may represent mistakes that arise in the process of recording empirically a person’s feelings
about a particular value. In either case, the error is conceptualized as a random disturbance,
with an expected value of zero.
Measurement error will be present in any manifestation of feelings about a given value.
For example, if a person is asked to state the importance of value i, then his or her response
3
will reflect both the utility of, and the error associated with, that value. This can be shown as
U(V ali ) + ei . Empirically, it is impossible to separate the expression into its two constituent
elements.
For a given individual, the choice between any two values (say, i and j) is determined by
comparing his or her feelings of importance about (or utility for) the two values. The actual
choice would be generated as follows:


 V ali
Choice(V ali , V alj ) =

 V al
j
: U(V ali ) + ei > U(V alj ) + ej
(1)
: U(V ali ) + ei < U(V alj ) + ej
From expression (1), it is easy to see how the error associated with the two values can lead
to an expressed choice that is inconsistent with a person’s true relative utilities for those
values. For example, if U(V ali ) > U(V alj ), but either ej is a sufficiently large positive value
or ej is a sufficiently small negative value, then the sum of the utilities and the errors would
lead to a choice that contradicts the comparison of the utilities, alone.
One strategy for dealing with the error is to have the individual repeat the choice several
(say, m) times, producing Choice(V ali , V alj )1 , Choice(V ali , V alj )2 , . . . , Choice(V ali , V alj )m .
We assume that the utility for each value is constant across the replications, but that the
errors are random draws from their respective distributions. The dominant pairwise choice
for that person would be obtained by summing across the m replications:
Choice(V ali , V alj )Dom


 V ali
=

 V al
j
: mU(V ali ) +
: mU(V ali ) +
m
P
m
P
ei > mU(V alj ) +
ei < mU(V alj ) +
m
P
m
P
ej
(2)
ej
Since the errors are random, the specific realizations of the error terms should tend to cancel
out across replications. Therefore, each summed error in expression (2) should tend to be
closer to the expected value of the error (that is, zero) than would any single error term.
With replicated pairwise choices, the outcome is dominated by the summed utilities rather
than the errors. Hence, an individual’s dominant empirical choice between values i and j
4
should tend to be consistent with his or her relative utilities across those values, even if some
of the specific replications produce choices that are inconsistent.
Now, if a person has crystallized feelings regarding the p values under consideration, then
two conditions should be met: First, the utility differences across separate values should be
sufficiently large to distinguish between them. In other words, that individual recognizes
which values are more important and which values are less important to him- or herself.
Second, the absolute values of the various error terms should be small, since the person is
quite sure about his or her feelings. And, the net result is that replicated choices for a pair
of values— Choice(V ali , V alj )1 ,Choice(V ali , V alj )2 , . . . , Choice(V ali , V alj )m — should be
consistent with each other, as well as with the underlying relative utilities. Stated differently,
an individual who is certain about his or her personal value orientations should tend to make
the same choice every time they are confronted with a specific pair of values.
The situation is very different when either ambivalence or indifference is present. Again,
ambivalence exists when a person cannot choose between two alternatives because he or
she considers both of the alternatives to be very important. Therefore, if an individual is
ambivalent about values i and j, then U(V ali ) = U(V alj ) and the level of utility is high
(i.e., the specific quantification of U(·) is large).
In contrast, indifference exists when a person cares so little about the alternatives that
he or she cannot make a clear choice between them. If a person is indifferent about values i
and j, then it is still the case that U(V ali ) = U(V alj ). Now, however, the level of utility is
low (i.e., the specific quantification of U(·) is small).
The impact of ambivalence or indifference on an empirical value choice can be seen very
easily if we rearrange expression (1) as follows:


 V ali
Choice(V ali , V alj ) =

 V al
j
: U(V ali ) − U(V alj ) > ej − ei
: U(V ali ) − U(V alj ) < ej − ei
5
(3)
Under ambivalence or indifference, the arithmetic differences between the utilities that
appear on the left-hand side of the inequalities in expression (3) are all equal to zero. Therefore, the empirical choice between values i and j is determined completely by the error or,
more specifically, the differences in the errors associated with each of the two values.
Replicating the pairwise choice does not change the situation, because the individual’s
utilities are constant. Therefore, the utility difference for the two values is zero on each
separate replication. But, the errors are random draws so their values will vary across the
replications. Sometimes, the specific manifestation of ei will be larger than the manifestation
of ej , so the person will choose value i over value j. Other times, the randomly-drawn ei
will be less than the corresponding ej , leading to a choice of value j over value i. The net
result is that both ambivalence and indifference will generate inconsistencies across separate
pairwise value choices. Thus, inconsistent value choices could be an empirical manifestation
of ambivalence or indifference with respect to the values under consideration.
A Model of Value Structure
Again, the traditional theoretical position maintains that individuals possess ordered
value structures, in the sense that each person can rank the separate values according to
their degree of subjective importance. With p values, each individual would array them as
follows:
V al1 > V al2 > . . . > V alp−1 > V alp
Where V al1 is the value that the person considers most important, V al2 is the second-most
important value, and so on, down to V alp which the individual considers to be the least
important value out of the set under consideration. Once again, it is useful to consider an
explicit theory about the process that generates this rank-ordering for an individual.
We will assume that a person’s value structure is constructed from his or her dominant
pairwise value choices. In order to produce a fully-ordered hierarchy, the pairwise choices
within every possible subset of three values must be transitive. The property of transitivity
6
means that there is a sort of cumulation across separate choices within a triad such that,
given any two choices, the third choice is known. For example, assume that value i is chosen
over value j, and that value j is chosen over value k. Transitivity occurs when i is also
chosen over k. In contrast, the choices would be intransitive if (given the first two choices)
k were chosen over i.
In order to see why transitivity is required for the ordering, let us assign a score to
each value, corresponding to the number of times that value is chosen over any other value.
Assume that a person’s choices among values i, j, and k are transitive, as laid out above. In
this case, value i receives a score of 2 because it is chosen over two other values (j and k),
value j receives a score of 1 because it is only chosen over one other value (that is, k), and
value k is scored zero because it is not chosen over any other values. Thus, the three values
can be fully ordered, or arranged hierarchically, according to their scores (Peffley, Knigge,
Hurwitz 2001).
Now, let us assume that the choices among values i, j, and k are intransitive, with i
chosen over j, j chosen over k, and k chosen over i). In this case, i receives a score of 1,
because it is chosen over j. Value j also receives a score of 1, because it is chosen over k.
And, value k receives a score of 1 as well, because it is chosen over i. Hence, all three values
receive the same score because each one is only chosen over one other value. As a result, the
values cannot be ordered according to their scores.
The idea of rank scores can be generalized across the full set of values. In other words,
each value can be assigned a score ranging from zero to p − 1, corresponding to the number
of times that value is chosen over any of the other values. If an individual’s choices are
transitive in all distinct triads of values drawn from a set of p values, then each of those p
values will be assigned a different rank score from the other p − 1 values. If any subsets of
three pairwise choices are intransitive, then the values within that triad will be assigned the
same rank score; of course, this means that it is impossible to construct a full preference
7
ordering across the entire set of p values. In this manner, transitivity across pairwise value
choices can be used as a strong test for the existence of hierarchical value structures
If the traditional theoretical perspective is, in fact, a valid representation of human value
choices, then the most likely sources of intransitivity lie in the errors that are inevitably
involved in the expression of those choices.2 The randomly-drawn error component on any
particular pairwise choice could contradict the choices made on two other value pairs, thereby
generating an intransitivity. So, for example, assume that an individual possesses an array
of ordered utilities across three values, such that:
U(V ali ) > U(V alj ) > U(V alk )
And, further assume that the person chooses V ali over V alj , and V alj over V alk . Finally,
assume that the errors associated with the choice between V ali and V alk are such that:
U(V ali ) − U(V alk ) < ek − ei
As shown back in equation (3), the preceding configuration of utilities and errors would cause
the individual to choose value k over value i, thereby generating an intransitivity within the
triad composed of values i, j, and k.
It is impossible to “remove” the error associated with any particular pairwise value choice.
However, the net impact of the error should be minimized with the dominant pairwise choices,
since the aggregated random “noise” should tend toward zero across the replications that
go into each dominant choice. Still, the pervasive existence of measurement error probably
means that this effect will never be eliminated entirely even if people possess starkly different
utilities for the respective values.
But, intransitivities should be particularly likely to occur in the presence of ambivalence
or indifference. As explained earlier, either of these two conditions implies that the utilities
across the affected values are equal, and the difference between them is zero. As a result,
the individual pairwise choices and the dominant choice across the two values will both be
8
determined entirely by the error. The latter is determined by random draws from the relevant probability distributions, and there is no reason to predict transitivity in the resultant
choices.
Thus, another effect of ambivalence and indifference is a greater likelihood of intransitivity
and the attendant inability to combine pairwise choices into fully-articulated, hierarchical,
value structures. This, in turn, poses serious challenges for traditional psychological theories
which hold that such value structures are a universal component of human behavior (Rokeach
1973; Schwarz and Bilsky 1987). Therefore, it is critically important to determine whether
the recent challenges based upon ambivalence and indifference have merit, or whether the
traditional theoretical understanding provides the more accurate depiction of individual value
orientations.
Previous Research
The existing empirical evidence is ambiguous and largely indeterminate. First, and perhaps most fundamentally, the very existence of orderly value structures has never been
rigorously tested (Hitlin and Piliavin 2004; Goren n.d.). Some studies simply focus on single
values, taken separately from other values (e.g., Feldman 1988). In other cases, researchers
ask subjects to rank-order a set of values (e.g., Rokeach 1973; Ball-Rokeach, Rokeach, Grube
1984). Both of these approaches bypass any direct assessment of consistency or transitivity
in value choices. The one line of work that does attempt to construct preference orders from
pairwise value choices (Jacoby 2002; 2006) is based upon a very limited dataset which precludes evaluation of the measurement error that undoubtedly exists in expressed statements
of choices between values (Miethe 1985; McCarty and Shrum 2000).
A second, and closely related, set of studies raises questions about the effectiveness of
asking people to make explicit choices between values (Alwin and Krosnick 1985; Miller,
Wynn, Ulrich, Marti 2001). This work points to both the practical difficulties involved
in having subjects rank-order sets of values and the validity of the resultant distinctions
between values (e.g., Schwartz 1994; Maio, Roese, Seligman, Katz 1996). Instead, these
9
analysts suggest that ratings of separate values provide superior measurement of individual
value orientations.
Third, most studies of ambivalence never directly test individual willingness or ability to
make choices between specific values. Instead, they examine indirect evidence based upon
the supposed consequences of value ambivalence. For example, Feldman and Zaller (1992)
infer that self-professed liberals are more ambivalent than conservatives through the terms
that they use to justify their own expressed opinions. Alvarez and Brehm (2002) argue
that value ambivalence should induce heteroscedasticity in models of attitude formation,
since ambivalent individuals should experience more difficulty in “translating” their value
orientation into specific opinions toward stimulus objects like political issues.
Finally, several lines of research have produced results which may call into question the
widespread existence of stable value choices. For example, Maio and Olson (1998) argue that
many people regard values as “truism,” or general beliefs that have very little in the way of
cognitive underpinnings (also see Bernard et al. 2003). Accordingly, it is fairly easy to get
people to change their expressed value preferences, thereby generating inconsistent choices
(also see Goren, Federico, Kittilson 2006; 2009).
In a similar vein, several issue framing studies indicate that varied presentations of a
political controversy can affect support for different values in ways that lead to opinion
change (e.g., Nelson et al. 1997; Grant and Rudolph 2003). More generally, priming specific
values may increase the degree to which individuals choose those values over others (Seligman
and Katz 1996). But, there is also opposing evidence: Nelson et al. (1997, p. 56) report that
preference orders across multiple values do not exhibit systematic variability across different
frames of an issue. And, Jacoby (2008) reports that priming specific values does not cause
people to choose those values more frequently over other values. In summary, the existing
evidence on ambivalence, indifference, and the existence of value hierarchies is inconclusive.
10
STUDY DESIGN AND DATA COLLECTION
In order to analyze consistency and transitivity in value preferences, it will be necessary to
observe replicated pairwise value choices. But, such data are not usually collected. Therefore,
this study relies upon a new set of items, administered as part of an internet survey that was
supported by the program for Time-Sharing Experiments in the Social Sciences.3 As we will
see, respondents made choices among subsets of values in a manner that enables empirical
tests of consistency and hierarchical value structuring. Such tests are impossible with more
traditional data collection strategies such as rating single values, rank-ordering a full set of
values, or making comparisons between all possible pairs from a value set.
Before proceeding, we need to consider exactly which values will comprise the pool from
which we will ask people to make choices. This is a serious problem, because the potential
range of values is very broad (e.g., Kuklinski 2001). For present purposes, we will focus on
a set of five values with immediate political relevance. Two values, liberty and equality, are
centerpieces of American political culture (e.g., McClosky and Zaller 1984; Feldman 1988). A
third value, economic security, is widely believed to be a precondition for meaningful participation in the modern “marketplace” of socioeconomic interactions (e.g., Hochschild 1995).
Social order is a fourth value which is drawn from the traditional conservative emphasis on
strict norms of human behavior and social interaction (e.g., Rossiter 1962). And, morality is
a value which drives much of the supposed “culture war” in modern American society (e.g.,
Hunter 1994; Layman and Carmines 1997). Virtually all contemporary domestic political
issues can be traced to the problematic achievement of these five values. So, it is definitely
important to consider how ordinary citizens feel about them, along with their willingness or
ability to make clear choices among the values.
Choices among the preceding five values were elicited from the internet survey respondents, using the method of triads (Coombs 1964; Weller and Romney 1988). Respondents
were first shown a screen that introduced, and provided a brief definition for, each of the
five values (Figure 1). After that, they were shown a series of ten screens. On each screen, a
11
distinct combination of three values (i.e., “a triad”) was listed. For each triad, respondents
were asked to indicate which one of the three values is most important, and which one of the
three is least important. Figure 2 shows an example of this screen. Given the nature of the
internet survey, respondents could make the selections very easily, by clicking radio buttons
with the computer mouse. The ten triads (i.e., all possible subsets of three values from the
five) also are shown in Figure 2. The order of the triads was varied randomly, as was the
order of the three values within each triad.
The individuals’ responses to each triad can be broken down into three pairwise choices.
For example, assume a triad containing values i, j, and k, with a respondent stating that i
is most important and k is least important. This, in turn, implies that i is more important
than j, j is more important than k, and i is more important than k. In the full set of ten
triads created from the five values, each pair of values appears in three different triads. For
example, “liberty” and “equality” appear in the first three triads listed in Figure 2, while
“liberty” and “morality” appear in the third, fifth, and six triads, and so on. Therefore, three
replications of each respondent’s choice between each value pair are recovered. This allows
an assessment of consistency in pairwise choices (i.e., how many times does the respondent
make the same choice for each pair?).
The value choices within any triad are transitive by construction. But, that is not problematic for present purposes, because we are interested in transitivity across the dominant
pairwise value choices, rather than across any of the single replications of each pairwise
choice. For each value pair, we estimate the dominant choice by taking the value that is
chosen over the other value in the pair on two or three of the replications. Since the resultant
dominant choice is constructed from pairwise choices made across different triads, it is not
constrained to be transitive with respect to any of the other dominant pairwise value choices.
Therefore, we can perform empirical tests of transitivity in the dominant value choices.
The method of triads provides a very efficient strategy for obtaining replicated pairwise
value choices. For one thing, it requires fewer survey items than presenting the respondents
12
with all possible pairs of values and asking them to choose the more important value in
each pair. With five values, there are ten distinct pairs. And, to produce the same level
of replication as the triads, each pair would have to be repeated three times, generating a
total of thirty survey items. Again, only ten triads are necessary, with two distinct responses
in each triad (i.e., the most- and least-important values). Furthermore, respondents could
easily remember their choices across replicated pairs of values. This is more difficult with
the triads, since there are different subsets of values in each triad and respondents cannot
look back at their choices on previous triads. For these reasons, the triads comprise a very
practical and efficient approach to eliciting value choices with properties that can be tested
empirically (Gulliksen and Tucker 1961).
EMPIRICAL RESULTS
Let us begin the empirical analysis with some descriptive information about the distribution of value choices in American public opinion. In the dataset, the replicated pairwise
choices are used to determine each person’s dominant choice for each value pair. We then
assign rank scores to each value, summarizing the respective values’ relative positions within
each person’s value structure. Recall that a value’s rank score represents the number of times
that value is the dominant choice over other values in that person’s full set of choices; so,
larger scores indicate more important values and vice versa. With the Knowledge Networks
data, there are five values, so the scores for each value can range from zero to four.
Variability in Value Importance
Table 1 shows the distribution of rank scores for each of the four values. The most
striking feature of the table is the wide variability among the preferences. At the individual
level, there does not seem to be general agreement about which values are most (or least)
important. Morality received the largest number of high ( “4”) scores but even so, only
about one-third of the respondents considered this value to be more important than any
13
other. And, very few people placed equality or social order very highly; less than eight
percent of the respondents had each of these values at the top of their rankings. At the
other extreme, social order received the largest number of zero scores. But, again, only
about 30% of the respondents ranked this value in last place. Overall, it is fair to say that
each of the four values shows up in every possible position within the value hierarchies of a
substantial number of people.
The dot plot shown in Figure 3 summarizes succinctly the aggregate distribution of
importance rankings for each value. Specifically, the horizontal position of the point plotted
within each row corresponds to the mean score for that value; the solid bar around each point
represents a 95% confidence interval. From the figure, it can be seen that morality, economic
security, and liberty are considered to be the most important values. Their mean rank scores
are 2.36, 2.32, and 2.29, respectively. However, as can be discerned from the overlapping
confidence intervals in Figure 3, the differences between these means are not statistically
significant. The remaining two values, equality and social order, have significantly lower
mean rank scores, at 1.58 and 1.45, respectively. Again, the difference between these two
means is not statistically significant.
At the aggregate level, Americans do not seem to differentiate fully among the different
values. Instead, public opinion sorts values into two distinct sets: One group that is relatively
more important, composed of morality, liberty, and economic security and a second group
that is usually viewed as less important, equality and social order.
Consistency in Pairwise Value Choices
Table 2 presents basic data on consistency in the pairwise value choices. Each row of
the table represents one of the ten distinct value pairs that can be derived from the set of
five values. Specific pairwise value choices are extracted from the responses to the triads, as
explained in the previous section. Thus, there are three choices available for each value pair.
Beginning at the left side of the table, the first four columns represent the number of
times the first-mentioned value in that row is chosen over the second-mentioned value, across
14
the three replications for that pair. The rightmost column within each row summarizes the
amount of consistency in choices involving that value pair; that is, the percentage of times
respondents chose the same value across all three replications of that particular value pair.
The entries in Table 2 show that the respondents are highly consistent in their value
choices. In every case, 70% or more make the same pairwise choice across all three replications. Stated somewhat differently, the percentage of respondents who exhibit some degree
of inconsistency ranges from a high of 29.58% (for choices between equality and morality)
or just under one-third of the sample, to a low of 19.57% (for choices between morality and
social order) or just under one-fifth of the sample
As a standard of comparison for the entries in Table 2, consider an admittedly unrealistic
situation in which everyone is indifferent between two values (due to ambivalence, failure to
recognize any potential conflict between the values, and so on). In that case, the probability
of choosing one value over the other on a single choice is 0.50. And, the probability of making
consistent choices across all three replications is only 0.25.4 In other words, if people did
not maintain real preferences between a given pair of values, then only one-fourth of the
sample should be exhibiting consistent choices across the three replications of each choice.
But, again, the empirical levels of consistency are much higher than this stringent standard.
This, in turn, suggests that most people really do have crystallized preferences across the
values examined in this study.
Are Individual Value Choices Transitive?
For each survey respondent, the full set of dominant pairwise choices can be examined for
transitivity. The use of dominant pairwise choices means that we can assess transitivity as a
separate phenomenon from consistency in the choices. In other words, a person’s dominant
choices could be transitive, even if there are some inconsistencies across the replications of
each choice. And conversely, a person could be perfectly consistent in his or her choices
across the replications but still show an intransitive set of choices involving that value pair.
15
Table 3 provides the basic data on transitivity among individual value choices. The first
row of the table shows that only 12.48% of the respondents exhibited any intransitivity
among all of their dominant pairwise choices. Stated differently, nearly nine-tenths of the
respondents maintain fully-ordered (i.e., completely transitive) value preferences. The remaining rows in the table give the proportion of intransitive choices on all subsets of three
values. Parenthetically, note that some people are intransitive on more than one triad; hence
the proportions intransitive on the particular triads sum to more than the 12.48% figure given
above.
The information in Table 3 shows a very obvious pattern: On any given triad, only a
tiny minority of the respondents give intransitive choices among the values. The number of
intransitivities never rises above about three percent on any value triad, and it is generally
smaller than that. Clearly, the vast majority of the respondents have no difficulty providing
a complete ordering of the five values. This, in turn, provides strong confirmation for the
widespread existence of fully-differentiated hierarchical value structures within the mass
public.
Testing Possible Sources of Inconsistency and Intransitivity
As we have just seen, a sizable majority of the survey respondents are consistent in
their choices within any pair of values. And, overwhelming majorities of respondents exhibit
transitive sets of pairwise value choices. Still, there remain nontrivial numbers of inconsistent
and intransitive choices within the data. And, it is important to determine why they exist.
Are the inconsistencies and intransitivities due to “simple” measurement error, ambivalence,
or indifference?
In order to answer the preceding question, we must determine where the inconsistent and
intransitive choices occur within individual value hierarchies and across survey respondents.
First, measurement error (as we are conceiving of it in the present context) is random, so
there should be no pattern to any inconsistency or intransitivity that it generates. Second, ambivalence involves conflict among strongly-held values. Therefore, it should lead to
16
inconsistent and intransitive choices among more important values and within people who
are knowledgeable enough to recognize the contradictions. According to Feldman and Zaller (1992), liberals should also experience more ambivalence than conservatives. Therefore,
nonuniform and non-ordered choices should be more prevalent among their ranks. Third,
indifference implies that the individual does not care very much about the values under consideration. So, problematic choices should occur among less important values, and within
people who are less likely to be sophisticated and engaged in the survey response context.
These three predictions about patterns of inconsistent or intransitive responses are tested
by fitting a logistic regression model to the TESS data. In fact, the model is fitted twice:
Once for inconsistent pairwise responses and once for intransitive subsets of three choices.5
The dependent variable in each case is the probability of an inconsistent choice or an intransitive set of three choices. So, the units of analysis for this regression model are composed of
specific pairs of values, and specific subsets of three value pairs. There are ten such choices,
and ten such subsets of three, for each survey respondent. Of course, any given respondent’s
choices will be nonindependent. Therefore, the analysis relies upon robust standard errors,
clustered by respondent, to take this into account.6
The independent variables are limited a bit by what is available in the TESS instrument.
But, there should be sufficient information to test the relevant hypotheses. The first independent variable is the mean rank of the two values involved in the pairwise choice, or the
three values in a given subset. The ambivalence and indifference hypotheses lead to differing
predictions about this variable’s impact on inconsistency and intransitivity. Ambivalence
should generate inconsistent and intransitive choices among more highly-ranked values. Indifference should produce these kinds of responses among values that are regarded as less
important.
The second independent variable is education, operationalized as two dummy variables
for respondents who didn’t complete high school, and for those with formal schooling beyond high school. Again, the two hypotheses lead to conflicting predictions about the signs
17
of the coefficients on these variables. Ambivalence should lead to inconsistency and intransitivity among more highly educated respondents, while indifference should concentrate such
problematic choices among those with lower levels of education.
The third variable in the equation is ideology, measured on a seven-point scale. The
variable is coded so that larger values indicate more liberal self-placements. Therefore, if
liberals truly are more ambivalent than conservatives (as suggested by Feldman and Zaller
1992), then the coefficient on this variable should be positive.
The fourth independent variable is race, represented by three dummy variables for respondents who identified themselves as African American, Hispanic, or members of another
nonwhite racial group. Here, too, varying predictions can be made. On the one hand, minority racial identification has been used as a social status indicator and it has also been
associated with lower levels of engagement in survey interviews; these could increase inconsistency and intransitivity in their responses. On the other hand, minority communities may
have distinct subcultures which may lead to more crystallized feelings about some values;
that could increase both consistency and transitivity.
A dummy variable for female respondents is included in order to pick up any potential
gender-based differences in choice behavior. Family income (in thousands of dollars) is
also included as a social status indicator; the general expectation is that higher income
should correspond to greater clarity in value choices although one could, perhaps, make an
ambivalence-based argument to the contrary.
Finally, dummy variables are included for four of the values (social order is omitted).
These variables are scored one if that value is included in the pair or subset of three, and
zero otherwise. The reasoning for including these variables is that Americans “across the
board” may simply find it more difficult to deal with some values rather than others.
Table 4 presents the maximum likelihood estimates, odds ratios, and observed probability
values for the model predicting inconsistency in pairwise value choices.7 First, note that the
model does not fit the data all that well. The Wald Chi-square statistic is significant, but
18
the pseudo R2 is tiny, at 0.019. Thus, the likelihood for the fitted model is not that much
higher than that for the null model, in which all coefficients except the intercept are equal
to zero. This kind of result is most compatible with the measurement error interpretation,
which predicts that inconsistencies should be random and unpredictable. The relatively
small values of the fit statistics suggest that is precisely the case, here.
Turning to the independent variables, notice first that the coefficient for the mean value
rank is statistically different from zero in the negative direction. This shows that people are
less likely to express inconsistent choices across the replications on pairs that include values
they consider to be more important. Similarly, the coefficient on the dummy variable for
high levels of education is also significantly negative. Thus, more schooling corresponds to
a lower probability of inconsistent choices. Both of these results support the indifference
hypothesis, rather than an ambivalence-based explanation for the observed inconsistencies
in pairwise choices.
The interpretation is complicated a bit by the coefficient for ideology, which is positive
and statistically significant. This shows that liberals are more likely to be inconsistent
in their value choices than conservatives. And, such a result is consistent with Feldman
and Zaller’s (1992) argument that liberals are relatively ambivalent about the core values
underlying American political culture.
Two additional variables also show significant effects: The negative coefficient for African
Americans shows that individuals from this racial group are more likely to express consistent
value choices. And, the dummy variable for the value, “equality,” has a positive coefficient,
showing that inconsistent choices across replications are especially common when the pairwise
comparison involves this particular value. Neither of these results are unreasonable; indeed,
they are probably very understandable given the existence of distinctive societal orientations
among the Black community and the troublesome status of equality in American social and
political thought.
19
Table 5 shows the maximum likelihood estimates for the equation predicting intransitivity
across subsets of three value choices. Once again, the model fit is attenuated, with a small
(although significant) Chi-square value and a low pseudo R2 (0.031). Just as was the case
in the analysis of inconsistencies, this supports the measurement error interpretation of
intransitive value choices.
There is only one independent variable in Table 5 with a coefficient that is statistically
distinguishable from zero: The dummy variable for high levels of education has a relatively
large negative impact on the probability of an inconsistency. And, once again, this is more
consistent with the indifference hypothesis than the ambivalence hypothesis.
A Closer Look at Ideology and Equality
Overall, the evidence from Tables 4 and 5 supports the view that inconsistencies and
intransitivities in value choices are due more to measurement error and indifference than
to feelings of ambivalence about pairs of values. But, there are two results which might
suggest otherwise: The statistically significant, positive, coefficients for ideology and the
dummy variable for equality in Table 4. Again, empirical inconsistencies are more prevalent
among more liberal respondents, and when the value pair involves equality. While both of
these results are consistent with the ambivalence hypothesis, they could also be due to other
processes. For example, liberals may simply be less attuned to core values than conservatives
or moderates. And, the American public may just give less thought to equality than to
liberty, morality, and other values. In both cases, the result would probably be high levels
of inconsistency, even though ambivalence is not present.
Let us consider these possibilities more closely. If the inconsistent choices exhibited
by liberals are truly due to ambivalence, then those inconsistencies should occur among the
values that liberals consider to be particularly important. But, this does not appear to be the
case: Looking only at the pairwise choices made by those people who identify themselves as
liberals, the correlation between the dummy variable indicating the presence of inconsistent
choices in a pairwise comparison and the average rank score of the two values in that pair is
20
miniscule (and not statistically significant), at -0.010. Liberals tend to be more inconsistent
in their choices, regardless how important they consider the values to be. Hence, they do
not appear to feel ambivalent about particular values.
The results are a bit different for the inconsistencies involving equality. Again, if this is
ambivalence, then inconsistent choices should be more common when equality is considered
to be particularly important— even though Table 4 showed that inconsistencies tend to
occur among less important values. To test this, we can correlate each person’s rank score
for equality with a dummy variable indicating whether he or she makes inconsistent choices
(across the replications) on any of the four value pairs that involve equality. Here, the
empirical value of the correlation coefficient (Pearson’s r) is 0.305. Thus, inconsistent choices
involving equality versus other values are more common when equality, itself, is considered
relatively important. This is precisely what we would expect if it is ambivalence regarding
equality that is generating the inconsistent choices.
CONCLUSION
An explicit model of value choice, such as that offered in this paper, is useful because it
enables relatively precise statements about the consequences of phenomena like ambivalence
and indifference. The model-based predictions can be compared against citizens’ actual value
choices. The latter are, themselves, obtained from an internet survey instrument using the
method of triads. The resultant information is used to test for consistency in, and transitivity
across, value choices. The ability to do this is critically important for evaluating the validity
of several theories that purport to represent the ways that people think about core values.
Without the kinds of empirical tests that are enabled by the unique data employed here,
the quality of value orientations and the existence of rank-ordered value structures would
have to remain as untested assumptions. In fact, the results obtained from this study lead
to several conclusions.
First, the distribution of value preferences reveals both consensus and conflict in the
ways that Americans think about fundamental principles like liberty and equality. There
21
appears to be wide agreement that morality, liberty, and economic security comprise the most
important values, with equality and social order falling at somewhat less salient positions
in public esteem. But, these aggregate patterns emerge from enormous variability among
individual choices among the values. All five of these values are highly important for some
subsets of the public, and less so for others.
Second, the data from the internet survey show that people do, generally, make consistent choices among values. That, in turn, attests to the viability of values as a psychological
construct. If values were only “truisms,” without strong cognitive and affective underpinnings, then we would expect people to move frequently from one to the other when asked
to choose between them. But, that just did not happen with these data: A clear majority
of the survey respondents made the same choice across the three replications of each value
pair. So, this provides strong evidence against criticisms that have been aimed toward the
values concept in the recent literature.
Third, individual value choices are almost always transitive. This means that people
really do differentiate among values in ways that enable them to say which ones are more
important and which ones are less important. And, that is consistent with the existence of
hierarchical value structures throughout the mass public. This study thus provides critical
empirical evidence to support the predominant scholarly understanding of the way that
people think about values.
Fourth, those inconsistencies and intransitivities which do exist in the data appear to
result mainly from a combination of measurement error and indifference. The impact of
measurement error comes as no real surprise, since it is pervasive in all survey data. Political scientists are now coming to recognize how errors of measurement attenuate the empirical
relationships between distinct responses to different items (Ansolabehere, Rodden, Snyder
2008), and that appears to be occurring with expressed value choices as well. Similarly, low
levels of interest and sophistication are known to “loosen the connections” between distinct
22
political orientations (e.g., Sniderman, Brody, Tetlock 1991); these phenomena seem to exert
a similar effect on individual feelings about values.
Finally, ambivalence is not a widely-ranging phenomenon with respect to values. Instead,
its effects seem to be concentrated on choices involving one particular value— equality. With
the other four values, feelings of greater importance correspond to higher levels of consistency
in choices. But, with equality, exactly the opposite occurs. Apparently, those who have the
greatest affinity toward equality also have the greatest difficulty in expressing consistent
choices between it and other values.
In conclusion, the theoretical status of values stems from their potential to exert pervasive
influences on many aspects of human behavior. But, before we can disentangle the relations
between values and other politically-relevant phenomena, it is important to develop a firm
understanding of the ways that people think about the values, themselves. This study makes
useful progress in that direction. Most people do maintain hierarchically structured value
choices. Citizens’ orderly preferences among different values generally are not obstructed by
factors like ambivalence, indifference, or measurement error. The latter phenomena certainly
do exist in American public opinion about values; however, they are relatively uncommon
and do not compromise the conceptual status of rank-ordered value preferences. Such evidence vindicates psychological theories and scholarly understandings that have been widely
accepted, but seldom tested, over the years.
23
NOTES
1. Of course, these utilities vary from one individual to the next. So, the expression for the
utility, itself, could have another subscript indexing people. For example, the j th person’s
utility for value i could be shown as U(V ali )j . Here, however, it should be clear that the
discussion is referring to a single individual, so the second subscript is omitted to keep the
notation as simple as possible.
2. It is possible that the utility-based model of value choice outlined above may not be a valid
representation of the way that people actually think about values. For example, the utility
associated with a particular value might vary across choices. In that case, there would be
no reason to expect transitivity at all. And, the fundamental theoretical representation of
human value preferences would be called into serious question.
3. The internet survey was conducted by Knowledge Networks, Inc. Data collection was carried out from June 16 through June 21, 2005. A total of 1000 surveys were fielded, and 649
respondents completed the interview schedule. Thus, the response rate was about 65%.
4. Assuming that the probability of choosing one value over the other is 0.50, there are eight
equally-likely outcomes across three replications of a given pairwise choice. Of these, two
outcomes involve the same choice (of one value or the other) across all three replications.
Hence, 2/8 = 0.25.
5.
The occurrence of intransitivity is related empirically to the occurrence of inconsistent
choices. In other words, if an individual exhibits nonuniform choices between two values,
then it is more likely that the same value pair will be involved in an intransitive set of choices
with at least one other value. The pattern is remarkably consistent across different values,
even though the exact percentages vary a bit. As an example, consider one pair of values,
liberty and equality. Across the entire sample, about 93.5% of the respondents gave fully
transitive dominant choices involving these two values; only 6.5% showed any intransitivity
on the dominant choices that included these two values plus one of the other three values
from the set of five. But, among people who gave fully consistent choices between liberty
and equality (i.e., they always picked on or the other as more important across the three
replications), 97.5% gave fully transitive dominant choices. Among those who showed any
inconsistency across replications, only 83.0% gave fully transitive dominant choices.
6. The analysis could also be carried out as a multilevel model, with the pairwise value choices
(or subsets of three choices) nested within individual survey respondents. When multilevel
logistic regression models for the determinants of inconsistent and intransitive choices are
fitted to the TESS data, the empirical results are virtually identical to those reported here;
the substantive interpretation would not change at all. A separate report on the multilevel
analysis is available from the author.
7. The logistic regression equations are estimated using information from only 468 survey respondents out of the 649 for whom complete value choices are available. The attrition is due
to the survey question about ideology, which included an option for people who said they
do not think of themselves in these terms; anyone who chose this response is excluded from
the analysis. On the other hand, recoding these respondents to the neutral “Moderate or
middle of the road” category and including them in the logistic regressions does not lead to
any different substantive conclusions.
25
REFERENCES
Alvarez, R. Michael and John Brehm. (2002) Hard Choices, Easy Answers. Princeton, NJ:
Princeton University Press.
Alwin, Duane F. and Jon A. Krosnick. (1985) “The Measurement of Values in Surveys: A
Comparison of Ratings and Rankings.” Public Opinion Quarterly 49: 535-552.
Ansolabehere, Stephen; Jonathan Rodden, James M. Snyder, Jr. (2008) “The Strength of
Issues: Using Multiple Measures to Gauge Preference Stability, Ideological Constraint,
and Issue Voting.” American Political Science Review 102: 215-232.
Ball-Rokeach, Sandra J.; Milton Rokeach; Joel W. Grube. (1984) The Great American
Values Test: Influence Behavior and Belief Through Television. New York: Free Press.
Bernard, Mark M.; Gregory R. Maio; James M. Olson. (2003) “Effects of Introspection
about Reasons for Values: Extending Research on Values-as-Truisms.” Social Cognition 21: 1-25.
Coombs, Clyde H. (1964) A Theory of Data. New York: Wiley.
Davis, Darren W. and Brian D. Silver. (2004) “Civil Liberties vs. Security: Public Opinion
in the Context of the Terrorist Attacks on America.” American Journal of Political
Science 48: 28-46.
Feldman, Stanley. (2003) “Values, Ideology, and the Structure of Political Attitudes.” In
Oxford Handbook of Political Psychology. David O. Sears, Leonie Huddy, Robert Jervis
(Editors). New York: Oxford University Press.
Feldman, Stanley and John Zaller. (1992) “The Political Culture of Ambivalence: Ideological Responses to the Welfare State.” American Journal of Political Science 36:
268-307.
Goren, Paul. (n.d.) ”Personal Value Systems and Public Opinion: A Preliminary Report
on Some New Value Items.” University of Minnesota. Typescript/
Goren, Paul; Christopher M. Federico; Miki Caul Kittilson. (2006) “Party, Values, and the
Belief System Core.” Paper presented at the 2006 Annual Meetings of the American
Political Science Association.
Goren, Paul; Christopher M. Federico; Miki Caul Kittilson. (2009) “Source Cues, Partisan
Identities, and Political Value Expression.” American Journal of Political Science 53:
805-820.
Grant, J. Tobin and Thomas J. Rudolph. (2003) “Value Conflict, Group Affect, and the
Issue of Campaign Finance.” American Journal of Political Science 47: 453-469.
Gulliksen, Harold and Ledyard R. Tucker. (1961) “A General Procedure for Obtaining
Paired Comparisons from Multiple Rank Orders.” Psychometrika 26: 173-183.
Hansson, Sven Ove. (2001) The Structure of Values and Norms. Cambridge, UK: Cambridge University Press.
Hitlin, Steven and Jane Allyn Piliavin. (2004) “Values: Reviving a Dormant Concept.”
Annual Review of Sociology 30: 359-393.
Hochschild, Jennifer L. (1995) Facing Up to the American Dream: Race, Class, and the
Soul of the Nation. Princeton, NJ: Princeton University Press.
Hunter, James Davison. (1994) Culture Wars: The Struggle to Define America. New York:
Basic Books.
Jacoby, William G. (2002) Core Values and Political Attitudes. In Barbara Norrander and
Clyde Wilcox (Editors), Understanding Public Opinion (Second Edition). Washington,
DC: Congressional Quarterly.
Jacoby, William G. (2006) “Value Choices in the American Public.” American Journal of
Political Science 50: 706-723.
Jacoby, William G. (2008) “Testing for Hierarchical Structure and Priming Effects among
Individual Value Choices.” Paper presented at the 2008 Annual Meetings of the Midwest Political Science Association.
Kamakura, Wagner A. and Jose Alfonso Mazzon. (1991) “Value Segmentation: A Model
for the Measurement of Values and Value Systems.” Journal of Consumer Research
18: 208-218.
Katona, George. (1975) Psychological Economics. New York: Elsevier.
Kinder, Donald R. (1983) “Diversity and Complexity in American Public Opinion.” In
Ada W. Finifter (Editor), Political Science: The State of the Discipline. Washington,
D.C.: American Political Science Association.
Kuklinski, James H. (2001) Introduction: Political Values.” In Citizens and Politics: Perspectives from Political Psychology. James H. Kuklinski (Editor). Cambridge, UK:
Cambridge University Press.
Layman, Geoffrey C. and Edward G. Carmines. (1997) “Cultural Conflict in American Politics: Religious Traditionalism, Postmaterialism, and U.S. Political Behavior.” Journal
of Politics 59: 751-777.
Maio, Gregory R. and James M. Olson. (1998) Values as Truisms: Evidence and Implications.” Journal of Personality and Social Psychology 74: 294-311.
Maio, Gregory R.; Neal J. Roese; Clive Seligman; Albert Katz. (1996) “Rankings, Ratings,
and the Measurement of Values: Evidence for the Superior Validity of Ratings.” Basic
and Applied Social Psychology 18: 171-181.
McCann, James A. (1997) Electoral Choices and Core Value Change: The 1992 Presidential
Campaign.” American Journal of Political Science 41: 564-583.
27
McCarty, John A. and L. J. Shrum. (2000) “Measurement of Personal Values in Research.”
Public Opinion Quarterly 64: 271-298.
McClosky, Herbert and John Zaller. (1984) The American Ethos: Public Attitudes Toward
Capitalism and Democracy. Cambridge, MA: Harvard University Press.
Miethe, Terance D. (1985) “The Validity and Reliability of Value Measurements.” Journal
of Psychology 119: 441-453.
Miller, Arthur; Tor Wynn; Phil Ullrich; Mollie Marti. (2001) “Concept and Measurement
Artifact in Multiple Values and Value Conflict Models.” Political Research Quarterly
54: 407-419.
Nelson, Thomas E.; Rosalee A. Clawson; Zoe M. Oxley. (1997) “Media Framing of a Civil
Liberties Conflict and Its Effect on Tolerance.” American Political Science Review 91:
567-583.
Peffley, Mark; Pia Knigge; Jon Hurwitz. (2001) “A Multiple Values Model of Political
Tolerance.” Political Research Quarterly 54: 379-406.
Rokeach, Milton. (1973) The Nature of Human Values. New York: Free Press.
Rokeach, Milton. (1979) Understanding Human Values: Individual and Societal. New York:
Free Press.
Rossiter, Clinton. (1962) Conservatism in America: The Thankless Persuasion. New York:
Vintage Books.
Schwartz, Shalom H. (1992) “Universals in the Content and Structure of Values: Theoretical
Advances and Empirical Tests in 20 Countries.” In Mark P. Zanna (Editor), Advances
in Experimental Social Psychology. Orlando, FL: Academic Press.
Schwartz, Shalom H. (1996) “Value Priorities and Behavior: Applying a Theory of Integrated Value Systems.” In Clive Seligman, James M. Olson, Mark P. Zanna (Editors),
he Psychology of Values: The Ontario Symposium, Volume 8. Mahwah, NJ: Lawrence
Erlbaum Associates.
Schwartz, Shalom H. and Wolfgang Bilsky. (1987) “Toward a Universal Psychological
Structure of Human Values.” Journal of Personality and Social Psychology 53: 550562.
Seligman, Clive and Albert N. Katz. (1996) “The Dynamics of Value Systems.” In The
Psychology of Values: The Ontario Symposium, Volume 8. Clive Seligman, James M.
Olson, Mark P. Zanna (Editors). Mahwah, NJ: Lawrence Erlbaum Associates.
Seligman, Clive; James M. Olson; Mark P. Zanna. (1996) The Psychology of Values: The
Ontario Symposium, Volume 8. Mahwah, NJ: Lawrence Erlbaum.
Sniderman, Paul M.; Richard A. Brody; Philip E. Tetlock. (1991) Reasoning and Choice:
Explorations in Political Psychology. Cambridge, UK: Cambridge University Press.
28
Tetlock, Philip E. (1986) “A Value Pluralism Model of Ideological Reasoning.” Journal of
Personality and Social Psychology 50: 819-827.
Verplanken, Bas and Rob. W. Holland. (2002) “Motivated Decision Making: Effects of Activism and Self-Centrality on Values in Choices and Behavior.” Journal of Personality
and Social Psychology 82: 434-447.
Weller, Susan C. and A. Kimball Romney. (1988) Systematic Data Collection. Newbury
Park, CA: Sage.
29
Table 1: The distribution of rank scores for each value.
Value:
Rank
Score:
Liberty
Equality
Economic
Security
Morality
Social
Order
Zero:
14.48%
23.88%
9.09%
16.18%
30.20%
One:
15.10%
27.43%
20.80%
15.41%
25.89%
Two:
20.96%
23.57%
22.65%
16.80%
19.72%
Three:
26.19%
17.41%
23.88%
19.11%
16.80%
Four:
23.27%
7.71%
23.57%
32.51%
7.40%
Note: For each person, the rank score gives the number of times the value in that column dominates other values in the replicated preferential choices. Cell entries give
the percentage of respondents with each rank score for each value. The number of
observations is 649.
Table 2: Consistency across replications in pairwise value choices.
Number of times the first value
is chosen over the second value
across the three replications:
Zero
One
Two
Three
Percentage making
consistent choices
across all three
replications
Liberty versus
equality
25.12%
11.56%
15.72%
47.61%
72.73%
Liberty versus
economic security
38.67%
12.02%
12.33%
36.98%
75.65%
Liberty versus
morality
41.91%
10.02%
10.63%
37.44%
79.35%
Liberty versus
social order
21.57%
10.48%
11.40%
56.55%
78.12%
Equality versus
economic security
52.39%
14.64%
10.02%
22.96%
75.35%
Equality versus
morality
56.09%
10.02%
19.57%
14.33%
70.42%
Equality versus
social order
33.13%
12.79%
12.63%
41.45%
74.58%
Economic security versus
morality
44.68%
7.24%
12.48%
35.59%
80.27%
Economic security versus
social order
22.03%
11.71%
15.87%
50.39%
72.42%
Morality versus
social order
25.27%
8.32%
11.25%
55.16%
80.43%
Note: In each row, the entries in the four leftmost columns indicate the percentage of respondents
that selected the first-mentioned value zero, one, two, or three times across the replications
of that pairwise choice. The rightmost column gives the percentage of respondents that
made fully consistent choices across all three replications. The number of observations is
649.
Table 3: Transitivity in pairwise value choices.
Value triad
Percentage whose
choices across value
pairs are intransitive
Any intransitive choices
across all ten value triads
12.48%
Liberty, equality, economic security
2.47%
Liberty, equality, morality
3.39%
Liberty, equality, social order
1.85%
Liberty, economic security, morality
0.77%
Liberty, economic security, social order
2.16%
Liberty, morality, social order
1.08%
Equality, economic security, morality
2.16%
Equality, economic security, social order
2.77%
Equality, morality, social order
1.08%
Economic security, morality, social order
1.69%
Note: The entry in the first row shows the percentage of respondents with any intransitivity on
any of the subsets of three values (i.e., triads). The entries in the remaining rows of the
table give the percentage of respondents whose choices on that specific triad are intransitive.
All percentages are based upon 649 observations.
Table 4: Predicting inconsistent pairwise value choices.
Maximum
Likelihood
Estimate
Odds
Ratio
Observed
Probability
Value
Average
rank
-0.171
(0.051)
0.843
0.001
Low
Education
-0.133
(0.168)
0.875
0.426
High
Education
-0.520
(0.116)
0.595
0.000
Ideological
self-placement
0.084
(0.031)
1.087
0.008
African
American
-0.393
(0.172)
0.677
0.024
Hispanic
-0.480
(0.241)
0.619
0.050
Other
race
-0.200
(0.241)
0.818
0.406
Female
0.087
(0.010)
1.091
0.381
Family
income
-0.006
(0.016)
0.994
0.717
Liberty
0.109
(0.098)
1.115
0.268
Equality
0.219
(0.092)
1.245
0.018
Economic
security
0.120
(0.093)
1.128
0.093
Morality
-0.054
(0.095)
0.947
0.569
Constant
-0.737
(0.273
—
0.007
Note: This table reports the results from a logistic regression analysis. The Wald Chi-square
statistic for this equation is 56.57 with 13 df, producing a probability value of 0.000.
The pseudo R2 is 0.019. The units of analysis are pairwise value choices. Entries
in leftmost column are maximum likelihood coefficient estimates and the figures in
parentheses are robust standard errors, clustered on individual survey respondents.
The number of observations for this analysis is 4680 (10 pairwise value choices for
each of 468 survey respondents).
Table 5: Predicting intransitive sets of value choices.
Maximum
Likelihood
Estimate
Odds
Ratio
Observed
Probability
Value
Average
rank
0.119
(0.289)
1.127
0.680
Low
Education
-0.424
(0.551)
0.654
0.441
High
Education
-1.051
(0.345)
0.350
0.002
Ideological
self-placement
0.008
(0.097)
1.008
0.938
African
American
-0.251
(0.506)
0.778
0.620
Hispanic
-0.322
(0.723)
0.725
0.656
Other
race
0.522
(0.801)
1.685
0.515
Female
0.124
(0.321)
1.132
0.699
Family
income
0.033
(0.047)
1.034
0.484
Liberty
0.196
(0.346)
1.217
0.570
Equality
0.430
(0.355)
1.536
0.226
Economic
security
0.099
(0.291)
1.104
0.733
Morality
-0.259
(0.348)
0.772
0.455
Constant
-4.192
(1.072
—
0.000
Note: This table reports the results from a logistic regression analysis. The Wald Chi-square
statistic for this equation is 22.57 with 13 df, producing a probability value of 0.054.
The pseudo R2 is 0.031. The units of analysis are pairwise value choices. Entries
in leftmost column are maximum likelihood coefficient estimates and the figures in
parentheses are robust standard errors, clustered on individual survey respondents.
The number of observations for this analysis is 4680 (10 pairwise value choices for
each of 468 survey respondents).
Figure 1: Introduction screen for value triads from Knowledge Networks web survey.
We’d like to ask you about some things that are important for our society, such as liberty,
equality, economic security, morality, and social order. First, here is what we mean by these
ideas:
By LIBERTY we mean a guarantee of the widest freedom possible for everyone to act
and think as they consider most appropriate.
By EQUALITY we mean narrowing the gap in wealth and power between the rich and
the poor.
By ECONOMIC SECURITY we mean the guarantee of a steady job and a decent
income.
By MORALITY we mean people living according to the rules that constitute decent
human behavior.
By SOCIAL ORDER we mean being able to live in an orderly and peaceful society
where the laws are respected and enforced.
All five of these ideas are important, but sometimes we have to choose between what is more
important and what is less important. And, the specific choices we make sometimes depend
upon the comparisons we have to make.
On the next few screens, we will show you these ideas in sets of three. For each set, please
use the mouse to indicate the idea that you think is most important of the three, and also
the idea that you think is least important of the three. In some cases, you might think all
three of the ideas are very important, but please try to indicate the ones you think are most
and least important if you had to choose between them.
If you absolutely cannot decide which ideas are most or least important, then just click the
box at the bottom of the screen, and move on to the next set of three ideas.
Figure 2: Format of value triad items from Knowledge Networks web survey.
After viewing the introductory screen for the values (shown in Figure 1), respondents moved
on to another screen which presented the first value triad in a format like the following.
The MOST
IMPORTANT
idea from these
three is:
The LEAST
IMPORTANT
idea from these
three is:
Liberty
Equality
Economic Security
Respondents used the mouse to click on one box within each column, in order to indicate their
choices for the most and least important values within the triad. The remaining nine triads
were then presented using an identical format. Each triad appeared on a separate screen.
The order of the triads, and the order of the values within each triad were randomized.
Respondents were not permitted to move backward and revisit previously-completed triads.
The full set of value triads is as follows:
Liberty, Equality, Economic security
Liberty, Equality, Social order
Liberty, Equality, Morality
Liberty, Economic security, Social order
Liberty, Economic security, Morality
Liberty, Social order, Morality
Equality, Economic security, Social order
Equality, Economic security, Morality
Equality, Social order, Morality
Economic security, Social order, Morality
Figure 3: Mean rank scores for each value.
Morality
●
Economic Security
●
Liberty
●
Equality
●
Social Order
●
0
1
2
3
4
Mean importance score
Note: Points in the dot plot represent mean rank scores (calculated across 649 internet
survey respondents) for each value. The error bars around the points represent the
95% confidence intervals for the respective means.