JOURNAL OF THE ACADEMY
Holbrook
10.1177/0092070305281627
OF
et al.
MARKETING
/ EXPERT JUDGMENT
SCIENCE
ARTICLE
WINTER 2006
Audience Judgments as the
Potential Missing Link
Between Expert Judgments
and Audience Appeal:
An Illustration Based on Musical
Recordings of “My Funny Valentine”
Morris B. Holbrook
Columbia University
Kathleen T. Lacher
Tallahassee, Florida
Michael S. LaTour
University of Nevada, Las Vegas
Previous research on the arts, entertainment, and other
cultural objects has found, at most, a weak link between
expert judgments of aesthetic excellence and audience appeal to nonexpert consumers. However, this tendency for
audience appeal only weakly to reflect expert judgments of
excellence raises the question of how this fragile relationship might be mediated by audience judgments of excellence. As the first study to examine the potential intervening role of audience judgments, the present article
investigates the links between expert judgments, audience
judgments, and audience appeal in an illustrative case
based on 200 recordings of the song “My Funny Valentine.” The results support a scenario in which audience appeal is weakly related to expert judgments through the
hitherto neglected intervening role of audience judgments
so as to suggest refinements in our approaches to marketing entertainment, the arts, or other cultural offerings,
Journal of the Academy of Marketing Science.
Volume 34, No. 1, pages 8-18.
DOI: 10.1177/0092070305281627
Copyright © 2006 by Academy of Marketing Science.
as well as various consumer services, durables, or
nondurables.
Keywords: expert judgment; audience judgment; audience appeal; music, entertainment, the arts
For more than two millennia, various commentators
ranging from Plato to Mayor Rudolph Giuliani have
debated the nature of taste, the standards for cultural preferences, and the question of whether consumers have the
power to discriminate what is good, valuable, or otherwise
worthwhile in the realm of entertainment, the arts, and
other cultural offerings (for a review of this literature, see
Holbrook 1999). Many cultural commentators from the
ideological left, the conservative right, and a continuum of
positions in between have argued that mass audiences lack
the training and refinement needed to appreciate artistic
excellence, that popularity requires ease of accessibility,
and therefore that the quest for profitability forces producers of pop culture to commercialize their offerings by
dumbing them down in order to reach a broader audience
Downloaded from jam.sagepub.com at PENNSYLVANIA STATE UNIV on February 20, 2016
Holbrook et al. / EXPERT JUDGMENT 9
(Brantlinger 1983). Some wear their elitist credentials
with a certain degree of defiant pride (Henry 1994). Others ironically poke fun at what they regard as the lovable
excesses of kitsch (Stern and Stern 1990). Still others subscribe to what might be regarded as an emerging consensus among sociologists (as found especially in the writings
of Bourdieu 1983, 1984, 1986, 1993).
The Sociological Consensus
This sociological consensus suggests that, historically,
a cultural hierarchy appeared when members of the more
upscale social classes found ways to sacralize various art
forms such as the theater (serious drama), the fine arts
(painting and sculpture), or classical music (symphony orchestras) in a manner that helped to distance themselves
from the more downscale social classes (Levine 1988).
According to Bourdieu (1984), such class distinctions
depend on—while simultaneously reinforcing—the relative distribution of both economic and cultural capital in a
pattern that creates homologies between class membership and habits of cultural consumption. For example,
education—one aspect of the “habitus” determined by but
also reinforcing of the class structure—closely parallels a
whole system of class-related distinctions in cultural tastes
wherein (e.g.):
High Education
Art
Creative Integrity
Bach
:
:
:
:
Low Education
Entertainment
Commercialization
Beatles
Sociologists sometimes question the degree to which
Bourdieu’s European observations characterize the cultural scene in America today (Erickson 1996; Gartman
1991), but most would concede that—in the United States,
as elsewhere—strong links do indeed occur between education and cultural tastes (DiMaggio, Useem, and Brown
1978; Gans 1974).
The Cultural Field
Further work by Bourdieu (1983, 1986, 1993) has
delved more deeply into parallel phenomena that occur at
the level of a specific cultural field—for example, music,
literature, the theater, or motion pictures. In this context,
Bourdieu suggests that, at a more micro level of analysis,
comparable processes involving unequal distribution of
the relevant cultural capital also tend to manifest themselves. Specifically, within a given cultural field, certain
participants—by virtue of both formal training and informal assimilation—acquire higher levels of the knowledge,
expertise, and sensitivity needed to appreciate the more
refined distinctions associated with what passes for excellence in that particular cultural area. In other words,
a potentially small, mostly self-selected, and specially
trained group of experts or connoisseurs adopts standards
of evaluation that are recognized by themselves and by
others as valid criteria for excellence. Such cognoscenti
claim a legitimate faculty for discriminating tastes that sets
them apart from the more nonexpert consumers who have
not attained so high a level of cultural capital in that particular cultural field.
One example of this socially constructed process of
valorization concerns the difference between professional
critics and journalistic reviewers (e.g., in such cultural
fields as film, theater, the fine arts, or music). The former
(e.g., Leonard Maltin in the case of film) are assumed to
adhere to the accepted field-specific standards of artistic
excellence in judging the enduring aspects of aesthetic
value; the latter (e.g., Gene Shalit, the movie reviewer for
NBC News) are assumed to cater to the comparatively
less-refined tastes of the mass audience by praising what
they anticipate large numbers of their less discerning readers or viewers will tend to enjoy (Shrum 1991).
Previous Findings
Those who subscribe to the sociological picture just
painted—including, quite conspicuously, Bourdieu himself
(1983, p. 330; 1984, p. 271; 1986, p. 152; 1993, p. 164)—
tend to assume the existence of a negative correlation
across cultural objects between expert judgments (e.g., by
professional critics) and audience appeal (e.g., to nonexpert consumers). However, two major problems with
this conceptualization are that (1) it has seldom been tested
empirically, and (2) when it has been tested empirically it
has not received support from the data.
Specifically, the previous literature has seldom addressed the issue posed here in a direct manner (for a
review, please see Holbrook 1999). Here, we are primarily
concerned with legitimized “good taste” as represented by
the critical judgments of professional critics. One common departure from the present focus reflects the aforementioned difference between evaluations by professional
critics (intended to reflect the aesthetic worth of a cultural
offering from the vantage point of a long-run perspective)
as opposed to assessments by journalistic reviewers (who
aim their appraisals at influencing, anticipating, or otherwise matching the short-term preferences that guide audience appeal to the readers who buy the print media or
the viewers who watch the television programs in which
their reviews appear) (Holbrook 1999). Various studies—
especially in the area of motion pictures—have pursued
the latter focus and have measured the contribution of
journalistic reviews to influencing and/or predicting a
film’s box-office revenues (for reviews, please see
Basuroy, Chatterjee, and Ravid 2003; Eliashberg and
Shugan 1997). In such studies, the relationship between
journalistic reviews and box-office revenues has tended to
Downloaded from jam.sagepub.com at PENNSYLVANIA STATE UNIV on February 20, 2016
10
JOURNAL OF THE ACADEMY OF MARKETING SCIENCE
be statistically significant but rather weak (with explained
variances of less, often much less, than 10%). For example, Eliashberg and Shugan (1997) showed that the percentages of positive or negative journalistic reviews made
statistically significant but weak contributions to explaining box-office revenues across an 8-week period (β = .306,
t = 2.584)—with week-by-week effects that increased
from the first 4 to the last 4 weeks, suggesting that reviews
play the role of predictors more than influencers of audience tastes. Meanwhile, in a variety of different approaches to measuring the impact of journalistic reviews
on box office success, Ravid (1999) found a positive effect
of attention (number of reviews) but no effect of favorability
(percentage of good reviews). Following up, in a variety of
tests regressing box-office revenues on the percentages of
positive journalistic reviews across an 8-week period,
Basuroy et al. (2003) reported significant but weak effects
of reviews on revenues (.14 < β < .21, 2.96 < t < 3.56).
By contrast with these studies of the film industry, the
present research addresses the question of how the aesthetic excellence in a work of art or some other cultural
offering (as judged by professional experts) is related to
the product’s audience appeal (as experienced by nonexpert consumers). Here, the question is not whether
reviews influence, predict, or otherwise cause popularity
but rather whether some quality that prompts (un)favorable critical assessments of a cultural offering by professional experts also leads to its (un)popularity among
nonexpert consumers so that a relationship appears across
cultural objects between expert judgments and audience
appeal. For this purpose, judgments by professional critics
or other experts must replace assessments by journalistic
reviewers as the relevant standard for comparison.
In this connection, previous studies have again found
statistically significant but only weak relationships between expert judgments and consumer appeal (with explained variances below 10%). For example, studying the
responses of experts and novices to 20 black-and-white
graphic prints, Schindler, Holbrook, and Greenleaf (1989)
showed that, across four repeated exposures spread 1 week
apart, the preferences of novices converged toward those
of experts in the sense that the partial correlation of preferences between the two in Week 4, controlling for original
novice preferences in Week 1, was a significant but weak
r = .31. In the case of films, Wallace, Seigerman, and
Holbrook (1993) reported a rather confusing nonmonotonic (U-shaped) relationship of film revenues to critical
ratings, with significant linear and quadratic terms that
suggested highest revenues for both the “best” and the
“worst” films. However, in a study of motion pictures that
focused on better developed measures of assessments by
professional critics and nonexpert consumers, Holbrook
(1999) again found that the correlation between expert
judgments and audience appeal was significant but weak
at r = .25. To the best of our knowledge, no previous study
WINTER 2006
has examined this issue in the area of major concern to the
present article, namely, musical performances.
Competing Explanations:
Four Scenarios
As indicated thus far, previous research has raised conceptual questions about the relationship between aesthetic
excellence (as judged by professional experts) and audience
appeal (the taste preferences of nonexpert consumers)—
with emerging answers that appear to suggest a significant
but weak association between expert judgments and audience appeal, usually with correlations or standardized
betas in the vicinity of .30 or less (which represents an
explained variance of about .302 or below 10%). However,
all previous work has ignored a potential intervening variable that may mediate the weak relationship between
expert judgments and audience appeal—helping to
account for the slippage between the two—namely, evaluations of aesthetic excellence by nonexpert consumers or
what we will call audience judgment.
Conceptually, it appears clear that, when considering
the potential intervening role of audience judgment (AJ),
one of various mediating effects might account for the
overall weak association between expert judgment (EJ)
and audience appeal (AA). We might summarize the leading possibilities under the four scenarios presented in
Figure 1.
Scenario 1. In Scenario 1, the links between EJ and AJ
and between AJ and AA are both moderately positive, but
the overall multiplicative contribution of EJ to AA—a
moderate plus multiplied by a moderate plus—turns out to
be fairly weak. In other words, under Scenario 1, nonexpert consumers have some understanding of what is
judged by experts to be excellent and display some tendency to like what they know to be critically judged as excellent, but the multiplicative combination of these two
moderately positive links produces only a weakly positive
overall relationship between EJ and AA. In effect, the consumer says, “I am partially aware of what experts value
and . . . up to a point . . . that’s what I like.” For example, in
the area of music, Scenario 1 might lead a consumer to say,
“I am pretty sure that experts evaluate Gerry Mulligan
more favorably than Kenny Rogers, and I also tend somewhat to prefer the former to the latter some of the time.”
We have picked Kenny Rogers and Gerry Mulligan for
purposes of illustrating Scenario 1—among countless
other possibilities—because these two performers illustrate the polls of taste currently under examination. Professional music critics have extolled the virtues to be found in
the oeuvre of Mr. Mulligan, while Mr. Rogers might be
considered something of a pop icon. A subscriber to
Scenario 1 would argue that people sense Mulligan is a
“better” musician and tend to like Mulligan accordingly.
Downloaded from jam.sagepub.com at PENNSYLVANIA STATE UNIV on February 20, 2016
Holbrook et al. / EXPERT JUDGMENT 11
FIGURE 1
stylings do not give a hoot about their hero’s estimation in
the eyes of professional critics. Rather, when compared
with the music of (say) Gerry Mulligan, they have a sort of
visceral response to Mr. Rogers that encourages a strong
liking for his work without the need to think very much
about it. A subscriber to Scenario 2 would argue that people do not know or care whether Mr. Rogers is a better musician than Mr. Mulligan because all they know or want to
know is that they enjoy Kenny and not Gerry.
Scenario 2. Scenario 2 assumes that, even if AJ is positively related to AA, the nonexpert consumer has no clear
sense of what is considered excellent under the critical
norms of professional EJ. Hence, with little or no contribution of EJ to AJ, EJ has even less indirect mediated impact
on AA. In essence, the consumer says, “I don’t know much
about aesthetics or art or the standards of expert evaluation, but I know what I personally enjoy, and that’s what I
like.” Scenario 2 is somewhat unflattering to the nonexpert
consumer in that it assumes ignorance of EJ. It thereby
echoes the “disrespectful” or “politically incorrect” viewpoint expressed by cultural commentators who have held
elitist views concerning the nature of the cultural hierarchy
(as described earlier). Scenario 2 asserts that nonexpert consumers tend to lack knowledge relevant to a particular cultural field and therefore do not know or perhaps do not
even care what cultural objects are highly regarded by critical experts but rather may gravitate toward enjoying the
(presumably) simpler and more easily accessible pleasures that their comparative lack of refinement (in a particular cultural field) permits them to appreciate. Thus, for
such consumers, only weak relationships would appear
between EJ and either AJ or AA. However, Scenario 2
does at least leave open the possibility that nonexpert consumers form personal opinions on aesthetic worth and
shape their preferences accordingly. Colloquially, for the
case of music, the motto of one who conformed to Scenario 2 might be: “I don’t know much about music, but I do
know what I like, and I don’t like Gerry Mulligan as well as
Kenny Rogers.” This motto would suggest that those fans
who swoon over Kenny Rogers’s good looks or the starstruck romantics who thrill to his warm and earthy vocal
Scenario 3. By contrast, Scenario 3 assumes—even
less flatteringly in some ways—that nonexpert consumers
do not care about what they think is excellent but, rather,
form their preferences independently of their own aesthetic assessments so that no more than a near-zero relationship appears between AJ and AA. Hence, even if the
link between EJ and AJ is positive, only a weak association appears between EJ and AA. In effect, under Scenario
3, the consumer says, “I like some things that I think are
poor and dislike some things that I think are excellent because, hey, pleasure and not quality is what counts in my
overall enjoyment.” If anything, Scenario 3 is even more
unflattering to nonexpert consumers than Scenario 2 because it assumes that, in their formation of cultural preferences, people are untrue to their own aesthetic judgments
and are therefore untrue to themselves. Ironically enough,
we have seen this view expressed by various friends and
colleagues, by students, by reviewers, and by others anxious to preserve the “dignity” of the “common man” by attacking the political correctness of even raising questions
about the potential differences in judgments between critical experts and nonexpert consumers. Almost paradoxically, Scenario 3 asserts that nonexpert consumers might
know perfectly well what objects in a cultural field are
prized by critical experts for their excellence but that—
despite an awareness of such critically sanctioned merit
(which, by itself, would encourage favorable affective responses)—these audience members happen to enjoy the
simpler pleasures of life to be found in more easily accessible forms of pop culture. Presumably, the two effects
combine in such a way that no strong relationship appears
between expert judgments and audience appeal. Colloquially, for the case of music in particular, the motto of one
who conformed to Scenario 3 might be: “I know a lot
about music and know that Gerry Mulligan is ‘better’ than
Kenny Rogers, but I also know what I like, and I like
Kenny Rogers better than Gerry Mulligan.”
Scenario 4. Finally, Scenario 4 bears perhaps the closest resemblance to the phenomenon envisioned by
Bourdieu (1983, 1986, 1993) insofar as EJ makes a direct
negative contribution to AA, canceling what would otherwise be an overall positive mediated indirect relationship.
Here, indirectly, people tend to like what they know they
“should” like, but beyond this indirect positive effect,
they have an averse reaction to what the critics regard as
Downloaded from jam.sagepub.com at PENNSYLVANIA STATE UNIV on February 20, 2016
12
JOURNAL OF THE ACADEMY OF MARKETING SCIENCE
excellent—as if they were to say, “I try to like what’s good
for me, but I don’t find it as enjoyable as the more trashy
stuff.” In the case of music, the motto of someone who
conforms to Scenario 4 might be: “I know that the experts
favor Gerry Mulligan, and I appreciate his talents, but I’m
not going to let any professional critic force me to listen to
Gerry Mulligan’s long-winded and boring solos or stop
me from enjoying Kenny Rogers.”
The Research Question
We shall regard the choice among Scenarios 1 through
4 as a research question to be addressed by empirical
investigation, rather than by pontificating or proselytizing
for a particular cultural viewpoint. It appears to us that, in
the absence of much supporting evidence, some aspects of
our four scenarios are espoused for ideological reasons by
liberal, highly educated, well-meaning intellectuals with
strong egalitarian values who tend to project their own
sophistication onto the general population. Such individuals reflect the shared values of academia and just naturally
assume that other people also know what is regarded by
experts as excellent but may tend to enjoy something else.
Other aspects of our four scenarios jibe more closely with
the elitist views of cultural commentators who contend
that most people do not know what experts regard as
excellent and would not enjoy it if they did.
We propose that, rather than pursuing the sorts of cultural polemics that have appeared in the literature reviewed earlier, the applicability of the four scenarios
should be investigated empirically in the form of the following research question:
Research Question: Which scenario best describes the
overall relationship between expert judgment and
audience appeal in general and best captures the
potential mediating role of audience judgment in
particular?
As explained earlier, any of the four scenarios would account for the one solid piece of evidence that we seem to
have, namely, the statistically significant but weak correlation between EJ and AA observed in previous research. But the scenarios differ dramatically in terms of an
addressable empirical issue regarding the heretofore unexamined potential intervening link via AJ. AJ does indeed play a mediating role that weakens the overall
relationship between EJ and AA, this answer to our research question may help to resolve the puzzle raised by
earlier studies.
Preview
In sum, previous studies have measured EJ and AA,
finding significant but weak links between the two. But
WINTER 2006
these studies have not measured judgments of aesthetic
excellence by nonexpert consumers, making it impossible
to decide among the competing explanations represented
by Scenarios 1 through 4. Furthermore, despite the enormous amount of ink spilled in cultural commentary on various aspects of the issue, we do not find strong a priori
conceptual reasons for favoring one scenario over the others. Rather, we believe, the choice between Scenarios 1
through 4 suggests a research question that requires an
answer based on empirical investigation. This empirical
answer must hinge on the measurement not only of EJ and
AA but also of a hitherto neglected but potentially relevant
mediator, namely, AJ. Accordingly, this study is the first—
to the best of our knowledge—to collect all three measures
of appreciation in a way that permits us to address the
aforementioned research question.
METHOD
Stimulus Objects
It appears clear that the research question just raised
applies with special force to the cases of films or the fine
arts as studied previously (references cited earlier) and to
the case of our musical example. In the latter connection,
artists ranging from Gerry Mulligan to Kenny Rogers and
beyond have drawn on partially overlapping repertoires to
make recordings of tunes such as one by Rodgers and Hart
that has become a well-known and oft-played jazz-andpop standard, namely, “My Funny Valentine” (“MFV”). It
makes sense to study how experts and consumers respond
to just one piece such as “MFV” because this allows us
to control for most extraneous aspects of the music not
directly related to the actual performers and performances
themselves (lyrics, melody, harmony, composers, historical origins, familiarity of the song, and so forth).
Accordingly, to create a set of artistic offerings that
were as similar as possible in ways other than those relevant to performance-based, taste-related assessments by
critical experts versus nonexpert consumers, we assembled stimulus objects based on one-chorus excerpts from
200 different recorded versions of “MFV.” These performances covered a wide spectrum of stylistic differences
that ranged from pure jazz instrumentals (Chet Baker,
Miles Davis, Bill Evans, Paul Desmond, Art Farmer, Jim
Hall, Art Pepper, Gary Burton, Keith Jarrett, Zoot Sims,
Gerry Mulligan, Erroll Garner, Stan Getz, J. J. Johnson,
Lee Konitz, Hampton Hawes, Milt Jackson, Wynton
Marsalis) to jazz-oriented singing (Chet Baker, Ella Fitzgerald, Sarah Vaughan, Ernestine Anderson, Carol Kidd,
Carmen McRae, Anita O’Day) to commercial pop singing
(Frank Sinatra, Tony Bennett, Billy Eckstine, Julie
Andrews, Mary Martin, Gordon MacRae, Johnny Mathis,
Downloaded from jam.sagepub.com at PENNSYLVANIA STATE UNIV on February 20, 2016
Holbrook et al. / EXPERT JUDGMENT 13
Anita Baker, Michelle Pfeiffer, Dinah Shore, Barbra
Streisand, Carly Simon, Eileen Farrell, Kenny Rogers,
Linda Ronstadt) to easy listening (Roger Williams, Peter
Nero, Ray Anthony, Jackie Gleason, Andre Kostelanetz,
Hugo Montenegro, 101 Strings, John Williams and the
Boston Pops Orchestra) to rock (Van Morrison, Jerry Garcia, Elvis Costello, Nico, Miranda Sex Garden, the Stylistics, the Whispers). Every different version that we could
locate on compact disc was used—including multiple performances by some artists—the point being to present a
broad diversity of stylistic approaches that would be likely
to differ dramatically in both their evaluative appraisal by
critical experts and their audience appeal to nonexpert
consumers while controlling for extraneous differences
due to the identity of the song itself. Interested readers may
obtain a full list of the 200 recordings of “MFV” by contacting the first author.
One-chorus excerpts from the various recordings—
each comprising the main statement of the melody in that
particular performance—were recorded on separate cassette tapes and presented in random orders to three different groups of judges concerned with providing three different sorts of ratings.
Ratings of AA, AJ, and EJ
The first two separate groups of judges consisted of college students at two major southern universities and were
randomly allocated with no overlap between the two
groups. These were “nonexpert” consumers in the sense
that they were recruited from those who possessed no special knowledge of music beyond that common in the general population.
AJ—Excellence. The second separate randomly assigned group of nonexpert judges also listened in random
orders but rated the “MFV” performances for their “aesthetic character” (defined as artistic creativity and technical precision) on a 9-point scale from 1 (poor) to 9
(excellent). The exact wording of the questionnaire items
was as follows:
We are interested in finding out your opinion of the
aesthetic character of each song, such as the artistic
creativity and the technical precision. Please circle
the number that best reflects your judgment for each
song.
Poor
Song 1
Song 2
Song 3
•
•
•
Song 29
Excellent
1
1
1
2
2
2
3
3
3
4
4
4
5
5
5
6
6
6
7
7
7
8
8
8
9
9
9
1
2
3
4
5
6
7
8
9
EJ—Excellence. A third group of judges consisted of
highly trained experts drawn from the faculty and graduate
students in music at another major southern university.
These expert judges possessed degrees of musical education at or beyond the levels typical of professional critics in
the arts and reflective of the standards for legitimacy applicable in this particular cultural field. These critical experts
rated the “MFV” performances on the same 9-point scale
of “aesthetic character” (defined as artistic creativity and
technical precision) from 1 (poor) to 9 (excellent).
Groups of Judges and Pseudojudges
AA—Enjoyment. Listening in random orders, one group
of nonexpert judges rated the “MFV” performances for
how “enjoyable” (defined as entertaining and appealing)
they found the listening experience on a 9-point scale from
1 (did not enjoy at all) to 9 (enjoyed very much). The exact
wording of the questionnaire items was as follows:
We are interested in finding out how enjoyable each
song was for you to listen to. For example, how entertaining and appealing was it?
Did not
enjoy
at all
Song 1
Song 2
Song 3
•
•
•
Song 29
Enjoyed
very
much
1
1
1
2
2
2
3
3
3
4
4
4
5
5
5
6
6
6
7
7
7
8
8
8
9
9
9
1
2
3
4
5
6
7
8
9
Nonexpert consumers. The sizes of the two separate
groups of nonexpert consumers just described were determined by the large number of stimulus objects to be rated.
Based on past experience, the number of individuals
needed to rate each piece with adequate multi-item reliability was set at 12 judges. However, to avoid undue fatigue, each participant was asked to rate only 28 or 29
(randomly assigned) “MFV” performances from the full
set of 200. This meant that 7 judges were needed to cover
the entire set of 200 performances once so that two separate randomly assigned groups of 12 × 7 = 84 judges were
required for the two different rating tasks to measure AA
and AJ. Within each of the two tasks, 84 judges were randomly combined in sets of 7 each to form 12 “pseudojudges” whose ratings could be assessed for interjudge
reliability via a 12-item coefficient alpha (α). Toward this
end, scores were first standardized (M = 0.0, SD = 1.0)
across objects within each judge’s ratings. These standardized scores were next combined to compile ratings by the
12 pseudojudges in that task group. The ratings by these
Downloaded from jam.sagepub.com at PENNSYLVANIA STATE UNIV on February 20, 2016
14
JOURNAL OF THE ACADEMY OF MARKETING SCIENCE
12 pseudojudges were then summed to form a 12-item index with each pseudojudge constituting an “item.” Finally,
the two 12-item indices of AA and AJ were assessed for reliability via coefficient alpha (α).
Critical experts. The expert judges were treated in a
similar fashion except that—because they were expected
to be higher than nonexpert consumers in the homogeneity
of their critical evaluations and because they required a
small payment to participate ($25)—they were somewhat
fewer in number. Specifically, still reflecting issues of fatigue, 56 expert judges each evaluated a randomly assigned subset of 28 or 29 “MFV” performances. Their
evaluative ratings, again standardized within each judge,
were combined into 8 expert pseudojudges whose eightitem summative index of EJ was assessed for reliability via
coefficient alpha (α) in the manner previously described.
Analysis
The recursive model representing the four scenarios
described earlier was estimated by ordinary least squares
(OLS) regression as a simple three-variable path analysis.
Specifically, any relationship between EJ and AA
(hypothesized by all scenarios to be small or zero) was
assumed to consist of (1) a potential indirect association
via AJ as an intervening link and (2) a potential remaining
direct relationship of EJ and AA when controlling for any
mediating role of AJ. To rephrase our earlier research
question, we wish to determine which of the four scenarios
best describes the relationship between EJ and AA in general and best captures the potential mediating role of AJ
in . To recast the usual conditions for a mediated relationship of the type examined here, AJ would be viewed as
“mediating” the expected weak relationship between EJ
and AA to the extent that (1) rEJ, AA > 0, (2) rEJ, AJ >> 0, (3)
rAJ, AA >> 0, and (4) rEJ, AA.AJ ≤ 0 (i.e., a zero or negative
direct contribution of EJ to AA appears when controlling
for AJ) (Baron and Kenny 1986; Pedhazur 1982).
To recapitulate briefly, in accord with the previous research reviewed earlier, all four scenarios assume that rEJ,
is, at most, weakly positive. But they differ in their asAA
sumptions about why this is the case, as follows:
Scenario 1—because rEJ, AJ and rAJ, AA are both only moderately positive
Scenario 2—because rEJ, AJ is near zero (even if rAJ, AA is
positive)
Scenario 3—because rAJ, AA is near zero (even if rEJ, AJ is
positive)
Scenario 4—because both rEJ, AJ and rAJ, AA are positive
but rEJ, AA.AJ is negative
Notice that—like all the other studies reviewed earlier—
the potential relationships examined here are inherently
WINTER 2006
correlational in nature rather than the outcomes of experimental manipulations, thereby inevitably raising issues
concerning possible alternative explanations. Notice also,
however, that, in this analysis, AA, AJ, and EJ are all measured on data gathered from separate groups of judges.
This means that any relationships found deserve to be considered free from methods artifacts such as consistency biases due to the same individual answering coherently to
multiple questions (Holbrook and Batra 1987; Olney,
Holbrook, and Batra 1991). Furthermore, as in the other
research reviewed earlier, treating a large number of stimulus objects (200 “MFV” performances) rather than people (consumers and experts) as the relevant units of
analysis has the advantage of avoiding alternative explanations that may occur when only a small number of experimental treatments (say Movie A vs. Movie B or Song 1 vs.
Song 2) are used in a between-subjects design (Jackson
1992).
RESULTS
Reliabilities for the Multi-Judge
Indices of AA, AJ, and EJ
As assessed by coefficient alpha (α), the interjudge
reliabilities for the 12-item indices of AA and AJ were as
follows: αAA = .59 and αAJ = .75, respectively. Internal consistency for the eight-item index of EJ was αEJ = .71. None
of these indices were improved by the removal of any item
or by the use of normalized in place of standardized scores.
These levels of internal consistency indicate a satisfactory
degree of homogeneity among judges across objects and
suggest that interjudge reliabilities are acceptable for the
three rating tasks of interest (Nunnally 1967).
OLS Path Analysis
Results for the simple three-variable path analysis
described earlier may be portrayed by the relationships
shown in Figure 2 (with all coefficients significant at p <
.001, except in the one case noted). As indicated, we find a
rather modest positive association between EJ and AA via
a correlation of rEJ, AA = .365 (t = 5.516)—corresponding to
a fairly small explained variance of 13.3 percent. This
weak positive association between EJ and AA (r = .365,
r2 = .133) can be decomposed into indirect links from EJ to
AJ (rEJ, AJ = .548, t = 9.216) and from AJ to AA (rAJ, AA =
.614, t = 10.957; βAJ, AA.EJ = .592, t = 8.821) with a nonsignificant direct link from EJ to AA (βEJ, AA.AJ = .040, t =
0.603, ns) and with an overall fit of R = .615 (R2 = .379,
F2,197 = 60.019). Notice that, as would be expected from the
logic of the path analysis, the overall link between EJ and
Downloaded from jam.sagepub.com at PENNSYLVANIA STATE UNIV on February 20, 2016
Holbrook et al. / EXPERT JUDGMENT 15
FIGURE 2
AA (.365) can be decomposed into its indirect component
via AJ (.548 × .592 = .324) plus its direct component
(.040) where .324 + .040 = .364.
relationship might suggest that, to some extent, experts and
consumers use some of the same criteria as standards for
their assessments of excellence and that, to some extent,
they share perceptions of the degrees to which various performances succeed in terms of these shared criteria. For
example, such common standards might include playing
in tune, singing on key, well-synchronized ensembles,
steady rhythm, and so forth. However, because the experts
share additional criteria among themselves—as well as
perceptions that require more refined standards of evaluation, more discriminating sensibilities, or more subtle
powers of analysis—their judgments do deviate from
those of nonexpert consumers in various ways. These
might include attention paid by experts to security of intonation, gracefulness of phrasing, choice of judicious
tempo, appropriateness of rubato, and so forth. Hence,
overall and in support of Scenario 1, the findings suggest a
moderately positive link between EJ and AJ.
DISCUSSION
Limitations and Directions
for Future Research
Summary
As in virtually any study of the type presented here,
various limitations warrant mention as potential bases for
future research. To repeat a point mentioned previously,
our support for Scenario 1—like the findings in all the
other studies reviewed earlier—is, at best, correlational in
nature and therefore does not permit the sort of strict
causal interpretation that would result from an experimental design. Indeed, one might imagine alternative scenarios consistent with the present pattern of findings. For
example, it could be that, via a sort of halo effect, both
experts and consumers base their judgments of excellence
on what they enjoy. If it happens that experts and consumers enjoy the same things, this could account for the association between expert judgments and audience judgments. Because our data did not include a measure of
expert appeal (enjoyability to trained professionals), we
cannot definitively rule out this alternative scenario (beyond suggesting that basing judgments of excellence on
feelings of enjoyment appears to run counter to the conventional norms of professional expertise).
However, one reviewer has proposed that the alternative scenario just described would predict that AA should
mediate the relationship between EJ and AJ so that if the
EJ-AJ relationship does not completely disappear when
controlling for AA, this would strengthen the argument for
discounting the alternative scenario. Accordingly, we explored this possibility by testing the alternative scenario in
which EJ ⇒ AA ⇒ AJ. The relevant direct relationship
between EJ and AJ when controlling for AA is βEJ, AJ.AA =
.373 (t = 6.889, p < .001). This direct relationship meets
the reviewer’s criterion for discounting the alternative
scenario in favor of accepting Scenario 1.
The results just reported support Scenario 1 at the
expense of Scenarios 2, 3, and 4. Specifically, in accord
with Scenario 1, AA shows a moderately positive association with AJ, which in turn shows a moderately positive
association with EJ. In the case of our illustrative example
based on 200 performances of “MFV,” nonexpert consumers do tend to enjoy what they perceive to be excellent and
do tend to perceive as excellent what the experts judge to
be high in aesthetic worth. When combined multiplicatively, these two relationships produce a weak overall positive association between AA and EJ. This finding contradicts the assumption by Scenario 2 that people do not
recognize the merit of what is regarded by experts as excellent. It also contradicts Scenario 3’s assumption that people do not enjoy what they themselves regard as excellent.
Furthermore, contrary to Scenario 4, we do not find a negative link between EJ and AA when holding the intervening link via AJ constant. This absence of a canceling effect
of EJ on AA when controlling for AJ means that the overall relationship between EJ and AA is slightly more positive than found in previous studies, thereby supporting
Scenario 1. Apparently, in the illustrative case of “MFV,”
the recognition of aesthetic excellence (as assessed by the
expert judgments) is positively but weakly associated with
liking (audience appeal) via the intervening recognition of
aesthetic merit (audience judgments, as assessed by nonexpert consumers) so that rEJ, AA = .365 (rEJ, AA2 = .133).
We might speculate on the nature of the underlying process that accounts for the moderately positive relationship
between expert judgments and audience judgments. This
Downloaded from jam.sagepub.com at PENNSYLVANIA STATE UNIV on February 20, 2016
16
JOURNAL OF THE ACADEMY OF MARKETING SCIENCE
Nevertheless, as already noted, the correlational nature
of our study argues against the permissibility of causal
attributions. For this and other reasons, it follows that
future research should investigate experimental designs—
involving the controlled manipulation of aesthetic excellence, perhaps with the inclusion of a measure for expert
appeal—so as to establish a clear chain of causality from
expert judgments to audience appeal. Subject to the need
for such future refinements of methodology, the present
correlational findings in support of Scenario 1 suggest
some tentative managerial implications.
Managerial Implications
The sorts of managerial implications derived from the
research reported here differ considerably from those
stemming from the aforementioned studies of how film
reviews affect box-office scores—most obviously because
the present investigation focuses on recorded music rather
than films, but also because it deals with EJ rather than
with journalistic reviews. Specifically, in the present case,
it appears that aesthetic excellence in a musical performance (as judged by experts) contributes significantly but
weakly to AA (as assessed by nonexpert consumers). This
finding carries managerial implications both within the
context of cultural products in general or musical offerings
in particular and also for goods and services that fall outside the sphere of aesthetic experiences.
The first clear managerial implication is that improvements in the aesthetic merits of artistic offerings in general
and music in particular will produce increases in audience
appeal but that, because this relationship is fairly weak, the
quality improvements must be large to encourage sizable
gains in popularity. Perhaps this means that creative perfectionism has a hitherto untapped commercial value or,
put differently, that recording artists or producers of other
art forms should strive for a level of excellence that some
commentators feel has disappeared from the current musical scene or from other artistic venues because of considerations of cost and concessions to commercialism.
Second, because the link between AJ and AA represents an internal psychological process, it appears fruitless
for marketing managers to try to alter this relationship by
attempting to rearrange the psyches of consumers (who
will presumably continue to like what they perceive to be
excellent, but only to a moderate extent). Rather, it appears
more promising for marketers to attempt to strengthen the
relationship between EJ and AJ so as to leverage the indirect connection (via AJ) between EJ and AA. Such a
leveraging of the link between EJ and AJ might result from
various initiatives that fall under the potential control of
marketers.
As one approach, marketers could promote artistic offerings such as music in ways that explain how and why they
have prompted favorable responses from expert critics—
WINTER 2006
perhaps with an emphasis on specific aesthetic details that
support the validity of such claims. Obvious places to feature such information would include the liner notes on CD
packages, blurbs on book jackets, catalogs for museum
exhibitions, programs for theatrical performances, or the
advertisements that appear in the arts-related media—as in
a recent ad from DownBeat magazine (August 2004) that
quoted a critic to tout the merits of a new recording by the
saxophonist Andy Parsons called Flip!: “Parsons’ tunes
provide ample room for meaningful dialog and interaction
among the players who make accessible music out of challenging lines and changes” (p. 8).
Via a second approach, marketers could foster the
acceptance of such quality-related information by promoting the desirability of learning to appreciate the “finer
things in life” or, in other words, of strengthening the association between EJ and AJ. For example, Starbucks currently sells compilation CDs of material selected by
respected artists such as Ray Charles, Sheryl Crow, and
Tony Bennett. Theaters post reviews by professional critics in front of the box office. Along similar lines, record
stores might display a Top-10 shelf of most critically
acclaimed recordings in the vicinity of the rack containing
the Top-10 best-sellers. Book retailers might stock the
book-award prize winners next to the most popular selections. Such measures might encourage patrons to note the
differences between EJ and AA and to adjust their judgments of aesthetic excellence accordingly.
A third approach could encourage or even sponsor educational efforts intended to teach nonexpert consumers
about the critical standards for excellence in a particular
cultural field such as music. One illustration would be the
sorts of educational programs for children that Wynton
Marsalis conducts as part of his jazz activities at Lincoln
Center in New York City—reminiscent of Leonard
Bernstein’s Young People’s Concerts a half-century ago.
This viewpoint might sound altruistic until we stop to realize that education—in the form of what Bourdieu (1984)
called “cultural capital”—is the main tool a society has
to ensure that aesthetic excellence, rather than going unrewarded, will receive the commercial success that it so
richly deserves but all too seldom attains.
Such observations recall the motto of Sy Sims in the
retail-clothing business: “An educated consumer is our
best customer.” In this spirit, it appears clear that the findings of the present study also carry potential implications
for any number of marketing situations beyond the confines of the cultural context examined thus far and, indeed,
beyond the purely aesthetic aspects of the consumption
experience. For example, in the service industries, marketers need to ask questions about the extent to which objective manipulations of service quality as judged by experts
encourage increases in customer satisfaction or loyalty
and about the degree to which such effects are mediated by
consumers’ quality-related perceptual judgments. In the
Downloaded from jam.sagepub.com at PENNSYLVANIA STATE UNIV on February 20, 2016
Holbrook et al. / EXPERT JUDGMENT 17
case of consumer durables, producers might wonder about
the connection between independent quality ratings such
as those provided by (say) J. D. Power in the auto industry
and consumer judgments of excellence and, from there,
overall ratings of liking or satisfaction. Similarly, in the
marketing of nondurables, we might wonder how strongly
the quality ratings provided by (e.g.) Consumer Reports
explain consumer quality judgments and, from there, satisfaction or preference ratings. In all such situations—
analogous to our findings for the case of music in this
study—we might anticipate opportunities based on
strengthening the relationship between EJ (ratings of
excellence) and AJ (perceptions of quality), for example,
via advertisements featuring knowledgeable endorsements by well-known experts (as opposed to those by
celebrities who have no clear claim to product-related
expertise). (This is the potential difference between
Michael Jordan endorsing Nike shoes or Gatorade sports
drink as opposed to Hanes underwear or McDonald’s fast
food.)
Conclusion
We conclude that—in the illustrative case pursued here,
involving 200 recorded versions of “My Funny Valentine”
and in support of Scenario 1—nonexpert consumers do
recognize and enjoy excellence as assessed by professional experts, albeit to a limited degree in which AJ mediates the significantly positive but weak association
between AA and EJ. Clearly, such a finding requires generalization to other kinds of cultural objects in other sorts
of contexts and potentially to the broader range of consumer goods and services. To the extent that support for
Scenario 1 reappears in future research, it suggests viable
approaches to the more effective marketing of entertainment and the arts, as well as possible applications to other
kinds of products, perhaps even in ways that might prove
beneficial to society.
ACKNOWLEDGMENTS
The authors are listed in alphabetical order and contributed equally to this project. Morris B. Holbrook gratefully
acknowledges the support of the Columbia Business
School’s Faculty Research Fund. Michael S. LaTour gratefully acknowledges the financial support of the Torchmark
Discretionary Fund of the Auburn University College of
Business. Please address correspondence to Morris B.
Holbrook, Apt. 5H, 140 Riverside Drive, New York, NY
10024 (phone: 212-873-7324; e-mail: mbh3@
columbia.edu).
REFERENCES
Baron, Reuben M. and David A. Kenny. 1986. “The Moderator-Mediator
Variable Distinction in Social Psychological Research: Conceptual,
Strategic, and Statistical Considerations.” Journal of Personality and
Social Psychology 51 (December): 1173-1182.
Basuroy, Suman, Subimal Chatterjee, and S. Abraham Ravid. 2003.
“How Critical Are Critical Reviews? The Box Office Effects of Film
Critics, Star Power, and Budgets.” Journal of Marketing 67 (October): 103-117.
Bourdieu, Pierre. 1983. “The Field of Cultural Production, or: The Economic World Reversed.” Poetics 12 (November): 311-356.
⎯⎯⎯. 1984. Distinction: A Social Critique of the Judgement of Taste.
Cambridge, MA: Harvard University Press.
⎯⎯⎯. 1986. “The Production of Belief: Contribution to an Economy of
Symbolic Goods.” In Media, Culture and Society: A Critical Reader.
Eds. Richard Collins, James Curran, Nicholas Garnham, Paddy
Scannell, Philip Schlesinger, and Colin Sparks. London: Sage, 131163.
⎯⎯⎯. 1993. The Field of Cultural Production: Essays on Art and Literature. Ed. Randal Johnson. New York: Columbia University Press.
Brantlinger, Patrick. 1983. Bread & Circuses: Theories of Mass Culture
as Social Decay. Ithaca, NY: Cornell University Press.
DiMaggio, Paul, Michael Useem, and Paula Brown. 1978. Audience
Studies of the Performing Arts and Museums. Washington, DC: National Endowment for the Arts.
Eliashberg, Jehoshua and Steven M. Shugan. 1997. “Film Critics: Influencers or Predictors?” Journal of Marketing 61 (April): 68-78.
Erickson, Bonnie H. 1996. “Culture, Class, and Connections.” American
Journal of Sociology 102 (July): 217-251.
Gans, Herbert J. 1974. Popular Culture and High Culture: An Analysis
and Evaluation of Taste. New York: Basic Books.
Gartman, David. 1991. “Culture as Class Symbolization or Mass Reification? A Critique of Bourdieu’s Distinction.” American Journal of
Sociology 97 (September): 421-447.
Henry, William A. III. 1994. In Defense of Elitism. New York:
Doubleday.
Holbrook, Morris B. 1999. “Popular Appeal Versus Expert Judgments of
Motion Pictures.” Journal of Consumer Research 26 (September):
144-155.
⎯⎯⎯ and Rajeev Batra. 1987. “Assessing the Role of Emotions as
Mediators of Consumer Responses to Advertising.” Journal of Consumer Research 14 (December): 404-420.
Jackson, Sally. 1992. Message Effects Research. New York: Guilford.
Levine, Lawrence W. 1988. Highbrow/Lowbrow: The Emergence of Cultural Hierarchy in America. Cambridge, MA: Harvard University
Press.
Nunnally, Jum C. 1967. Psychometric Theory. New York: McGraw-Hill.
Olney, Thomas J., Morris B. Holbrook, and Rajeev Batra. 1991. “Consumer Responses to Advertising: The Effects of Ad Content, Emotions, and Attitude Toward the Ad on Viewing Time.” Journal of
Consumer Research 17 (March): 440-453.
Pedhazur, Elazar J. 1982. Multiple Regression in Behavioral Research:
Explanation and Prediction. 2d ed. New York: Holt, Rinehart &
Winston.
Ravid, S. Abraham. 1999. “Information, Blockbusters, and Stars: A
Study of the Film Industry.” Journal of Business 72 (4): 463-492.
Schindler, Robert M., Morris B. Holbrook, and Eric A. Greenleaf. 1989.
“Using Connoisseurs to Predict Mass Tastes.” Marketing Letters 1
(1): 47-54.
Shrum, Wesley. 1991. “Critics and Publics: Cultural Mediation in Highbrow and Popular Performing Arts.” American Journal of Sociology
97 (September): 347-375.
Stern, Jane and Michael Stern. 1990. The Encyclopedia of Bad Taste.
New York: HarperCollins.
Wallace, W. Timothy, Alan Seigerman, and Morris B. Holbrook. 1993.
“The Role of Actors and Actresses in the Success of Films: How
Much Is a Movie Star Worth?” Journal of Cultural Economics 17
(June): 1-27.
Downloaded from jam.sagepub.com at PENNSYLVANIA STATE UNIV on February 20, 2016
18
JOURNAL OF THE ACADEMY OF MARKETING SCIENCE
ABOUT THE AUTHORS
Morris B. Holbrook ([email protected]) is the Dillard Professor of Marketing, Graduate School of Business, Columbia
University, New York. Holbrook graduated from Harvard College with a B.A. in English (1965) and received his M.B.A.
(1967) and Ph.D. (1975) in marketing from the Columbia Business School where, since 1975, he has taught courses in Marketing Strategy, Research Methods, Consumer Behavior, and
Commercial Communication in the Culture of Consumption. His
research has covered a wide variety of topics with a special focus
on communication in general and on aesthetics, semiotics, hermeneutics, art, entertainment, music, motion pictures, nostalgia,
and stereography in particular.
Kathleen T. Lacher ([email protected]) lives in Tallahassee, Florida, where she has a consulting business. She received
her B.M.E. in choral music (1978) and her Ph.D. in business
administration—marketing (1991), both from Florida State Uni-
WINTER 2006
versity. She taught at Auburn University, Auburn, Alabama, and
Georgia Southwestern State University, Americus, teaching
courses in Consumer Behavior, Research Methods, and Strategy.
Her research covers consumer behavior, using both quantitative
and qualitative methods. She performs with the Tallahassee
Community Chorus, which debuted at Carnegie Hall in 2004,
and holds the position of Secretary for the Board of Directors at
the Tallahassee Habitat for Humanity.
Michael S. LaTour ([email protected]) is a professor of
marketing and chair, Department of Marketing, College of Business, University of Nevada, Las Vegas. He earned his Ph.D.
(1986) in business administration from the University of Mississippi with a major in marketing. He graduated with multiple honors. His research has covered a variety of topics including
psychophysiological response to promotional stimuli, gender issues in advertising, advertising ethics, cross-cultural consumer
behavior, industrial buyer behavior, and consumer memory of
advertising stimuli and product experience.
Downloaded from jam.sagepub.com at PENNSYLVANIA STATE UNIV on February 20, 2016