The impact of setting on wine tasting experiments: Is the
process of wine tasting inherently flawed?
American Association of Wine Economists
10th Annual Conference
June 21 - 25, 2016
Bordeaux, France
Geoffrey Lewis
Professorial Fellow, Melbourne Business School
[email protected]
Steve Charters MW
Professor & Head of Research, School of Wine & Spirits Business
Burgundy School of Business
[email protected]
Benoît Lecat
Wine and Viticulture Department Head
California Polytechnic State University
[email protected]
1
Prior Research
This research is an extension of work done in Melbourne (Lewis & Zalan,
2014) and in Dijon (Lewis, Lecat & Zalan, 2015) where we demonstrated
that 'objective characteristics' (brand, label and price) overwhelm subjective
appreciation of the wine.
These experiments, following earlier work by Plassman (Plassman et al.,
2008), involved manipulation – the tasters were presented with 5 wines, but
in fact only 3 wines were involved, two of the wines being presented twice.
With the Dijon research the manipulation went to the extent of re-labelling
two of the wines, with the tasters pouring the wine from the mis-labelled
bottles.
The earlier study was conducted at the School of Wine & Spirit Business
(Groupe ESC Dijon Bougogne) in 2014 with two experimental groups:
1.
English-language program students in the Master of Wine Business and
Master of Wine Management programs (n=31), and
2.
Students in the French-language program CIVS (Commerce International
des Vins et Spiritueux), the leading post-graduate wine business program in
France (n=27)
2
2014 Research Design
The experiment was structured in two rounds:
Round 1 - Can you identify the Village Appellation?
The participants were asked to identify the Village appellation of three red
Burgundies, indicating their Rating (scale 1-6) and Willingness to Pay.
This round was conducted to get participants used to the process and
to make them less likely to suspect experimental manipulation.
Round 2 - The Burgundy Appellation Challenge
The participants were presented with five red Burgundy wines from the
same producer and vintage and were asked to assess the classification
(Grand Cru, Premier Cru, Village Cru, Bourgogne Regional Appellation) of
the wines.
The question was posed whether some classifications were over/underrated or over/under-priced and how they valued these wines based on
quality, regardless of the classification.
They were again asked to rate the wines (on a scale of 1-6) and indicate
their WTP.
3
Round 2 – Presented and True Prices
Wine
Presented Price
(on the tasting sheet)
1
€48 Grand Cru
2
€31 Premier Cru
3
€31 Premier Cru
4
€20 Village Appellation
5
€8 Regional Appellation
4
Round 2 – Presented and True Prices
Wine
Presented Price
(on the tasting sheet)
1
€48 Grand Cru
2
€31 Premier Cru
3
€31 Premier Cru
4
€20 Village Appellation
5
€8 Regional Appellation
True Price
(Estate price)
€48 Grand Cru
€31 Premier Cru
€8 Regional Appellation
5
Round 2 – Presented and True Prices
Wine
Presented Price
(on the tasting sheet)
True Price
(Estate price)
1
€48 Grand Cru
€48 Grand Cru
2
€31 Premier Cru
€8 Regional Appellation
3
€31 Premier Cru
€31 Premier Cru
4
€20 Village Appellation
€48 Grand Cru
5
€8 Regional Appellation
€8 Regional Appellation
We anchored Wines 1, 3 and 5, and manipulated the price of Wines 2 and 4, unlike the Australian
experiment, where we anchored Wines 2, 3 and 4, and manipulated the price of Wines 1 and 5.
6
Group 1: (English Language Program) Results
(n=31)
Presented Price
WTP
Rating
60.00
6.00
50.00
5.00
40.00
4.00
30.00
3.00
20.00
2.00
10.00
1.00
0.00
Rating (1-6)
Euros per bottle
True Price
0.00
1
2
3
4
5
Wines
7
Group 2: (French Language Program) Results
(n=27)
Presented Price
WTP
Rating
60.00
6.00
50.00
5.00
40.00
4.00
30.00
3.00
20.00
2.00
10.00
1.00
0.00
0.00
1
2
3
4
Rating (1 - 6)
Euros per bottle
True Price
5
Wines
8
Results from the 2014 Dijon experiment
The results from the 2014 experiments confirmed that, even with a group of
experienced tasters (WSET 2/3), objective characteristics overwhelmed
subjective assessment of the wine. Ratings and Willingness to Pay were
driven by the appellation, labelling and price of the wines.
The largely French-speaking CIVS students' assessment (Ratings) of the
wines was more accurate than the other students, but their Willingness to
Pay did not fully* reflect their Ratings. Hence, the 2015 AAWE conference
paper was entitled:
"Do the French have better palates … but no
better sense of value?"
* Their WTP for Wine 4 was higher than Wines 2 & 3, but not as high as for Wine 1 (even though they Rated
Wine 4 higher than Wine 1).
9
Repeating the Experiment in a 'Natural Setting'
In 2014 the experiment was conducted at ESC Dijon, School of Wine & Spirits
Business in the typical WSET-style wine-tasting that the tasters were familiar
with.
In a letter to the Journal of Wine Economics, commenting on the work of
Plassman et al., Jeffrey Postman, a New York cardiologist, offered the
following observation:
Some wines are much better than others, but the satisfaction one gets
from them is so much more nuanced in a social setting than in a blind tasting,
the latter is but a pale shadow of the former. The message that I take home
from the Plassman experiment is that blind tasting has little to do with the real
life experience of tasting wine.
Based on Postman's challenge, we hypothesized that we would get quite
different results if we replicated the 2014 Dijon experiments in a 'natural
setting'. To that end we conducted an experiment with the same group of
tasters – the class of 2016 instead of the 2014 cohort – using exactly the same
experimental structure, but under the guise of a 'French Food & Wine
Experience' where the students tasted the wines while enjoying a meal and
considering the question of how the wines matched with the food.
10
Repeating the Experiment in a 'Natural Setting'
The experiment was conducted at a well-respected Dijon restaurant, Dame
d'Aquitaine and presented to the students as a "French Food and Wine
Experience" to celebrate the completion of their course.
One luncheon was held for English-speaking students and one, a week later,
with the French-speaking students. The invitation indicated we would be
exploring how wine and food went together.
Students arrived at the restaurant and seated themselves at tables of 6-8. At
each table there was a member of the ESC Dijon faculty, ostensively to act as
'host', but in fact to act as a participant-observer. The participant-observers
were briefed prior to the experiment and were informed that some
manipulation was involved, but they were not informed of the details.
To avoid participant-observers influencing the student responses to the wine,
during the introduction to the luncheon, students were told that we had
specifically asked the faculty members not to share their opinions about the
wines.
11
Repeating the Experiment in a 'Natural Setting'
As in the 2014 study, the experiment was structured in two rounds:
Round 1 – Can you identify the White Wine Variety?
Three wine wines (Sancerre, Bourgogne and Alsace) were served with the entrée.
The participants were asked to identify the region of three white wines, indicate their
Rating of the wines (scale 1-6) and their Willingness to Pay.
This round was conducted to get participants used to the process and to make
them less likely to suspect experimental manipulation.
This round was debriefed and a prize was awarded to the student whose WTP was
closest to the retail prices of the wines.
Round 2 – The Burgundy Appellation Challenge
The participants were presented with five red Burgundy wines from the same
producer and vintage (ranging from Grand Cru to Bourgogne Regional Appellation)
and were asked to assess how the wines matched with the main course.
The question was posed whether some classifications were over/under-rated
or over/under-priced and how they valued these wines based on quality,
regardless of the classification.
They were again asked to rate the wines (on a scale of 1-6) and indicate their WTP.
12
Structure of the Tasting
Presented Price
70.00
Echezeaux Grand Cru 61€
60.00
Price (Euros)
50.00
Nuits-Saint-Georges 1er Cru
Les Boudots 36€
40.00
Nuits-Saint-Georges 1er Cru
Les Damodes 36€
30.00
Vosne Romanée
Village Appellation 28€
20.00
10.00
Bourgogne Regional
Appellation 9€
0.00
0
1
2
3
Wines
4
5
6
13
2016 Dijon Results – Ratings and True Price
True Price
Presented Price
Rating-French
Rating-Non French
70.00
6.00
60.00
5.00
4.00
40.00
3.00
30.00
Rating (1-6)
Price & WTP (Euros)
50.00
2.00
20.00
1.00
10.00
0.00
0.00
0
1
2
3
Wines
4
5
6
14
2016 Dijon Results – WTP and True Price
True Price
Presented Price
WTP-French
WTP-Non French
70.00
60.00
Price & WTP (Euros)
50.00
40.00
30.00
20.00
10.00
0.00
0
1
2
3
Wines
4
5
6
15
Conclusions
1. The 'natural wine consumption setting' of a restaurant meal enjoyed with
friends did nothing to reduce the impact of objective wine characteristics.
The tasters did not detect the manipulation, even though the similarity of
some of the wines were discussed around some of the tables.
2. Again, the French tasters up-rated the Village Appellation (actually the
Grand Cru), but not to the extent the tasters in the 2014 experiment did,
who actually rated the Village Appellation more highly than the Grand Cru.
3. This result suggests (subject to further statistical analysis), contrary to our
expectation, that a formal WSET-style setting may be a better setting for
assessing wines that a 'natural wine consumption setting'.
4. But in both settings objective characteristics (appellation, label & price)
overwhelmed the tasters' subjective assessment of the wines.
The question remains as to whether our experimental design
accurately tested Postman's proposition and this offers
possibilities for further research.
16
Thank you!
Discussion
17