Stimulus Materials
Step 1: The development of the written scenarios for the pre-test study
With reference to the original research by Haidt and colleagues and the applied research on
media by Tamborini and colleagues, our research team developed six written scenarios
designed to represent morality. Five of the scenarios were written to violate each of MFT’s
five moral domains (harm/care, fairness/reciprocity, authority/resect, ingroup/loyalty, and
purity/sanctity) and one scenario was used as a control / foil scenario.
The scenarios were constructed in a discursive process by the principal researchers, coming
from different cultural backgrounds to allow each scenario to be applicable in German and
English language. Scenarios were then discussed with fellow researchers not involved in the
study. For each of the five moral scenarios, we focused on elements that had previously been
discussed in MFT research such as addressing fairness with a distribution of benefits problem
or addressing issues of loyalty with a family conflict. Each scenario was created with the
intention to be used in a virtual world.
Each scenario was constructed in a similar pattern, using a short story of approximately 500
characters. To control for potential gender effects, each moral scenario’s main character was
male. For the foil scenario we used a female character. We used popular, everyday names for
the protagonists of our stories such as “Christian Reynolds” (English) or “Christian Huber”
(German). Names were adjusted for the country they were used.
Step 2: Testing the written scenarios
In order to test in how far our people recognized the (potential) violation of each of the five
moral modules in our scenarios, we carried out a pre-test among a convenient sample of
college students (N = 328 for Germany and N = 148 for the US). Participants took part in the
story voluntarily and were gathered from communication courses from two countries. Sample
characteristics were similar (Mage= 23.2 (SD= 3.0), 76% female, n =323, USA: Mage= 22.5,
SD= 5.6, 65% female, n= 125). This pre-test was part of a larger study on media preferences.
We relied on an online questionnaire that contained the six written scenarios. Questionnaires
were identical for both languages.
2
The introduction of the scenario testing reads as follows (English version):
In the following section of the survey, you will read six short stories. In each of the stories, the
main character behaves in a way that some people might believe is morally questionable.
After reading each story, we will ask you to give a very brief evaluation of the main
characters.
For each statement, respond by telling us your level of agreement: "absolutely agree" (1) to
"absolutely disagree" (4)
He has treated another person unfairly (fairness/ reciprocity)1
He has done something disgusting (purity/ sanctity)
He has caused another person physical harm (harm/ care)
He acted disrespectful to an authority figure (authority/ respect)
He has betrayed his family and/or friends (in-group/ loyalty)
Results for each of the scenarios are presented below in the order used in the questionnaire
and were later recommended as the order the participants in the experimental study should
take. Scenarios are presented in English – German versions are available on request. We
provide: a) the descriptive statistics for each of the five moral modules. We marked the
statistics for the relevant scenario in bold. We then carried out a paired t-test comparing the
means of the recognition of the intended module with the module that had the next highest
recognition score. In order to accept a scenario we required the recognition score to be the
lowest (= highest recognition) and to be significantly different from the next lowest
recognition score (= next highest recognition).
We carried out each analysis separately for the samples in the two countries. We did not carry
out any analysis for the foil scenario. Here, we expected no recognition of any moral module.
Descriptive figures already indicate that this was successful.
Scenario „Harm/ Care“
"The Angry Bartender"
Christian Reynolds is a bartender at a small pub. Two of his customers are arguing angrily
with each other, which has caused other customers to feel uncomfortable and leave the pub.
Although Mr. Reynolds has asked the two several times to calm down, they continue to argue.
During their dispute a chair is broken, and for Mr. Reynolds, this is too much. Angrily, he
throws a bottle at the men, striking one of them on the shoulder, and brutally throws them out
of the pub.
Module
Authority/ Respect
Fairness / Reciprocity
Harm/ Care
In-group/ Loyalty
Purity/ Sanctity
1
Germany
3.63 (.62)
2.65 (.85)
1.33 (0.55)
3.84 (.44)
3.48 (.78)
Mean (SD)
US
3.22 (.92)
2.64 (.86)
1.57 (.78)
3.39 (.77)
3.05 (.99)
Remarks in parenthesis added to emphasize the moral module that each response item
corresponded to.
3
T-Test
Germany (H/C vs. F/R): T = -27.05, df = 311, p <.001
US (H/C vs. F/R): T = -12.21, df = 123, p <.001
Scenario “Authority/ Respect”
"The Old House"
Lucas Williams works for a demolition company in a small town. He receives an order from
the mayor of the town to demolish an old, dilapidated building. When Mr. Williams asks why
the building is to be demolished, he gets no answer. He is only told that the mayor wishes it to
be destroyed. However, Mr. Williams knows the building well from his childhood and knows
of a citizen's initiative to try and save it from demolition. Against the mayor's wishes, Mr.
Williams deliberately delays the building demolition to give the citizen's group time to save it.
Please rate Mr. Williams' behavior by responding to each of the following statements.
Setting (visual template) in the simulation (experimental study): town center
Module
Authority/ Respect
Fairness / Reciprocity
Harm/ Care
In-group/ Loyalty
Purity/ Sanctity
Germany
2.39 (.78)
3.28 (.80)
3.96 (.27)
3.65 (.68)
3.96 (.19)
Mean (SD)
US
2.22 (.86)
3.06 (.90)
3.70 (.71)
3.56 (.78)
3.66 (.71)
T-Test
Germany (A/R vs. F/R): T = -15.63, df = 300, p <.001
US (A/R vs. F/R): T = -8.26, df = 110, p <.001
Scenario “Fairness/ Reciprocity”
"In the Orchard"
Mr. Meyer owns a large apple orchard on the outskirts of a small town. One day he asks two
neighbor children - William and Bernard - to help him with the harvest. He made an
agreement with the boys to pay them each $10 to help him gather up apples, and both
children quickly went to work. As the boys are working, Bernard - because he is taller than
William - is able to pick many more apples because he can reach the higher branches on the
orchard trees. At the end of the day, Bernard has picked twice as many apples as William,
and Mr. Meyer pays Bernard $15 for his work. For William, he tells the boy that he has no
more money left over, so the boy gets nothing.
Module
Authority/ Respect
Fairness / Reciprocity
Harm/ Care
In-group/ Loyalty
Purity/ Sanctity
Germany
3.55 (.77)
1.05 (.25)
3.27 (1.09)
2.99 (1.07)
2.89 (1.07)
Mean (SD)
US
3.23 (1.03)
1.24 (.64)
3.47 (.85)
2.18 (1.07)
2.39 (1.09)
4
T-Test
Germany (F/R vs. P/S): T = -29.59, df = 302, p <.001
US (F/R vs. I/L): T = -7.50, df = 90, p <.001
Scenario “Foil”
"The Wet Book”
When she was in college, Sarah Schultz used to study with a famous professor who is going to
be retiring in a few days. To celebrate his retirement, Sarah decides to talk to some former
classmates and prepare a scrapbook of the professor. While walking to a friend’s house,
Sarah trips over a wet curb and the book falls in a puddle of mud. The book is nearly
destroyed before Sarah and her friend are able to dry the pages and continue working on the
project.
Module
Authority/ Respect
Fairness / Reciprocity
Harm/ Care
In-group/ Loyalty
Purity/ Sanctity
Germany
3.93 (.31)
3.94 (.30)
3.87 (.44)
3.88 (.44)
3.94 (.30)
Mean (SD)
US
3.73 (.64)
3.74 (.63)
3.74 (.63)
3.60 (.82)
3.72 (.61)
Scenario “In-group/ loyalty”
"The Family Business"
John Smith and his brother Martin run a family business together that was founded by their
elderly father. John handles the accounting for the business, while Martin handles customer
service. Recently, John has been offered a new job overseas, but it would require him to leave
his family - and his brother Martin - behind. John knows that Martin cannot run the family
business by himself, but John decides that the opportunity to work overseas is too great to
pass up. Despite Martin's wishes, John leaves the family business, and Martin is forced to
close it down.
Module
Authority/ Respect
Fairness / Reciprocity
Harm/ Care
In-group/ Loyalty
Purity/ Sanctity
Germany
3.30 (.85)
2.28 (.94)
3.26 (1.07)
1.77 (.72)
3.39 (.77)
Mean (SD)
US
3.23 (.81)
2.26 (.81)
3.20 (1.05)
1.88 (.71)
3.10 (.91)
T-Test
Germany (I/L vs. F/R): T = -10.73, df = 305, p <.001
US (I/L vs. F/R): T = -6.03, df = 117, p <.001
Scenario “Purity/ Sanctity”
"A Bath in the Mud"
Mark Sanders is a teacher in the town high school. Outside of town, there is an old swamp
that is rumored to have a special mud that is very relaxing to bathe in. However, because the
5
swamp is polluted (it sits near an open sewage drain), few people will actually go near it. But
Mark has had a very exhausting week at the high school and just wants to feel good. One
afternoon after school he visits the swamp, undresses and climbs into the putrid swamp mud.
Module
Authority/ Respect
Fairness / Reciprocity
Harm/ Care
In-group/ Loyalty
Purity/ Sanctity
Germany
3.83 (.49)
3.99 (.11)
3.98 (.18)
3.96 (.24)
3.03 (.95)
Mean (SD)
US
3.51 (.86)
3.63 (.79)
3.59 (.80)
3.63 (.74)
1.93 (.85)
T-Test
Germany (P/S vs. A/R): T = -14.39, df = 313, p <.001
US (P/S vs. A/R): T = -10.10, df = 66, p <.001
Summary of pre-tests
Participants of the pre-test study recognized that the character in our story had violated the
intended moral module more strongly than they recognized a violation for a moral module
whose violation was not intended. This finding was found for all cases in both countries.
Notably we were not concerned with how strongly each of the scenarios violated a moral
module, as there is no objective measure of this. According to MFT the recognition of a moral
module violation depends on the fact that this moral module is salient, only if a salient module
is addressed should people positively identify a behavior as a violation of this module. For
instance, if purity/ sanctity is not important to you, you will be less likely to recognize a
certain behavior as a moral violation – as behaving unnaturally for instance – compared to
someone for whom purity/ sanctity is more important. For example, this can – at least on a
descriptive level – be seen for the purity/ sanctity scenario. Based on the recognition score in
the German sample (M = 3.03, SD = .95) one could argue that our manipulation was not
successful. However, in the American sample, the recognition score is much better (M = 1.93,
SD = .85) and even lower than the scales mean of 2.52. Additionally, even Germans
significantly recognized the scenario to violate purity/ sanctity more than any other of the four
remaining modules. This may be explained by the fact that Americans in our sample we much
more salient in the purity/ sanctity dimension than Germans (T = 8.12, df = 209.66, p <.001)
and might thus be more likely to recognize the scenario as a violation, whereas the Germans
only recognized that it is more likely a violation of purity than a violation of any other
module.
Step 3: Programming the content
The six scenarios from the pre-test were used as a script for creating a virtual world.
This was done using the AURORA® toolset of the computer game Neverwinter Nights. The
AURORA® toolset allowed us to generate a medieval-looking virtual world environment
2
One could indeed argue that one way to test the successful violation of a moral module would be to test the
score indicated by the participants against the median score of 2.5 (for a 1to 4 scale). However, as there is no
absolute score for a violation, one can only make a comparison with the other modules.
6
comparable to popular role-playing games such as World of Warcraft and Runescape. All
game characters – both avatars for the players and non-player-characters (NPC) – were human
characters from the game’s built-in character library. A game narrative world was created in
which participants were told to visit six former students of a professor in order to gather up
photo albums for a party in the professor’s honor. The six students represented the six NPC
episodes, one for each moral scenario including the foil scenario. For both languages,
identical versions translated by the research team were employed. Programming was carried
out by the same research team that developed the scenarios.
Each scenario consisted of the player making contact with one of the NPCs and
initiating a dialog with them. This dialog told the story presented in the written scenarios with
the NPC acting as the protagonist. The dialogues were structured in the same way for each of
the scenario and contained a greeting section, a description of the problem, and one choice
option by the player on what to say. Regardless which option was chosen, the conversation
continued the same way, until the player was confronted with the final decision. At the end of
each dialogue, participants could instruct the NPC either to violate a moral module by telling
the character to behave as described in the pre-tested scenarios or they could chose to tell the
NPC not to violate the relevant moral module. In the intervention condition, the NPC declared
that he would act as indicated by the player. In the no intervention condition, the NPC always
declared to commit a violation regardless of the player’s suggestion. Participants’
conversations and instructions to the NPC were logged covertly using the game’s inventory
function.
NPCs were presented in-game with an environment and appearance that would fit
their stories. For instance: In the harm/care scenario players had to visit a tavern that was
programmed using the built-in library for a town-house setting. A room was constructed that
contained tables, chairs, large oak barrels and a bar with bottles. The two arguing customers
were wandering around broken furniture. The player could interact with both characters but if
they interacted with the non-target NPC – one of the background characters – they were only
given a short reply (“Leave me alone”); this directed the player to interact with the target NPC
to begin the scenario conversation. For the purity/scenario a swamp was programmed that
included animated rats and rotting animal corpses along with the target NPC. For the fairness/
reciprocity scenario the two children were standing in an orchard, greeting the player. While
the children gave short replies, dialogue was only possible with the NPC protagonist. Sample
pictures of the game design and screenshots are added here:
Sample Pictures
7
The toolbox surface using showing the game map, showing one map area. The
fairness/reciprocity plays in the centre left quadrant, the foil scenario plays in the centre, the
swamp for the purity/ sanctity scenario is visual in the top right corner. The ingroup/ loyalty
scenario plays inside the house in the lower right corner.
The introduction, tutorial with a female avatar. Setting: interior
Scene from the fairness/ reciprocity scenario using a male avatar. NPS is in the back (dark
figure). Children and orchard visible centre right. Dialogue box with two choices in the left
upper corner. Setting: Outside.
8
Harm/ care scenario using a female avatar. NPC to the left, arguing customers to the right.
Setting: Interior
Purity/ sanctity scenario using a female avatar. NPC in front of swamp, dialogue box in the
upper left corner. Setting: Outside.
Limitations of our stimulus materials
We have to acknowledge that using the AURORA© toolset set presented certain restrictions.
For instance, we were not able to program the visual results of each violation, but rather we
could only control dialogue regarding each. The players in the game could not see, for
example, that in the harm/care scenario a character was hit by a bottle or in the purity/sanctity
scenario that the NPC actually went in the swamp; rather, they were informed that this would
be the case through the game’s dialogue and the NPC’s stated intensions. This might have
made the violations less obtrusive but were due to limited programming capabilities, however
we point out that similar video games often use dialogue as the base for narrative.
Additionally, while programming the game our research team had to mitigate between
internal and external validity of the study. For example, we used character archetypes from
the gaming library, and we gave each character a stylized portrait not revealing any further
details (see pictures above in the dialogue boxes). While this kept player avatars consistent
9
across the sample, it does not allow us to examine for the effects of character identification or
appearance on the behaviors in (or enjoyment of) the video game. Also, in order to keep the
gaming simulation enjoyable, we abstained from using only one character model but used
character models that fitted the relevant story which introduces potential character effects.
Nonetheless, we stress the fact that by programming our stimulus material we could control
more circumstances than usual in related research, even if not all elements were kept constant
(settings, character appearance etc.).