Do presupposition triggers influence the felicity of voice mismatched
sentences?
Margreet Vogelzang
Bachelor’s thesis
s1648497
[email protected]
Artificial Intelligence department
School of Computing and Cognition
University of Groningen
Advisor: Jennifer Spenader
something which also happened before, and is
mentioned in the first part of the sentence. The presuppiosition trigger therefore makes an explicit link
between the clauses and so the relationship between
the two parts of the sentence is emphasized.
Little research has been done however on the influence of presupposition triggers on different types
of sentences. An experiment of Arregui et al. (2006)
shows that sentences containing verb phrase ellipsis
with ‘flawed’ antecedents can be improved by adding
a presupposition trigger.
Different presupposition triggers are appropriate for
different types of sentences. The type of relationship
between two clauses is called the type of coherence
relation or the coherence type. Hume (1748) distinguishes three types of coherence relations, naming
resemblance, cause-effect and contiguity in time or place.
Sentences (1) and (2) are both sentences with a resemblance relation, since there are similar events described
in both clauses of the sentence. In this experiment
only two of the coherence relations will be taken into
account; the resemblance and the cause-effect relation.
Examples of these coherence relations are stated in
sentence (3)-(6) [(3) and (4) as given by Kehler et al.
(2008) p.12, example (11)].
Abstract
Kehler (2002) claims that in sentences with resemblance relations parallel syntactic structures are preferred, while for cause-effect relations this is irrelevant,
thus voice mismatches are dispreferred in resemblance
relations. Adding a presupposition trigger like too might
help repair the damage made by the mismatch, making
such sentences more felicitous.
In a sentences judgment task experiment, we asked
Dutch speakers to judge the felicity of resemblance and
cause-effect sentences with and without syntactic voice
mismatches. An additional factor is the presence or absence of the presupposition triggers too (ook) in resemblance relations and toch in cause-effect relations.
The results show that adding a presupposition trigger
doesn’t affect matched resemblance sentences, but significantly improves mismatched resemblance sentences.
Adding a presupposition trigger also doesn’t significantly improve matched or mismatched cause-effect sentences. Both resemblance and cause-effect sentences are
not significantly worsened by mismatch, so our results
can’t be said to support Kehler’s theory for Dutch sentences.
Keywords: presupposition triggers, anaphors, coherence relations, pronoun interpretation
1
Introduction
Presupposition triggers are words that trigger an
implicit assumption about the world, generally treated
as background information, and whose meaning is
unaffected by logical operators. Examples are words
like too and again; an example senctence is shown in (1).
(3) Samuel threatened Justin with a knife, and
Erin blindfolded him. [resemblance]
(4) Samuel threatened Justin with a knife, and Erin
stopped him. [cause-effect]
(5) Justin was threatened by Samuel with a knife, and
Erin blindfolded him. [resemblance]
(6) Justin was threatened by Samuel with a knife, and
Erin stopped him. [cause-effect]
(1) John got cake with his coffee again.
In this sentence, the presupposition trigger again
gives you the presupposition that at some time in
the past John got cake with his coffee too, at least
once. Some presupposition triggers can be bound
anaphorically, like again in sentence (2).
Noticable is the use of a pronoun in all the examples. The pronoun in the second clause of each
sentence refers to a noun in the first clause. The
pronoun is therefore another way to make sure the
reader links the second clause back to the first one; to
know what the pronouns antecedent is.
Theoretically, the pronouns in sentence (3)-(6) could
(2) John got cake with his coffee on Thursday
and he got cake with his coffee again today.
In this case, the second part of the sentence states
1
refer to both Samuel and Justin, as they are both male.
People have a certain way of interpreting the pronoun
in the different sentences though, that depends on a
variety of features of the sentence, e.g. the grammatical
role of the pronoun, the coherence relation between the
two clauses or the context in which the sentence places
the pronoun.
Kehler (2002) and Kehler et al. (2008) have argued
that the type of coherence relation between two clauses
influences pronoun interpretation. They claim pronouns in resemblance relations such as (3) have different interpretation preferences than pronouns in causeeffect relations like (4).
Simple rules to explain this difference in interpretation preference are not easily formulated. E.g., a possible heuristic is to prefer to interpret pronouns as having an antecedent of the same grammatical role as the
pronoun. But this heuristic can’t explain the difference
in interpretation as the pronoun carries the same grammatical role in sentence (3) and (4), but in sentence
(3) the pronoun refers to the object of the first clause
(Justin) while in (4) it refers to the subject (Samuel).
According to Kehler et al. (2008), simple strategies
for pronoun interpretation that say e.g. object pronouns prefer object antecedents, work for resemblance
relations like (3), but are irrelevant for cause-effect
relations like (4), and so preferences for pronoun interpretation can be changed by changing the coherence
relation (confirmed by Wolf et al. (2004)). In sentences
with a resemblance relation a parallel structure is preferred, and that explains why mismatched sentences
like (5), in which the voice of the two clauses doesn’t
match, sound less felicitous than sentences like (3), in
which both clauses are active. Also, a more felicitous
sentence than (5) would be (7), because both clauses
are passive.
effect sentences, as both Kehler et al. (2008) and Frazier
and Clifton Jr. (2006) didn’t use a presupposition trigger.
As mentioned before, Arregui et al. (2006) found
that sentences containing verb phrase ellipsis with
‘flawed’ antecedents are interpreted as better when
a presupposition trigger is present. Also, Hendriks
(2004) suggested that the presence of a presupposition
trigger like too might help repair the damage made
by the voice mismatch, making such sentences more
felicitous. The presence of a presupposition trigger
could give the hearer an alternative way to recognize
how the events occuring in the clauses are linked. An
example of a mismatched sentence with a presupposition trigger is (8).
(8) Justin was threatened by Samuel with a knife,
and Erin blindfolded him too. [resemblance]
In a sentence judgment task experiment, we ask
Dutch speakers to judge the felicity of resemblance
and cause-effect sentences with and without voice
mismatches. An additional factor is the presence or
absence of the presupposition trigger too (in Dutch ook)
and the Dutch presupposition trigger toch. Testing in
Dutch is particularly useful because the Dutch trigger
toch marks denial of expectation, a type of cause-effect
relation, and a presupposition trigger to mark such a
relation doesn’t exist in English.
Like the sentences used in the experiment by Kehler
et al. (2008) and Frazier and Clifton Jr. (2006) our test
sentences contain an anaphoric device in the second
clause to force the reader to make an explicit link between the two clauses (Frazier and Clifton Jr. (2006)
used verb phrase ellipsis, Kehler et al. (2008) used pronouns). Our sentences contain an object pronoun so
we can study how the presence of the presupposition
trigger influences the pronoun interpretation and the
felicity of the sentence.
In this way we can test if, as Frazier and Clifton Jr.
(2006) found, mismatched sentences are infelicitious
regardless of the type of coherence realtion, or if, as
Kehler et al. (2008) state, only the mismatched resemblance sentences are infelicitious and mismatching
doesn’t influence the felicity of cause-effect sentences.
We can also check if presupposition triggers can repair
the damage done by the mismatch.
Our prediction is that the presence of the presupposition trigger will improve the mismatched sentences
because it offers listeners an alternative means for interpreting the coherence relation. Also, our expectation is that mismatch will degrade sentences with resemblance relations but will not affect sentences with
cause-effect relations, because only in sentences with
resemblance relations a parallel structure is preferred.
(7) Justin was threatened by Samuel with a knife,
and he was blindfolded by Erin. [resemblance]
An explanation for the loss in felicity in sentences
with a resemblance relation when there is no parallel
syntactic structure could be that it makes the anaphor
interpretation difficult because simple strategies are no
longer possible.
In a study testing Kehler’s theories with verb phrase
ellipsis, Frazier and Clifton Jr. (2006) unexpectedly
found that people disliked syntactic mismatches regardless of whether they appeared in resemblance or
cause-effect sentences. The difference between these
experiments was, as noted by Frazier and Clifton Jr.
(2006), that their sentences with a resemblance relation didn’t have the presupposition trigger too in them,
while most of Kehlers sentences did. This however
would not explain the difference in results in the cause2
2 Experiment 1:
Pronoun interpretation preference
2.1
conjunction, to indicate the type of coherence relation
between the two clauses (resemblance with and (in
Dutch en) or cause-effect with but (in Dutch maar)).
This second clause contains a female subject and a
male object pronoun, which will refer to one of the
male nouns in the first clause. The transitive verb
connecting the subject and object in the second clause
can either be a synonym (in resemblance relations) or
an antonym (in cause-effect relations) of the verb in
the first clause.
Introduction
As abovementioned, all sentences can be interpreted in
two different ways; grammatically, the pronoun could
refer to both the subject and the object in the first
clause. The sentences were carefully formulated so the
context would link the pronoun to the object in the first
clause. Experiment 1 was done as a pretest to make
sure that in all the experimental sentences the pronoun
referred to the object, so that if we asked participants
how felicitous a sentence was in experiment 2, all participants judging the sentence were using the same interpretation, in which the pronoun refers to the object.
The participants were presented with the sentences and
had to report which noun they thought the pronoun referred to; the subject or the object.
2.2
2.4
The experiment used a 2x2x2 within-subjects design, varying the presence of a presupposition trigger, the kind of coherence relation and the voice
match/mismatch. This means each sentence had eight
different forms, like can be seen in (9)-(16). In total
there were 28 different test sentences and 32 fillers (the
test sentences can be found in appendix A). Each participant was shown 60 sentences; one variant of each
test sentence and the 32 fillers. This means each participant saw each test condition two to four times.
For the experiment a total of eight different lists were
created, so that every list contained one of the conditions of a test sentence. For each participant one list
was selected randomly. The list of sentences presented
to the participant was pseudo-randomized, so that the
test sentences and the fillers were mixed and there were
no long sequences of the same experimental condition
type. This minimizes possible sequence effects in the
lists.
Participants
20 students and teachers from the University of
Groningen participated in this study. All participants
were native speakers of Dutch.
2.3
Design
Materials
An example of the eight forms of a sentence used in
this experiment, with their translations, are (9)-(16).
(9) Harry bedreigde Lars, en Mieke intimideerde
hem. [Harry threatened Lars, and Mieke intimidated
him.]
(10) Harry bedreigde Lars, en Mieke intimideerde hem
ook. [Harry threatened Lars, and Mieke intimidated
him too.]
(11) Harry bedreigde Lars, maar Mieke redde hem.
[Harry threatened Lars, but Mieke saved him.]
(12) Harry bedreigde Lars, maar Mieke redde hem
toch. [Harry threatened Lars, but Mieke saved him
toch.]
(13) Lars werd door Harry bedreigd, en Mieke intimideerde hem. [Lars was threatened by Harry, and
Mieke intimidated him.]
(14) Lars werd door Harry bedreigd, en Mieke intimideerde hem ook. [Lars was threatened by Harry,
and Mieke intimidated him too.]
(15) Lars werd door Harry bedreigd, maar Mieke
redde hem. [Lars was threatened by Harry, but Mieke
saved him.]
(16) Lars werd door Harry bedreigd, maar Mieke
redde hem toch. [Lars was threatened by Harry, but
Mieke saved him toch.]
2.5
Procedure
The participants were tested individually in an online
test. First they were presented with a list of questions about what languages they spoke, where they
had lived and what their native language was. Then
they received an instruction saying they should select
the name of the person they thought the pronoun was
most likely to refer to.
When starting with the actual test, the sentences
were shown one by one, with the question ‘What does
he/him refer to?’, and the three possible answers, the
first two being the names of the subject and object in
randomized order and the last one being the option
‘both are possible’. The instructions given before the
start of the test said they could only mark ‘both are
possible’ if they thought it was just as likely that the
pronoun would refer to the subject as to the object.
The participant selected an answer and then pressed
the ‘next’ button to go to the next question. All 60
sentences were presented this way. The complete experiment took approximately ten minutes.
All sentences consist of two clauses, that are connected by a conjunction. In the first clause, the subject
and object of a transitive verb are male, referred to
by a proper name. The second clause starts with the
3
2.6
Results
When starting with the actual test, the sentences
were shown one by one, with the question ‘How felicitous is this sentence?’, and the seven possible answers.
The participant selected an answer and then pressed
the ‘next’ button to go to the next question. 60 sentences were presented this way. The complete experiment took approximately fifteen minutes.
Because participants who filled out ‘both are possible’
thought both options were equally likely, those examples may show the same felicity of the sentence in both
interpretations. So, what should be checked is if there
are sentences that are interpreted only in the ‘wrong’
way (pronoun referring to the subject). In 22 out of
the 28 sentences this was the case, but only six sentences exeeded a 10% margin (detailed information can
be found in appendix B). These sentences will be taken
out of the test set; 22 sentences remain to be evaluated
as test sentences in experiment 2.
3.6
Prior to further analysis, seven participants were excluded because they took the test in less than five minutes; this is not enough time to read and evaluate all
60 sentences seriously. Also, four participants were excluded because their results showed long repeated sequences of the same answers, which shows they did
not take the test seriously. So, after exclusion, 38 participants remained for analysis.
Because participants were asked to judge the felicity
of a sentence on a Likert scale with seven grades, the
anwers, ranging from ‘very bad’ to ‘very good’, can be
converted to grades from respectively one to seven.
Using the abovementioned grades, a within-subject
analysis of variance (ANOVA) was done to evaluate
the effects between the three conditions. Significant effects were only found between the presence of a presupposition trigger and the kind of coherence relation
(F (1, 37) = 17.600; p < 0.001). Detailed information can
be found in appendix C.1.
3 Experiment 2:
Judging the felicity
3.1
Introduction
In the second experiment participants were asked to
judge on a Likert scale with seven grades, how felicitous a sentence sounds. The goal of this experiment is
to test whether or not presupposition triggers influence
the felicity of the mismatched sentences. We can also
check if the influence of mismatching is seen in sentences with a resemblance relation and sentences with
a cause-effect relation, or only in sentences with one of
the two coherence relations.
3.2
Results
Participants
48 students from the Psychology Department at the
University of Groningen participated in this study for
course credit. All participants were native speakers of
Dutch.
3.3
Materials
The type and build of the sentences used in this experiment are the same as used in experiment 1.
3.4
Design
The design is the same as that of experiment 1. There
were only 22 test sentences used in experiment 2
though, because six sentences were excluded on the basis of the results of experiment 1. All the test sentences
used can be found in appendix A.
3.5
Figure 1: resemblance relation; matched and mismatched, with and without presupposition triggers
Procedure
The participants were tested individually in an online
test. First they were presented with a list of questions about what languages they spoke, where they had
lived and what their native language was. Then they
received an instruction saying they should judge the
felicity of a sentence according to how felicitous they
though most people would find the sentence. They
were given seven options, presented horizontally on a
Likert scale with seven grades, running from ‘very bad’
to ‘very good’, with ‘neutral’ in the middle.
An ad hoc paired t-test shows that mismatched resemblance sentences improved significantly by adding
a presupposition trigger (t(37) = −5.439; p < 0.001). It
also shows that the felicity of mismatched cause-effect
sentences does not significantly change by adding a
presupposition trigger (t(37) = 1.761; p = 0.087). Check
appendix C.3 for detailed information on the outcomes
of the t-test. The descriptive statistics can be found in
4
and sentences of the sub-class result relations as examples of cause-effect relations, so they check the coherence relations as stated by Hume (1748) for a more
specific category. In our experiment, parallel relations,
referred to as resemblance, and denial of expectation
relations, referred to as cause-effect, are used in the
test sentences. Denial of expectation and result relations are different sub-classes of cause-effect relations,
but it could be that sentences belonging to the different sub-classes have slightly different rules as to what
the interpretation preferences of the anaphora are or
how felicitous sentences with specific relations naturally sound.
It should also be noted that it is hard to extract valid
conclusions from these results, since the number of participants and the number of sentences that remained
for the analysis was lower than expected and hoped
for. This was partially caused by doing the test online;
it gives participants more freedom to fill out nonsense.
Also, a number of effects that did show were not significant. With a larger number of sentences and participants these effects might have been significant, but
for now we’ll have to base our conclusions only on the
significant effects. Also, some sentences are naturally
more felicitous than others, e.g. because some verbs
are used more often, and it has proved to be difficult to
come up with many good test sentences.
If we also take a look at the difference in size between the effects that were not significant, it is clear the
improvement of matched resemblance sentences over
mismatched resemblance sentences without a presupposition trigger is larger than the the improvement of
matched over mismatched cause-effect sentences without a presupposition trigger; matched cause-effect sentences were even slightly worse than their mismatched
forms. We still can’t draw any definite conclusions
from these results, as they are not significant, but it
can be said that the results do gravitate towards supporting Kehler’s theory more than towards supporting
Frazier and Clifton’s theory.
Furthermore, the results show that matched sentences with a resemblance relation are hardly affected
by adding a presupposition trigger. The reason for
this could be that the matched sentences were already
good, so adding a presupposition trigger did not make
a difference. Mismatched resemblance sentences however improve significantly by adding a presupposition
trigger, supporting the theory of Hendriks (2004) and
reproducing the results of Arregui et al. (2006), only
now using a pronoun as an anaphoric device instead
of verb phrase ellipsis. Adding a presupposition trigger to mismatched cause-effect sentences does not have
a significant effect though.
What will remain unclear is why adding a presupposition tigger to a matched or mismatched cause-effect
Figure 2: cause-effect relation; matched and mismatched, with and without presupposition triggers
appendix C.2. The resulted means are also shown in
figure 1 and 2.
Unlike to our expectations, no significant difference
is found between matched resemblance sentences and
mismatched resemblance sentences without presupposition triggers (t(37) = 1.941; p = 0.060). The difference
in felicity between mismatched resemblance sentences
and matched resemblance sentences with presupposition triggers is also not significant (t(37) = −1.899; p =
0.065).
In the cause-effect sentences there is no large effect
to be seen between the matched and mismatched conditions both with and without a presupposition trigger
(resp. t(37) = −1.417; p = 0.165 and t(37) = −1.433; p =
0.160). The mismatched conditions were even slightly
better than their matched forms (figure 2).
4
General results & discussion
As mentioned before, Kehler (2002) and Kehler et al.
(2008) state that mismatching will decrease the felicity of sentences with a resemblance relation, beacuse it
makes it harder to correctly interpret the pronoun, but
will not affect the felicity of sentences with a causeeffect relation. Our results do not show a significant
difference in felicity between matched and mismatched
resemblance sentences without the presence of a presupposition trigger. Also, no difference is found between matched and mismatched cause-effect sentences,
both with and without a presupposition trigger. Because these results are not significant, they can’t be
used as evidence for the theory of Kehler (2002) and
Kehler et al. (2008), or for the theory of Frazier and
Clifton Jr. (2006), who state mismatching will decrease
the felicity of both resemblance and cause-effect sentences.
What should be mentioned is that in their experiment, Kehler et al. (2008) use sentences of the sub-class
parallel relations as examples of resemblance relations
5
contrastive topics. Journal of Semantics, 21(2), 133–
153.
Hume, D. (1748). An inquiry concerning human understanding (1995edition ed.). The Liberal Arts Press,
New York.
Kehler, A. (2002). Coherence, reference, and the theory
of grammar. CSLI Publications.
Kehler, A., Kertz, L., Rohde, H., & Elman, J. L. (2008).
Coherence and coreference revisited. Journal of Semantics, 25, 1–44.
Wolf, F., Gibson, E., & Desmet, T. (2004). Discourse coherence and pronoun resolution. Language and Cognitive Processes, 19(6), 665–675.
sentence does not improve the sentence siginificantly.
An explanation for the difference in effect between the
two presupposition triggers used in this experiment
could be that some presupposition triggers are more
commonly used than others, and ook is more commonly used in Dutch than toch. But that does not explain why toch is still used in Dutch, while it doesn’t
make sentences more felicitous. It might be that some
or all sentences were not appropriate for the use of toch,
because of the coherence relation or for some other reason, but new research would have to be done on what
kind of sentences toch is normally used in in Dutch.
What also can’t be explained by the theories mentioned in the introduction is why matched resemblance
sentences with presupposition triggers are not more felicitous than mismatched resemblance sentences with
presupposition triggers. Because even though the presupposotion trigger gives an alternative mean for interpreting the coherence relation, the mismatch is still
present, so it would be expected that matched sentences will still be more felicitous than mismatched
sentences. It might be the case that the reader is so
glad to recieve another way to discover the coherence
relation between the two clauses that he/she grades
the sentence very high, but a matched version should
of course always sound more felicitous than a mismatched version of a sentence.
Concluding, our results show that a presupposition trigger significantly improves the felicity of mismatched resemblance sentences, so the presence of the
presupposition trigger too can help repair some of the
damage made by the voice mismatch. Futhermore, our
results show that both resemblance and cause-effect
sentences are not significantly worsened by mismatch,
so they are not in support of the theory of Kehler
(2002) and Kehler et al. (2008) that mismatching only
affects the felicity of sentences with a resemblance relation, but also don’t support the theory of Frazier and
Clifton Jr. (2006), who state mismatching will worsen
both resemblance and cause-effect sentences. Finally,
adding a presupposition trigger doesn’t significantly
improve the felicity of cause-effect sentences, and more
research should be done on the effect of the presupposition trigger toch on different types of sentences.
References
Arregui, A., Clifton Jr., C., Frazier, L., & Moulton, K.
(2006). Processing elided verb phrases with flawed
antecedents: The recycling hypothesis. Journal of
Memory and Language, 55(2), 232–246.
Frazier, L., & Clifton Jr., C. (2006). Ellipsis and discourse coherence. Linguistics and Philosophy, 29, 315–
346.
Hendriks, P. (2004). Coherence relations, ellipsis, and
6
Appendices
A
Test sentences
#.
1. resemblance, match, without presupposition.
2. resemblance, match, with presupposition.
3. cause-effect, match, without presupposition.
4. cause-effect, match, with presupposition.
5. resemblance, mismatch, without presupposition.
6. resemblance, mismatch, with presupposition.
7. cause-effect, mismatch, without presupposition.
8. cause-effect, mismatch, with presupposition.
1.
(a) Henk moedigde Stijn aan, en Floor prikkelde hem.
(b) Henk moedigde Stijn aan, en Floor prikkelde hem ook.
(c) Henk moedigde Stijn aan, maar Floor ontmoedigde hem.
(d) Henk moedigde Stijn aan, maar Floor ontmoedigde hem toch.
(e) Stijn werd door Henk aangemoedigd, en Floor prikkelde hem.
(f) Stijn werd door Henk aangemoedigd, en Floor prikkelde hem ook.
(g) Stijn werd door Henk aangemoedigd, maar Floor ontmoedigde hem.
(h) Stijn werd door Henk aangemoedigd, maar Floor ontmoedigde hem toch.
2.
(a) Sander pakte Jelmer, en Cindy hield hem vast.
(b) Sander pakte Jelmer, en Cindy hield hem ook vast.
(c) Sander pakte Jelmer, maar Cindy liet hem vrij.
(d) Sander pakte Jelmer, maar Cindy liet hem toch vrij.
(e) Jelmer werd door Sander gepakt, en Cindy hield hem vast.
(f) Jelmer werd door Sander gepakt, en Cindy hield hem ook vast.
(g) Jelmer werd door Sander gepakt, maar Cindy liet hem vrij.
(h) Jelmer werd door Sander gepakt, maar Cindy liet hem toch vrij.
3.
(a) Robin wees Dirk af, en Tamara verliet hem.
(b) Robin wees Dirk af, en Tamara verliet hem ook.
(c) Robin wees Dirk af, maar Tamara accepteerde hem.
(d) Robin wees Dirk af, maar Tamara accepteerde hem toch.
(e) Dirk werd door Robin afgewezen, en Tamara verliet hem.
(f) Dirk werd door Robin afgewezen, en Tamara verliet hem ook.
(g) Dirk werd door Robin afgewezen, maar Tamara accepteerde hem.
(h) Dirk werd door Robin afgewezen, maar Tamara accepteerde hem toch.
4.
(a) Peter liet Karel achter, en Laura negeerde hem.
(b) Peter liet Karel achter, en Laura negeerde hem ook.
(c) Peter liet Karel achter, maar Laura bleef bij hem.
(d) Peter liet Karel achter, maar Laura bleef toch bij hem.
(e) Karel werd door Peter achtergelaten, en Laura negeerde hem.
(f) Karel werd door Peter achtergelaten, en Laura negeerde hem ook.
(g) Karel werd door Peter achtergelaten, maar Laura bleef bij hem.
(h) Karel werd door Peter achtergelaten, maar Laura bleef toch bij hem.
5.
(a) Tim kraakte Willem af, en Marrit haalde hem neer.
(b) Tim kraakte Willem af, en Marrit haalde hem ook neer.
1
(c) Tim kraakte Willem af, maar Marrit prees hem.
(d) Tim kraakte Willem af, maar Marrit prees hem toch.
(e) Willem werd door Tim afgekraakt, en Marrit haalde hem neer.
(f) Willem werd door Tim afgekraakt, en Marrit haalde hem ook neer.
(g) Willem werd door Tim afgekraakt, maar Marrit prees hem.
(h) Willem werd door Tim afgekraakt, maar Marrit prees hem toch.
6.
(a) Thijs prees Ivo, en Anne verheerlijkte hem.
(b) Thijs prees Ivo, en Anne verheerlijkte hem ook.
(c) Thijs prees Ivo, maar Anne haalde hem neer.
(d) Thijs prees Ivo, maar Anne haalde hem toch neer.
(e) Ivo werd door Thijs geprezen, en Anne verheerlijkte hem.
(f) Ivo werd door Thijs geprezen, en Anne verheerlijkte hem ook.
(g) Ivo werd door Thijs geprezen, maar Anne haalde hem neer.
(h) Ivo werd door Thijs geprezen, maar Anne haalde hem toch neer.
7.
(a) Mik accepteerde Herman, en Naomi aanvaardde hem.
(b) Mik accepteerde Herman, en Naomi aanvaardde hem ook.
(c) Mik accepteerde Herman, maar Naomi wees hem af.
(d) Mik accepteerde Herman, maar Naomi wees hem toch af.
(e) Herman werd door Mik geaccepteerd, en Naomi aanvaardde hem.
(f) Herman werd door Mik geaccepteerd, en Naomi aanvaardde hem ook.
(g) Herman werd door Mik geaccepteerd, maar Naomi wees hem af.
(h) Herman werd door Mik geaccepteerd, maar Naomi wees hem toch af.
8.
(a) Harry bedreigde Lars, en Mieke intimideerde hem.
(b) Harry bedreigde Lars, en Mieke intimideerde hem ook.
(c) Harry bedreigde Lars, maar Mieke redde hem.
(d) Harry bedreigde Lars, maar Mieke redde hem toch.
(e) Lars werd door Harry bedreigd, en Mieke intimideerde hem.
(f) Lars werd door Harry bedreigd, en Mieke intimideerde hem ook.
(g) Lars werd door Harry bedreigd, maar Mieke redde hem.
(h) Lars werd door Harry bedreigd, maar Mieke redde hem toch.
9.
(a) Samuel trapte Justin, en Marja schopte hem.
(b) Samuel trapte Justin, en Marja schopte hem ook.
(c) Samuel trapte Justin, maar Marja verdedigde hem.
(d) Samuel trapte Justin, maar Marja verdedigde hem toch.
(e) Justin werd door Samuel getrapt, en Marja schopte hem.
(f) Justin werd door Samuel getrapt, en Marja schopte hem ook
(g) Justin werd door Samuel getrapt, maar Marja verdedigde hem.
(h) Justin werd door Samuel getrapt, maar Marja verdedigde hem toch.
10.
(a) Leon misleidde Simon, en Rianne verraadde hem.
(b) Leon misleidde Simon, en Rianne verraadde hem ook.
(c) Leon misleidde Simon, maar Rianne waarschuwde hem.
(d) Leon misleidde Simon, maar Rianne waarschuwde hem toch.
(e) Simon werd door Leon misleid, en Rianne verraadde hem.
(f) Simon werd door Leon misleid, en Rianne verraadde hem ook.
(g) Simon werd door Leon misleid, maar Rianne waarschuwde hem.
(h) Simon werd door Leon misleid, maar Rianne waarschuwde hem toch.
2
11. *
(a) Frank bestal Nick, en Petra bedroog hem.
(b) Frank bestal Nick, en Petra bedroog hem ook.
(c) Frank bestal Nick, maar Petra beschermde hem.
(d) Frank bestal Nick, maar Petra beschermde hem toch.
(e) Nick werd door Frank bestolen, en Petra bedroog hem.
(f) Nick werd door Frank bestolen, en Petra bedroog hem ook.
(g) Nick werd door Frank bestolen, maar Petra beschermde hem.
(h) Nick werd door Frank bestolen, maar Petra beschermde hem toch.
12. *
(a) Rick steunde Maarten, en Lieke hielp hem.
(b) Rick steunde Maarten, en Lieke hielp hem ook.
(c) Rick steunde Maarten, maar Lieke bekritiseerde hem.
(d) Rick steunde Maarten, maar Lieke bekritiseerde hem toch.
(e) Maarten werd door Rick gesteund, en Lieke hielp hem.
(f) Maarten werd door Rick gesteund, en Lieke hielp hem ook.
(g) Maarten werd door Rick gesteund, maar Lieke bekritiseerde hem.
(h) Maarten werd door Rick gesteund, maar Lieke bekritiseerde hem toch.
13.
(a) Bas verdedigde Koen, en Julia beschermde hem.
(b) Bas verdedigde Koen, en Julia beschermde hem ook.
(c) Bas verdedigde Koen, maar Julia viel hem aan.
(d) Bas verdedigde Koen, maar Julia viel hem toch aan.
(e) Koen werd door Bas verdedigd, en Julia beschermde hem.
(f) Koen werd door Bas verdedigd, en Julia beschermde hem ook.
(g) Koen werd door Bas verdedigd, maar Julia viel hem aan.
(h) Koen werd door Bas verdedigd, maar Julia viel hem toch aan.
14.
(a) Tijn dankte Cas af, en Hanna keerde hem de rug toe.
(b) Tijn dankte Cas af, en Hanna keerde hem ook de rug toe.
(c) Tijn dankte Cas af, maar Hanna charmeerde hem.
(d) Tijn dankte Cas af, maar Hanna charmeerde hem toch.
(e) Cas werd door Tijn afgedankt, en Hanna keerde hem de rug toe.
(f) Cas werd door Tijn afgedankt, en Hanna keerde hem ook de rug toe.
(g) Cas werd door Tijn afgedankt, maar Hanna charmeerde hem.
(h) Cas werd door Tijn afgedankt, maar Hanna charmeerde hem toch.
15.
(a) Chris waardeerde Ben, en Margo bewonderde hem.
(b) Chris waardeerde Ben, en Margo bewonderde hem ook.
(c) Chris waardeerde Ben, maar Margo minachtte hem.
(d) Chris waardeerde Ben, maar Margo minachtte hem toch.
(e) Ben werd door Chris gewaardeerd, en Margo bewonderde hem.
(f) Ben werd door Chris gewaardeerd, en Margo bewonderde hem ook.
(g) Ben werd door Chris gewaardeerd, maar Margo minachtte hem.
(h) Ben werd door Chris gewaardeerd, maar Margo minachtte hem toch.
16.
(a) Erik bond Vik vast, en Mirjam ketende hem.
(b) Erik bond Vik vast, en Mirjam ketende hem ook.
(c) Erik bond Vik vast, maar Mirjam maakte hem los.
3
(d) Erik bond Vik vast, maar Mirjam maakte hem toch los.
(e) Vik werd door Erik vastgebonden, en Mirjam ketende hem.
(f) Vik werd door Erik vastgebonden, en Mirjam ketende hem ook.
(g) Vik werd door Erik vastgebonden, maar Mirjam maakte hem los.
(h) Vik werd door Erik vastgebonden, maar Mirjam maakte hem toch los.
17. *
(a) Joost aaide Tom, en Ina streelde hem.
(b) Joost aaide Tom, en Ina streelde hem ook.
(c) Joost aaide Tom, maar Ina verfoeide hem.
(d) Joost aaide Tom, maar Ina verfoeide hem toch.
(e) Tom werd door Joost geaaid, en Ina streelde hem.
(f) Tom werd door Joost geaaid, en Ina streelde hem ook.
(g) Tom werd door Joost geaaid, maar Ina verfoeide hem.
(h) Tom werd door Joost geaaid, maar Ina verfoeide hem toch.
18.
(a) Rutger haatte Frits, en Ellen verafschuwde hem.
(b) Rutger haatte Frits, en Ellen verafschuwde hem ook.
(c) Rutger haatte Frits, maar Ellen had hem lief.
(d) Rutger haatte Frits, maar Ellen had hem toch lief.
(e) Frits werd door Rutger gehaat, en Ellen verafschuwde hem.
(f) Frits werd door Rutger gehaat, en Ellen verafschuwde hem ook.
(g) Frits werd door Rutger gehaat, maar Ellen had hem lief.
(h) Frits werd door Rutger gehaat, maar Ellen had hem toch lief.
19.
(a) Pim nodigde Hugo uit, en Wilma vroeg hem mee.
(b) Pim nodigde Hugo uit, en Wilma vroeg hem ook mee.
(c) Pim nodigde Hugo uit, maar Wilma negeerde hem.
(d) Pim nodigde Hugo uit, maar Wilma negeerde hem toch.
(e) Hugo werd door Pim uitgenodigd, en Wilma vroeg hem mee.
(f) Hugo werd door Pim uitgenodigd, en Wilma vroeg hem ook mee.
(g) Hugo werd door Pim uitgenodigd, maar Wilma negeerde hem.
(h) Hugo werd door Pim uitgenodigd, maar Wilma negeerde hem toch.
20.
(a) Roy liet Erwin toe, en Sarah keurde hem goed.
(b) Roy liet Erwin toe, en Sarah keurde hem ook goed.
(c) Roy liet Erwin toe, maar Sarah sloot hem buiten.
(d) Roy liet Erwin toe, maar Sarah sloot hem toch buiten.
(e) Erwin werd door Roy toegelaten, en Sarah keurde hem goed.
(f) Erwin werd door Roy toegelaten, en Sarah keurde hem ook goed.
(g) Erwin werd door Roy toegelaten, maar Sarah sloot hem buiten.
(h) Erwin werd door Roy toegelaten, maar Sarah sloot hem toch buiten.
21.
(a) Olivier verbeterde Oscar, en Iris corrigeerde hem.
(b) Olivier verbeterde Oscar, en Iris corrigeerde hem ook.
(c) Olivier verbeterde Oscar, maar Iris beloonde hem.
(d) Olivier verbeterde Oscar, maar Iris beloonde hem toch.
(e) Oscar werd door Olivier verbeterd, en Iris corrigeerde hem.
(f) Oscar werd door Olivier verbeterd, en Iris corrigeerde hem ook.
(g) Oscar werd door Olivier verbeterd, maar Iris beloonde hem.
(h) Oscar werd door Olivier verbeterd, maar Iris beloonde hem toch.
4
22.
(a) Thomas stalkte Klaas, en Karin viel hem lastig.
(b) Thomas stalkte Klaas, en Karin viel hem ook lastig.
(c) Thomas stalkte Klaas, maar Karin alarmeerde hem.
(d) Thomas stalkte Klaas, maar Karin alarmeerde hem toch.
(e) Klaas werd door Thomas gestalkt, en Karin viel hem lastig.
(f) Klaas werd door Thomas gestalkt, en Karin viel hem ook lastig.
(g) Klaas werd door Thomas gestalkt, maar Karin alarmeerde hem.
(h) Klaas werd door Thomas gestalkt, maar Karin alarmeerde hem toch.
23. *
(a) Gert beledigde Fred, en Bette kwetste hem.
(b) Gert beledigde Fred, en Bette kwetste hem ook.
(c) Gert beledigde Fred, maar Bette behoedde hem.
(d) Gert beledigde Fred, maar Bette behoedde hem toch.
(e) Fred werd door Gert beledigd, en Bette kwetste hem.
(f) Fred werd door Gert beledigd, en Bette kwetste hem ook.
(g) Fred werd door Gert beledigd, maar Bette behoedde hem.
(h) Fred werd door Gert beledigd, maar Bette behoedde hem toch.
24. *
(a) Guus dankte Aaron, en Lisa loofde hem.
(b) Guus dankte Aaron, en Lisa loofde hem ook.
(c) Guus dankte Aaron, maar Lisa verachtte hem.
(d) Guus dankte Aaron, maar Lisa verachtte hem toch.
(e) Aaron werd door Guus gedankt, en Lisa loofde hem.
(f) Aaron werd door Guus gedankt, en Lisa loofde hem ook.
(g) Aaron werd door Guus gedankt, maar Lisa verachtte hem.
(h) Aaron werd door Guus gedankt, maar Lisa verachtte hem toch.
25. *
(a) Floris joeg Kelvin op, en Maria jutte hem op.
(b) Floris joeg Kelvin op, en Maria jutte hem ook op.
(c) Floris joeg Kelvin op, maar Maria remde hem af.
(d) Floris joeg Kelvin op, maar Maria remde hem toch af.
(e) Kelvin werd door Floris opgejaagd, en Maria jutte hem op.
(f) Kelvin werd door Floris opgejaagd, en Maria jutte hem ook op.
(g) Kelvin werd door Floris opgejaagd, maar Maria remde hem af.
(h) Kelvin werd door Floris opgejaagd, maar Maria remde hem toch af.
26.
(a) Kees berispte Kay, en Renske vermaande hem.
(b) Kees berispte Kay, en Renske vermaande hem ook.
(c) Kees berispte Kay, maar Renske complimenteerde hem.
(d) Kees berispte Kay, maar Renske complimenteerde hem toch.
(e) Kay werd door Kees berispt, en Renske vermaande hem.
(f) Kay werd door Kees berispt, en Renske vermaande hem ook.
(g) Kay werd door Kees berispt, maar Renske complimenteerde hem.
(h) Kay werd door Kees berispt, maar Renske complimenteerde hem toch.
27.
(a) Gerard ondervroeg Sjors, en Marloes verhoorde hem.
(b) Gerard ondervroeg Sjors, en Marloes verhoorde hem ook.
5
(c) Gerard ondervroeg Sjors, maar Marloes stelde hem op zijn gemak.
(d) Gerard ondervroeg Sjors, maar Marloes stelde hem toch op zijn gemak.
(e) Sjors werd door Gerard ondervraagd, en Marloes verhoorde hem.
(f) Sjors werd door Gerard ondervraagd, en Marloes verhoorde hem ook.
(g) Sjors werd door Gerard ondervraagd, maar Marloes stelde hem op zijn gemak.
(h) Sjors werd door Gerard ondervraagd, maar Marloes stelde hem toch op zijn gemak.
28.
(a) Stefan verdacht Juri, en Roos beschuldigde hem.
(b) Stefan verdacht Juri, en Roos beschuldigde hem ook.
(c) Stefan verdacht Juri, maar Roos vertrouwde hem.
(d) Stefan verdacht Juri, maar Roos vertrouwde hem toch.
(e) Juri werd door Stefan verdacht, en Roos beschuldigde hem.
(f) Juri werd door Stefan verdacht, en Roos beschuldigde hem ook.
(g) Juri werd door Stefan verdacht, maar Roos vertrouwde hem.
(h) Juri werd door Stefan verdacht, maar Roos vertrouwde hem toch.
* removed from the test set in experiment 2
6
B
Results experiment 1
Answers are shown per question and subject. The last column shows the number of ones per sentence, which are the
number of answers that say the pronoun refers to the subject. The sentences with more than two ones were removed
from the test set in experiment 2.
question
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
subject
1
2
2
2
2
1
2
2
2
2
2
1
3
2
2
2
2
2
3
2
2
2
2
2
2
3
2
2
2
2
2
2
2
2
3
2
2
1
3
3
3
2
3
2
2
2
3
2
2
2
3
2
2
3
3
2
2
2
3
2
2
2
2
3
2
2
3
2
3
2
2
3
2
2
2
2
2
2
2
1
2
2
2
2
2
2
2
4
2
2
2
2
3
2
2
3
3
2
2
2
2
2
2
2
2
2
2
2
2
2
2
3
2
2
2
2
5
2
2
2
3
2
2
3
3
2
2
2
2
2
3
3
3
3
1
2
2
2
2
3
1
3
3
3
2
6
3
2
3
2
2
2
2
2
2
2
2
1
2
2
2
2
2
2
2
2
2
2
2
1
2
2
2
2
7
2
2
2
2
2
3
2
2
2
2
2
2
2
3
2
2
2
2
2
2
2
2
2
2
2
2
2
2
8
2
2
2
2
2
2
2
2
2
2
1
1
2
2
1
2
2
2
2
2
2
2
1
2
2
2
2
2
9
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
1
2
3
2
2
2
3
3
2
1
2
3
2
10
2
2
3
2
2
1
2
2
2
3
1
1
2
3
2
2
2
2
2
1
2
2
3
3
2
2
3
2
7
11
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
1
2
2
1
1
1
2
2
2
12
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
13
2
2
1
3
3
1
2
2
3
3
3
3
2
2
2
2
3
2
1
3
2
2
3
3
2
1
3
2
14
2
2
2
2
2
2
2
2
2
2
3
2
1
2
2
2
2
2
2
3
1
2
2
2
1
2
2
2
15
2
2
2
2
2
2
2
2
1
1
2
2
2
2
2
2
1
2
3
2
2
2
2
2
3
3
2
2
16
2
2
1
2
2
2
2
2
3
2
2
2
2
1
1
1
1
2
2
2
2
2
2
1
2
2
2
2
17
2
2
2
2
2
2
2
2
1
3
2
2
2
2
2
2
1
2
2
2
2
2
2
2
1
2
2
2
18
2
2
2
2
3
2
2
2
3
2
3
2
3
2
2
2
2
2
2
2
3
2
2
2
2
2
2
2
19
2
2
2
2
2
2
3
2
3
2
2
2
2
2
2
2
2
1
2
2
2
1
2
2
2
2
2
2
20
3
2
2
2
1
2
1
2
3
2
3
2
1
2
2
2
3
2
2
2
3
2
1
2
3
2
2
2
# 1s
0
0
2
0
2
2
1
0
2
1
3
3
2
1
2
2
3
2
1
2
2
1
3
4
4
1
0
0
C
C.1
Results experiment 2
ANOVA
8
C.2
Descriptive statistics
9
C.3
Paired samples t-test
10