Cogn Comput
DOI 10.1007/s12559-010-9068-x
Agreement and its Multimodal Communication in Debates:
A Qualitative Analysis
Isabella Poggi • Francesca D’Errico
Laura Vincze
•
Received: 15 April 2010 / Accepted: 8 August 2010
Springer Science+Business Media, LLC 2010
Abstract The paper defines the notion of agreement from
a cognitive point of view and analyses types of agreement
signals in TV debates. Agreement is defined as a relation of
identity, similarity or congruence between the opinions of
two or more persons, by contrasting it with confirmation
and admission, and the connected notions of proposal,
assessment and opinion are overviewed. Research is then
presented on the multimodal signals of agreement in
debates from the Canal 9 and the AMI corpora; different
ways to express agreement are singled out in extensive
discourse, single words and body signals, and analysed
through an annotation scheme of multimodal data. Different types of agreement are illustrated, including true,
indirect and apparent agreement.
Keywords Agreement Opinion Social signals Multimodality
Social Signals of Agreement
Recognizing and interpreting the signals through which
people inform each other of their reciprocal relations and
attitudes is an essential capacity for social life, and Intelligent Systems, from dialogue systems to Embodied
I. Poggi F. D’Errico (&) L. Vincze
Roma Tre University, Via del Castro Pretorio 20,
00185 Rome, Italy
e-mail: [email protected]
I. Poggi
e-mail: [email protected]
L. Vincze
e-mail: [email protected]
Agents, should be capable of sensing, interpreting and
delivering social signals.
Social signals are perceivable stimuli that, either directly
or indirectly, convey information concerning social
actions, social interaction, attitudes, social emotions and
social relations [1].
A specific kind of social signals are those of agreement
and disagreement, through which participants in a communicative interaction express if they share the same
opinions, they accept each other’s proposals and have
convergent or divergent goals, attitudes and feelings.
When people discuss or argue, it is relevant, for both
those who participate in the interaction and those who
observe it from outside, to understand what are the relationships between participants and whether they agree or
disagree; to do so, they must be able to catch and interpret
social signals of agreement and disagreement, which can be
expressed in different communicative modalities—words,
gesture, intonation, face, gaze, head movements, posture.
To agree, one may look at you while smiling, clap hands,
tell you ‘‘bravo!’’ or simply nod. Yet, in some cases,
apparent agreement in fact conceals underlying hostility, or
simply polite but hypocritical interaction.
In this paper, we define the notions of agreement and
other connected notions such as opinion and proposal and
report a study on real cases of actual and apparent, direct
and indirect agreement conveyed through verbal and nonverbal social signals.
Agreement: a Semantic Analysis
In the context of the European Net of Excellence SSPNet
(Social Signal Processing Network), we are investigating
the social signals of agreement and disagreement in public
123
Cogn Comput
debates. One of our objectives is to describe the recurrent
ways in which agreement is expressed through verbal and
nonverbal behaviour in debates and to allow the construction of systems for the automatic recognition of social
signals of agreement.
Before finding and analysing signals of agreement in
debate, we want to define the notion of agreement
according to a model of mind, social action and communication in terms of goals and beliefs [2, 3]. As a first step
to achieve it, we explore the semantic area of this word.
The Logical Structure of Agreement
In the model we adopt [4], from a logical point of view,
‘‘agreement’’ is a predicate with three arguments. More
specifically, since a predicate is defined as a property if
referred to one argument, and a relation if referred to more
than one, ‘‘agreement’’ is a relation among three arguments. The arguments are two Agents and an Object: let us
call them A, B and O. O is a ‘‘cognitive’’ object, i.e., a
representation in some mind, while A and B are two
individual or collective Agents (e.g., two persons or two
groups) having O represented in their minds; therefore, the
proposition underlying the notion of ‘‘agreement’’ is
‘‘A agrees with B about O’’. This means that there is
‘‘agreement’’ between A and B about O when there is a
relation of identity or similarity, or anyway of congruence,
harmony, accordance, between A and B concerning O, that
is, when both A and B consider O in the same way (e.g., as
true or false, good or bad, or both).
Agree and Agreement
The English verb to agree has two different meanings, let
us call them agree 1 and agree 2, which differ from each
other in terms of two dimensions.
The first is the dimension of their ‘‘Aktionsart’’, the
‘‘aspect’’ or ‘‘character’’ of action [5–7], since agree 1 is a
(mental) state while agree 2 is a (social or communicative)
action.
In its first reading of agree 1, to agree is a stative verb
meaning that ‘‘some person has the same opinion as
another person’’. If I say ‘‘I agree with you’’, this means
that my opinion is the same as the one you have just
expressed. There is a coincidence between the opinions of
two persons, but this is a preceding steady state that needs
not be brought about by some process of conviction or
persuasion: we ‘‘just happen’’ to agree with each other,
that’s all.
In the reading of agree 2, instead, the meaning of to
agree can be more clearly conveyed by its synonymical
expression to come to an agreement: a verb phrase meaning that ‘‘two persons, by talking with each other, finally
123
come to an agreement about some course of action’’; in a
sense, ‘‘they negotiate until they set a goal that is shared by
both’’. For instance, if I say ‘‘I agreed with Maria about
meeting at 5 o’clock’’, this means that there was a process
of common decision-making, and possibly a communicative interaction, at the end of which the same goal (meeting
at 5 o’clock) has come to be accepted in both minds of
Maria and myself: a goal agreed upon.
Therefore, agree 2 differs from agree 1 in two senses: first
because in agree 2, at the beginning of the process, for the
two Agents there is not the same similarity as is the case for
agree 1, but, on the contrary, to agree mentions a process
that in the end results in such a similarity. Second, agree 2
differs from agree 1 because the achieved or previous similarity is between goals, not between opinions.1
The English noun agreement also has three different
meanings. In one of its readings, agreement is simply ‘‘the
fact that two or more persons have similar feelings or
similar opinions’’. In another reading, though, an agreement is ‘‘the result of a process through which two or more
persons have found an agreement’’. This result may be
either an abstract object—the fact that two persons formerly disagreeing finally have the same opinion or share
the same goal—or a very concrete object, e.g., a contract: a
sheet of paper where the agreement is written.
In conclusion, the notions in the area of agreement vary
along two dimensions:
1.
2.
the nature of agreement (in terms of its ‘‘Aktionsart’’,
its aspect or ‘‘character’’), which may be a state, a
process, or the result of a process;
the ‘‘objects’’ that are in agreement, which may be
opinions or goals or (in the case of Italian ‘‘andare
d’accordo’’, see footnote 1) emotions.
The difference between agree 1 (to be in agreement) and
agree 2 (to come to an agreement) has an intriguing
implication. People can come to an agreement about a goal.
Suppose in a negotiation, A has goal GA, B has goal GB,
and the result of their coming to an agreement is that both
convene that pursuing goal GB is better for the moment,
but that later GA will be pursued too. In this case, A agreed
1
In Italian, these two meanings of ‘‘to agree’’ respectively
correspond to two different verb phrases: essere d’accordo—the
stative meaning, a preceding steady state—and mettersi d’accordo—a
social action aimed at coming to an agreement. A third Italian phrase
including ‘‘d’accordo’’ is ‘‘andare d’accordo’’, meaning that two
persons generally have the same goals, they have consonant feelings,
they are empathic to each other, so they generally need not argue or
discuss to make shared decisions. If I say: ‘‘Maria ed io andiamo
d’accordo’’ (Maria and I have a feeling with each other), it means that
we feel well together, we have similar feelings, and that there is no
conflict between us; so, no reason of striving to come to an agreement.
In this case, the similarity is not one of opinions nor of goals, but one
of emotions or feelings.
Cogn Comput
(in the sense of ‘‘came to an agreement’’) about pursuing
GB even if his goal was in fact GA.
Yet, while this can happen for a goal, it cannot for an
opinion: about an opinion you can only agree in the sense
of being (already) in agreement, not in the sense of coming
to an agreement: you cannot negotiate or come to an
agreement about an opinion! While goals that we do not
feel as ours can be assumed out of necessity, convenience
or opportunism, as sub-goals of our own goals, this is not
so for our beliefs, and among them, for our opinions.
What do We Agree About?
What is the ‘‘cognitive object’’ O about which two persons
may agree? What can we agree about? We can say
‘‘I agree’’ after a sentence like ‘‘I think that Napoleon was
a great man’’ or ‘‘I propose that all teachers give a home
assignment about Napoleon’’, but not after a question like
‘‘Did Napoleon die in 1821?’’ nor after a statement like
‘‘Napoleon died in 1821’’, unless someone challenges this
as not a factual belief but a questionable statement. In sum,
we may not agree about a statement concerning a ‘‘factual’’
belief, i.e., about a simply informative speech act, but only
about speech acts like a proposal, an assessment (i.e., the
expression of some evaluation), or finally the expression of
an opinion. What are these three speech acts and what do
they have in common?
Proposal
According to our model [2, 3, 8], both action and communication are aimed at bringing about goals: every action
is a means for a goal, but every goal may in turn be a
means, a sub-goal, to one or more super-ordinate goals
(supergoals), with actions and goals forming complex
hierarchical structures called plans. In these terms, a proposal is a speech act (an action performed through language), more specifically a request for action, i.e., a speech
act by which a Sender asks an Addressee to pursue some
goal. As opposed to other requests for action (order, advice,
imploration and the like), a proposal is characterized by
three features: in asking to pursue that goal,
1.
2.
3.
the Sender implies that the proposed goal is shared by
Sender and Addressee: it is not only a means for
supergoals of the Sender, but also for the Addressee’s;
the Sender implies either that s/he does not have more
power than the Addressee nor power over the
Addressee, or that, in any case, he does not make
appeal to his/her power over the Addressee;
as a consequence of 1 and 2, when the Sender makes a
proposal, the Addressee is free to accept it or not
(to opt for pursuing that goal or not) and his
acceptance implies that he approves the proposal,
i.e., he also believes, as implied by the Sender, that the
proposed goal is a means for his goals too.
Assessment
An assessment [9] is a speech act of information that
conveys the expression of an evaluation. In the adopted
framework, evaluation is defined [10, 11] as a belief about
whether and how much some object, person, event or other
entity have, or provide someone with, the power to achieve
some goal. The evaluation is positive when they allow one
to achieve a goal, negative when they prevent from
achieving it, and any evaluation is conceived of with
respect to some goal: not only utilitarian or selfish aims but
also ethical, social, aesthetical ends. Typical expressions of
evaluation in verbal languages are evaluative adjectives,
like good, stupid, ugly, right, generous, as well as verbs
like approve, dislike, accuse, blame, that mention evaluations with respect to different goals. Yet, evaluation is
conveyed also in indirect ways: for example, helping a
child to complete a task he could accomplish autonomously
may convey you do not judge him able to do it by himself
[12].
Opinion
The expression of an opinion is an informative speech act
too. But to define it, we must wonder what an opinion is
and what is its difference from a ‘‘factual’’ belief.
According to some definitions in Social Psychology, an
opinion is an evaluative assertion concerning a questionable issue, showing features of instability, plasticity and
specificity [13]. It is sometimes viewed as a subset of the
complex notion of attitude, a positive or negative orientation towards some object, involving thoughts and affective
responses [14]. Opinion differs from attitude first because
in attitudes, the emotional-affective dimension is salient
(this attracts me/rejects me), so they are based in some way
on a global stance towards a class of objects, while opinion
is simply a ‘‘cognitive state, a lower form of knowledge’’
[15] yielding a specific response (this is right/wrong) to a
particular issue of collective concern [16].
According to socio-constructionist views [17], opinions
are forms of thought that are not based on the classical
principles of logic typical of scientific thought (noncontradiction, identity, causality); rather, they rely on forms of
social thought, like commonsense, that are based instead on
principles like similarity or regularity: here causality is not
demonstrated but leans on the perception of repetition of an
event and finds its foundation in social consensus, not in
123
Cogn Comput
empirical verification. This is why, for socio-constructionists, the opinion is ‘‘socially constructed’’, that is, it is
not a pre-existing thought held by one person, but one that
comes to be constructed through communicative
interaction.
A Cognitive Notion of Opinion
Though the notion of opinion is widely used in Social
Psychology and Sociology and in schools of Journalism,
except for some work on opinion dynamics in the area of
Social Simulation [18], the very nature of opinion has not
been precisely defined from a cognitive point of view.
Here, we propose a tentative definition of opinion in terms
of a cognitive model of mind and social interaction [3, 19].
People have goals and pursue them by making plans of
action, based on beliefs, that is, representations of the outer
and inner world. A belief is a propositional or sensorimotor
representation of objects, persons or events of the external
world, or of mental states of ourselves or others, acquired
through perception or communication, or generated
through inference. About beliefs, we have meta-beliefs
concerning their level of certainty, that is, how they are
likely to be an adequate representation of reality; this level
of certainty may either stem from the very source of a
belief (e.g., beliefs acquired through perception are felt as
the most certain of all) or be checked against previous
beliefs and tested through reasoning and inferential processing. To the extent to which a new incoming belief is
compatible with previously assumed beliefs, i.e., it is not
contradicted by them and even might be inferred on the
basis of them, the previous beliefs are good arguments for
the new one (they support it), and the new and previous
beliefs enhance each other’s level of certainty.
In terms of this model, an ‘‘opinion’’ can be distinguished from a ‘‘factual’’ belief for at least the following
features:
1.
‘‘Facts’’ and ‘‘opinions’’ generally differ for their level
of certainty. In general, we are less certain of an
opinion than we are of a factual belief (although this is
not a distinctive feature of opinions, since sometimes
people hold their opinions with a very high level of
certainty: they feel very confident about them). In any
case, it is quite different for me if:
a. I think that x
b. I believe that x
c. I am convinced that x
d. I am sure that x
e. I know that x
Of course, an important difference between the beliefs
x mentioned in these sentences is an increase in the level of
123
certainty I credit them: this is clear, for instance, with
‘‘I think’’ as opposed to ‘‘I believe’’ and ‘‘I am convinced’’
as against ‘‘I am sure’’. These four expressions all take into
consideration the possibility of the uncertainty of a belief,
just because they attribute it different levels of it, even if
this level is in any case significantly high. The last phrase
instead, ‘‘I know’’, does not take this uncertainty into
account. Furthermore, a more radical difference distinguishes this expression: saying ‘‘I know’’ implies a notion
of ‘‘objectivity’’ of some belief. Something that people
‘‘know’’ is seen as something that can find some shared
evidence of either a perceptual or a logical kind and that
cannot be questioned by anybody. In other words, saying
‘‘I know that x’’ at least implies (even if, probably, as a
necessary but not yet a sufficient condition) that I believe
as true something that I believe someone else (or possibly
everyone else) takes as true. In saying ‘‘I know’’, I presuppose that some belief exists which is already known and
taken for granted by some people and I assert that I have
already come to believe it before now. On the contrary, if I
say ‘‘I think’’, ‘‘I believe’’, ‘‘I’m convinced’’ or even ‘‘I am
sure’’, I imply this is something I believe myself, but also
something I ‘‘subjectively’’ believe, in that I do not pretend
it to be an (already) commonly shared belief.
2.
The difference in the level of certainty of ‘‘opinions’’
versus ‘‘factual’’ beliefs may be partly due to the fact
that an opinion is a belief concerning abstract objects.
While a ‘‘fact’’ is like a ‘‘concrete’’ object, in that it
has a physical appearance, one people can perceive
through their senses, an opinion is a belief entertained
in our mind, characterized by the following features:
a.
it cannot, by definition, be acquired through
perception;
b. it is the result of some inference, a ‘‘conclusion’’
drawn, possibly, on the basis of empirical facts,
but not empirical (perceivable) per se;
c. one who has an opinion cannot find any empirical
evidence; at most, one can find logical arguments
for it. This is why an opinion cannot be the object
of verbal expressions like ‘‘I know’’, but rather of
ones like ‘‘I think’’, ‘‘I believe’’, ‘‘in my view’’.
3.
In general, an opinion is a belief concerning something
about which people may have different positions,
different points of view, in either a perceptual or
pragmatic sense. To have different ‘‘views’’ means that
one judges something with reference to one’s standing
point. So I may ‘‘think’’ an object as rectangular if I
see it obliquely from my angle, but if another sees it
frontally, he may view it as a square object. In the
same way, if I ‘‘see’’ some action from the point of
view of my goals or values, and you do so on the basis
Cogn Comput
of your goals or values, we may have different beliefs
about it. So an opinion, in the pragmatic sense, is a
belief concerning values or goals, as witnessed by the
fact that in the expression of opinions people often use
deontic expressions, like ‘‘you should’’ ‘‘one has to’’,
‘‘you may not’’ and so on. In this case, the opinion is a
belief conceived of starting from a particular, let’s say,
ideological standing point, one’s worldview. In fact, if
two persons have different goals or different values,
they may view things from different standing points,
and the result of what they come to believe may be
different.
Our tentative definition of opinion is then the following:
an opinion is a subjective belief, that is, a belief that one
knows is not necessarily shared and has no empirical
evidence for; it is a belief stemming from taking a particular
(subjective) position, in either a perceptual or pragmatic
sense. In the former sense, an opinion is a hypothesis, that is,
a belief not yet completely confirmed by empirical evidence.
In the pragmatic sense, an opinion is a belief concerning
goals or values. And since an evaluation is a belief concerning how much something may favour the attainment of
some goal, an evaluation is a type of opinion too.
A Definition of Agreement
On the basis of the above exploration, we define as
agreement the fact that there is a relation of identity,
similarity or congruence between the mental states of two
or more persons, where these mental states are in any case
opinions. In fact, also when I agree about your proposal,
this means I agree with your opinion that the goal you
propose is the most convenient in the current situation. If I
agree about your assessment, I agree with the evaluation
you expressed, and evaluations are a sub-case of opinions.
Thus, B agrees with A when B assumes that his/her
opinion is the same, similar or in any case congruent (in the
same line, not conflicting) with the opinion of A.
Instead, when the mental state in common between A
and B is a goal or an emotion (like is the case for the Italian
phrase ‘‘andare d’accordo’’), we should better talk of
syntonization [20] or of an empathic relation, rather than of
agreement proper.
Furthermore, agreement, i.e., the fact that people share
some opinion, may either be an initial state—a state of
coincidence of mental states casually occurring between
two persons—or the result of a process through which
some person B, who initially had an opinion OB different,
opposite, contradictory with respect to A’s opinion OA,
after discussing and arguing, finally comes to have the
same opinion OA as A. In this case, B has been convinced
of opinion OA, and agreement is the result of a process of
conviction: B either has been persuaded by A or has persuaded oneself during, or thanks to, or independent of the
discussion with A. In fact, persuasion is a process through
which one comes to change his future behaviour (one is
‘‘persuaded to’’ do something), thanks to the fact that one
comes to change one’s opinion (one is ‘‘convinced of’’
something) [8].
According to this definition, agreement is an internal
mental state of B: B’s assumption about his having the
same opinion as A. This assumption may be communicated
by B to A or to others through a speech act or a communicative nonverbal act, that we call ‘‘expression of
agreement’’.
Mental States and their Communication
We have contended that an expression of agreement may
follow a speech act of proposal, an assessment, or the
expression of an opinion. Now it is important to stress that
these are speech acts, i.e., acts communicating some
internal mental state; but we must distinguish the communicative act from the internal mental state it expresses.
So, a proposal expresses a goal, an assessment expresses an
evaluation. Also, an opinion is not immediately the communication of some belief but, at a first level, simply a
belief, a bare internal mental state, that may or may not be
communicated to others.
Now, for agreement as well, we must draw the same
distinction: agreement is a mental state, while the expression of agreement is a behaviour aimed at communicating
one’s agreement with a mental state of another.
Agree, Confirm, Admit
To have a clearer view of the expression of agreement, the
communicative act of agreeing may be usefully contrasted
with those of confirming and admitting, which have some
properties in common and some contrasting with it. Let us
first define the notion of confirmation.
To achieve our goals, we need reliable and certain factual information, and according to the laws of human
communication [19], we generally (though sometimes
fallaciously!) believe that a belief is more certain if others
believe it true. So if we have some belief but are not certain
of it and we want to know if someone else has the same
belief, we may phrase it either by an informative sentence
with markers of uncertainty (e.g., ‘‘I think John arrived
yesterday’’) or by an interrogative sentence, namely a yes/
no question (e.g., ‘‘Did John arrive yesterday?’’). In both
cases, we treat that belief as a hypothesis, i.e., a belief we
are not certain of, and wait/ask whether the other confirms
it (yes) or disconfirms it (no) [21]. Replying ‘‘yes’’ or
123
Cogn Comput
simply nodding to an information or a yes/no question
means: ‘‘I also have the same belief that you put as a
hypothesis’’ and counts as ‘‘I confirm what you say’’ [22].
A confirmation is a communicative act through which B
informs A that B also has the same belief mentioned by A
and considers it true, and by doing so raises the level of
certainty A may credit to that belief, reinforcing its
reliability.
This ‘‘sameness’’ of beliefs is what is common to confirmation and agreement, with the difference that in
agreement, the belief B shares with A is not a factual belief
but an opinion.
To be aware of the difference between agreement and
confirmation is important because some communicative
acts as saying ‘‘yes’’ or nodding are ambiguous between the
two: a yes and a nod may follow either a statement or an
expression of opinion, and depending on the previous
statement, they must be interpreted as confirmation or
agreement, respectively.
Another speech act with which agreeing may be confused is admitting. Admitting has certain traits in common
with confirmation, in that also by admitting B informs A
that B has the same belief mentioned by A, but it contains
an extra element: B is in some way confirming a hypothesis
which is not in his interest to mention; he admits something
by confirming it only when he has no other choice than
doing so.
Verbal and Body Signals of Agreement in Political
Debates: a Qualitative Study
In this paper, we report a qualitative study aimed at finding
signals of agreement in debates.
The hypothesis of our research is that agreement may be
conveyed both by verbal and body signals and by their
combination: so our research issue was to find expressions
of agreement in debates and to draw a typology of them
from the points of view of both the verbal and body signals
and of their specific semantic import.
Corpora
To see how people express agreement, we analysed fragments of debates from the Canal 9 and AMI corpora, both
available on the portal of the SSPNet, the ‘‘Social Signal
Processing’’ European Network of Excellence (http://
sspnet.eu/).
Canal 9 is a corpus of political debates collected by the
IDIAP of Martigny (Switzerland). It includes 72 debates
held since 2004 through 2006 at Canal 9, a TV Emitter in
the Canton Valais, where two or four persons, facing each
other and representing opposite positions about political,
123
cultural or social issues, discuss in French at the presence
of a Moderator [23].
AMI corpus is a multimodal data set of 100 h of meeting
recordings in English, some meetings naturally occurring
and some elicited, with participants playing different roles
in a design team [24].
The two corpora differ in recording features, participants, goal of the debate, but also as to the likeliness of
finding cases of agreement in the two. In many Canal 9
debates, it is not easy to find real cases of agreement. First,
by definition, since in public debates, a sort of meta-rule of
media communication holds according to which each participant has the role of being a representative of a particular
position, it would be awkward, possibly incorrect, that
during the debate one changes one’s opinion by showing
convinced by a representative of the opposite party. So
each participant is bound to maintain his/her own point
and, if possible, never to acknowledge the other is right.
Even, out of entertainment purposes, a hot disagreement
may be welcome.
Discussions in the AMI corpus, on the contrary, are not
broadcasted and then more private: people may become
allied or confederates while discussing about some topic,
and later disagree about another, or they may be convinced
by another participant without losing face or falling short
of their own social role. So in the AMI corpus, it is easier to
find cases of agreement.
To find expressions of agreement, we selected four
debates from Canal 9 and one from AMI corpus, for a total
of 5 h, and we analysed them both as to their verbal content
and to the participants’ multimodal behaviour. The former
was analysed in terms of speech acts and discourse analysis, while the body signals were described and classified
in terms of the annotation scheme presented in Table 1.
Multimodality of Agreement: an Annotation Scheme
To analyse the multimodal communication of agreement in
the Canal 9 and AMI debates, we used the annotation
scheme shown in Table 1. Here, column 1 mentions the
time in the video and the Sender of the behaviour under
analysis; col. 2 reports the verbal behaviour and col. 4
behaviour in other modalities, distinguishing head, gaze,
mouth and intonation where present; columns 3 and 5
attribute a meaning to the behaviour of cols. 2 and 4,
respectively, possibly mentioning after an arrow (?),
besides its literal meaning, its indirect meaning, i.e., what
the Sender wants to be inferred from the meaning literally
conveyed.
Table 1 illustrates the analysis of a fragment from the
‘‘Héliski’’ debate of the Canal 9 corpus.
In this debate, participants argue for or against continuing to carry skiers high on the mountains, in the Canton
Cogn Comput
Table 1 Annotation scheme of agreement in verbal and nonverbal modalities
Time,
Sender
Verbal behaviour
8.27
Ah, oui, ça peut se me´riter,
Pouget Oh yes, one can deserve
8.28
Meaning
Body behaviour
Meaning
I agree I rephrase what
you said ? I agree
Head: Slightly inclined
leftward-downward
I am thinking
Gaze: Eyes gazing rightdownward, not at
Interlocutor
I am reflecting on what I really think of this
and how to answer
ça ce me´rite la montagne,
I repeat what you
one deserves the mountain, said ? I agree
Head: turns head rightward, I address you I agree
towards M then nods
Gaze: looks rightward at M
then closes eyelids rapidly
8.29
Effectivement actually
I acknowledge what you Head: raised, back of the
mean is true ? I agree
head slightly down
I address you I agree
I am proud ? I do not submit, I am going to
counter-argue ? I partially disagree
Gaze: Eyes staring off into
space
I am reflecting (that this may be right) ? I
acknowledge ? I agree
Intonation: Raising
intonation
I am not finished ? I am going to counterargue ? I partially disagree
Valais, by a service of helicopters (Héliski). This service is
strongly contrasted by ecologists (represented by Darbellay, a green deputy, and Bally, a member of Mountain
Wilderness) and strongly supported by tour operators
(Bornet) and helicopter pilots (Pouget). Darbellay and
Bornet are on the opposite (spatial and ideological) sides in
the debate.
(1) Canal 9, Héliski
Isabelle Gay, the Moderator of the Héliski debate,
asks Pouget, the helicopter pilot:
‘‘Mais, Monsieur Pouget, à quoi ça rime finalement
d’offrir une si grande facilite´ de la montagne? Est-ce
que ça ce me´rite pas la montagne?’’.
Pouget answers: ‘‘Ah, oui, ça peut se me´riter, ça ce
me´rite la montagne, effectivement’’.
(Isabelle Gay:
‘‘But, Mr. Pouget, what’s the point of offering such a
high ease for the mountain? Shouldn’t the mountain
be deserved?’’
Pouget answers: Oh, yes, one may deserve, one
deserves the mountain, actually).
At time 8.27, Pouget (col. 1) says oui (yes) (col. 2),
meaning he agrees (col. 3) with what the Moderator meant
with her previous rhetorical question. Then, he rephrases her
sentence (ça peut se me´riter = one can deserve, col. 2),
which counts as a part of discourse displaying agreement (see
‘‘Verbal Markers’’). At the same time, his head and gaze (col.
4) convey he is reflecting on his own opinion and on how to
answer (col. 5). At 8.28, he agrees by both verbal and body
behaviour: he literally repeats—hence he agrees with—the
Moderator’s sentence (cols. 2 and 3); turning head and gaze
rightward, he explicitly addresses her, and by his nod and his
eyelid closing (col. 4), he conveys agreement (5) [25]. At
8.29, Pouget conveys two different, almost contradictory
meanings with verbal and body behaviour. He agrees by his
word effectivement (= actually, col. 2), and by gaze staring
off into space, a signal of reflection [3] that in this context
indirectly implies acknowledgement, hence agreement. But
his head and intonation communicate he is only partially
agreeing with M. His raised (proud) head means he does not
completely submit to the idea that to enjoy mountain one
must gain it through effort and fatigue (deserve it, as M
suggested), and also his raising intonation, conveying his
argument is not finished, anticipates that his agreement is
only partial (see ‘‘Apparent Agreement’’).
Results
During a debate, a participant may express agreement in at
least three ways:
1.
2.
3.
by a sentence or discourse (a sequence of two or more
sentences) that expresses some opinion which is
similar or congruent with an opinion previously
expressed by another participant;
by verbal expressions containing specific words
(‘‘agreement markers’’), like ok, I agree, oui or others:
by body signals, like nods, smiles, or by some gestures
or gaze signals.
These verbal and body expressions convey various types
of agreement, which we can distinguish from a semantic
point of view along two dimensions: real/apparent and
stronger/weaker.
Along the real/apparent dimension, we distinguish True,
Indirect and Apparent agreement. Sometimes, in fact, a
123
Cogn Comput
body or verbal expression of agreement can be taken at
face value (true agreement); and within this, at times, no
apparent signal of agreement is produced, but substantive
agreement can be inferred from the global meaning of what
is literally communicated by words or body signals (indirect agreement); sometimes, finally, agreement is only
local, partial or hypocritical, while actually it masks indirect disagreement (apparent agreement). Further, within
true agreement, we can distinguish stronger and weaker
forms (enhanced and unwilling agreement).
In the following, we present these types of agreement
while distinguishing, where possible, how they are conveyed
by three types of signal: discourse, words and body signals.
True Agreement
We call ‘‘true agreement’’ the cases in which a participant
expresses his/her sharing of the same opinion with another
participant. Let us see how true agreement is expressed in
discourse, verbal markers and body signals.
Agreement by Discourse
Agreement is expressed by discourse when a sentence or
sequence of sentences of one participant either literally
repeats or rephrases an opinion expressed earlier by
another participant.
Here is a case of ‘‘literal repetition’’: in a debate about
palliative care, the sentence of a participant is repeated by
one of the opposite side with the same words.
(2) Canal 9, Soins Palliatifs
02.54 M.: Alors, Damian Koenig, je reprends vos
termes, les soins palliatifs—un de´fit pour le futur?
Pour le futur? Aujourd’hui ça veut dire que c’est quoi
alors? C’est pas un de´fit?
03.04 Koenig: Non, c’est un ve´ritable de´fit de´jà à
partir d’aujourd’hui,
…..
08.05 M.: Madame Berthouzoz, vous eˆtes d’accord,
c’est un ve´ritable de´fit, les soins palliatifs?
08.07 Berthouzoz: C’est un ve´ritable de´fit…
02.54 M.: So, Damian Koenig, I come back to your
words, palliative care—a challenge for the future?
For the future? So, what are they today? Is it not a
challenge?
03.04 Koenig: No, it is a true challenge right today,
….
……
08.05 M.: Madame Berthouzoz, do you agree, is it a
true challenge, palliative care?
123
08.07 Berthouzoz : It is a true challenge.
Sometimes, the latter participant does not literally repeat
what the former said, but conveys a very similar meaning
in other words: a ‘‘synonymical rephrasing’’. Here is an
example in a debate about ‘‘Invalidity Insurance’’, the
insurance to be provided by the State to disabled people,
where two participants (Delessert, a member of the Right
Radical Party, and Chevrier, a member of the Centre Right
Democratic Party) are for reducing the insurance, while
Rossini, a leftist politician, and Richoz, the President of the
Association for the Blind and Visually Impaired People,
are against this reduction.
(3) Canal 9, Assurance Invalidité
To the Moderator’s question addressed to Richoz,
what are his expectations from the Invalidity insurance, Richoz answers:
18.17 Richoz: Moi j’attends qu’on ne remet pas aux
calendes grecques les de´cisions concernant le financement de l’ Assurance Invalidite´.
……
19.27 Chevrier: Je trouve qu’il y a effectivement des
mesures d’e´conomie qui sont acceptables mais qui
nous devons tre`s rapidement, et je vous rejoins.
18.17 Richoz: I expect that the decisions regarding
the financing of Invalidity Insurance won’t be postponed for too long.
19.27 Chevrier: I think there are economic measures
which are acceptable but we have to [adopt] them
very rapidly, and I agree with you.
Richoz expresses the hope that decisions about the
financing of the Insurance does not come too late
(qu’on ne remet pas aux calends greques), and one
minute later Chevrier, who is on the opposite side,
rephrases the same concept (nous devons tre`s
rapidement).
Verbal Markers
Agreement can be expressed by ‘‘agreement markers’’:
words or constructions that contain the meaning of agreement in their very semantic content, like d’accord (ok), oui
(yes), vous avez absolument raison (you are absolutely
right), nous sommes d’accord (we agree), je vous rejoins
(I join you [in believing x]), effectivement (in fact), (bien)
e´videmment (obviously), tout-à-fait (absolutely).
A frequent ‘‘agreement marker’’ is ‘‘d’accord’’ (= ok;
I agree), which though, to mean agreement, must be used in
a performative way, that is, as if meant by the same person
who is speaking. Like in these cases of the Héliski debate:
Cogn Comput
(4)
3.24. D’accord, ils sont en infraction avec la loi
(Ok, they are in violation of the law)
(5)
6.42. Nous sommes d’accord
(We agree)
(6)
14.02. D’accord, alors
(Ok, then)
(7)
17.43. Je suis d’accord avec vous…
(I agree with you)
In (4) and (6), ‘‘d’accord’’ is used as an interjection, i.e.,
an isolated utterance, with no syntactic relations to other
words in the same sentence, that by itself conveys the
meaning of a whole speech act [3]. In (5) and (7), it anyway
makes part of a speech act stating that the Speaker shares
the same opinion of the Interlocutor.
Yet, sometimes ‘‘d’accord’’ is not an expression of
agreement by the one who is uttering it, but simply reports
that people in the debate are agreeing about something.
(8)
18.30. Pour ça vous eˆtes d’accord
(So you agree)
(9)
18.50. Oui, parce qu’on est bien d’accord
(Yes, because one agrees)
Also in the following example a sentence containing
d’accord expresses agreement.
(10) Canal 9, Hèliski
06. 24 V. Bornet: Non, je ne partage pas cet avis.
C’est clair que l’environnement est une composante
essentielle, je comprends tout à fait le propos de
Monsieur Darbellay, on doit y prendre garde, on doit
faire attention à l’environnement, on doit aussi tole´rer que des activite´s composent avec l’environnement, puisque finalement on se trouve là dans un
de´cor qui est fe´erique et qui permet de pre´senter la
nature aussi.
06.42 Darbellay: Nous sommes d’accord, disons, il y
a des zones où il y a une activite´ touristique intensives où il y a beaucoup de te´le´skis, de remonte´es
me´caniques, et.c, où il y a beaucoup de gens qui sont
pre´sents. Dans ces zones-là, continuons a pratiquer
l’he´liski, du moins encore pour un certain nombre
d’anne´es, disons…
06. 24 V. Bornet: No, I don’t share the same opinion.
It is clear that the environment is an essential
component, I completely comprehend M. Darbellay’s
proposal. One should be careful, one should pay
attention to environment, but one should also tolerate
that some activities come to terms, combine with the
environment, because one finally finds himself in a
fairy-tale landscape and which allows to present
nature as well..
06.42 Darbellay: We agree, let’s say, there are zones
where there is an intensive touristic activity, where there
are a lot of teleski, of mechanic steps and so on, where a
lot of people are present. In those zones, let us go on with
héliski, at least still, let’s say, for some years…
In this debate, the tour operator Bornet argues in favour
of the Héliski, while the green deputy Darbellay is against
it. Yet, in this example, Bornet acknowledges that the
environment is essential and says he understands the reasons of Darbellay’s proposal. Therefore, Darbellay
acknowledges that he and Bornet share the same opinion
(nous sommes d’accord = we agree). But seeing Bornet’s
acceptance of ecological reasons as a positive nontrivial act
of negotiation by a tour operator, Darbellay feels he should
perform an act of reciprocity, and he acknowledges some
right to the tourist agencies to exploit nature for touristic
purposes in some zones.
In this case, a verbal expression of agreement is in fact a
cue to real agreement. However, some of the expressions
mentioned above are polysemic, that is, they may convey
different types of meanings, some of which actually do not
refer to agreement in the strict sense. For example, both
‘‘d’accord’’ in French and ‘‘ok’’ in English sometimes
simply convey one has understood, but not necessarily one
agrees. This is often the case for some utterances of the
Moderator, Isabelle Gay, in some of Canal 9 debates.
(11) Canal 9, Héliski
20. 43 Pouget: Alors, la formation totale d’un pilote,
je parle à partir du moment où il a son brevet de
pilote d’he´licopte`re, pour eˆtre pilote forme´ professionnel, ça coûte encore une fois, au moins entre cent
mille et cent cinquante mille euro par pilote.
20. 57 Modérateur: D’accord. Le Grand Conseil…
20. 43 Pouget: So, a pilot’s training, I’m talking since
he gets his licence of helicopter pilot, to be a professional formed pilot, it costs at least between
100.000 and 150.000 euros per pilot.
20. 57 Moderator: Ok. The great Council…
Here the Moderator’s response, d’accord, counts more
as an acknowledgement of what Pouget has just said than
as showing agreement. It is like saying ‘‘Ok, I see’’, not
quite as ‘‘I agree’’, ‘‘you’re right’’. In sum, this is more a
signal of backchannel than of real agreement.
123
Cogn Comput
In other cases, again, Gay uses ‘‘d’accord’’ just for her
role of Moderator, to stop the escalation of competitive
utterances between conversationalists.
(12) Canal 9, Héliski
13.50 Darbellay: Mais vous voyez, il y a beaucoup de
gens qui font de la peau-de-phoque et qui disent:
Mois je vais y aller dans la nature! Il vont pas e´tudier
la carte pour savoir quelles sont les trajectoires des
he´licopte`res…
13.56 Pouget: On sait exactement où ce sont ces
he´licopte`res.
13.58 Darbellay: Non non, vous le savez, vous le
savez tre`s bien, mais la majorite´ des peau-dephoqueurs…
14.02 Modérateur: D’accord, alors, on a parle´ des
nuisances sonores, on a parle´ de l’environnement, je
reste avec vous M. Pouget, parque pour vous
l’He´liski repre´sente, je l’ai dit, plusieurs choses, la
plus importante à vos yeux se situe l’entrainement des
pilotes. C’est-à-dire, expliquez-nous.
13.50 Darbellay: But you see, there are many people
who practice the ‘seal skin’, who say: I’m going into
the wild. They do not study the map to know what are
the helicopters’ routes.
13.56 Pouget: People know where precisely the
helicopters are.
13.58 Darbellay: No, no, you know, you do know it
well, but the majority of the seal skin people….
14.02 Moderator: Ok, so, we have talked about noise
perturbations, we have talked about the environment,
I stay with you M. Pouget, since for you the Héliski
represents, as I said, a lot of things, the most
important being, in your view, the pilots’ training.
That is, please explain to us.
Pouget, the helicopter pilot, and Darbellay, the ecologist
deputy, are arguing harder and harder. Darbellay says that
the tourists going on the mountains with the seal skin do so
to enjoy unpolluted nature, hence they are disturbed by
hearing noise and breathing pollution from the helicopter.
Pouget counter-argues that there are specific itineraries for
the helicopter and that it is sufficient not to walk on those
routes not to be disturbed. But Darbellay says it is absurd
that one should study a map to avoid helicopters. Seeing
they are escalating, the Moderator stops them with a
‘‘d’accord’’ (14.02) and then starts interviewing Pouget on
another topic.
Body Signals of Agreement
As witnessed by literature on nonverbal behaviour (see [26]
for a survey), typical signals of agreement are smile,
123
eyebrow raising and nods. In our data, also some gestures
may convey agreement.
(13) Canal 9, Assurance Invalidité
28.31 Chevrier: Parce qu’il y a une nouvelle effectivement culture qui doit s’instaurer, sûrement.
Richoz moves his right hand forward, as if presenting
and showing something
Chevrier: on doit l’ame´liorer
Richoz nods.
Chevrier: on doit accentuer cette collaboration entre
ces deux milieux, parce qu’ils ne sont pas du tout
antinomiques.
28.31: Chevrier: Because there is a new culture which
has to be established, of course,
Richoz moves his right hand forward, as if presenting
and showing something
Chevrier: we have to improve it,
Richoz nods.
Chevrier: we have to emphasise this collaboration
between these two environments, as they are not
completely opposed.
Here, Richoz agrees with his opponent Chevrier not only
by nodding at his sentence, but also by a gesture with open
palm moving laterally; this counts as claiming that something is ‘obvious’ and that about it nothing further needs or
can be added [27]. Therefore, it is like admitting, accepting
what has been said.
In another fragment too, Richoz’s hands, previously
pressed against the desk, suddenly raise with open palms
up, in a gesture which means ‘‘It’s evident’’, ‘‘It’s obvious’’
[28], underlining that the listener totally agrees with the
speaker on this matter.
Body signals, just like words, can be polysemic. For
example, like ‘‘d’accord’’ or ‘‘oui’’ (and ok or yes), also
nodding and smiling may have very different interpretations, depending on the context in which they are produced,
but also on specific aspects of its performance. Namely, a
nod has various possible meanings, among which ones of
agreement, but also ones of bare confirmation or backchannel [22].
Eye and mouth behaviours are also important in conveying agreement. Within gaze signals, the closing of the
eyelids is relevant [25]. When agreeing with the present
Speaker, the Interlocutor’s nods are often accompanied by
rapid blinks, or by wide open eyes, usually with raised eyebrows, that emphasize the extent to which one agrees with
the other: eyelid behaviour as a nod intensifier. Yet, blinks as
cues of agreement can appear also by themselves, without
being accompanied by a nod. In the debate on ‘‘Nurseries:
Mothers as nursery workers’’ of the Canal 9 corpus, a woman
performs rapid repeated blinks while gazing at the Speaker,
Cogn Comput
and the combination of these two signals expresses agreement with what the present Speaker is saying.
Coming to mouth behaviour, smiling is the most
recurrent signal of agreement, but also lip pressing conveys
it. In the same debate about nurseries, the Interlocutor
provides a backchannel to the present Speaker by a double
very rapid short nod and by raising and pressing lower lip.
The nod may simply be a backchannel like ‘‘I see what you
mean’’, but pressing lips conveys something like an
approval, so together they mean agreement.
Stronger and Weaker forms of True Agreement
Agreement can be expressed in stronger or weaker forms.
Enhanced Agreement
The agreement expressed in debates may be, so to speak,
enhanced: people do not simply communicate, they share
other’s opinion, they do something more, providing additional arguments to support it. Here is a case we call
‘‘collaborative agreement’’. In the Canal 9 debate about
‘‘Brain drain’’ in Valais, the participants are Rappaz, a TV
reporter, and three researchers from important institutes,
one of them, Ferrez, being the Head of a prestigious
research Institute, IDIAP.
(14) Canal 9, Fuite des cerveaux
29.42 Chiara Meichtry : Il y a des tre`s belles re´ussites… (There are very nice achievements…)
Rappaz raises and lowers eyebrows fast, while
turning towards Ferrez, as if pointing him to
Meichtry as one of the achievements.
Meichtry: l’IDIAP, je veux dire, enfin, il y a beaucoup
de choses. (Idiap, I mean, well, there are a lot of things).
Rappaz by raising and lowering eyebrows solicits
attention from Meichtry, then performs a deictic body
movement by turning towards Ferrez, as if reminding to
Meichtry, who was speaking of achievements of the Valais:
‘‘here is an example of achievement’’. He is not only
agreeing with Meichtry, but even helping her to find one
more example of what she is saying, i.e., to remind that
Ferrez is a living demonstration of an achievement in
Valais. She accepts this help and concludes mentioning the
IDIAP.
Here is another multimodal example of something more
than agreement: we call it ‘‘complicity agreement’’.
(15) Canal 9, Fuite des cerveaux
23.53 Rappaz : En termes de logement, Gene`ve n’est
pas effectivement la panace´e… (In what housing is
concerned, Geneva is not exactly a panacea).
Crettenand listens to him with lowered head, he
smiles and raises one eyebrow.
The smile with single eyebrow raising is a signal of
complicity. Actually, complicity implies having the same
goals as another and doing things while in agreement with
him, possibly even at the expense of someone else, furtively or secretly [25].
Unwilling Agreement
Sometimes you agree, but you would not like to. Here is
an example of ‘‘unwilling agreement’’. In the ‘‘Héliski’’
debate, Pouget is arguing that forbidding the Héliski
would cause unemployment, as Héliski is a major source
of income to the helicopter company. According to Darbellay instead, Héliski would bring only an additional
profit of 5% of the total gain, so the helicopter company
could actually be dismissed. But Pouget argues that
depending on the season, whether rainy or not, they do
more or less Héliski, reaching even an additional profit of
15–20%:
(16) Canal 9, Héliski
25.42 Pouget: …donc je dis que ça varie entre cinq
ou quinze ou vingt pour cent dans toute l’activite´.
(Then I say it varies between 5 and 15–20% during
the whole activity)
M.: C’est e´norme vingt pour cent, Monsieur Darbellay. (It’s enormous 20%, M.Darbellay).
Darbellay: Steps slightly backwards, lowers head,
joins hands as if praying.
By stepping back and lowering head, Darbellay is
showing submission, while by joining hands, he looks like
apologizing for having so strongly argued against Pouget.
Globally this counts as an admission. ‘‘In front of such
arguments, I must admit I agree’’.
Indirect Agreement
Sometimes, people communicate and they agree in indirect
ways by signals whose literal meaning is not one of
agreement. In this example from the ‘‘Brain drain’’ debate,
Rappaz, the journalist, has just said that the newspapers out
of the canton speak badly of Valais.
(17) Canal 9, Fuite des cerveaux
33.26 Crettenand: …On parle d’image.
With a deictic hand gesture and deictic gaze,
Crettenand points at Rappaz and at the same time
smiles. He is so alluding to what Rappaz has just
said).
123
Cogn Comput
C’est clair que si le Valaisan hors canton lit que des
journaux qui ne sont pas Valaisans et qui parlent des
faits et des gestes particuliers, des mœurs valaisannes,
c’est clair qu’il va pas se faire une bonne image, eh?
Crettenand: … We speak about image.
With a deictic hand gesture and deictic gaze,
Crettenand points at Rappaz and at the same time
smiles. He is so alluding to what Rappaz has just
said).
It’s obvious that if the people of Valais who live
outside the canton read only papers which don’t come
from Valais and which speak badly about Valais, it’s
obvious that they won’t have a good impression about
it, eh?
By simply pointing at Rappaz through deictic gesture
and deictic gaze, Crettenand is indirectly implying he
refers to what Rappaz said, and the smile conveys a positive attitude towards him. Even if none of these signals per
se means ‘‘I agree with him’’, their combination indirectly
conveys agreement with what Rappaz has said.
Apparent Agreement
In the Canal 9 corpus, due to the reasons mentioned in
‘‘Verbal and Body Signals of Agreement in Political
Debates: A Qualitative Study’’, it is very rare to find cases
of agreement. People confront each other strongly, almost
never conceding each other to be right. So, also among the
few cases of apparent agreement, both verbal and nonverbal—for example, sentences containing words like oui
or d’accord, head nods or smiles—some are not really
cases of agreement: the Speaker is simply providing a
confirmation, namely, either giving a positive answer to a
yes/no question or reaffirming a factual information mentioned by the previous Speaker. It is very rare to find cases
in which a yes or a nod convey true agreement with some
opinion, evaluation or proposal.
Moreover, in the ‘‘Héliski’’ debate, even when a Speaker
says ‘‘I agree’’, his agreement is only local, partial, because
after expressing agreement he immediately provides an
opposite argument. Take the case described in Table 1.
Pouget, the helicopter pilot, answers Isabelle Gay, the
Moderator.
(18) Canal 9,Héliski
08. 20 Modérateur: Mais, M. Pouget, à quoi ça rime
finalement d’offrir une si grande facilite´ de la
montagne? Est-ce que ça ce me´rite pas la montagne?
08. 28 Pouget: Ah, oui, ça peut se me´riter, ça ce
me´rite la montagne, effectivement. Maintenant, je
123
dois dire que si l’on arrive aussi à faire plaisir à des
personnes qui ne pratiquent pas la montagne à peaux-de-phoque, qui n’ont pas les moyens de le faire,
je pense à des personnes qui sont âge´es, qui ne
peuvent plus pratiquer la peau-de-phoque, on peut
aussi les amener en montagne pour leur faire plaisir.
08. 20 Moderator: But, Mr. Pouget, what’s the point
of offering such a high ease for the mountain?
Shouldn’t the mountain be deserved?
08. 28 Pouget: Oh, yes, one may deserve, one
deserves the mountain, actually. Now, I should say
that if you also can let some people feel a pleasure,
people who don’t practice the mountain by seal-skin,
who cannot do it, I mean aged people, people who
cannot practice the seal-skin, you can take them on
the mountain to let them feel this pleasure.
As evidenced by the verbal content of this fragment, this
is an example of partial, or better, apparent agreement. The
Moderator asks Pouget: ‘‘Shouldn’t the mountain be
deserved?’’. A rhetorical question, i.e., one aimed at
obtaining a positive answer. Pouget first says ‘‘Oui’’ (yes).
Yet, in this context, this is not necessarily an expression of
agreement but rather a positive answer to the Moderator’s
yes/no question.
Moreover, Pouget immediately extenuates his positive
answer, and the resulting expression of agreement, by using
the modal verb ‘‘on peut’’ (one may). Again, he accepts the
idea, because he repeats the Moderator’s sentence with no
modal verb, and finally adds the adverb ‘‘effectivement’’ (in
fact), which has a meaning of agreement. However, this is
pronounced with a suspensive intonation, a raising pitch
that signals he has not finished speaking (and arguing).
Furthermore, immediately after, he utters another adverb,
‘‘maintenant’’ (now), and the idiomatic expression ‘‘je dois
dire’’, which both may have an adversative meaning: they
signal a contrast with the previous direction of reasoning,
by stopping the inference that could be drawn from the
preceding sentence.
Actually, the mountain should be deserved; but some
people cannot afford it, so letting them have the pleasure of
mountain is an important thing. Thus, if in the first part of
his sentence Pouget seems to agree about a negative
evaluation of Héliski (not requiring the mountain to be
deserved), in the second part, almost in an understatement,
but in a very effective way, he argues for a positive aspect
of Héliski: that it allows people who can’t afford the
mountain to enjoy it all the same.
Finally, that this fragment is a case of only partial and
apparent agreement is also clear from Pouget’s multimodal
behaviour, as results from Table 1.
Cogn Comput
Conclusion
In this work, we have defined the notion of agreement from
a cognitive point of view, while connecting it to other
notions like proposal, assessment, opinion and contrasting
agreement with confirmation and admission. We have
distinguished various types of agreement and quasi-agreement and found several ways to express it by words and
body multimodal communication in TV debates.
Agreement may be expressed in direct and indirect
ways, but also apparent agreement may conceal or precede
utter disagreement. We may convey we agree by repeating
or rephrasing what the other has said, or by using verbal
and body markers—words like ‘‘yes’’, ‘‘ok’’, but also
smiles, head nods and eyelid closings. On the other hand, a
verbal marker like ‘‘I must say’’ or a vocal signal like
raising intonation may anticipate that our agreement is only
partial and that disagreement is following.
Our analysis has shown how the same types of
meaning can be communicated by different modalities:
words and body signals may work both in parallel, as
synonymous with each other, and in a relation of reciprocal enhancement, with their combination in the end
expressing the globality of the meaning intended by the
speaker and with the various aspects of meaning distributed across modalities. In such cases, even incongruity of
meaning can be attributed to signals conveying contradictory meanings.
Of course, additional questions might come to mind
about our classification of agreements. For example, is
there a specific correspondence between the two typologies
outlined, on the signal and on the meaning side? One might
wonder, for example, if the choice between literal repetition and synonymical rephrasing is possibly related to
differences between weak and strong agreement. The
answer is not trivial, since in a sense it is more likely that a
rephrasing, requiring heavier cognitive work, only occurs
for stronger and enhanced agreement, while a literal repetition calls for less deep processing and less commitment
to the stated proposition. In subsequent work, with quantitative analysis carried out on more numerous data, such
questions might find an answer.
Understanding when other people agree with us or
with each other is, even too obviously, a fundamental
skill for our social life. Yet, signals of agreement are not
always explicit or straightforward, they may be subtle or
indirect, or else unreliable, concealing disagreement
under apparent agreement. Constructing systems to detect
and interpret agreement requires extensive quantitative
analysis of multimodal data; but this must be preceded
by careful qualitative analysis of real interaction to allow
capturing all the nuances of agreement signals in
everyday life.
Acknowledgments This research is supported by the Seventh
Framework Program, European Network of Excellence SSPNet
(Social Signal Processing Network), Grant Agreement Number
231287.
References
1. Vinciarelli A, Pantic M, Heylen D, Pelachaud C, Poggi I,
D’Errico F. Social signal processing: bridging the gap between
social animal and unsocial machine. 2010 (forthcoming).
2. Conte R, Castelfranchi C. Cognitive and social action. University
College of London Press: London; 1995.
3. Poggi I. Mind, hands, face and body. A goal and belief view of
multimodal communication. Berlin: Weidler; 2007.
4. Parisi D, Antinucci F. Elementi di grammatica. Torino: Boringhieri; 1973.
5. Vendler Z. Verbs and times. In: Linguistics in philosophy. Ithaca
New York; 1967. pp. 97–121.
6. Comrie B. Universali del linguaggio e tipologia linguistica.
Bologna; 1983.
7. Tavosanis M, Gasperetti M. Comunicare. Milano: Apogeo; 2004.
8. Poggi I. The goals of persuasion. Pragmat Cogn. 2005;13:
298–335.
9. Ogden R. Phonetics and social action in agreements and disagreements. J Pragmat. 2006;38:1752–75.
10. Castelfranchi C, Poggi I. Blushing as a discourse: was Darwin
wrong? In: Crozier R, editor. Shyness and embarrassment. Perspectives from social psychology. New York: Cambridge
University Press; 1990. pp. 230–51.
11. Miceli M, Castelfranchi C. The role of evaluation in cognition
and social interaction. In: Dautenhahn K editor. Human cognition
and agent technology. Amsterdam: John Benjamins; 2000.
pp. 225–61.
12. Poggi I, D’Errico F. Social signals and the action—cognition
loop. The case of overhelp and evaluation. Proceedings of the 1St
IEEE International Workshop on Social Signal Processing,
Amsterdam, 13 September 2009.
13. Stoetzel J. Théorie des opinions, Paris: Puf; 1943.
14. Moscovici S. Social influence and social change, London: Academic Press; 1976.
15. Price V. Public opinion. London: Sage; 1992.
16. Wiebe GD. Some implications of separating opinion from attitudes. Public Opin Q. 1953;17:328–52.
17. Billig M. Ideology and opinions, London: Sage; 1991.
18. Amblard F, Deffuant G. The role of network topology on
extremism propagation with the relative agreement opinion
dynamics. Physica A. 2004; 343:725–38.
19. Castelfranchi C, Poggi I. Bugie, finzioni, sotterfugi. Per una
scienza dell’inganno. Roma: Carocci; 1998.
20. Stern DN. The interpersonal world of the infant, New York: Basic
Book; 1985.
21. Poggi I, Castelfranchi C, Parisi D. Answers, replies and reactions.
In: Parret H, Sbisà M, Verschueren J editors. Possibilities and
limitations of pragmatics. Asterdam: John Benjamins; 1982.
pp. 569–91.
22. Poggi I, D’Errico F, Vincze L. Types of nods. The polysemy of a
social signal. In: Calzolari N, Choukri K, Maegaard B, Mariani L,
Odijk J, Piperidis S, Rosner M, Tapias D editors. Proceedings of
the 7th international conference on language resources and
evaluation, 17–23 May 2010, Valletta, Malta; 2010.
23. Vinciarelli A, Dielmann A, Favre S, Salamin H. Canal9: a
database of political debates for analysis of social interactions. In:
Proceedings of the international conference on affective computing and intelligent interaction (IEEE International Workshop
123
Cogn Comput
on Social Signal Processing), Amsterdam, Netherlands; 2009.
pp. 1–4.
24. Carletta J, Evert S, Heid U, Kilgour J, Robertson J, Voormann H.
The NITE XML Toolkit: flexible annotation for multi-modal
language data. Behav Res Methods Instrum Comput. 2003;
3:353–63.
25. Vincze L, Poggi I. Communicative functions of eye closing
behaviours. Second international workshop on social signal processing, Firenze; 2010.
123
26. Bousmalis K, Mehu M, Pantic M. Spotting agreement and disagreement: a survey of nonverbal audiovisual cues and tools.
Proceedings of the IEEE International Conference on Affective
Computing and Intelligent Interfaces. 2009; 2:121–9.
27. Kendon A. Gesture: visible action as utterance. Cambridge:
Cambridge University Press; 2004.
28. Müller C. Forms and uses of the Palm Up Open Hand: a case of a
gesture family? In: Mueller C, Posner R editors. The semantics
and pragmatics of everyday gestures. Berlin: Weidler; 2004.