Building Trust in PRVAs by User Inner State Transition
through Agent State Transition
Tetsuya Matsui
National Institute of Informatics
2-1-2 Hitotsubashi, Chiyoda, Tokyo
101-8430, Japan
[email protected]
Seiji Yamada
National Institute of Informatics/Sokendai/
Tokyo Institute of Technology
2-1-2 Hitotsubashi, Chiyoda, Tokyo
101-8430, Japan
[email protected]
Moon’s study focused on the trust that was constructed within
the interchange between users and computers. It takes a long
time to construct this trust. In this paper, we aimed for users’
trust to be constructed by only perceiving product information
that PRVAs gave. This trust needs little time and simple technology. We aimed for user trust to be constructed by PRVAs
expressing agent motion and utterance.
ABSTRACT
In this research, we aim to suggest a method for designing
trustworthy PRVAs (product recommendation virtual agents).
We define an agent’s trustworthiness as being operated by user
emotion and knowledgeableness perceived by humans. Also,
we suggest a user inner state transition model for increasing
trust. To increase trust, we aim to cause user emotion to transition to positive by using emotional contagion and to cause user
knowledgeableness perceived to become higher by increasing
an agent’s knowledge. We carried out two experiments to
inspect this model. In experiment 1, the PRVAs recommended
package tours and became highly knowledgeable in the latter
half of ten recommendations. In experiment 2, the PRVAs recommended the same package tours and expressed a positive
emotion in the latter half. As a result, participants’ inner states
transitioned as we expected, and it was proved that this model
was valuable for PRVA recommendation.
USER TRUST STATE TRANSITION MODEL AND USER
STATE TRANSITION OPERATORS
User Trust State Transition Model
We describe a user trust state as the user emotion state and
knowledgeableness perceived state. Our basic hypothesis is
that “positive emotion and high knowledgeableness perceived
brings high trust."
Dunn and Schweizer showed that how much people trust
their partners depended on familiarity and emotion [?]. They
showed that positive people tended to trust partners when
partners were unfamiliar with the truster. Myers and Tingley
studied negative emotion [?] . They showed negative people trust their partners less than positive people. From these
studies, the emotion state seemed to affect the trust state.
ACM Classification Keywords
H.5.2. User Interfaces
Author Keywords
product recommendation virtual agent, user inner state,
emotional contagion, trustworthy, anthropomorphic agent
We aimed to cause a user’s emotion state to transition by using
emotional contagion. Emotional contagion is a phenomenon
in which a speaker’s emotion is spread to a partner [?]. Tsai et
al. showed that emotional contagion can be caused between
users and virtual anthropomorphic agents [?].
INTRODUCTION
PRVAs, product recommendation virtual agents, are agents
that take the role of clerks and advisers in online stores. In
this paper, we experimented to design trustworthy PRVAs to
increase users’ buying motivation. Terada et al. showed that
the appearance of PRVAs affected the recommendation result
[?]. Kamei et al. experimented with robot clerks and showed
that customers feel more familiarity with robots who navigated them toward the store rather than robots that only recommended lunch [?]. Moon et al. showed that self-disclosure,
exchanging information with each other, induced users to give
their information to computers [?].
“Knowledgeable" is one aspect of intelligence. Geven et al.
showed that users perceived more intelligence and trustworthiness with real agents rather than cartoon-like agents [?]. Also,
Mimoun et al. indicated that a lack of intelligence was one of
the most crucial problems with anthropomorphic agents [?].
In our experiment, we focused on knowledgeableness, one
aspect of intelligence. From these prior pieces of research,
it is clear that positive emotion and high knowledgeableness
perceived brings high trust.
Permission to make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for profit or commercial advantage and that copies bear this notice and the full citation
on the first page. Copyrights for components of this work owned by others than ACM
must be honored. Abstracting with credit is permitted. To copy otherwise, or republish,
to post on servers or to redistribute to lists, requires prior specific permission and/or a
fee. Request permissions from [email protected].
In our model, we described a user’s emotion state and knowledgeableness perceived state as { +} and { − }. State { +}
means a positive or high state, and state { − } contains a
negative, low, and neutral state. We defined a user’s basic
state to be { − − }. The left value means the emotion state,
and the right value means the knowledgeableness perceived
state. Thus, state { − − } means a negative or neutral emotion
HAI ’16, October 04-07, 2016, Biopolis, Singapore
© 2016 ACM. ISBN 978-1-4503-4508-8/16/10. . . $15.00
DOI: http://dx.doi.org/10.1145/2974804.2974816
111
Figure 1. User inner state transition model
Figure 2. Agent with positive emotion transition operators and agent
without operators (neutral emotion)
and low or neutral knowledgeableness perceived. If a user’s
emotion transitions to positive, the user state transitions to {
+ − }. If a user’s knowledgeableness perceived transitions to
high, the user state transitions to { − + }.
Also, we executed smooth utterance with text to speech software, VOCELOID+ Yuzuki Yukari EX2 . Figure ?? shows the
agent with and without emotion transition operators.
The PRVAs recommended package tours to Japanese Middle
Ages castles. They recommended ten tours by taking turns,
and the destination changed for each recommendation. The
order of destinations was random for each participant. We
defined each recommendation as Rn, and n means the recommendation number (1 to 10). Each recommendation took no
longer than one minute.
Our goal state was { + + }, that is, positive emotion and high
knowledgeableness perceived. Users having this state seem to
trust PRVAs more than when having the other three states. We
show this model in Figure ??.
Positive emotion and high knowledgeableness are instinctively
effective for PRVAs or any agent; however, it is not clear
whether positive emotion and high knowledgeableness work
together. In this paper, we inspected this point.
From R1 to R5, the PRVAs executed only positive emotion
operators. The PRVAs made recommendations with smiles
and cute gestures; however, their utterances contained only
location and expense information. Also, from R6 to R7, the
PRVAs executed positive emotion transition operators and high
knowledgeableness perceived transition operators. They made
recommendations with smiles, cute gestures, and knowledge
on location, expense, history, and architecture. Thus, from
our hypothesis, from R1 to R5, the participants’ state was { +
− }, positive emotion and low knowledgeableness perceived.
Also, from R6 to R7, the participants’ state was { + + }. Thus,
knowledgeableness perceived transitioned to high.
User State Transition Operators
The aim of this research was to increase users’ trust by causing
only the agent state to transition. Thus, positive emotion and
high knowledgeableness perceived were executed with only
the agent state. Positive emotion was executed by “agent’s
smile" and “cute gesture" in order to cause positive emotional
contagion [?]. High knowledgeableness perceived was executed by “much product knowledge."
We carried out two experiments to prove that these transition
operators were effective in user inner state transition and that
our transition model was valuable for designing PRVAs.
After watching each recommendation, participants were asked
to answer some questions. In this paper, we focus on the result
for two of the questions as follows.
EXPERIMENT 1: EXECUTING HIGH KNOWLEDGEABLENESS PERCEIVED
• Q1: Did you feel happy when you watched the movie?
Experiment 1: Materials and Method
Participants
• Q2: Did you feel that the agent has correct knowledge?
Fifteen Japanese participants were recruited for experiment
1. They were aged between 20 and 39, for an average of 29.0
and SD was 5.7. There were seven males and eight females;
however, one female’s data was removed from analysis because of machine trouble. Each participant spent about 20 to
30 minutes for the whole experiment.
• Q3: Did you feel that the agent have considerable persuasive
power?
Question Q1 was the indicator of the user’s emotion, and Q2
was the indicator of the user’s knowledgeableness perceived.
Q3 was the indicator of the agent’s trustworthy perceived by
user. All participants were asked to answer “Yes" or “No" for
these questions. Also, all participants were asked to answer
the same questions before recommendations after watching
PRVAs standing without any motion and utterance.
Task
The PRVAs and recommendation systems were constructed
with MMDAgent 1 , the free dialogue agent “Mei," and a platform that is distributed by the Nagoya Institute of Technology.
1 http://www.mmdagent.jp/
2 http://www.ah−soft.com/voiceroid/yukari/
112
Figure 3. Ratio of participants’ answers for Q1 and Q2 in experiment 1
Figure 4. Ratio of participants’ answers for Q1 and Q2 in experiment 2
Experiment 1: Result
perceived. Also, from R6 to R7, the participants’ state was { +
+ }. Thus, emotion transitioned to positive.
We conducted a statistical analysis based on the ratio of the
number of participants who answered “Yes" for each question. The x−axis indicates each questions. We conducted a
chi-square test for every successive recommendations. If there
were significant differences, the number of users whose inner
state transitioned increased significantly between those two
recommendations. For the result of Q1, there were no significant differences. For the result of Q2, there was a significant
difference between R5 and R6 (p<0.05).
Experiment 2: Result
We conducted the same analysis as in experiment 1. For the
result of Q1, there was a significant difference between R5 and
R6 (p<0.05). For the result of Q2, there were no significant
differences.
The left graph of Figure ?? means the ratio of the number
of participants who answered “Yes" for Q1 after R5 and R6.
The right graph of Figure ?? means the ratio of the number of
participants who answered “Yes" for Q2 after R5 and R6.
The left graph of Figure ?? means the ratio of the number
of participants who answered “Yes" for Q1 after R5 and R6.
The right graph of Figure ?? means the ratio of the number of
participants who answered “Yes" for Q2 after R5 and R6.
Regarding Q3, we conducted a chi-square test between the
average of the ratio of the participants who answered “Yes" in
the first five recommendations and the ratio of the participants
who answered “Yes" in the latter five recommendations. As a
result, there was a significant difference between R5 and R6
(p<0.05).
Regarding Q3, we conducted a chi-square test between the
average of the ratio of the participants who answered “Yes" in
the first five recommendations and the ratio of the participants
who answered “Yes" in the latter five recommendations. As a
result, there was a significant difference between R5 and R6
(p<0.01).
DISCUSSION
The result shows that our hypothesis and model is proper. The
right graph of Figure ?? shows that our knowledgeableness
perceived transition operators worked according to our assumption. This graph shows that many participants felt that
the agents imparted historical and architectural knowledge was
more knowledgeable than agents who imparted only location
and expense information. This result seems to be axiomatic;
however, historical and architectural knowledge seemed to
not be more important information than location and expense
information for many users. It was possible that participants
judged the knowledgeable PRVAs as being redundant. This
result contradicted this expectation. Also, this result seems
to suggest that an “informative" agent is perceived knowledgeable/intelligent. However, knowledgeable PRVAs may
not only impart many pieces of knowledge but also proper
information to users.
EXPERIMENT 2: EXECUTING POSITIVE EMOTION
Experiment 2: Materials and Method
Participants
Fifteen Japanese participants were recruited for experiment 2.
They were aged between 20 and 39, for an average of 29.3 and
SD is 6.9. There were eight males and seven females. Each
participant spent about 30 minutes for the whole experiment.
Task
The PRVAs, recommendation products, recommendation format, and questions for experiment 2 were the same as in experiment 1. All participants watched the recommendations for
package tours to Japanese castles. The difference was what
transition operators were executed for each recommendation.
From R1 to R5, the PRVAs executed only high knowledgeableness perceived operators. The PRVAs’ recommendations
contained historical and architectural knowledge; however,
they stayed expressionless without making any gestures. Also,
from R6 to R7, the PRVAs executed high knowledgeableness
perceived transition operators and positive emotion transition
operators. They made recommendations with historical and
architectural knowledge, smiles, and cute gestures. Thus, from
our hypothesis, from R1 to R5, the participants’ state was { −
+ }, negative or neutral emotion and high knowledgeableness
The left graph of Figure ?? shows that our emotion transition
operators worked according to our assumption. This graph
shows that smiles and cute gestures were effective for emotional contagion between agents and users. This effect was
already reported for 3D game characters and users [?] and
robots and users [?]. However, there are scarcely any studies
about emotional contagion between PRVAs and users. This
result suggests that emotional contagion is valid for PRVAs to
increase their own trustworthyness.
113
CONCLUSION
In this research, we suggested a model of user inner state transition regarding trustworthiness and experimented to inspect
this model. This model supposes that the user inner state can
be operated by only changing the agent state. The result of
experiments showed that this hypothesis was proper and that
we can cause user the inner state to transition in an arbitrary
phase. The emotion transition operators and the knowledgeableness perceived transition operators immediately worked
after executing operators. Also, trustworthiness can be increased by only an agent’s positive emotion and knowledge
without complex interactions.
This result suggests a valuable new design method for PRVAs.
This method does not need any particular equipment and prerecommendation phase. Also, this result suggests that complex
social relationships can be constructed without bidirectional
communications.
Figure 5. Observed transition paths in two experiments
ACKNOWLEDGMENT
This study was partially supported by JSPS KAKENHI "Cognitive Interaction Design" (No.26118005).
The left graph of Figure ?? and the right graph of Figure ??
show no significant differences. However, in both figures, the
ratio of R5 showed high values. The participants’ emotion
state kept at { + } between R5 and R6 in experiment 1, and
their knowledgeableness perceived state kept at { + } between
R5 and R6 in experiment 2.
REFERENCES
1. Jennifer R Dunn and Maurice E Schweitzer. 2005. Feeling and believing:
the influence of emotion on trust. Journal of personality and social
psychology 88, 5 (2005), 736.
2. Arjan Geven, Johann Schrammel, and Manfred Tscheligi. 2006.
Interacting with embodied agents that can see: how vision-enabled
agents can assist in spatial tasks. In Proceedings of NordiCHI 2006.
135–144.
The result of Q3 shows the emotion state and knowledgeableness perceived state transitions effected to trustworthy state.
When emotion state and knowledgeableness perceived state
transited to { + }, trustworthy state also transited to positive.
3. Elaine Hatfield, John T Cacioppo, and Richard L Rapson. 1994.
Emotional contagion. Cambridge university press.
4. Koji Kamei, Tetsushi Ikeda, Masahiro Shiomi, Hiroyuki Kidokoro,
Akira Utsumi, Kazuhiko Shinozawa, Takahiro Miyashita, and Norihiro
Hagita. 2012. Cooperative customer navigation between robots outside
and inside a retail shop?an implementation on the ubiquitous market
platform. annals of telecommunications-annales des télécommunications
67, 7-8 (2012), 329–340.
From these result, we constructed two different participants’
transition paths as shown in Figure ??. From this, we suggested that the transition operators worked immediately for
user inner state transition. In general, long and complex interaction (for example, self-disclosure [?]) was regarded as the
essential phase constructing trust or other social relationships
between an agent and user. Our result suggested that trust
could be constructed by only expressing the agent state. This
method takes little time and can be executed on finite media,
for example, small screens or simple structure machines.
5. Mohammed Slim Ben Mimoun, Ingrid Poncin, and Marion Garnier.
2012. Case study?Embodied virtual agents: An analysis on reasons for
failure. Journal of Retailing and Consumer services 19, 6 (2012),
605–612.
6. Youngme Moon. 2000. Intimate exchanges: Using computers to elicit
self-disclosure from consumers. Journal of consumer research 26, 4
(2000), 323–339.
7. Dan Myers and Dustin Tingley. 2011. The influence of emotion on trust.
Unpublished PhD thesis.) Princeton, NJ: Princeton University (2011).
Also, these results suggested that emotion and knowledgeableness perceived can be operated independently. In both
experiments 1 and 2, participants’ emotion and knowledgeableness perceived transitioned independently. This is valuable
for constructing agents for varying aims.
8. Hiroki Ozono, Motoki Watabe, Sakiko Yoshikawa, Satoshi Nakashima,
Nicholas O Rule, Nalini Ambady, Reginald B Adams Jr, and others.
2010. What’s in a Smile? Cultural Differences in the Effects of Smiling
on Judgments of Trustworthiness. Letters on Evolutionary Behavioral
Science 1, 1 (2010), 15–18.
This research contained some limitations. First, we did not
use biological signals (for example, brain waves, perspiration,
and heartbeat) to measure the user inner state. This is our future works. Second, although PRVAs can work passing across
borders, the notion of trust depends on cultures. Ozono et
al. showed that both Japanese and American people judged
partners as being trustworthy by observing their smiles; however, they focused on the other part of smiling faces [?]. This
study suggests that we need to customize an agent’s smiling
depending on the culture.
9. Kazunori Terada, Liang Jing, and Seiji Yamada. 2015. Effects of Agent
Appearance on Customer Buying Motivations on Online Shopping Sites.
In Proceedings of CHI ’15. 929–934.
10. Jason Tsai, Emma Bowring, Stacy Marsella, Wendy Wood, and Milind
Tambe. 2012. A study of emotional contagion with virtual characters. In
Intelligent Virtual Agents. Springer, 81–88.
11. Junchao Xu, Joost Broekens, Koen Hindriks, and Mark A Neerincx.
2015. Mood contagion of robot body language in human robot
interaction. Autonomous Agents and Multi-Agent Systems 29, 6 (2015),
1216–1248.
114