Emoticon Recommendation in Microblog using
Affective Trajectory Model
Wei-Bin Liang, Hsien-Chang Wang∗ , Yi-An Chu and Chung-Hsien Wu
Department of Computer Science and Information Engineering, National Cheng Kung University, Tainan, Taiwan
E-mail: [email protected]
∗ Department of Information Management, Chang Jung Christian University, Tainan, Taiwan
E-mail: [email protected]
Abstract—An emoticon is a metacommunicative pictorial
representation which is widely used in text-based online
communication such as Plurk and Facebook to convey the user’s
emotions. However, these social networks still lack a mechanism
to provide appropriate emoticon recommendation according to
the input posts. Therefore, this paper develops an approach to
emoticon recommendation in microblog. Generally, a blog post
is composed of at least one emotional topic. Therefore, topic
tracking is the key information for emoticon recommendation.
In this paper, a fixed-size window is first employed to segment
a post into a number of segments. Then, these segments are
projected to emoticon profiles in the emoticon space through
latent Dirichlet allocation (LDA). An affective trajectory model
characterizing the emoticon profiles of the segment sequence
is proposed to construct a recommendation model based on
k-medoids algorithm. Finally, emoticon recommendation can be
realized by similarity measure based on Hausdorff distance.
To evaluate the performance of our proposed approach, the
experimental data were crawled from Plurk for training and
evaluation. The results show the effectiveness of the proposed
approach.
Index
Terms:
emotion
recognition,
recommendation, microblog, trajectory
emoticon
I. I NTRODUCTION
One of the primary functions of the internet is to
connect people online. Recently, hundreds of millions of
people around the world actively used social media such
as Plurk and Facebook for communication or information
sharing. Therefore, there is great interest in affective
computing [1] of the social media across a variety of
domains, for example, health monitoring
[2][3] and
product recommendation [4][5]. Emotions play an important
role to realize the application of these domains because
emotion affects human intelligence, rational decision making,
social interaction, perception and more. Although emotion
recognition can be achieved from speech, facial expression,
physiological signal and text, text-based emotional analysis
does not lose its popularity. Previous research for emotion
recognition from texts considered a variety of textual contents,
for example, microblogs, stories [6][7], news [8], and spoken
dialogs [9].
An emoticon is a metacommunicative pictorial
representation composed of strings of symbols and has
been widely used in text-based online communication to
convey emotions. The use of emoticons contributes to
978-616-361-823-8 © 2014 APSIPA
Today, buy one Starbucks coffee get one free
Haha the experimental results are not good
Fig. 1.
Two example posts with emoticons
facilitation of the social interactions, especially in microblogs.
Figure 1 demonstrates the example posts with emoticons.
Compared to conventional text-based communication, the
bottom example has ambiguity of emotion because the words
Haha and good are often used to indicate the happiness but
the word not presents opposite meaning.
Two main and mostly used approaches for emotional
modeling are categorical and dimensional [10]representations.
The categorical approaches is based on classification of
emotions as basic emotions such as happiness, sadness and
boredom that are hard-wired in human brain and recognized
universally. Social networks nowadays has already provided
various emoticons more than Ekman’s six basic emotions [11].
On the other hand, within the dimensional approach,
researchers argue that emotional states are not independent
from one another but related in a systematic manner. To
determine the emotional dimensions is a challenging issue;
however, the two dimensions, arousal and valence, has shown
to cover the majority of affect variability and been widely
used [12][13]. One of the famous work is afffective norms for
English words (ANEW) [14] that comprises a set of normative
emotional ratings for English words; however, how to rate and
assess emotions is still questionable. Emotion classification
approaches can be broadly classified into lexicon-based
approach and learning-based approach. An emotional lexicon
comprises a set of affective terms for each emotion, for
example, ANEW, LIWC and WordNet-Affect [15]. The
lexicon-based approach is a straightforward solution; however,
ambiguity of word sense and the lack of linguistic information
result in implicit expression of emotions. Another way
of emotion classification is to train a model to classify
input observations into different emotions. Machine learning
approaches such as Naı̈ve Bayes, support vector machine [16],
vector space models e.g., LSA, pLSA [17], and latent Dirichlet
allocation (LDA) [8].
Current mechanism of social network is that users manually
pick a proper emoticon from the supplied emoticon pool.
APSIPA 2014
Sometimes, it is hard to select and there are a lot of mental
burden for users. Therefore, the goal of this paper is to develop
a text-based automatic emoticon recommendation system.
First, we will perform emoticon analysis through affective
text-mining for input posts. Then, a recommendation model
considering the advantage of either categorical emotions or
dimensional emotions will be trained.
Post with Primary Emoticon
Liang - feeling Joy
Good night, I will go to sleep
Although I am hungry and exhausted
I completed my work
rest things will be to do tomorrow
Fluctuation
Fig. 3. Example of emotional fluctuation that different segments of post can
be tagged as different emoticons
II. S YSTEM OVERVIEW
Figure 2 illustrates the framework of the proposed emoticon
recommendation system in which the top panel is the training
phase and the bottom panel is the test phase. For an input post
from the collected microblog corpus, we assume that one post
comprises the primary emoticon and the embedded emoticons
because of emotional fluctuation. Moreover, blog post often
lack explicit punctuation marks so that the sentences even
paragraphs are not easy to identify. To alleviate this problem,
a blog post is regarded as a emotion trajectory. Therefore,
the first step is to segment the input post into a number
of segments using a fixed-size sliding window. In addition,
ambiguity of word sense also confuses the perception of a blog
post. So, each post segment is then projected to the emoticon
profiles in the emoticon space. To project the post segments,
the state-of-the-art of natural language processing - LDA is
employed to obtain the emoticon profiles. Thus, the system can
estimate the probability of each emoticon for an input post.
The problem for the generated emoticon trajectory is that the
sizes of trajectories are not equal for all trajectories. Moreover,
for an unknown post, we do not know which segment can
represent the primary emoticon. In other words, the posts of
two emoticons may be projected to the same trajectory. In the
proposed system, the k-medoids clustering with the Hausdorff
distance metric is employed to cluster the trajectories in the
next step. Hence, the centroids and the occurrence probability
of the j-th trajectory ej conditioned on the i-th cluster ci is
obtained. To measure the confidence to a trajectory cluster, the
Gaussian density is employed to model a probability to each
input post. In the affective trajectory modeling, both of the
centroids of the clustered trajectories E with the parameters of
Gaussian density and the occurrence probabilities are included
into the affective trajectory model. The last procedure of this
system is to recommend the emoticon e∗ by concluding the
information from occurrence probability of the j-th emoticon
conditioned on the i-th cluster and the information from the
likelihood of the j-th emoticon estimated by the Gaussian
mixture model based on the Hausdorff distance.
III. A FFECTIVE E MOTICON R ECOMMENDATION M ODEL
Give an input post W , the goal of our system is to
recommend the most likely emoticon e∗ by estimating
probability of P r(ej |W ) for j-th emoticon ej over all
emoticons ΩE . Moreover, due to ambiguity of word sense,
all emoticons are further clustered. Therefore, P r(ej |W ) is
written as
e∗ = arg
= arg
max
P r(ej |ci , W )
max
P r(W |ej )P r(ej |ci )
P
ej P r(W |ej )P r(ej |ci )
ej ∈ΩE ;ci ∈ΩC
ej ∈ΩE ;ci ∈ΩC
(1)
where ci is the i-th cluster in cluster space ΩC . In this paper,
the evidence in Eq.(1) is assumed to be equal and ignored so
that the formula is further written as
e∗ = arg
max
ej ∈ΩE ;ci ∈ΩC
P r(W |ej )P r(ej |ci ).
(2)
Eq.(2) presents the recommendation model which estimate the
probability of emoticon ej and ej is occurred underlying the
cluster ci for input post W . We will describe how to compute
these two terms in detail in the following sections.
IV. F EATURE E XTRACTION
A. Post Segmentation
For text-based online communication, the post are often
organized by emotion fluctuation. In other words, a post may
comprise not only the primary emoticon but also the number
of embedded emoticons. Moreover, the sentence of primary
emoticon can be allocated at any position. Therefore, it is not
easy to capture the real affective state of a post. Figure 3
demonstrates an example that a post was tagged one primary
emoticon whereas more than one emoticon can be tagged at
segments. In this paper, we assume that the context of a post
is a trajectory of emoticon. However, blog posts often lack
explicit punctuation marks so that sentences even paragraphs
are not easy to identify. Herein, a fixed-size sliding window
is employed to segment a post into M segments, that is,
W → {t1 , t2 , . . . , tM }
(3)
B. Affective Trajectory Mapping
After post segmentation, the m-th segment tm is mapped
to an emoticon profile and defined as


P r(tm |e1 )
 P r(tm |e2 ) 


sm = 
(4)

..


.
P r(tm |e|E| )
where |E| denotes the number of emoticon.
To compute the j-th element P r(tm |ej ) of emoticon profile
sm , we employ the LDA to obtain the word sense of
Fig. 2.
Framework of the proposed emoticon recommendation system
Fig. 5.
Example of similarity measure between two affective trajectories
be employed directly for our task in which the major problem
is the different lengths of two emoticon trajectories.
B. Distance measure
Fig. 4.
Concept of LDA-based affective text-mining
emoticons. Figure 4 illustrates the concept of the LDA-based
emoticon profile generation. The latent variable Z is employed
to connect the relation between emoticons and all words.
Hence, the term P r(tm |ej ) is further written as
P r(tm |ej ) = P r(tm |Z)P r(Z|ej )
(5)
For simplicity, the affective trajectory of the input post W will
be denoted as S.
V. A FFECTIVE T RAJECTORY M ODELING
A. Trajectory Clustering
Even the post is projected to emoticon profile, the effect for
emoticons having similar trajectories is inevitable. To alleviate
this problem, k-medoids clustering scheme with a distance
matrix [18] is employed to group the trajectories in to I
clusters. In contrast to the k-means algorithm, k-medoids
chooses datapoints as medoids (or center) and work with
arbitrary matrix of distance between datapoints instead of
L2 -norm. In common with most other similarity measure
such as L2 -norm and cosine measure, the recommendation
system adopts the Hausdorff distance (HD) for similarity
estimation. Figure 5 illustrate an example why L2 -norm cannot
The Hausdorff distance (HD) is a metric between two sets
of points extracted from the object model and the test data.
The classical HD measure [19] between two trajectories Sk
and Sl is defined as
H(Sk , Sl ) = max(h(Sk , Sl ), h(Sl , Sk ))
(6)
h(Sk , Sl ) = max min ka − bk
(7)
where
a∈Sk b∈Sl
Hereby h(Sk , Sl ) is called the directed Hausdorff distance
from set Sk to set Sl with some underlying L2 -norm k · k
on the points of Si and Sj .
To compute the term P r(W |ej ) in Eq.(2) underlying cluster
Cj , Gaussian density is utilized to quantize the HD-based
similarity measure to confidence measure. Hence,
P r(W |ej ) = P r(S|ej )
= N (H(S, Sij )|Λij = {µij , σij })
(8)
where N (sm |Λij ) is the Gaussian density with mean vector
µij and standard deviation σij .
C. Trajectory Distribution
The last step of the recommendation model is to represent
trajectory distributions by a large sparse matrix. In this
paper, this matrix can be simply constructed after trajectory
clustering. Therefore, the probability of the j-th trajectory
Fig. 6. Search bar of Plurk. We can collect emoticon “hungry” by the query
sentence with “(hungry)”
Fig. 8.
Fig. 7.
Top-k recommendation accuracy (%) with different window sizes
Distribution of 21 emoticons
conditioned on i-th cluster P r(ej |ci ) can be computed by
nij
P r(ej |ci ) = P
(9)
i nij
where nij is the number of trajectories of the j-th emoticon
in the i-th cluster.
VI. E VALUATION
Fig. 9.
Comparison between LDA and our proposed approach
A. Data Collection
To evaluate the proposed method of the proposed emoticon
recommendation system, we collected over 5,000 blog posts
from the social network, Plurk (http://www.plurk.com/top/),
which is one of most popular microblogging service in Taiwan.
Plurk supports the search function, so we can collected the
posts containing one emoticon only. Figure 6 demonstrates the
post collection containing one emoticon by the query sentence
with string“(hungry)”. Then, a crawler was implemented to
download and parse the query results. Finally, twenty-one most
frequent emoticons as shown in Figure 7 were selected to for
evaluation.
For the affective trajectory mapping, LDA was implemented
by R package, topicmodels, and trained on the collected data
and the number of topics was set to ten heuristically.
B. Results
Because of emotional fluctuation, top-k most likely
emoticons will be recommended according to the input
post in our proposed system. The first experiment was to
evaluate the effect of the window size for post segmentation.
Figure 8 shows the recommendation accuracy with different
window sizes. The results reveal that the best recommendation
performance of the proposed system achieved 76.05%
accuracy where the window size was thirty for top-5
recommendation. It is reasonable that a Chinese post consists
of about thirty words.
Based on the window size of 30 words, figure 9 shows
the comparison between our proposed approach and the
LDA-based baseline. This experiment was designed to evaluate
the performance of affective trajectory. Because we cannot
guarantee that the blog posts are always organized without
emotional fluctuations, the LDA-based baseline is confused
by the ambiguity of word sense.
VII. C ONCLUSIONS
An emoticon recommendation system based on affective
trajectory is proposed, implemented and evaluated. The main
idea is that the blog posts are segmented into a sequence
of segments using a sliding window, mapped to an affective
trajectory by LDA-based method and the k-medoids clustering
algorithm. Moreover, a sophisticated similarity measure of
trajectory based on the Hausdorff distance is used to estimate
the distance between two trajectories. Evaluation results
show that the proposed approach using affective trajectory
outperformed the systems based on LDA-based method.
R EFERENCES
[1] R. W. Picard, Affective Computing. MIT Press, 1997.
[2] X. Wang, C. Zhang, Y. Ji, L. Sun, L. Wu, and Z. Bao, “A depression
detection model based on sentiment analysis in micro-blog social
network,” in Trends and Applications in Knowledge Discovery and Data
Mining, ser. Lecture Notes in Computer Science, J. Li, L. Cao, C. Wang,
K. Tan, B. Liu, J. Pei, and V. Tseng, Eds. Springer Berlin Heidelberg,
2013, vol. 7867, pp. 201–213.
[3] L. Coviello, Y. Sohn, A. D. I. Kramer, C. Marlow, M. Franceschetti,
N. A. Christakis, and J. H. Fowler, “Detecting emotional contagion in
massive social networks,” PLoS ONE, vol. 9, no. 3, p. 90315, 03 2014.
[4] R. Dong, M. P. O’Mahony, M. Schaal, K. McCarthy, and B. Smyth,
“Sentimental product recommendation,” in Proc. ACM Conference on
Recommender Systems, ser. RecSys’13. ACM, 2013, pp. 411–414.
[5] R. Dong, M. Schaal, M. O’Mahony, K. McCarthy, and B. Smyth,
“Opinionated product recommendation,” in Case-Based Reasoning
Research and Development, ser. Lecture Notes in Computer Science,
S. Delany and S. Ontañón, Eds. Springer Berlin Heidelberg, 2013, vol.
7969, pp. 44–58.
[6] S. M. Kim, A. Valitutti, and R. A. Calvo, “Evaluation of unsupervised
emotion models to textual affect recognition,” in Proc. NAACL HLT 2010
Workshop on Computational Approaches to Analysis and Generation of
Emotion in Text, ser. CAAGET’10. Association for Computational
Linguistics, 2010, pp. 62–70.
[7] A. Agrawal and A. An, “Unsupervised emotion detection from text using
semantic and syntactic relations,” in Proc. IEEE/WIC/ACM International
Joint Conferences on Web Intelligence and Intelligent Agent Technology
- Volume 01, ser. WI-IAT’12.
IEEE Computer Society, 2012, pp.
346–353.
[8] S. Bao, S. Xu, L. Zhang, R. Yan, Z. Su, D. Han, and Y. Yu, “Mining
social emotions from affective text,” Knowledge and Data Engineering,
IEEE Transactions on, vol. 24, no. 9, pp. 1658–1670, Sep 2012.
[9] W.-B. Liang, C.-H. Wu, and M.-H. Sheng, “Affective-cognitive dialogue
act detection in an error-aware spoken dialogue system,” in Signal
and Information Processing Association Annual Summit and Conference
(APSIPA), 2013 Asia-Pacific, Oct 2013, pp. 1–6.
[10] R. Calvo and S. D’Mello, “Affect detection: An interdisciplinary review
of models, methods, and their applications,” IEEE Tran. Affective
Computing, vol. 1, no. 1, pp. 18–37, Jan 2010.
[11] P. Ekman, “An argument for basic emotions,” Cognition & Emotion,
vol. 6, no. 3–4, pp. 169–200, 1992.
[12] W.-L. Wei, C.-H. Wu, and J.-C. Lin, “A regression approach to
affective rating of chinese words from anew,” in Affective Computing
and Intelligent Interaction, ser. Lecture Notes in Computer Science,
S. D’Mello, A. Graesser, B. Schuller, and J.-C. Martin, Eds. Springer
Berlin Heidelberg, 2011, vol. 6975, pp. 121–131.
[13] C.-H. Wu, W.-L. Wei, J.-C. Lin, and W.-Y. Lee, “Speaking effect removal
on emotion recognition from facial expressions based on eigenface
conversion,” Multimedia, IEEE Transactions on, vol. 15, no. 8, pp.
1732–1744, Dec 2013.
[14] M. Bradley and P. Lang, “Affective norms for english words (anew):
Stimuli, instruction manual and affective ratings,” The Center for
Research in Psychophysiology, University of Florida, Tech. Rep., 1999.
[15] C. Strapparava and A. Valitutti, “Wordnet-affect: an affective extension
of wordnet,” in Proc. International Conference on Language Resources
and Evaluation, ser. LRE’2004, 2004, pp. 1083–1086.
[16] S. Aman and S. Szpakowicz, “Using roget’s thesaurus for fine-grained
emotion recognition,” in Proc. International Joint Conference on Natural
Language Processing (IJCNLP), 2008, pp. 312–318.
[17] S. M. Kim, A. Valitutti, and R. A. Calvo, “Evaluation of unsupervised
emotion models to textual affect recognition,” in Proceedings of the
NAACL HLT 2010 Workshop on Computational Approaches to Analysis
and Generation of Emotion in Text, ser. CAAGET ’10. Stroudsburg,
PA, USA: Association for Computational Linguistics, 2010, pp. 62–70.
[18] L. Kaufman and P. Rousseeuw, “Clustering by means of medoids,” in
Statistical Data Analysis Based on the L1-Norm and Related Methods,
Y. Dodge, Ed. North-Holland, 1987, pp. 405–416.
[19] D. Huttenlocher, G. Klanderman, and W. Rucklidge, “Comparing images
using the hausdorff distance,” IEEE Transactions on Pattern Analysis
and Machine Intelligence, vol. 15, no. 9, pp. 850–863, Sep 1993.