JOURNAL OF INFORMATION SCIENCE AND ENGINEERING 29, 647-662 (2013)
Building a Graded Chinese Sentiment Dictionary
Based on Commonsense Knowledge
for Sentiment Analysis of Song Lyrics
HUI-HSIN WU1, ANGELA CHARNG-RURNG TSAI1, RICHARD TZONG-HAN TSAI2,+
AND JANE YUNG-JEN HSU1,+
1
Department of Computer Science and Information Engineering
National Taiwan University
Taipei, 116 Taiwan
2
Department of Computer Science and Engineering
Yuan Ze University
Chungli, 320 Taiwan
With the rise of social media, sentiment analysis has become a popular research
field in recent years. Sentiment dictionaries are vital for this task; however, there are few
available in Chinese. Translated English sentiment dictionaries are often inaccurate or
lacking vocabulary. Moreover, many dictionaries provide only binary polarity values or
no values at all. In this paper, we integrate several common sentiment dictionaries into a
larger dictionary. We then describe an algorithm that can extrapolate sentiment values
for the empty entries in this dictionary from the annotated ones using the relationships in
Chinese ConceptNet. The resulting iSentiDictionary is a Chinese sentiment dictionary
with 28,248 concepts annotated with sentiment values. To test our dictionary, we construct a dictionary-based song lyrics sentiment analysis system. We compare performance of iSentiDictionary with that of three translated sentiment dictionaries: ANEW,
SenticNet and SentiWordNet. Experimental results show that using iSentiDictionary
achieves the best results in terms of both Word Coverage and Absolute Sentiment Error.
We believe this is because of iSentiDictionary’s high word coverage for Mandarin pop
songs.
Keywords: sentiment analysis, sentiment dictionary, common sense knowledge, song
lyrics analysis
1. INTRODUCTION
Sentiment analysis refers to the application of natural language processing to identify and extract subjective information from source materials. It aims to determine the
writer’s attitude to some topic or the overall contextual polarity of a document3. A critical technology is using a sentiment dictionary to identify sentiment words in documents.
Broadly speaking, sentiment dictionaries can be divided into three types: those containing only sentiment words [1, 2], those containing words with sentiment polarities [3-6],
and those containing words annotated with sentiment values [7-9]. The dictionary compiled for this work belongs to the latter type. Note that the dictionary with sentiment values contains most information among the three kinds.
In order to build sentiment dictionaries with broader vocabulary, several researchers
Received February 7, 2012; accepted June 21, 2012.
Communicated by Toyoaki Nishida.
+
Corresponding author.
3
http://en.wikipedia.org/wiki/Sentiment_analysis/
647
648
HUI-HSIN WU, CHARNG-RURNG TSAI, TZONG-HAN TSAI AND JANE YUNG-JEN HSU
[2, 4, 6, 9, 10] have developed methods for calculating the sentiment values/polarities of
‘valueless’ words from those in existing dictionaries automatically. WordNet-Affect [4],
SentiWordNet [10] and HowNet-VSA [6] use synonym relationships to expand existing
vocabulary.
Using such WordNet-type dictionaries, one can identify sentiments for text in which
opinions are explicitly expressed using positive and negative terms (e.g., good/bad, nice/
nasty, excellent/awful). But for sentiments expressed implicitly through context (e.g., a
comment on Lady Gaga’s new song is “A Christmas gift”), these keyword dictionaries
are not effective. Researchers in MIT’s Media Lab have developed a frame-work for
teaching computers commonsense knowledge called ConceptNet [11], a semantic network in which objects of commonsense concepts are represented as nodes. Concepts are
expressed in natural language with sets of related words and phrases: for example, “take
a picture”, “taking pictures”, “to take pictures”, and “you take a picture” are various surface forms of the same Concept. In addition, ConceptNet contains more various types of
relationships between two concepts than WordNet. The relationships can be used to populate sentiment values between valued nodes and valueless nodes. For example, if word
A and B have an is-A relationship, B’s sentiment value is very likely to be correlated to
A’s value. In 2009, Cambria [9] built a sentiment dictionary SenticNet, which uses dimensionality reduction to infer the polarity of common sense concepts in ConceptNet
and hence provides a public resource for mining opinions from natural language text at a
semantic, rather than just syntactic, level. They demonstrated SenticNet to be superior to
SentiWordNet in sentiment analysis when analyzing patients’ opinions.
To our knowledge, there are only two Chinese sentiment dictionaries with annotated
sentiment values [12, 13]. Neither are publicly available. In addition, these dictionaries
only contain 1000-2000 words, which may be insufficient for sentiment analysis of diverse natural language texts, such as blogs or lyrics.
The goal of this paper is to build a Chinese sentiment dictionary with a much larger
vocabulary than existing ones. We collected our sentiment seeds from existing sentiment
dictionaries. The seeds were classified into four categories: S-Value, S-Polarity, S-Ambiguity and S-Unknown. Then, we proposed a self-training sentiment spreading activation to spread the sentiment values on ConceptNet. The self-training method takes advantage of the different information provided by the four kinds of seeds and stops concepts with incorrect sentiment values from propagating their values to other concepts. Finally, we derived iSentiDictionary, which is a Chinese sentiment dictionary with 28,248
concepts, each having a sentiment value between  1 and + 1.
We review existing sentiment dictionaries according to their characteristics in section 2. ConceptNet and related works are introduced in section 3. We describe our proposed method in section 4. The experiments are shown in section 5. In section 6, we illustrate how we employ the compiled dictionary to estimate songs’ sentiment values
based on their lyrics. The conclusion is given in section 7.
2. EXISTING SENTIMENT DICTIONARIES
We introduce nine common sentiment dictionaries used as our sentiment seeds in
this section: ANEW, SentiWordNet, SenticNet, General Inquirer-Emotion (GI-Emotion),
Wordnet-Affect, National Taiwan University Sentiment Dictionary (NTUSD), HowNet-
BUILDING A SENTIMENT DICTIONARY BASED ON CONCEPTNET FOR SENTIMENT ANALYSIS OF LYRICS 649
Vocabulary for Sentiment Analysis (HowNet-VSA), WeFeelFine and Never Ending Language Learner-Emotion (NELL-Emotion).
2.1 Simple Sentiment Word-List Dictionaries
The simplest type of sentiment dictionary consists of a list of sentiment words with
no annotation to indicate opinion polarity. The We Feel Fine project [1] has been collecting human “feelings” from a variety of major blog publishers since 2006. The system
scours blog posts for the phrases “I feel” or “I’m feeling”, and then compares containing
sentences against a list of 5000 predefined sentiment words. We refer to this list as the
WeFeelFine dictionary.
The Never-Ending Language Learner (NELL) [2] is an intelligent computer agent
that is constantly extracting web data to populate a growing structured knowledge base.
Never Ending Language Learner-Emotion (NELL-Emotion) contains 3,109 English concepts learned by the Never-Ending Language Learner.
2.2 Dictionaries with Sentiment Polarities
The General Inquirer (GI) [3] was one of the first sentiment dictionaries. It has 11,787
English “words” (different meanings of a word are counted separately) and 182 labels.
The 3,682 English words labeled as “positive outlook” or “negative outlook” in GI can
be considered a sentiment dictionary (GI-Emotion).
WordNet-Affect [4] marks some synsets in WordNet with sentiment labels, e.g.,
EMOTION, MOOD, COGNITIVE STATE, PHYSICAL STATE. We use a sub-dictionary comprised of the 1,587 English words with EMOTION labels in WordNet-Affect in
our work. There are 492 words in the dictionary with the label “positive-emotion”, 895
words labeled “negative-emotion”, 22 words labeled “neutral-emotion” and 141 words
with the “ambiguous-emotion” label. The remaining 37 words have been labeled as more
than 2 emotions.
National Taiwan University Sentiment Dictionary (NTUSD) [5] is a Chinese sentiment dictionary derived from GI and the Chinese Network Sentiment Dictionary (CNSD).
Ku et al. translated GI and merged the result with CNSD. Then, they used the synonym
relations to derive the NTUSD sentiment dictionary, which has 11,088 Chinese words.
HowNet [6] was developed in 1988 based on the idea that knowledge consists of the
relations between concepts and the relations between the attributes of concepts. HowNet
Vocabulary for Sentiment Analysis (HowNet-VSA) is a sentiment dictionary with both
Chinese and English contents released in 2007. We use four different sentiment categories
from HowNet-VSA. The number of concepts in each category is listed below in Table 1.
Table 1. Number of concepts in each HowNet-VSA category.
Category
English
Chinese
Positive Emotion
769
833
Negative Emotion
1,011
1,254
Positive Opinion
3,594
3,730
Negative Opinion
3,563
3,116
650
HUI-HSIN WU, CHARNG-RURNG TSAI, TZONG-HAN TSAI AND JANE YUNG-JEN HSU
2.3 Dictionaries with Sentiment Values
The Affective Norms for English Words (ANEW) [7] provides a set of normative
emotional ratings for a collection of 1,034 English words. Each word in ANEW is rated
in terms of pleasure, arousal and dominance respectively with a value ranging from 1 to
9. The value represents the degree of the word in the corresponding dimension. ANEW
was compiled manually in 1999 and has been widely used in the sentiment analysis area
since [14, 15].
SentiWordNet [8] assigns each synset of WordNet three sentiment values: positivity,
negativity and objectivity. Each sentiment value ranges from 0 to 1, and the summation
of the sentiment values of a synset is 1. There are a total 147,306 English words in SentiWordNet. 96,155 of them are labeled as objective words (objectivity value equals to 1).
SenticNet [9] was developed by Cambria et al. in 2010, which contains 5,273 English concepts. Each concept is labeled with a sentiment value between  1 and + 1 to represent the degree of pleasantness of the concept. Cambria et al. used the sentiment keywords and the corresponding values defined in the Hourglass of Emotions as seeds and
derived the sentiment values of other concepts on the AffectiveSpace [16]. The AffectiveSpace is a vector space composed of WordNet and ConceptNet.
3. COMMONSENSE KNOWLEDGE BASE
We chose ConceptNet, a commonsense knowledge ontology, as our sentiment value
propagation ontology. We introduce ConceptNet and related works in this section.
Commonsense knowledge is general and widely applicable knowledge about the
everyday world that is possessed by all people. In AI research, the term is used to refer to
the millions of basic facts and understandings possessed by most people. For many years,
the MIT Open Mind Common Sense (OMCS) [17] project has devoted their energies to
encoding commonsense knowledge into machine readable forms.
3.1 ConceptNet
ConceptNet [11] is a unique resource in that it captures a wide range of commonsense concepts and relations built by OMCS. There are 15 kinds of relations and each has
a polarity attribute. ConceptNet focuses on a richer (though still very pragmatic) set of
semantic relations (e.g., EffectOf, DesireOf, CapableOf) between compound concepts (e.g.,
“buy food,” “drive car”) rather than focusing on formal taxonomies of words. It contains
over 1 million English commonsense sentences collected from the OMCS website.
To add Chinese concepts to ConceptNet, we created a community-based social game
to collect sentences that contain assertions of two concepts [18]. Since 2010, this ongoing
project has collected over 250,000 sentences. The assertions are verified via question-answering between players [18]. English ConceptNet was used as a guide for generating
questions to ask users to improve the coverage of Chinese concepts within resource
bounds [19]. These verified Chinese commonsense assertions are then used to construct a
directed graph. The nodes of this graph are concepts, and labeled edges are relations between two concepts.
BUILDING A SENTIMENT DICTIONARY BASED ON CONCEPTNET FOR SENTIMENT ANALYSIS OF LYRICS 651
3.2 Sentiment Analysis using ConceptNet
Previous studies have used ConceptNet to derive sentiment values for concepts.
Their basic idea is that “related concepts share common sentiment.” For example, the concepts “blood,” “hurt” and “ambulance” are linked to the concept “car accident” through
ConceptNet edges. Therefore, the sentiment value of “car accident” should be similar to
its related concepts.
Liu et al. [20] used the Ekman model to represent emotion in a vector of six dimensions [happiness, sadness, anger, fear, disgust, surprise]. They started with seed words
and applied a “spreading activation” algorithm on ConceptNet to spread the sentiment
values for each concept.
Cambria et al. developed SenticNet, a publicly available semantic resource built
from ConceptNet and WordNet-Affect. SenticNet uses dimensionality reduction to infer
the polarity of common sense concepts. The calculation of concept polarity is based on
the assumption that relative distances between concepts in AffectiveSpace, are directly
proportional to their polarity degree difference.
Our approach is based on the work of Liu et al. [20]. We combine multiple existing
dictionaries, generate seed words, and improve the basic spreading activation by leveraging the information provided in the dictionaries.
4. OUR SENTIMENT DICTIONARY: ISENTIDICTIONARY
To compile a high-coverage Chinese sentiment word dictionary in which every
word has a sentiment value, we must integrate the dictionaries mentioned in section 2,
find accurate translations for each sentiment word, and spread sentiment values from
words with values to words without values using the relationships between words.
We first collect sentiment seeds from multiple dictionaries, pre-classified into the
three types. Then, we propose a self-learning sentiment-spreading activation method
based on the base sentiment-spreading activation method proposed by Liu et al. [20]. Our
system’s architecture is shown in Fig. 1.
Fig. 1. Architecture of proposed system.
652
HUI-HSIN WU, CHARNG-RURNG TSAI, TZONG-HAN TSAI AND JANE YUNG-JEN HSU
4.1 Sentiment Seed Collection
We collect sentiment seeds from the three kinds of dictionaries mentioned in section
2: dictionaries that collect sentiment words only, dictionaries with sentiment polarities
and dictionaries with sentiment values.
4.1.1 Translation of sentiment dictionaries
The first step of sentiment seed collection is to translate words from English to Chinese. We use Google Translate API2 and Yahoo Dictionary3 as our translation tools.
Given an English word, Google Translate outputs one of its translations while Yahoo
Dictionary outputs sets of possible translations. We designed a method to combine these
two translation tools’ results to exploit both Google’s high precision and Yahoo’s high
coverage. Given a English word e, suppose Google outputs g1 and Yahoo outputs y1, y2,
y3, ..., yi, ..., yn. If g1 appears in the translation set yi, we treat yi as e’s translation. Otherwise, g1 is used as e’s translation. Take “admired” for example, Google Translate outputs
“欽佩” and Yahoo Dictionary outputs “{欽佩、欣賞}、{稱讚, 誇獎}、{對...不勝佩服}”. We
choose “{欽佩、欣賞}” as the translation result because the translation set contains “欽佩”.
4.1.2 Expanding sentiment dictionaries
We filter three kinds of words which appear before the original words to expand all
sentiment dictionaries. (a) Sentiment superfluous words: concepts’ sentiment labels are
unchanged after filtering superfluous words, e.g., “覺得”(feel), “感覺”(feeling) and “使”
(make). (b) Sentiment degree words: concepts’ sentiment polarities are unchanged after
filtering degree words, e.g., “一些”(some), “非常”(very) and “極度”(extremely). (c) Negative words: concepts’ sentiment labels reverse after filtering negative words, e.g., “不
”(no), “不可”(not) and “不能”(can’t).
Table 2. Mapping from original tags in each dictionary to the sentiment polarities.
GI
WordNet-Affect
NTUSD
HowNet-VSA
Positive Sentiment
“positive outlook”
“positive-emotion”
“positive”
“positive emotion” or
“positive opinion”
Negative Sentiment
“negative outlook”
“negative-emotion”
“negative”
“negative emotion” or
“negative opinion”
4.1.3 Translation of Sentiment Dictionaries
For our dictionary, we use the same scheme as SenticNet to represent sentiment values: s ranges from  1 to + 1. For ANEW, we normalize the pleasure value as the sentiment value. For SentiWordNet, we regard the value of the “positive score” minus the
“negative score” as the sentiment value and then filter out the words with sentiment values equal to 0.
2
3
https://developers.google.com/translate/
http://tw.dictionary.yahoo.com/
BUILDING A SENTIMENT DICTIONARY BASED ON CONCEPTNET FOR SENTIMENT ANALYSIS OF LYRICS 653
For the dictionaries with polarity, the sentiments are divided into three categories:
positive, negative and ambiguous. The original labels in the dictionaries and the corresponding sentiment polarities are listed in Table 2. A concept with both positive and
negative sentiments is regarded as ambiguous.
For our sentiment seeds, we select all concepts that appear in at least two dictionaries. There are four different types of sentiment seeds: (a) S-Value: sentiment seeds labeled with sentiment values ranging from  1 to 1. The final sentiment value is the average of all sentiment values of the seed from different dictionaries. (b) S-Polarity: sentiment seeds labeled as positive or negative only. If the seed has more positive sentiment
labels, it is regarded as positive, whereas if it has more negative labels, it is regarded as
negative. (c) S-Ambiguous: sentiment seeds that have ambiguous sentiment. If the seed
has more ambiguous sentiment labels or the number of positive sentiment labels is equal
to number of negative sentiment labels, it is regarded as ambiguous. (d) S-Unknown:
sentiment seeds without annotation of polarity and value. Table 3 shows the number of
seeds in each type.
Table 3. The number of sentiment seeds in the four categories after merging the
dictionaries.
S-Value S-Polarity S-Ambiguity S-Unknown
Quantity 6,464
1,951
2,641
11
Total
11,067
4.2 Sentiment Spreading Activation
Sentiment expressions depend not only on the meanings of words, but also on their
semantics and contexts. Our approach is based on the work of Liu et al. [20], which relies on having a large-scale real-world knowledge base about people’s common affective
attitudes toward situations, things, people, and actions. We chose Chinese ConceptNet as
our knowledge base because of its breadth and coverage.
We use the sentiment seeds introduced in section 4.1 to spread sentiment values on
Chinese ConceptNet. In ConceptNet, each node represents a concept and each edge represents the relation between two concepts. The relation between two concepts is regarded
as a sentiment-spreading bridge. For example, the assertion: “紅色代表熱情” (Red symbolizes passion). “紅色” (red) is concept 1, “熱情” (passion) is concept 2 and “代表”
(symbolizes) is a relation between “紅色” (red) and “熱情” (passion). If we know the
sentiment value of “紅色” (red), we spread the sentiment value from “紅色” (red) to “熱
情” (passion).
In this section, we first introduce the base sentiment spreading activation proposed
by Liu et al. Then we propose our method, the self-training sentiment spreading activation. Our method avoids the problem of lower accuracy when multiple spreading passes
are carried out over ConceptNet in base sentiment spreading activation.
4.2.1 Base sentiment spreading activation
Liu et al. [20] regard sentiment propagation as spreading activation or undirected
inference on ConceptNet. The sentiment is represented as a six-degree vector [happiness,
654
HUI-HSIN WU, CHARNG-RURNG TSAI, TZONG-HAN TSAI AND JANE YUNG-JEN HSU
sadness, anger, fear, disgust, surprise] according to six basic emotions. For each sentiment seed, the vector is defined according to the basic emotion it belongs to: the value is
1 for the seed’s predefined sentiment degree and 0 for the other degrees. The seeds
propagate their sentiment values to neighboring concepts  i.e. the concepts in the same
assertions as the seeds. Negations are identified by checking if “not” appears. The propagating value is multiplied by  1 whenever a negated relation is detected.
Liu et al. use a second and third pass to improve the coverage of the dictionary. After each propagation, the sentiment values are discounted by a factor d. An example of
propagation for the base sentiment spreading activation model is given here. (The sentiment vector of happy is [1,0,0,0,0,0], the sentiment vector of surprising is [0,0,0,0,0,1]
and the discount factor d is 0.5.)
Something exciting is both happy and surprising.
(pass 1: exciting: [1,0,0,0,0,1])
Rollercoasters are exciting.
(pass 2: rollercoaster: [0.5,0,0,0,0,0.5]
Rollercoasters are typically found at amusement parks.
(pass 3: amusement park: [0.25,0,0,0,0,0.25])
4.2.2 Self-training sentiment spreading activation
Unlike the base sentiment spreading activation, in the self-training sentiment spreading activation, we define four types of sentiment seeds (S-Value, S-Polarity, S-Ambiguity and S-Unknown). We hope to learn more information from these seed groups and
solve the accuracy decay problem that the base sentiment spreading activation encountered.
In order to improve the coverage, the base sentiment spreading activation runs a second and a third pass, discounting the sentiment value by a factor d after each pass.
However, this method does not consider whether the concept has the ability to transfer
sentiments. If the sentiment value of this concept is wrong, then spreading its sentiment
value may cause more concepts to end up with the wrong sentiment values.
To solve this problem, we propose that only “sentiment concepts” are allowed to
transfer sentiment values. “Sentiment concepts” refers to all concepts of the S-Value, SPolarity, S-Ambiguity and S-Unknown types. Moreover, a concept in the S-Polarity
group loses its ability to transfer sentiment whenever its sentiment polarity is not consistent with its sentiment value. Concepts able to transfer sentiments are regarded as active
sentiment seeds.
An example of determining which concepts can spread sentiments is given in Fig. 2.
Assume the sentiment value of “快樂(happy)” is 0.8. After sentiment spreading, each
concept in “幸福(blessing)”, “睡不著(can’t fall asleep)”, “跳舞(dance)” and “形容詞
(adjective)” has a sentiment value 0.8 propagated from “快樂(happy)”. According to the
rule, “幸福(blessing)” and “跳舞(dance)” will be able to spread sentiments in the next
pass, while “睡不著(can’t fall asleep)” and “形容詞(adjective)” will not.
For each concept whose value is propagated from multiple concepts, the base sentiment spreading activation regards the average of all spread sentiment values as the final
sentiment value of this concept. However, this calculation method causes many concepts
to have a sentiment value close to zero (sentiment offset) in our implementation.
BUILDING A SENTIMENT DICTIONARY BASED ON CONCEPTNET FOR SENTIMENT ANALYSIS OF LYRICS 655
Fig. 2. An example of spreading sentiments on a small part of the Chinese ConceptNet.
After counting the number of neighboring seeds in the two polarities of each seed,
we find that the average number of positive neighbors of positive seeds is five times
higher than the average number of negative neighbors, and the average number of negative neighbors of negative seeds is four times higher than the average number of positive
neighbors.
Therefore, if the number of positive propagations is greater than the number of negative ones, we calculate the average of all positive sentiment propagation values as the
final sentiment value. Otherwise, we calculate the average of negative sentiment propagation values as the final sentiment value.
To summarize the whole self-training sentiment spreading activation, we first use SValue as the initial active sentiment seeds to stimulate sentiment spreading activation.
After each sentiment spreading activation pass, we filter out the newly propagated concepts which do not have the ability to spread sentiment values and add the remaining
ones to the active sentiment seeds for the next pass.
5. EXPERIMENT
Our ultimate goal is to create a Chinese sentiment dictionary with a large vocabulary and annotated with sentiment values and further improve the accuracy of Chinese
sentiment analysis. We first collect the sentiment seeds and propagate the sentiment values on the Chinese ConceptNet to get more sentiment items. We propose a self-learning
sentiment spreading activation to increase the coverage without decreasing the accuracy.
The input of this experiment is the four kinds of sentiment seeds listed in section
4.1.3 and Chinese ConceptNet. The output is a sentiment dictionary composed of a set of
concepts, each with a sentiment value that stands for the pleasure degree of the concept
ranging from  1 to + 1. We implemented two algorithms, the base sentiment spreading
activation (Base SSA) and our proposed method, the self-learning sentiment spreading
activation (SL SSA), and compared their results.
We use a 5-fold cross validation. The seeds in S-Value are randomly divided into
five equal-sized sets in every test. S-Value is composed of 6,464 seeds, so there are 5,171
training seeds and 1,293 test seeds. Then the experiment uses three values to evaluate the
result of sentiment spreading activation, the three values are:
656
HUI-HSIN WU, CHARNG-RURNG TSAI, TZONG-HAN TSAI AND JANE YUNG-JEN HSU
1. Mean Accuracy of Sentiment Polarity (%): The average of 5-fold test accuracies,
which only considers the sentiment polarity of the test seeds. A larger Mean Accuracy of Sentiment Polarity indicates the sentiment spreading activation has more sentiment concepts with the correct sentiment polarity.
2. Mean Error: The average of 5-fold test error values, which considers the mean distance between the result and ground truth sentiment value of the test seeds. A smaller
Mean Error represents that the result is more precise.
3. Coverage: The vocabulary size of the final sentiment dictionary.
The values in S-value are used as the ground truth sentiment values of the test seeds.
We ran three passes for each sentiment spreading algorithm because SL SSA did not find
any new seeds after three passes. The results are shown in Table 4. Passing more passes
in Base SSA gets a lower Sentiment Polarity Mean Accuracy and a higher Mean Error. In
contrast, our proposed method, SL SSA, maintains a stable Sentiment Polarity Mean
Accuracy and Error Value with different numbers of passes. The results show that SL
SSA allows more passes over the ConceptNet without lowering accuracy. The coverage
of Base SSA and SL SSA increase by spreading more passes. Although the coverage of
Base SSA is larger than SL SSA on two and three passes, Base SSA cannot ensure spreading accuracy. In summary, the experiment results show that our proposed method SL SSA
performs better than Base SSA.
Table 4. Evaluation of Base SSA and SL SSA.
Coverage
Pass 1
Pass 2
Pass 3
Mean Accuracy of
Mean Error
Sentiment Polarity (%)
BaseSSA
21,934
84.32
0.2896
SL SSA
21,887
84.46
0.2781
BaseSSA
77,843
79.41
0.3298
SL SSA
28,166
85.07
0.2700
BaseSSA
88,734
78.87
0.3330
SL SSA
28,248
85.28
0.2679
6. APPLICATION: SENTIMENT VALUE ESTIMATION BASED
ON SONG LYRICS
One application of sentiment dictionaries is using them to estimate sentiment values
for song lyrics. In the following paragraphs, we illustrate our approach to calculate a
song’s sentiment value based on a sentiment dictionary d.
For each song m, we first calculate the sentiment value of each line. Then, we aggregate the sentiment values of all lines to calculate m’s overall sentiment value. For any
line l in m, we calculate the sentiment values of all sentiment words in l. We first identify
sentiment words using d. For each matched term w0, we find the nearest preceding degree
word before w0 and the following degree word after w0. The set of degree words is selected from HowNet [6]. There are two types of degree words: preceding degree words
BUILDING A SENTIMENT DICTIONARY BASED ON CONCEPTNET FOR SENTIMENT ANALYSIS OF LYRICS 657
and following degree words, which are listed in Table 5. The nearest_preceding_degree(w0, l) refers to the amplifying coefficients of the nearest preceding degree word of
w0 in l, while the nearest_following_degree(w0, l) refers to the amplifying coefficients of
the nearest following degree word. The sentiment value of w0 in d is represented as sv(w0,
d). [w-3, w-1] represents the previous one to three words of w0 the in l.
Table 5. Amplifying coefficients of degree words.
Degree Words
Amplifying Coefficients
極度、極其、非常、極為、絕對、滔天、
十分、完完全全、萬般、萬分、無比、
最為、超級
很、多麼、分外、格外、好不、何等、
Preceding 相當、相當多的、太、超、頗為、更為
更加、特別、越加
或多或少、一些、稍、稍微、稍許、稍
為
Following
1.5
1.3
1.1
0.8
略微、略為、一點、有點、有些、輕度
0.6
不得了、不可開交
1.5
之極、之至
1.3
Then we use Eq. (1) to calculate w0’s sentiment value in l.
sv_unit(w0, l, d) = no_preceding_negative_word(w0, l)
*nearest_preceding_degree(w0, l)* sv(w0, d)
*nearest_following_degree(w0, l)
where
no_preceding_negative_word(w0, l)
 1.5, if there is at least a negative word appearing in [w1 , w3 ]
.

1, otherwise
(1)
Then we use Eqs. (2) and (3) to calculate the average positive and negative sentiment
values for l:
avg _ pos _ unit(l, d) 
 sv _ unit(w, l, d)
wW
W
 sv _ unit(w, l, d)
avg _ neg _ unit(l, d)  wW

W
(2)
(3)
658
HUI-HSIN WU, CHARNG-RURNG TSAI, TZONG-HAN TSAI AND JANE YUNG-JEN HSU
where W+ is the list composed of all words with positive sv_unit. W− is the list composed
of all words with negative sv_unit.
Then, we use Eq. (4) to calculate l’s sentiment value:
sv _ sentence(l, d)
 avg _ pos _ unit(l, d), if avg _ pos _ unit(l, d)  avg _ neg _ unit(l, d)

  0, if avg _ pos _ unit(l, d)  avg _ neg _ unit(l, d)

 avg _ neg _ unit(l, d), if avg _ pos _ unit(l, d)  avg _ neg _ unit(l, d)
(4)
Then, we can calculate average positive and negative sentiment values for a song m:
 sv _ sentence(l, d)
avg _ pos _ sent(m, d) 
lL
avg _ neg _ sent(m, d) 
lL
L
 sv _ sentence(l, d)
L
(5)
(6)
where L+ is the list composed of all sentences with positive sv_sentence. L− is the list
composed of all sentences with negative sv_sentence.
Then, we use Eq. (7) to calculate m’s sentiment value:
sv _ song(m, d)
 avg _ pos _ sent(m, d), if avg _ pos _ sent(m, d)  avg _ neg _ sent(m, d)

  0, if avg _ pos _ sent(m, d)  avg _ neg _ sent(m, d)

 avg _ neg _ sent(m, d), if avg _ pos _ sent(m, d)  avg _ neg _ sent(m, d)
(7)
The reason we calculate average positive and negative sentiment values separately
is based on the observation that if we calculate the average of all sentiment words, the
resulting value is usually very close to zero.
6.1 Experiment on Music Mood Estimation
We use the above dictionary-based approach to estimate sentiment values for the
660 Mandarin songs in the iPop dataset [21] and compare the effects of using different
sentiment dictionaries. Since ANEW, SenticNet, and SentiWordNet are English dictionaries, we employ Google Translation to translate the words in these dictionaries into
Chinese. We first report the coverage of words in these songs for these dictionaries. Then,
we compare the accuracies of sentiment value estimation using these dictionaries. We
define two evaluation metrics to measure the effectiveness of a given sentiment dictionary d for a song dataset SD.
The first is Word Coverage, which is defined as follows:
BUILDING A SENTIMENT DICTIONARY BASED ON CONCEPTNET FOR SENTIMENT ANALYSIS OF LYRICS 659
word _ coverage(d ,SD) 
# of distinct words in both SD and d
# of distinct words in SD
(8)
The second is Absolute Sentiment Error, which is defined as follows:
absolute _ sentiment _ error (d ,SD)


sv _ song (m, d )  gold _ standard _ sv(m)
mSD
(9)
SD
where gold standard sv(m) is the average human ratings of song m in the iPop dataset.
In Table 6, we can see that using our iSentiDictionary to predict iPop songs’ sentiment values is more accurate than using other dictionaries. We believe that this is because iSentiDictionary’s high word coverage for iPop songs. It is interesting to note that
although ANEW has lower coverage than SenticNet and SentiWordNet, its average absolute sentiment error is lower, probably because it is manually compiled while the others are automatically generated.
Table 6. Word coverage and absolute sentiment error of all dictionaries.
Absolute_sentiment_error
Dictionary d
word_coverage(d, iPop)
(d, iPop)
iSentiDictionary
32.69%
0.4568
Translated ANEW
6.30%
0.5858
Translated SenticNet
11.79%
0.6518
Translated SentiWordNet
11.97%
0.7315
7. CONCLUSION
This paper presents a novel method to compile a Chinese sentiment dictionary, iSentiDictionary. We collected our sentiment seeds from existing sentiment dictionaries.
The seeds were classified into four categories: S-Value, S-Polarity, S-Ambiguity and SUnknown. Then, we proposed a self-training sentiment spreading activation to spread the
sentiment values on ConceptNet. Finally, we derived iSentiDictionary, which is a Chinese sentiment dictionary with 28,248 concepts, each having a sentiment value between
 1 and + 1. The self-training method takes advantage of the different information provided by the four kinds of seeds and stops the concepts with a wrong sentiment value
from propagating their values to other concepts. Therefore, the proposed method
achieves a higher sentiment polarity accuracy and a lower sentiment value mean error
than the base sentiment spreading activation. Immediate future work will examine how to
reweight the values from different concepts. There is a score for each commonsense sentence in ConceptNet representing how many people input the sentence. We think this
score may be a good weighting function to help determine how the values from different
concepts should be combined. Moreover, the score is useful for filtering out noise in
ConceptNet.
660
HUI-HSIN WU, CHARNG-RURNG TSAI, TZONG-HAN TSAI AND JANE YUNG-JEN HSU
We also proposed a dictionary-based approach to estimate sentiment values for songs
using lyrics. Experimental results show that using our iSentiDictionary achieves the best
results in terms of both of our proposed evaluation metrics: Word Coverage and Absolute
Sentiment Error. We believe that this is because of iSentiDictionary’s high word coverage for iPop songs.
In summary, the main contributions of this paper are twofold: Firstly, we describe a
language-independent method of integrating existing sentiment dictionaries and populating valueless entries with values extrapolated from seed words that have sentiment values. Importantly, we have made our resulting iSentiDictionary publically available for
Chinese sentiment analysis research. Secondly, we apply our iSentiDictionary to lyricsbased sentiment value estimation. The experimental results show that iSentiDictionary
has the best coverage and the lowest average sentiment value error.
REFERENCES
1. S. D. Kamvar and J. Harris, “We feel fine and searching the emotional web,” in Proceedings of the 4th ACM International Conference on Web Search and Data Mining,
2011, pp. 117-126.
2. A. Carlson, J. Betteridge, B. Kisiel, B. Settles, E. R. Hruschk Jr., and T. M. Mitchell,
“Toward an architecture for never-ending language learning,” in Proceedings of the
24th AAAI Conference on Artificial Intelligence, 2010, pp. 1306-1313.
3. P. J. Stone, D. C. Dunphy, M. S. Smith, and D. M. Ogilvie, The General Inquirer: A
Computer Approach to Content Analysis, MIT Press, MA, 1966.
4. C. Strapparava and A. Valitutti, “WordNet-Affect: an affective extension of WordNet,” in Proceedings of the 4th International Conference on Language Resources
and Evaluation, 2004, pp. 1083-1086.
5. L.-W. Ku, Y.-T. Liang, and H.-H. Chen, “Opinion extraction, summarization and
tracking in news and blog corpora,” in Proceedings of the 2006 AAAI Spring Symposium Series on Computational Approaches to Analyzing Weblogs, 2006, pp. 100107.
6. Z. Dong and Q. Dong, How Net and the Computation of Meaning, River Edge, NJ,
World Scientific Publishing Co., Inc., 2006.
7. M. M. Bradley and P. J. Lang, “Affective norms for English words (ANEW) instruction manual and affective ratings,” Technical Report C-1, The Center for Research in
Psychophysiology, 1999.
8. S. Baccianella, A. Esuli, and F. Sebastiani, “SentiWordNet 3.0: An enhanced lexical
resource for sentiment analysis and opinion mining,” in Proceedings of the 7th International Conference on Language Resources and Evaluation, 2010, pp. 22002204.
9. E. Cambria, R. Speer, C. Havasi, and A. Hussain, “SenticNet: A publicly available
semantic resource for opinion mining,” in Proceedings of the AAAI Fall Symposium
Series on Commonsense Knowledge, 2010, pp. 14-18.
10. A. Esuli and F. Sebastiani, “SentiWordNet: A publicly available lexical resource for
opinion mining,” in Proceedings of the 5th International Conference on Language
Resources and Evaluation, 2006, pp. 417-422.
BUILDING A SENTIMENT DICTIONARY BASED ON CONCEPTNET FOR SENTIMENT ANALYSIS OF LYRICS 661
11. C. Havasi, R. Speer, and J. B. Alonso, “ConceptNet 3: a flexible, multilingual semantic network for common sense knowledge,” in Proceedings of the Recent Advances in Natural Language Processing, 2007.
12. Y. Wang, L. Zhou, and Y. Luo, “The pilot establishment and evaluation of Chinese
affective words system,” Chinese Mental Health, Vol. 23, 2008, pp. 608-612.
13. W.-L. Wei, C.-H. Wu, and J.-C. Lin, “A regression approach to affective rating of
Chinese words from ANEW,” in Proceedings of the 4th International Conference on
Affective Computing and Intelligent Interaction, Vol. Part II, 2011, pp. 121-131.
14. Y. Hu, X. Chen, and D. Yang, “Lyric-based song emotion detection with affective
lexicon and fuzzy clustering method,” in Proceedings of the 10th International Society for Music Information Retrieval Conference, 2009, pp. 123-128.
15. P. Dodds and C. Danforth, “Measuring the happiness of large-scale written expression: Songs, blogs, and presidents,” Journal of Happiness Studies, Vol. 11, 2010, pp.
441-456.
16. E. Cambria, A. Hussain, C. Havasi, and C. Eckl, “Affective space: Blending common sense and affective knowledge to perform emotive reasoning,” in Proceedings
of the 1st Workshop on Opinion Mining and Sentiment Analysis, 2009.
17. P. Singh, “The public acquisition of common sense knowledge,” in Proceedings of
the 2002 AAAI Spring Symposium Series on Acquiring Linguistic Knowledge for Information Access, 2002.
18. Y.-L. Kuo, K.-Y. Chiang, C.-W. Chan, J.-C. Lee, R. Wang, E. Shen, and J. Y.-J. Hsu,
“Community-based game design: experiments on social games for commonsense
data collection,” in Proceedings of the ACM SIGKDD Workshop on Human Computation, 2009, pp. 15-22.
19. Y.-L. Kuo and J. Y.-J. Hsu, “Resource-bounded crowd-sourcing of commonsense
knowledge,” in Proceedings of the 22nd International Joint Conference on Artificial
Intelligence, 2011, pp. 2470-2475.
20. H. Liu, H. Lieberman, and T. Selker, “A model of textual affect sensing using
real-world knowledge,” in Proceedings of the 8th International Conference on Intelligent User Interfaces, 2003, pp. 125-132.
21. W. R. Chu, R. T.-H. Tsai, Y.-S. Wu, H.-H. Wu, H.-Y. Chen, and J. Y.-J. Hsu,
“LAMP, a lyrics and audio mandopop dataset for music mood estimation: Dataset
compilation, system construction, and testing,” in Proceedings of the 15th Conference on Technologies and Applications of Artificial Intelligence, 2010, pp. 53-59.
Hui-Hsin Wu (吳蕙欣) is an engineer at HTC which is a
manufacturer of mobile phones. Her research interests include
sentiment analysis, music recommendation, and applications of
commonsense knowledge bases. Wu has an MS in computer science and information engineering from National Taiwan University. Contact her at [email protected].
662
HUI-HSIN WU, CHARNG-RURNG TSAI, TZONG-HAN TSAI AND JANE YUNG-JEN HSU
Angela Charng-Rurng Tsai (蔡長蓉) is an engineer at
KKBOX which is a provider of digital music subscription services in the Asia region. Her research interests include sentiment
analysis, music information retrieval, and applications of commonsense knowledge bases. Tsai has an MS in computer science
and information engineering from National Taiwan University.
Contact her at [email protected].
Richard Tzong-Han Tsai (蔡宗翰) is an Associate Professor of Computer Science and Engineering at Yuan Ze University.
Prof. Tsai has a Ph.D. in computer science and information engineering from National Taiwan University. His main research interests include natural language processing, cross-lingual information access, sentiment analysis, and context-based recommendation, etc. Prof. Tsai achieved great success in several international academic completions. Until now, he has published papers
in several top journals, including Bioinformatics, Nucleic Acids
Research, IEEE-ACM Transactions on Computational Biology
and Bioinformatics, Journal of the American Society for Information Science and Technology, IEEE Transactions on Audio, Speech and Language Processing, and Decision
Support Systems. His papers have been accepted by leading such as IJCAI, COLING,
and IJCNLP. Prof. Tsai has served on the editorial board of International Journal of
Computational Linguistics and Chinese Language Processing (IJCLCLP). He is a member of Association for Computational Linguistics (ACL), Association for Computational
Linguistics and Chinese Language Processing (ACLCLP), and a board member of Taiwanese Association for Artificial Intelligence (TAAI). Contact him at [email protected].
edu.tw.
Jane Yung-Jen Hsu (許永真) is a Chairman of Computer
Science and Information Engineering at National Taiwan University. Her research interests include intelligent multi-agent systems,
data mining, service oriented computing and web technology. Dr.
Hsu is on the editorial board of the International Journal of Service
Oriented Computing and Applications (published by Springer).
She has served on the editorial board of Intelligent Data Analysis
 An International Journal (published by Elsevier and IOS Press)
and the executive committee of the IEEE Technical Committee
on E-Commerce. She is actively involved in many key international conferences as organizers and members of the program committee. She is a member of AAAI, IEEE, ACM, Phi Tau Phi, and has been an executive committee member of
TAAI.