Information Processing and Management 51 (2015) 428–432
Contents lists available at ScienceDirect
Information Processing and Management
journal homepage: www.elsevier.com/locate/infoproman
Editorial
Sentiment analysis meets social media – Challenges and
solutions of the field in view of the current information sharing
context
In this introductory article, we briefly define the key concepts in Sentiment Analysis and describe present challenges faced
by research in the task. Subsequently, we introduce each of the papers in this volume – chosen from an open call for papers and
extended versions of the best papers presented at the 4th Workshop on Computational Approaches to Subjectivity, Sentiment
and Social Media Analysis (WASSA 2013)- and we describe their contribution to the advancement of the current research in
Sentiment Analysis. Finally, we conclude on the issues that have been tackled and those that remain open and reflect on the
possible future developments of the field.
Ó 2015 Published by Elsevier Ltd.
1. Introduction
In the past years, the quantity of user-generated contents on the Internet has grown at a very high rate. Whether on blogs,
wikis, forums, review sites, Social Media or Social Networks, users are actively contributing both to the factual contents displayed, but also providing their opinions on events, people, products, ideas, etc.
The growth in the quantity of subjective, opinionated information and the manner in which it is spread in near
real-time through the Internet has led to changes in the Economic, Politic and Social spheres. Opinions influence
the manner in which brands, people, organizations and events are perceived globally. Events being presented in the
news or Social Media lead to the voicing of strong opinions, which are able to motivate masses to action. The breadth
of this phenomenon, the rapidness with which opinions are created and spread and their large quantity has brought
about a growing need to develop systems that automatically detect and classify opinions, sentiments and attitudes in
these texts.
In Natural Language Processing, the field of Sentiment Analysis has emerged in the past decade as an answer to
this need. Although no generally applied definition exists in the literature, Sentiment Analysis is considered as the
computational task of automatically detecting and classifying sentiment from text, depending on its ‘‘polarity’’ or
‘‘orientation’’.
The orientation or polarity of the sentiment can be positive, negative or neutral, although different, more fine grained (e.g.
including ‘‘very positive’’, ‘‘very negative’’ or an intensity of 1 to 5) scales have also been considered.
Related tasks include ‘‘Opinion mining’’ (considered to be an equivalent task), ‘‘Review Mining’’, ‘‘Appraisal Mining’’.
These tasks are integrated in the broader area of ‘‘Affective Computing’’ Picard (1997) a branch of Artificial Intelligence
(AI) dealing with the design of systems and devices that can recognize, interpret, and process human affect. Sentiment analysis is also linked to ‘‘Subjectivity analysis’’, which some authors consider to be a step prior to the classification of sentiment.1 Subjectivity analysis regards the ‘‘linguistic expression of somebody’s opinions, sentiments, emotions, evaluations,
beliefs and speculations’’ Wiebe (1994).
Most work in the field has concentrated on creating and evaluating methods, tools and resources to discover whether a
specific ‘‘object’’ (person, product, organization, event, topic, etc.) is ‘‘regarded’’ in a positive or negative manner by a specific
‘‘source’’ (i.e. a person, an organization, a community, people in general, etc.).
1
For more information about the distinction between different terms, please see http://rua.ua.es/dspace/handle/10045/19437.
http://dx.doi.org/10.1016/j.ipm.2015.05.005
0306-4573/Ó 2015 Published by Elsevier Ltd.
Editorial / Information Processing and Management 51 (2015) 428–432
429
Whereas in the beginning the task was oriented mostly towards the classification of reviews (from online sales sites such
as Amazon.com, Booking.com, etc.), nowadays the focus of research in Sentiment Analysis is on retrieving, extracting and
classifying sentiments from Social Media sites or networks (especially Twitter and Facebook).
2. Main challenges in sentiment analysis
The term ‘‘sentiment’’ is defined by the Webster dictionary as suggesting a ‘‘settled opinion reflective of one’s feelings’’,
where ‘‘the term feeling points to a single component of emotion, denoting the subjective experience process, and is therefore only a small part of an emotion’’ Scherer (2005). The opinion thus denotes an evaluation of an object as a consequence of
which a certain feeling is conveyed.
As affective phenomena, emotions, feelings, sentiments, opinions, are difficult to identify and label, even for humans. The
computational task of automatically performing this task is thus faced with many challenges.
The first challenge is related to the necessity to have linguistic resources, both in terms of lexicons containing
sentiment-related words together with their associated polarity, as well as annotated corpora. Although such resources
exist, they are mainly created for English or, in the case of annotated corpora, done mostly for product reviews or
certain domains that are directly applicable to the industry. From here, a certain number of issues stem: the need
to create lexicons for sentiment in languages other than English, annotate corpora for text types other than reviews
and perform domain adaptation (i.e. employ annotated data from a domain as training to a different domain, without
losing performance). The human annotation of data is a cumbersome process. Therefore, the challenges resides in
employing existing resources for English to automatically obtain resources for other languages (through the use of dictionaries or machine translation systems).
The second challenge is related to the language employed in the new types of texts, such as microblogs. Here, due to the
limitation in characters, people often employ abbreviations. Additionally, language of microblogs contains special emotion
markers, like emoticons and slang expressions (usually in acronyms). Finally, there are specific markers of the topics conveyed, preceded by the hashtah (#) and mentions of users (preceded by @).
Further on, although emotions can be detected by from specific words, it is often the case that texts do no
explicitly contain such emotion-bearing words. Instead, special ‘‘triggers’’ of those emotions are mentioned (e.g. economic issues, the financial crisis, the war in Iraq, etc.). Especially in newspaper articles or marketing, the emotional
messages tend to contain triggers of emotions and not explicit mentions thereof. The challenge here resides in
identifying such potential triggers and associate them with an emotion label. This, of course, requires world knowledge or the use of high quantities of data on which statistical methods (usually distributional semantics) can be
employed.
Finally, dealing with human language and affect is not only difficult because of the inherent ambiguity of words, but also
because of the use of words in figurative senses and the use of irony or sarcasm, i.e. conveying a negative evaluation through
the use of a mostly positive choice of words. The performance of Sentiment Analysis systems is highly influenced by this
phenomenon, thus irony detection has become an important challenge related to the field.
In the light of the issues identified, in the next section we present the articles in this special issue and indicate their contribution to the challenges mentioned.
3. Articles included in the special issue and their contribution to current issues in sentiment analysis
The manuscripts contained in the present special issue were chosen both from an open call for papers. However, most of
them are the extended versions of the best articles (as reviewed by the Program Committee and by the journal reviewers)
that have been presented at the 4th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media
Analysis (WASSA 2013) – http://optima.jrc.it/wassa2013/. The event was organized in conjunction to the 2013
Conference of the North American Chapter of the Association for Computational Linguistics – Human Language
Technologies (NAACL-HLT 2013), June 14, 2013, in Atlanta (GA), U.S.A.
The workshop had a large number of submissions, which once again demonstrated the great interest in this area
and ensured the high quality of the papers selected for presentation and for the special issue. Thus, the manuscripts
included in this special issue bring about important contributions to the advancement of the state of the art in the
Sentiment Analysis area.
Following is a detailed description of the contribution brought by each of the articles:
3.1. Computational approaches for mining user’s opinions on the Web 2.0 – Gerald Petz, Michal Karpowicz, Harald Fürschuß,
Andreas Auinger, Václav Striteský, Andreas Holzinger
The paper sets out to investigate the differences between social media channels regarding opinion mining and to evaluate
the effectiveness of various text preprocessing algorithms as a subtask of opinion mining in these social media channels. In
430
Editorial / Information Processing and Management 51 (2015) 428–432
order to achieve this, the authors identify popular approaches and algorithms to carry out text pre-processing as a prior step
to sentiment analysis, identify differences between social media channels and thus entail impacts on opinion mining and
text pre-processing and evaluate of the effectiveness and properness of several algorithms that are tailored specifically
for the needs of the studied social media channels.
3.2. POS-RS: a random subspace method for sentiment classification based on part-of-speech analysis – Gang Wang, Zhu Zhang,
Jianshan Sun, Shanlin Yang, Catherine Larson
The article describes an enhanced Random Subspace method, POS-RS, for sentiment classification based on
part-of-speech analysis. In order to accomplish this, the authors employ ‘‘ensemble learning’’ with the Random Subspace
method, modifying the training dataset by modifying the feature space. Based on the Stanford Parser output, they distinguish content lexicon subspace and function lexicon subspace. When compared to the base learner – Support Vector
Machines, it is seen that POS-RS achieves the best performance through reducing bias and variance simultaneously.
These results illustrate that POS-RS can be used as a viable method for sentiment classification and has the potential of being
successfully applied to other text classification problems.
3.3. A Spanish semantic orientation approach to domain adaptation for polarity classification – M. Dolores Molina González,
Eugenio Martínez Cámara, M. Teresa Martín Valdivia, L. Alfonso Urenã López
The paper studies the integration of domain information in an already-existing lexicon for a Spanish polarity classification system. Different resources are generated from the original Spanish lexicon iSOL and using them, a series of
domain-dependent lexicons are built. Subsequently, the authors experiment with the lexicons and show their appropriateness and gain in performance in the chosen domains. All these resources are freely available for research
purposes.
3.4. Supervised sentiment analysis in Czech social media – Ivan Habernal, Tomás Ptáček, Josef Steinberger
This article presents and in-depth research on supervised machine learning methods for sentiment analysis of Czech
social media. The authors employ a large Facebook dataset containing 10,000 posts, accompanied by human annotation with
substantial agreement (Cohen’s 0.66). Subsequently, they evaluate various state-of-the-art features and classifiers as well as
different language-specific preprocessing techniques and feature selection algorithms. The results obtained showed that the
optimal combination is the use of unigrams, bigrams, POS features, emoticons, character n-grams and preprocessing techniques (unsupervised stemming and phonetic transcription). In addition, they report results in two other domains, where
significant improvements are obtained over the baseline.
3.5. Sentiment analysis system adaptation for multilingual processing: the case of tweets – Alexandra Balahur and Jos’e Manuel
Perea Ortega
In the past decade, the quantity of user-generated contents on the Internet has been growing exponentially. Social
Media platforms, such as Facebook, Twitter, Flickr, LinkedIn, etc., as well as commercial sites, like Amazon,
Booking.com, etc. offer their users the possibility to share their experiences and opinions on topics ranging from economics, to politics, products, VIPs and globally-critical events. This paper presents the experiments carried out with
the objective to develop a multilingual sentiment analysis system starting from initial evaluations of methods and
resources in two international evaluation campaigns for English and for Spanish. Subsequently, the authors describe
additional experiments to improve the performance of both Spanish and English systems, using multilingual
machine-translated data. The evaluations show that the use of hybrid features and multilingual, machine-translated
data (even from other languages) can help to better distinguish relevant features for sentiment classification and thus
increase the precision of sentiment analysis systems.
3.6. Sentiment, emotion, purpose, and style in electoral tweets – Saif Mohammad, Xiaodan Zhu, Svetlana Kiritchenko, Joel Martin
This paper analyzes electoral tweets for more subtlely expressed information such as sentiment (positive or negative), the emotion (joy, sadness, anger, etc.), the purpose or intent behind the tweet (to point out a mistake, to support, to ridicule, etc.), and the style of the tweet (simple statement, sarcasm, hyperbole, etc.). This is the first set of
tweets annotated for all the phenomena. The authors also developed two automatic statistical systems that use the
annotated data for training and predict emotion and purpose labels in new unseen tweets, thus establishing baseline
results for automatic systems on this new data.
Editorial / Information Processing and Management 51 (2015) 428–432
431
3.7. Mining affective text to improve social media item recommendation – Jianshan Sun, Gang Wang, Xusen Cheng, Yelin Fu
This article describes a sentiment-aware social media recommendation framework, referred to as SA_OCCF
(Sentiment-Aware One Class Collaborative Filtering), to recommend social media items for online users. It employs
sentiment analysis techniques to mine affective texts and leverage OCCF (One Class Collaborative Filtering) models
to provide personalized media item recommendations. The described sentiment classifier model is an ensemble
learning-based sentiment classification (ELSC) method. The system is evaluated using the TED dataset. The results
show that the proposed SA OCCF models outperform the baseline methods using a variety of recommendation accuracy metrics.
3.8. Signalling sarcasm: From hyperbole to hashtag – Florian Kunneman, Christine Liebrecht, Margot van Mulken, Antal van den
Bosch
This article describes and tests a system that detects sarcastic tweets. It is trained on a set of 406 thousand tweets,
harvested over time, marked by the hashtags ‘#sarcasm’, ‘#irony’, ‘#cynicism’, or ‘#not’ by the senders. The used classification algorithm is ‘‘Balanced Winnow’’ (Littlestone, 1988). The classifier is tested on a set of 2.25 million tweets
and correctly spot 309 of the 353 tweets that were explicitly marked with the hashtag, with the hashtag removed.
Testing the classifier on the top 250 of the tweets it ranked as most likely to be sarcastic, but that did not have a
sarcastic hashtag, it attains only a 35% average precision. Authors conclude that it is fairly hard to distinguish sarcastic
tweets from literally intended tweets in an open setting.
3.9. Detecting Positive and Negative Deceptive Opinions using PU-learning – Donato Hernández Fusilier, M.I., Manuel Montes-yGómez, Paolo Rosso, Rafael Guzmán Cabrera
In this paper, the authors apply a supervised approach to automatically identify deceptive and truthful reviews.
Existing approaches rely on large amounts of data on which the systems are trained. Based on a previous result
Hernández, Guzmán, Móntes y Gomez, and Rosso (2013), in the present work, the authors propose a method that
learns only from a few examples of deceptive opinions and a set of unlabelled data, employing a method called
PU-learning. In PU-learning, two sets of examples are available for training: the set P of positive instances, and a
set U, which is assumed to contain a mixture of both positive and negative examples, but without any label, as
opposed to other forms of semi-supervised learning, where it is assumed that the training set contains labelled
examples of both classes. Considering P as corresponding to the set of labeled deceptive opinions, and U as a set
of unlabeled review – containing a combination of deceptive and truthful opinions, the authors show that
PU-learning can be successfully employed to detect deceptive opinions, even when the training is done on a small
set of instances.
4. Conclusions and future directions of research in sentiment analysis
The articles presented in this special issue all deal with important challenges in the field of Sentiment Analysis, bringing
relevant contributions to tackling them. Nevertheless, in the endeavor to automatically deal with affective phenomena,
much remains to be done.
Regarding multilinguality, although machine translation systems can nowadays be reliably employed for certain languages, they still have a relatively low performance for less spoken ones. Thus, the challenge of creating linguistic resources
for Sentiment Analysis in these languages still remains.
Another issue is that communication is presently done predominantly through Social Media, where the traditional
tools for automatic language analysis lose much of their performance. The expressions employed, the variability of the
abbreviations and graphic symbols, as well as the reduced length of those texts make their processing difficult. On the
other hand, Social Media texts benefit from other types of markers (topic, mentions of people, sometimes sentiment
expressed), which could be exploited by Natural Language Processing applications and Sentiment Analysis, either to
obtain training data easily or to extend existing training datasets.
Finally, expressions of emotions are language, culture and event-specific. An interesting direction for future work
could therefore be to exploit the variety of sources from Social Media, as well as the profile of people contributing
and the meta-information from Social Media sources (links between people who are writing, geo-localization of messages, types of messages that are usually written by a specific user category, popularity of messages quantified by
likes, comments or retweets, etc.), to improve the detection of relevant content and help to better identify and classify
the opinions and sentiments expressed.
References
Hernández, D., Guzmán, R., Móntes y Gomez, M., & Rosso, P. (2013). Using PU-learning to detect deceptive opinion spam. In Proc. of the 4th Workshop on
Computational Approaches to Subjectivity, Sentiment and Social Media Analysis (pp. 38–45).
432
Editorial / Information Processing and Management 51 (2015) 428–432
Littlestone, N. (1988). Learning quickly when irrelevant attributes abound: A new linear-threshold algorithm. Machine Learning, 2, 285–318.
Picard, R. W. (1997). Affective computing. Cambridge, MA, USA: MIT Press.
Scherer, K. (2005). What are emotions? And how can they be measured? Social Science Information URL: <http://ssi.sagepub.com/cgi/content/ abstract/
44/4/695>.
Wiebe, J. M. (1994). Tracking point of view in narrative. Computational Linguistics, 20, 233–287.
Alexandra Balahur
Guillaume Jacquet
European Commission,
Joint Research Center (JRC) Institute for the Protection and Security of the Citizen (IPSC),
Via E. Fermi 2749,
T.P. 267, I-21027 Ispra, VA,
Italy
E-mail addresses: [email protected] (A. Balahur),
[email protected] (G. Jacquet)