Polish Journal
of Social Science
Vol. XI No. 1 2016
ELBLĄG UNIVERSITY OF HUMANITIES AND ECONOMY
Bogdan Pietrulewicz*, Justyna Leszcz, Karolina Magierek,
Aleksandra Wolska
Institute of Psychology, Kazimierz Wielki University in Bydgoszcz
The Effectiveness of Recognition
of Emotions Based on Facial Expressions
Abstract
Emotions play an important role in everyone’s life – they influence our decisions, judgments, cognitive or behavioral responses – in a more or less conscious way. They also
have a huge impact on homo socialis social interactions. Facial expressions are one of
many ways of displaying emotions (Ekman, 2009).
In this study, the authors sought to answer the question: does gender influence the effectiveness of recognition of emotions? The project included 120 respondents. The authors
assumed that there are significant differences between men and women in terms of the
efficiency of recognition of emotions. The statistical analysis of obtained empirical material has not confirmed the hypothesis.
Keywords: emotions, facial expressions, micro-expressions, emotional intelligence, efficiency of recognition of emotions.
QQ
Introduction
Emotions are one of the three components of attitudes. The curious aspect of emotions lies in that they determine human judgments, which are
ultimately expressed in actions (Mayer, Caruso, Salovey, 2000; Ekman, 2009).
There are many indicators based on which we can conclude about our own
emotional state, or the emotional state of other people. One of the strongest
indicators is our face (Ekman, Friesen, 2003). It is colloquially said that the
face is a kind of mirror of human emotions. The feelings of another person
* Correspondence
concerning this article should be addressed to Bogdan Pietrulewicz,
e-mail: [email protected].
180 Bogdan Pietrulewicz, Justyna Leszcz, Karolina Magierek, Aleksandra Wolska
are communicated by several key points on their face, such as the mouth,
nose, eyes or eyebrows (Ekman, Friesen, 1978).
The variety of micro-expressions which might occur on the human face
is very large. There are thousands of mimic compositions, but only a few of
them are universal. This means that almost every person – more or less effectively – associates micro-expressions with specific emotions, regardless of
gender, age or cultural background (Mayer, Geher, 1996).
QQ
What are emotions?
Emotions, speaking colloquially, may be defined as short and conscious
experienced of a pleasant or unpleasant nature. Scientifically speaking,
a dominant, universal definition of this term has still not been agreed (Mayer,
Salovey, Caruso, 2000).
Emotions are closely associated with physiological and psychological
stimulation, the character of which depends on the specific emotions. In
the neurobiological approach, emotions can be defined as complex action
programs, caused by external or internal stimuli (Ekman, 2001). A similar
view is represented by the majority of theoreticians and empirical experts –
emotions are nothing but reactions of the body, based on physiological and
psychological processes, formed in response to stimuli (Mayer, 2001; Ekman,
2016). Their function is to prepare the organism for an appropriate action.
In this sense, emotions are strictly a source of moods, as well as the feelings
of a human being. Emotional processes and emotions determine the importance of the subject, which is their source (Mayer, Salovey, Caruso, 2000).
Manifestations of emotions can be divided into four areas: bodily symptoms
(e.g. heartbeat or breathing, divided into involuntary and free); actions taken
(e.g. fight or flight); facial expressions (e.g. widening of the eyes or a frown)
and cognitive (e.g. convictions regarding the stimulus).
QQ
Facial expressions
The human face is a complicated system of muscles, encompassing more
than 40 structurally and functionally independent muscles, which may be operated independently of each other. Almost all the facial expressive muscles
are innervated by a single nerve – the facial nerve (cranial nerve VII), and
they are the only muscles in our body that are fixed to our skin, at least at one
The Effectiveness of Recognition of Emotions Based on Facial Expressions
181
end. As a result, they affect the appearance of the face directly (Cohn, Ying-li
Tian, Kanade, 2000).
There are 7 main areas that form the facial mimic system. They are the
nose, eyes, eyebrows and forehead, upper lip, lower lip and chin, mouth and
jaw. In addition, the eyebrows and forehead, the eyes, eyelids and lower part
of the face (cheeks, lips, chin and the lower part of the nose) have the ability to move independently of each other. In practice this means that the face
is capable of expressing several emotions at one time (Bradley, Greenwald,
Lang, 1992; Ekman, 2010).
Facial expressions are twofold in character – they may be involuntary
or intentional. By their nature, micro-expressions are not subject to will, because of the location of the facial nerve in the brain stem. In the process of
socialization, we learn socially acceptable behavior, discarding the unwanted
effects. In the case of the “evil eye” and “smiling lips,” we may therefore infer
that there is an ongoing struggle between “nature” and “culture,” which takes
place on the face (Mayer, 2001; Bible, 2009).
Depending on whether activity occurs in the motor cortex or the brainstem, facial expressions are voluntary or involuntary. The brain stem controls
the unconscious expressions, appearing spontaneously, and the motor cortex
is involved in the intentional, controlled and deliberately initiated facial expressions (Ekman, O’Sullivan, 2006).
The facial nerve connects most of the facial muscles to the brain, and
the same areas in the brainstem that trigger the appearance of micro-expressions, are also responsible for monitoring the emotional processing and
regulation (Ekman, 1992).
QQ
Basic emotions and micro-expressions
A Micro-expression is a mimic reaction, lasting about 0.4 seconds, appearing in response to a specific emotion. The fundamental emotions are: joy,
anger, surprise, fear, contempt and sorrow.
Conclusions on the inter-cultural character of certain emotions were put
forward in 1987, by Ekman and Friesen. In their study, they asked participants
to classify emotions based on facial expressions in an image. They found that
subjects were consistent in classification only of some emotions, regardless of
the origin of the person in the photo and the person examined (Ekman, 1999).
Years of research, initiated by Darwin and developed and continued by
Ekman and Friesen have shown that there are six universal emotions. Among
182 Bogdan Pietrulewicz, Justyna Leszcz, Karolina Magierek, Aleksandra Wolska
the researchers, however, there is an ongoing discussion on how different
emotions can be differentiated from each other (Ekman, 1990).
QQ
Emotional intelligence. Research overview
The issue of emotion is strictly linked to a construct being dynamically developed in psychology, that is emotional intelligence. Goleman (1999)
characterizes emotional intelligence as a set of five areas, consisting of:
understanding emotions, recognizing other people’s emotions, managing
emotions, self-motivating and maintaining interpersonal relationships.
Researchers Salovey and Mayer (1990) categorize emotional intelligence
as a set of emotional abilities. Matczak calls on their model in her studies,
highlighting two factors of emotional intelligence. The first is a cognitive factor, concerning the ability to recognize and understand emotions. The action
factor, in turn, is nothing else but the ability to use and regulate emotions
(Matczak, Piekarska, 2007).
There have been numerous studies on scientific grounds regarding
sex-resultant differences in emotional intelligence (Goleman, 1999). The
results of the largest projects in this field relate to the differences between
men and women in relation to the ability to perceive and recognize emotions.
The researchers found that women are more sensitive to non-verbal messages and recognize emotions thus communicated more easily and accurately
(Brody and Hall, 2005; Fabes and Martin, 1991; Hall and Matsumoto, 2004;
Leathers, 1992). Gender differences in this respect are inter-cultural and specifically address emotion recognition based on facial expressions (Leathers,
1992; Matczak, 2008; Piotrowska, 1997).
Numerous scientific studies confirm dominance of women in terms
of the ability to recognize emotions. The researchers explain the occurring
differences in inter-factor perspective, taking into account the role of evolutionary, biological and socio-cultural predictors (Barrett et al., 2000).
QQ
Study group and procedure
The aim of this study was to verify the effectiveness with which men and
women recognize emotions based on facial micro-expressions. The study encompassed 120 people, 65 women and 55 men, aged 18–65. The condition
The Effectiveness of Recognition of Emotions Based on Facial Expressions
183
for joining the research group was profession – other than a psychologist and
/ or psychotherapist and unrelated to study of emotional micro-expressions.
The study took place in accordance with the rules of ethics.
A recording of two people in social interaction with another person (restaurant employee – customer) was selected. Each recording was 30 seconds
long. Thanks to special software, allowing to analyze records second-by-second, two competent judges marked the dominant emotions of people in
every second of a recording. The judges’ marks were subsequently compared.
In case of inconsistent markings, the material in question was set aside and
then once again simultaneously presented to the judges. Material that elicited
doubts in primary taxation was marked by majority of votes or by means of
discussion.
Each of the 120 persons were presented in turn with the same two video
recordings. The researcher asked to define the emotion that a person feels
in the scene at the screen. Then, the recording was marked, frame by frame,
with the emotion chosen by the participant from a list of 5 basic emotions:
anger, disgust, joy, sadness, surprise, and neutral state. A common result was
calculated for N = 120 in the form of a percentage and then compared with
the responses of competent judges (assuming 100% efficiency).
QQ
Discussion and conclusions
This article describes the issue of recognition of emotions based on facial
micro-expressions. The authors decided to check whether there are significant differences between men and women in the effectiveness of recognizing
emotions based on facial micro-expressions.
The overall result achieved by all respondents for both recordings is
46,33% (first video 40.11%, second – 52.55%). People with humanistic professional profiles demonstrated less effectiveness in recognizing emotions
than employees of technical and economic sectors (49.15%: 51.03%). The
unemployed respondents reached the lowest score of the entire group (22%
efficiency). The best at recognizing emotions were people aged 30–39 years
old (47.47%), then whose aged 18–29 (46.34%) and finally those 40–65
years old (44.25%).
Statistical analysis showed no significant differences in the efficiency of
recognizing emotions between men and women. The study has not confirmed
the assumed hypothesis of the existence of statistically significant differences
between men and women in this respect.
184 Bogdan Pietrulewicz, Justyna Leszcz, Karolina Magierek, Aleksandra Wolska
There have been many empirical studies proving that women were more
effective in recognizing human emotions (Brody and Hall, 2005). It should
be emphasized that the results of some of them showed a predominance of
women only in the recognition of positive emotions (Matczak, 2006). In addition, in relation to negative emotions, such as anger, greater efficiency is
attributed to men (Elfenbein, Marsh and Ambady, 2002).
In the light of the results it is necessary to highlight the fact that the presented test videos were dominated by the neutral emotional state, joy and
disgust. Taking into account the results of previous explorations, the range
of emotions on film material should be expanded by negative emotions, such
as anger. Perhaps the assumed differences between the sexes would be confirmed when comparing the effectiveness in recognizing only positive or
negative emotions. It would also be advisable to expand the study group size
and further explore the topic presented to the reader herein.
QQ References
Barrett, L.F., Lane, R.D., Sechrest, L., Schwartz, G.E. (2000). Sex differences in emotional awareness. Personality and Social Psychology Bulletin, 26, p. 1027–1035.
Bible, E. (2009). Smiles and frowns are innate, not learned. Retrieved from San Francisco State University. http://www.sfsu.edu/news/2009/spring/1.html.
Bradley, M.M., Greenwald, M.K., Petry, M.C., Lang, P.J. (1992). Remembering pictures:
Pleasure and arousal in memory. Journal of Experimental Psychology: Learning,
Memory, & Cognition 18 (2), p. 379–390.
Brody, L.R., Hall, J.A. (2005). Płeć, emocje i ekspresja, [w:] M. Lewis i J.M. Haviland-Jones (red.). Psychologia emocji (s. 431–445). Gdańsk: GWP.
Cohn, J., Ying-li Tian, B., Kanade, T., (2000). Recognizing action units for facial expression analysis. In: IEEE Computer Society Conference on Computer Vision and
Pattern Recognition (CVPR’00), 1, June 2000.
Elfenbein, H.A., Marsh, A.A., Ambady, N. (2002). Emotional intelligence and the recognition of emotion from facial expressions. http://apps.olin.wustl.edu/faculty/
elfenbeinh/pdf/FacesChapter.PDF.
Feldman Barrett L., Salovey P. (eds), The wisdom in feeling (p. 144–166). New York,
Guilford Press, 2002.
Ekman, P. (2001). Smiling. In: Blakemore, C., Jennett, S. (eds), Oxford Companion to
the Body. London: Oxford University Press.
Ekman, P. (2009). Become Versed in Reading Faces. Entrepreneur. Retrieved from
http://www.entrepreneur.com/article/200934.
Ekman, P. (2009). Lie Catching and Micro Expressions. In: Martin, C. (ed.), The Philosophy of Deception. New York, NY: Oxford University Press
The Effectiveness of Recognition of Emotions Based on Facial Expressions
185
Ekman, P. (2010). Darwin’s Compassionate View of Human Nature. JAMA, 303 (6),
p. 557–558.
Ekman, P. (2016). What Scientists Who Study Emotion Agree About 11 (1), 31–34.
Ekman, P., Friesen, W.V. (2003). Unmasking the Face. Cambridge: Malor Books, p. 169.
Ekman, P., Friesen, W.V. (1978). W.V. Facial Action Coding System: A Technique for Measurement of Facial Movement. Consulting Psychologists Press Palo Alto, California.
Ekman, P., O’Sullivan, M. (2006). From Flawed Self-Assessment to Blatant Whoppers.
The Utility of Voluntary and Involuntary Behavior in Detecting Deception. Behavioral Science and the Law, 24 (5), 673–686.
Ekman, P. (1999). Basic Emotions. In: T. Dalgleish, M. Power. Handbook of Cognition
and Emotion. Sussex, UK: John Wiley & Sons Ltd.
Ekman, Pl. (1992). Facial Expressions of Emotion: An Old Controversy and New Findings. Philosophical Transactions of the Royal Society. (London) B335: 63–69.
Fabes, R.A., Martin, C.L. (1991). Gender and age stereotypes of emotionality. Personality and Social Psychology Bulletin, 17, 532–544.
Goleman, D. (1999). Inteligencja emocjonalna w praktyce. Poznań: Media Rodzina of
Poznań.
Hall, J.A., Matsumoto, D. (2004). Gender differences in judgments of multiple emotions from facial expressions. Emotion, 4, p. 201–206.
Leathers, D. (1992). Successful nonverbal communication: principles and applications. New York: Pearson Education, Inc.
Matczak, A. (2008). Do czego może być potrzebne pojęcie inteligencji emocjonalnej?
In: M. Śmieja, J. Orzechowski (red.), Inteligencja emocjonalna: fakty, mity, kontrowersje (s. 46–61). Warszawa: PWN.
Matczak, A. (2006). Natura i struktura inteligencji emocjonalnej. Psychologia. Etologia Genetyka, 13, 59–87.
Matczak, A., Piekarska, J. (2007). Korelaty wiedzy emocjonalnej mierzonej Testem
Rozumienia Emocji. Psychologia, Edukacja i Społeczeństwo, 4 (1), 67–82.
Mayer, J.D. (2001). A field guide to emotional intelligence, [w:] J. Ciarrochi, J.P. Forgas,
Mayer, J.D. (red.). Emotional intelligence and everyday life (s. 3–24). New York:
Psychology Press.
Mayer, J.D., Caruso, D., Salovey, P. (2000). Emotional intelligence meets traditional
standards for an intelligence. Intelligence, 27, p. 267–298.
Mayer, J.D., Geher, G. (1996). Emotional intelligence and the identification of emotion.
Intelligence, 22, s. 89–113.
Mayer, J.D., Salovey, P., Caruso, D. (2000). Models of emotional intelligence. In:
R. Sternberg (ed.), Handbook of intelligence (s. 396–420). Cambridge, UK: Cambridge University Press.
Mayer, J.D., Salovey, P., Caruso, D., Sitarenios, G. (2001). Emotional intelligence as
a standard intelligence. Emotion, 1, p. 232–242.
Piotrowska, A. (1997). Z badań nad inteligencję społeczną. Psychologia Wychowawcza, 40, s. 289–300.