KNOWLEDGE REPRESENTATION IN CHAT COMMUNICATION:
SEMIOTICS MEETS VIRTUAL REALITY
Prof. DSc. Elena Pozdnyakova
Moscow State Institute of International Relations (University), Russian Federation
ABSTRACT
The purpose of the research is to investigate how the traditional semiotic triad –
(symbols, indexes and icons) is represented in the hypertextual structure of chat
communication. It is hypothesized that virtual reality where chat communication is
going on will enrich the models of knowledge representation, as sign structures will be
significantly influenced by hypertext potential. Nowadays studies of the Internet as
hypertext are numerous. However, semiotic aspect of hypertexts has been researched
only fragmentarily. Not much has been done to investigate new forms of sign structures,
combinations of signs. Methods, employed in research, are: method of conceptual
analysis of composite signs; method of cognitive modeling of complex conceptual
categories; method of knowledge representation modeling. The results demonstrated
that the traditional triad of signs has been developed into a richer structure of text+icon
signs; text+emoticon (emoji) signs; text+sticker signs; text + signs of culture (music,
cinema, paintings, etc.). Knowledge representation structures: propositions, frames,
scenarios, categories have acquired new potential of knowledge structuring and
organizing. Communicators also widely used explanations, reasoning and inference.
Keywords: symbols, indexes, icons, knowledge representation, chat communication,
hypertext
INTRODUCTION
Social communication can be determined as transfer of knowledge, emotions, volition
in social space and time. Such philosophical categories as “ideal”, “material”,
“consciousness“, “sense”, “sign” are in the focus of the theory of social communication.
Semiotic approach defines communication as interrelation mediated by signs, sign
systems, languages, and codes. Information transfer in semiotic approach to
communication is nonlinear. It centers on relations, emerging in the process of
meanings delivery. Semiotics is a theory of how people interpret signs: iconic (sign
resembles referent), indexical (sign is causally associated with referent) and symbolic
(sign has an arbitrary relation to referent). Charles Pierce in his pragmatic-oriented sign
theory contributed much to the study of sing-based communication. [1].
For the last 30 years Internet-communication has been growing in size and power.
However semiotic aspect of hypertexts has been researched only fragmentarily. Not
much has been done to investigate new forms of sign structures, combinations of signs.
In this study we present chat communication of two people in Facebook Messenger. It is
hypothesized that virtual reality where chat communication is going on will enrich the
models of knowledge representation, as sign structures will be significantly influenced
by hypertext potential. In his theory Peirce emphasized that the three forms of signs are
not necessarily mutually exclusive. For Peirce, semiotics was a process of
understanding and not a structured system, so a sign under his model can be perceived
as an icon, symbol or index, or a combination of the three depending on its use and
interpretation [ibid.]. Now, witnessing the development of virtual reality, we can prove
Pierce’s idea of composite signs on a new hypertextual level.
Consequently, two questions will be discussed: 1) combinations of types of signs and
functions of composite signs in hypertext of chat communication; 2) knowledge
representation structures (frames) in chat communication.
MATERIALS AND METHODS
The material of the research are records of Facebook Messenger chat communication of
two people (a man and a woman), who never met in real life. They are middle class
people about 50, with higher education degrees, holding prestigious job positions. The
study was carried out with the consent of both. The period of correspondence was about
3 years, the number of messages – about 28000 (text messages, pictures, personal
photos, Facebook stickers, references to publications, You Tube, etc.). Usually public
chats become the material for research. In this case the personal chat was under study.
In this respect the material can be considered unusual and specific for internet
communication research. The article presents the semiotic and knowledge structures
analysis of a 10-day period of communication, containing 717 messages. Message is
understood as any sign activity of each of the interlocutors: sending a text message
(from a word to a number of sentences), a Facebook sticker (these two types of signs are
the most frequent); a picture, a screenshot, a personal photo, a reference to web pages
and to You Tube resources.
The following methods were used: 1) the method of conceptual analysis of composite
signs; 2) the method of cognitive modeling of complex conceptual categories; 3) the
method of knowledge representation modeling.
RESULTS
To answer the first question we had to decide where the border is between two or
several successive sign complexes, i.e. on what ground we consider some succession of
messages to be a complex sign?
Below are the examples from the material under study.
1. Photo of A. plus description
A. This is a panoramic shot. Hanging paths in Filevski park. We’ve spent half a day,
walking there. To the left of the steep goes a hanging road for adults named “The path
of courageous”. To the right is the one for children. B. Yes, I’m here
2. Reference to a web-site video of A plus Facebook sticker (Meer) plus answer of
B. – reference to a web-sitу video. The answer is topically connected with the message
of A. – “The rebellion of robots”. The introductory words of A.: “The rebellion of
robots has stated. Yet, we don’t give in”
http://smi2.ru/article/19544182
A. speaks about the rebellion of robots ironically, so after the reference a sticker is
placed (laughing to tears).
B. places the reference “Video of the day: when people trust robots too much”
(http://www.neurotechnologies.ru/article_news?id=231).
The complex sign of this type messaged by interlocutors via multimedia and the sticker
comes close to face-to-face communication when two people watch videos and react
emotionally
3.Greetings Good morning from B.
Good morning from A.
Saying “Good night”
4. The effect of presence
A. The whole day I’m in Rolf (auto-salon)
B. Good luck!
A. Taking this to tea (some pastries for the visitors of the salon). Sharing with you
5. Pictures
“Feeling alone” – this is the iconic sign, used by A. The meaning has become
conventional for the interlocutors.
“Have some rest” – this is the iconic sign , used by B.
6. Emoticons and stickers
I. Emoticons and vivid emotions. Iconic signs and indexes
A group of Finch emoticons is usually used by the interlocutors for stronger emotions.
Facebook messages help people connect with others every day, and our goal is to help
make those connections as rich as possible. Stickers are a fun, lightweight, and visual
way of letting your friend know how you feel. The Finch set moves and smiles at you.
When you first send one of the moving emoticons, it will go through its animation once,
and then again if you or your conversation partner taps on it [3].
Emoticons without inscription can be called composite iconic – indexing signs. We
believe that their indexing function stems from cause-result relation, as they represent
resulting emotion, caused by the previous message of a communication partner.
II. Facebook stickers with inscriptions are signs of even more complex character. The
conventional meaning moves to the inscription, leaving to the icon wider conceptual
context. As for indexing function – it results again from cause-result relation.
7. DEMOTIVATORS
Iconicity + conventionality + inference + reasoning.
In the beginning, those were motivational posters with the now classic
black background and text underneath a picture. At the end of the 20-th century
demotivators tended to be near exactly the same as the motivational counterparts.
Motivators tended to feature some artsy photograph and be “office friendly”. The
original demotivators then used similar artsy photos, but would just change the text to
be something depressing instead of uplifting [4].
Demotivators appear in the course of communication as ironical or sarcastic thought and
may be not closely connected with the context. But inference and reasoning allow the
interlocutors to connect the meaning of this complex sign to the concepts of previous
conversation.
DISCUSSION
Nowadays people have access to numerous contacts in the net. At a certain time point a
filtering process occurs in which a person narrows down (based on both conscious and
unconscious motivations) the range of potential contacts - and then chooses someone
with whom to develop a relationship in the form of on-line communication. Once the
relationship is underway, people start to negotiate emotional expression and conflict,
exchange knowledge, share past experience, etc. Given the limited dimension of a textonly style of communicating, how do they manage to overcome natural limitations?
As chat communication has matured technologically, people have received access to
more features to enhance messages. So Facebook Messenger makes it easy to attach
images, sounds, references, and almost any type of document to the message. The
interlocutor may perceive such attachments - especially images and references - as
presents.
Messenger enables interlocutors to insert images directly into the body of the message,
which adds another level of expressive complexity. People insert pictures in the course
of chat, among text messages, and this doesn’t disrupt the integrity of the message body.
The image can serve as a visual appendix to an idea within the message.
Senders also add hypertext links into their messages to provide a springboard for the
recipient to jump from the chat into the web. These links are "hot" and the receiver
reaches the link by simply clicking it within the message to reach the web page. “Such
links create a swift and easy transition between private space and public space, almost
as if the sender and reader are sitting side-by-side within the e-mail space, talking
privately, but then are able to open a door to step out into a worldwide public area of
information and entertainment” [2].
Now the second question about frame structure of the topic frequently discussed in the
experimental part of the char will be answered. To do this – we have to build a frame
with slots, containing conceptual structures of the topic “robots in the future of
humans”. This topic is vividly discussed by the communication partners. Many
composite
signs
were
used
for
representing
knowledge
structures.
humanoid robots are
a expensive and
nonprospective
solution
Brain megapower,
supported by supercomputers - that is
the best decision for
the future
Robots
in
future
For breakthrough in
technology there
should appear a
motive. At present
the path of
developmen is not
clear
Neuronets's research
can be another even
mor attactive way
Figure 1 A frame for the topic “Robots in future”
CONCLUSION
It can be stated, that virtual reality and the Internet have changed personal written
communication greatly. Now correspondents, unlike in traditional letter writing, can
apply new combinations of signs, can create hypertext according to their personal wish.
Research of semiotic activity in virtual communication is among high-priority topics in
cognitive science. Sign-creating activity is one of the unique characteristics of humans.
Ability to create signs is a natural human ability. However, the semiotic value of the
sign can’t remain unchanged either for sign-creators or for all the cases of sign use [5,
114]. The research demonstrated the sign creativity process in chat communication.
Several types of composite signs that can’t be found in traditional written
communication were investigated. All types of simple and composite signs bring the
possibility of sign-creation in a new environment of the hypertext. It was also proved
that knowledge of both communicators about the referent, according to Ch. Pierce, is
structured by an objective sign (signs) and by communicators’ interpretanta [6]
REFERENCES
[1] Atkin, A. Peirce’s Theory of Signs. (2010). The Stanford Encyclopedia of
Philosophy. Edward N. Zalta (ed.). Accessed March 3, 2016
[2] Suler, J. The Psychology of Avatars and Graphical Space in Multimedia Chat
Communities [Electronic resource] / J. Suler // The Psychology of Cyberspace /
Rider University. Official website. Access mode:
http://wwwusr.rider.edu/~suler/psycyber/psyav.html (reference date: 1.03.2016).
[3] http://techcrunch.com/2013/04/26/facebook-animated-stickers/
[4] http://www.todayifoundout.com/index.php/2010/03/the-origins-of-the-demotivatormeme
[5] Harris R. Rethinking writing, Athlone press,London, 2000, pp 254.
[6]Chandler D. Semiotics. The Basics, London, New York, Routledge, 2007 , pp307.