With the growing use of conversational user interfaces, stems the need for a better understanding of social and emotional characteristics embedded in the online dialogues. Specifically, textbase chatbots face the challenge of conveying human-like behavior while being restricted to one channel of interaction, such as texts. The aim of this paper is to investigate whether or not it is possible to normalize and formalize the use of emojis for a comprehensive and complete means of communication. In an effort to answer this question, and as a primary source, the paper will investigate the findings of a new study published in 2021 by the University of Virginia, How emoji and word embedding helps to unveil emotional transitions during online messaging. The study found that chatbots design can be enhanced to have the ability to understand the “affective meaning of emojis.” By that, chatbots will be more capable to understand the social and emotional state of the users and subsequently conduct a more naturalistic conversation with humans. The paper concludes by calling for more empirical research on chatbots using emojis for emotionally intelligent online conversations.
Throughout history, humans established relationships using explicit means of communication, such as words, and implicitly by using their body language. Most of these relationships were developed through face-to-face interactions. Body language delivers important visual clues to what is said. In fact, small clues such as facial expressions or gestures add a great deal of meaning to our words. In the last few decades and with the growing use of different forms of technology, people shifted to communicating through text and voice messaging in an online form. Chatbots, which is the common name of voice assistants and virtual assistants, as well as the text chat, is an important technology that has various implementations. Despite the widespread use of this service, especially to support businesses, chatbots technology still lacks efficiency due to the absence of body language. In this paper, I will explore the impact of using other informal means of communication in the chatbot texting service to replace body language and identify emotions. Using Emojis to infer emotional changes during chatbot texting, is one of the means I propose. In an effort to make textbase chatbots’ conversations with humans more efficient, I will try to answer the question on whether or not it is possible to normalize and formalize the use of emojis, for a comprehensive and enhanced means of communication. For this purpose, I will start by looking into the history of text chatbot service, and will then deblackbox the different layers, levels, and modules composing this technology. The aim of this paper is to contribute to finding solutions for the urgent challenges facing one of the most growing services in the field of technology today.
Definition of Chatbots
According to Lexico, a chatbot is “A computer program designed to simulate conversation with human users, especially over the internet.” Also, it is an artificial intelligence program, as well as a Human–Computer Interaction (HCI) model.” This program uses natural language processing (NLP) and sentiment analysis to conduct an online conversation with humans or other chatbots via text or oral speech. To describe this software, Micheal Mauldin, called it “ChatterBot” in 1994, after creating the first Verbot; Julia. Today, Chatbots is the common name of artificial conversation entities, interactive agents, smart bots, and digital assistants. Due to its flexibility, the chatbots’ digital assistants proved useful in many fields, such as education, healthcare and business industries. Also, it is used by organizations and governments on websites, in applications, and instant messaging platforms to promote products, ideas or services.
The interactivity of technology in combination with artificial intelligence (AI) have greatly improved the abilities of chatbots to emulate human conversations. However, chatbots are still unable to conduct conversational skills like humans do. This is due to the fact that chatbots today are not fully developed to infer their user’s emotional state. Progress is achieved everyday and chatbots are gradually getting more intelligent and more aware of their interlocutor’s feelings.
The Evolution History of Chatbots
What is called today as a benchmark for Artificial Intelligence (AI), “Turing test,” is rooted in Alan Turing’s well known paper that was published in 1950; Computing, Machinery and Intelligence. The overall idea of Turing’s paper is that machines too can think and are intelligent. We can consider this as the starting point of bots in general. For Turing, “a machine is intelligent when it can impersonate a human and can convince its interlocutor, in a real-time conversation, that they are interacting with a human.”
In 1966, The German computer scientist and professor at Massachusetts Institute of Technology (MIT), Joseph Weizenbaum, built on Turing’s idea to develop the first chatterbot program in the history of computer science; ELIZA. This program was designed to emulate a therapist who would ask open-ended questions and respond with follow-ups. The main idea behind this software is to make ELIZA’s users believe that they are conversing with a real human therapist. For this purpose, Weizenbaum programmed ELIZA to recognize some key words from the input and regenerate an answer using these keywords from a pre-programmed list of responses. Figure 1, illustrates a human conversation with ELIZA. It shows clearly how this program picks up a word, and responds by asking an open-ended question. For example, when the user said: “He says that I’m depressed much of the time,” ELIZA took the word “depressed” and used it to formulate its next response, “I am sorry to hear that you are depressed.” This case of open-ended questions created an illusion of understanding and having an interaction with a real human being, meanwhile the whole process was an automated one. PARRY is a more advanced copy of ELIZA that was founded in 1972. It was designed to act like a patient with schizophrenia. Like ELIZA, PARRY was a chatbot but with limited capabilities in terms of understanding language and expressing emotions. Add to it, PARRY was a slow respondent and couldn’t learn from the dialogue.
Figure 1: A human Conversation with ELIZA, Source: https://en.wikipedia.org/wiki/File:ELIZA_conversation.png
The British programmer Rollo Carpenter was the first pioneer to use AI for his chatbot; Jabberwacky, back in 1982. Carpenter aimed at simulating a natural human chat that can pass the Turing test. “Jabberwacky was written in CleverScript, a language based on spreadsheets that facilitated the development of chatbots, and it used contextual pattern matching to respond based on previous discussions.” Like his predecessors, Carpenter was not able to program Jabberwacky with high speed or to deal with large numbers of users.
The actual evolution of chatbot technology happened in 2001 when it was made available on messengers such as America Online (AOL) and Microsoft (MSN). This new generation of chatbots “retrieved information from databases about movie times, sports scores, stock prices, news, and weather.” The new improvement in this technology paved the way for a real development in machine intelligence and human–computer communication.
A new improvement to AI chatbots took place with the development of smart personal voice assistants, which were built into smartphones and home speaking devices. These voice assistants received voice commands, answered in digital voice and implemented tasks such as monitoring home automated devices, calendars, email and other applications. Multiple companies introduced their voice assistants; Apple SIRI (2010), IBM Watson (2011), Google Assistant (2012), Microsoft Cortana (2014) and Amazon Alexa (2014). The main distinction between the new generation of chatbots and the old ones is the quick meaningful response to the human interlocutor.
By all means, 2016 was the year of chatbots. In this year, there was a substantial development in AI technology, in addition to introducing the Internet of Things (IoT) to the field of chatbots. AI changed the way people communicated with service providers since “social media platforms allowed developers to create chatbots for their brand or service to enable customers to perform specific daily actions within their messaging applications.” The integration of chatbots in the IoT scenario opened the door wide for the implementation of such systems. Thanks to the development in natural language processing (NLP) and compared to ELIZA, today’s chatbots can share personal opinions and are more relevant in their conversation. However, they can be vague and misleading as well. The important point to note here is that chatbots are still being developed and as a technology, it hasn’t yet realized its fullest potential. This brief historical overview of the evolution of chatbots tells us that although the technology has experienced rapid developments, it is yet to promise us a world of possibilities, if properly utilized.
There are several ways to categorize chatbots (see Figure 2). First, they can be categorized according to their purpose as either assistants or for interlocutors. Assistant chatbots are developed to assist users in their daily activities, such as schedule an appointment, make a phone call, search for information on the internet and more. Second, Chatbots can also be grouped according to their communication technique, and this can be either via text, voice or image, or all of them together. Recently, chatbots can respond to a picture, comment and even express their emotions towards this picture. The third categorization is related to the chatbots’ knowledge domain, and it is the access range provided for the bots. Based on the scope of this access, a bot can be either generic or specific. While generic bots can in fact answer questions from any domain, the domain-specific chatbots respond only to questions about a specific knowledge domain. Interpersonal chatbots are also under the communication technique category and they are the bots that offer services without being a friendly companion. In addition, there are the Intrapersonal chatbots, which are close companions and live in their user’s domain. The Inter-agent chatbots are the ones that can communicate with other chatbots such as Alexa and Cortana. Fourth category is according to classification. Under this category, chatbots are classified into three main classes. The informative chatbots are used to give information to their user; these information are usually stored in a fixed source. The chat-based/conversational chatbots which conduct a natural conversation with their user like a human. Finally, the task-based chatbots that handle different functions and are excellent at requesting information and responding to the user appropriately. It is important to mention that the method that a chatbot uses to generate its response categorizes it into a rule-based, retrieval based, or a generative based chatbot. This paper will focus on the class of bots that use texts as means of communication.
The Chatbots Technology
Depending on the algorithms and techniques, there are two main approaches for developing chatbot technology; the pattern matching and the pattern recognition using machine learning (ML) algorithms. In what follows, I will provide a brief description of each technique, however, this paper is concerned with AI/ML pattern recognition chatbots.
Pattern Matching Model
This technique is used in rule-based chatbots, such as ELIZA, PARRY and Jabberwacky. In this case, chatbots “match the user input to a rule pattern and select a predefined answer from a set of responses with the use of pattern matching algorithms.” In contrast to knowledge-base chatbots, rule-based ones are unable to generate new answers because their knowledge comes from their developers who developed this knowledge in the shape of conversational patterns. Despite the fact that these bots are fast responding, however, their answers are automated and not spontaneous like the knowledge-base chatbots. There are three main languages used to develop chatbots with the pattern-matching technique; Artificial Intelligence Markup Language (AIML), Rivescript, and Chatscript.
Pattern Recognition Model: AI/ML Empowered
The main distinction between the pattern matching and pattern recognition bots, which is in more scientific words, rule-based and knowledge-based bots is the presence of Artificial Neural Networks (ANNs) algorithms in the latter case. By using AI/ML algorithms, these relatively new bots can extract the content from their users input using natural language processing (NLP) and the ability to learn from conversations. These bots need an extensive amount of Data training set as they do not rely on predefined response for every input. Today, developers use ANNs in the architecture of ML empowered chatbots. It is useful to mention here that retrieval-based chatbots use ANNs to select the most relevant response from a set of responses. Meanwhile, generative chatbots synthesize their reply using deep learning techniques. The focus of this paper is on the chatbots using deep learning methods since this is the dominant technology used in today’s chatbots.
Deblackboxing Chatbot technology
Uncovering the different layers, levels, and modules in the chatbots will help us to better understand this technology and the way it works. In fact, there are many designs that vary depending on the type of chatbot. The following description reveals the key design principles and main architecture that applies to all chatbots.
Figure 3 Demonstration of the general architecture for AI chatbot of the entire process. Source: How emoji and word embedding helps to unveil emotional transitions during online messaging
In an analysis of Figure 3, we can see the different layers of operation within a chatbot including the user interface layer, the user message analysis layer, the dialog management layer, the backend layer and finally, the response generation layer. The chatbot process begins when the software receives the user’s input through an application using text. The input is then sent to the user message analysis component to find the user’s intention following pattern matching or machine learning approaches. In this layer, Natural Language Processing (NLP) breaks the input down, comprehends its meaning, spell checks and corrects user spelling mistakes. The user’s language is identified and translated into the language of the chatbot with what is called Natural Language Understanding (NLU) which is a “subset of NLP that deals with the much narrower, but equally important facet of how to best handle unstructured inputs and convert them into a structured form that a machine can understand” and act accordingly. Then the dialog management layer controls and updates the conversation context. Also, it asks for follow-up questions after the intent is recognized. After the intent identification, the chatbot proceeds to respond or ask for information retrieval from the backend. The chatbot retrieves the information needed to fulfill the user’s intent from the Backend through external Application Performance Interfaces (APIs) calls or Database requests. Once the appropriate information is extracted, it gets forwarded to the Dialog Management Module and then to the Response Generation Module which uses Natural Language Generation (NLG) to turn structured data into text output that answers the main query.
The chatbots architecture is supported today with three important trends of technology; AI/ML algorithms, Big data and cloud computing systems. On one hand, AI/ML enable intelligent algorithms that are capable of learning on the go. These algorithms are the artificial neural networks (ANN) which are means of training data that empower chatbots’ outputs with greater “accuracy” (lower error rate). On the other hand, Big data provides AI/ML hungry ANN algorithms with a big amount of data which in turn enriches chatbot’s backend storage. Then the vast amount of AI trained chatbots’ output data needs the scalability and extensibility offered by cloud computing, in the shape of cheap extensible storage memories. This unique combination “offers huge advantages in terms of installation, configuration, updating, compatibility, costs and computational power” for chatbots.”
The above shows us that the chatbots technology is very complex and intricate. Nevertheless, it is at the same time flexible and can be easily further developed and upgraded with new layers. After analyzing the chatbots layers and gaining a better understanding of the role of each of these layers, we can in fact incorporate our desired upgrades and prepare for the new generation of charbots that are able to relate to the interlocutors’ emotions over text.
Discussion around the Main Argument:
As humans, we develop relationships through everyday face-to-face interactions. Body language delivers important visual clues to what we say. In fact, small clues such as facial expressions or gestures add a great deal of meaning to our words. In the 60s, Professor Albert Mehrabian formulated the 7-38-55% communication rule about the role of nonverbal communication and its impact during face-to-face exchanges, (see figure 4). According to this rule, “only 7% of communication of feelings and attitudes takes place through the words we use, while 38% takes place through tone and voice and the remaining 55% of communication take place through the body language we use.”
Figure 4: Theory of communication. Source: https://cisv.org/2019/06/06/two-key-parts-of-effective-communication/
In the last twenty years and with the growing use of different forms of technology, people shifted to communication through text and voice messaging in the online space. Chatbots is one of the important technologies that has various implementations. Despite the widespread use of this technology, chatbots still lack efficiency due to the absence of body language and ability to infer emotions, feelings and attitudes of its interlocutor. To solve this issue, many researchers proposed different scenarios. Our guide in this discussion is the primary source “How emoji and word embedding helps to unveil emotional transitions during online messaging.” This source is the first study of its kind, by the University of Virginia, and it suggests that using emojis and word embedding to model the emotional changes during social media interactions is an alternative approach to making the textbase chatbot technology more efficient. Also, the study advocates for the fact that extended affective dictionaries, which include emojis, will help in making chatbots work more efficiently. The study “explores the components of interaction in the context of messaging and provides an approach to model an individual’s emotion using a combination of words and emojis.” According to this study, detecting the user’s emotion during the dialogue session will improve chatbots’ ability to have a “more naturalistic communication with humans.”
Moeen Mostafavi and Michael D. Porter, the researchers who conducted this project, believe that tracking a chatbot user’s emotional state during the communication process needs a “dynamic model.” For this model they consulted the “Affect Control Theory (ACT) to track the changes in the emotional state of the user during his/her communication with a chatbot after every dialogue. Figure 5 demonstrates the interaction between a customer and a Chatbot using emojis. This interesting study concludes with an important finding: chatbots design can be enhanced to have the ability to understand the “affective meaning of emojis.” However, there is a need to extend dictionaries to support the researchers’ use of ACT to apply new designs for chatbots behaviors. The researchers claim that the increasing use of emojis in social media communication today will facilitate adding them to dictionaries to support the researchers’ efforts.
As this research paper demonstrates, the chatbots’ flexibility and the technological advances make it easy for the chatbot designers to incorporate the use of emojis in a more intelligent manner. This integration would increase this tool’s ability to understand, analyze and respond to the emotional changes of the human on the other end of the chat is experiencing. Nevertheless, I suggest that the challenge for this process is in building a rich foundation for these emojis in the dictionaries and which requires a collaboration at higher levels and more.
Figure 5: An interaction between a user and a Chatbot using emojis. Source: How emoji and word embedding helps to unveil emotional transitions during online messaging
Taking into account the significant human financial and capital investment committed to the development of chatbots and other AI-driven conversational user interfaces, it is necessary to understand this complex technology. The focus of the chatbot community has so far concentrated on the language factors, such as NLP. This paper argues that it is equally important to start heavily investing in the social and emotional factors in order to enhance the abilities of the textbase AI-driven chatbots. Chatbots have a long way to go before they realize their fullest potential and pass the Turing test. However, promising improvement surfaced in the last few years. The goal of this paper was to investigate whether or not it is possible to harness the use of an already available tool, such as emojis, to enhance the communication power of chatbots. A unique newly published primary source was investigated to help in answering this question. Understanding the evolution history of chatbots, their categories, and technology was important to deblackbox this complex technology. This clarity helps us realize that adding emojis to this complex process is not an easy one but still not impossible, given the additional support provided by three important technologies, AI/ML, Big data and Cloud computing. Using Emojis for chatbots involves applying modifications to the main structure of chatbot’s architecture by adding a new layer. Add to it, there will be a need to extend traditional dictionaries by adding emojis to support the process. The primary source found, by evidence, that implementing this new approach will definitely provide chatbot with new abilities to become more intelligent. To conclude, there is still a need for more empirical research on chatbots’ use of emojis as a leverage. The process is not easy but given the huge investment and growing need for chatbot in many fields, the potential for the outcomes of such research will be groundbreaking and will transform the human’s experience with chatbots as a tool of support.
Adamopoulou, Eleni, and Lefteris Moussiades. “Chatbots: History, Technology, and Applications.” Machine Learning with Applications 2 (December 15, 2020): 100006. https://doi.org/10.1016/j.mlwa.2020.100006.
The British Library. “Albert Mehrabian: Nonverbal Communication Thinker.” The British Library. Accessed May 8, 2021. https://www.bl.uk/people/albert-mehrabian.
“Chatbot.” In Wikipedia, May 6, 2021. https://en.wikipedia.org/w/index.php?title=Chatbot&oldid=1021732982.
“CHATBOT | Definition of CHATBOT by Oxford Dictionary on Lexico.Com Also Meaning of CHATBOT.” Accessed May 8, 2021. https://www.lexico.com/en/definition/chatbot.
Fernandes, Anush. “NLP, NLU, NLG and How Chatbots Work.” Medium, November 9, 2018. https://chatbotslife.com/nlp-nlu-nlg-and-how-chatbots-work-dd7861dfc9df.
Hoy, Matthew B. “Alexa, Siri, Cortana, and More: An Introduction to Voice Assistants.” Medical Reference Services Quarterly 37, no. 1 (January 2, 2018): 81–88. https://doi.org/10.1080/02763869.2018.1404391.
“Jabberwacky.” In Wikipedia, November 15, 2019.
Jurafsky, Daniel and James H. Martin. Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition, 2nd ed. (Upper Saddle River, N.J: Prentice Hall, 2008).
Kar, Rohan, and Rishin Haldar. “Applying Chatbots to the Internet of Things: Opportunities and Architectural Elements.” International Journal of Advanced Computer Science and Applications 7, no. 11 (2016): 8.
Mell, Peter, and Tim Grance. “The NIST Definition of Cloud Computing.” National Institute of Standards and Technology, September 28, 2011. https://doi.org/10.6028/NIST.SP.800-145.
Molnár, György, and Zoltán Szüts. “The Role of Chatbots in Formal Education.” In 2018 IEEE 16th International Symposium on Intelligent Systems and Informatics (SISY), 000197–000202, 2018. https://doi.org/10.1109/SISY.2018.8524609.
Mostafavi, Moeen, and Michael D. Porter. “How Emoji and Word Embedding Helps to Unveil Emotional Transitions during Online Messaging.” ArXiv:2104.11032 [Cs, Eess], March 23, 2021. http://arxiv.org/abs/2104.11032.
Shah, Huma, Kevin Warwick, Jordi Vallverdú, and Defeng Wu. “Can Machines Talk? Comparison of Eliza with Modern Dialogue Systems.” Computers in Human Behavior 58 (May 1, 2016): 278–95. https://doi.org/10.1016/j.chb.2016.01.004.
Shum, Heung-yeung, Xiao-dong He, and Di Li. “From Eliza to XiaoIce: Challenges and Opportunities with Social Chatbots.” Frontiers of Information Technology & Electronic Engineering 19, no. 1 (January 1, 2018): 10–26. https://doi.org/10.1631/FITEE.1700826.