When our emotions go digital
Researchers from the University of Lausanne and the EPFL are researching the way in which the Swiss population share emotions via instant messaging. The study, which is supported by the CROSS programme and is part of the National Centre of Competence in Research (NCCR) Evolving Language, is an innovative research topic that is particularly interesting for the preservation of Switzerland’s linguistic heritage.
By Emilie Wyss
In Collaboration with Aris Xanthos & Andrea Grütter
Laughing out loud, sobbing or babbling. Sharing our emotions is a natural phenomenon that we learn from birth, and it shows up in our language as well as in our body language. But how can we display our emotions when we are chatting via instant messaging without seeing or hearing each other?
The ACCOMOJI project, supported by the CROSS programme of the University of Lausanne and the EPFL, aims precisely at researching this topic. “We are examining the way in which the Swiss use emojis when they communicate with each other in an attempt to understand their role in managing social and emotional distance,” explains Aris Xanthos, senior lecturer and researcher at the University of Lausanne (UNIL). This study is taking place in parallel with the one that Aris Xanthos is carrying out within the NCCR Evolving Language, which investigates not only emojis, but also interjections, contractions and other forms of language specific to this mode of communication and which allow emotions to be expressed.
With ACCOMOJI, Aris Xanthos, together with Anita Auer (UNIL) and Robert West (EPFL) who are leading the project with him, used a citizen science approach in which the Swiss population was invited to annotate authentic WhatsApp conversation emojis in terms of function and emotional content. “We chose WhatsApp conversations because it is a tool that is widespread among the population and has become a standard of communication,” says Andrea Grütter, a PhD student in linguistics at the University of Zurich and research assistant at UNIL on the project. The researchers are particularly interested in accommodation phenomena: “Over time, speakers adapt their communication to their interlocutors. We are studying how this dynamic is reflected in the use of emojis over the course of conversations”, illustrates Aris Xanthos.
“We are appealing to the public because computers are not yet able to analyse certain aspects of messages reliably. How can the machine detect sarcasm, for example? Or the nature of the relationship between the two people interacting?”
Aris Xanthos
Preserving the Swiss linguistic heritage
Today more than ever, chat is king. First of all in our private circles, but also in the professional world where many colleagues use instant messaging tools. And the pandemic has exacerbated this trend even more. Whatsapp, for example, saw a 40% jump in usage in the last two weeks of March 2020, corresponding to the first social distancing measures for many countries.
This is not the first time that emojis have been put under the microscope by researchers. The originality of the ACCOMOJI project lies in the citizen science approach (using resources made available by the Citizen Science Center Zurich), as opposed to crowdsourcing or the use of language processing algorithms. It is also characterised by its focus on Swiss national languages and the use of a Swiss corpus. Thus, the data is taken from the What’s Up, Switzerland? corpus and was collected in the summer of 2014 by a team of linguists.
“Working on national languages is a real challenge because there are relatively few data and linguistic resources. This corpus is probably the largest that exists at the moment and yet it still contains too few conversations in national minority languages. But this is an additional motivation to work on this data and thus try to make a contribution to the Swiss linguistic heritage.“
Aris Xanthos
More recent conversations and in minority languages
In addition to emojis, various ways of expressing emotions are used in digital conversations, including:
- Repeating letters
yeeees
- Punctuation
yes. – yes!!
- Emoticons
yes :-))
- Interjections
yes haha
- GIFs (Animated image named after its computer format, .gif, short for Graphic Interchange Format)
“In a preliminary analysis we saw that the longer the conversation between two people goes on, the less emojis they use. There could be several reasons for this, such as sarcastic tones that need more clarification, for example,” tells Aris Xanthos. Furthermore, there seems to be a relationship between emojis, emoticons and letter repetitions: the more emojis there are, the less emoticons and letter repetitions there are.
This preliminary examination of the data confirms the importance of collecting more recent and linguistically diverse data. “It could very well be that the trend, of fewer emojis during a conversation, is a phenomenon related to the time period covered by the data, 2010-2014, at the very beginning of emojis on iOS and Android, and that it is not observed in more recent data,” reports Aris Xanthos.
What’s New Switzerland
This new data collection, entitled “What’s New Switzerland”, will take place within the National Centre of Competence in Research (NCCR) Evolving Language in 2022. This will be followed by a new call for participatory science and then a phase of computational data processing with a deep learning system. In other words, researchers will train machine learning models to detect emotions in digital communication.
Together, the results of the CROSS project and those of the NCCR Evolving Language will make it possible to assess the relevance of formal methods and models for dealing with the emotional aspects of digital communication, as well as to better understand how participatory science practices can be applied to linguistic research in the Swiss multilingual context.
Media Contact
Dr. Aris Xanthos
NCCR Evolving Language
Department of Language and Information Sciences (SLI)
University of Lausanne
021 692 30 25
Aris.Xanthos@unil.ch
Andrea Grütter
NCCR Evolving Language
English Department
University of Zurich
044 634 43 00
andrea.gruetter@es.uzh.ch
