Making sense of language through algorithms with Prof. Rico Sennrich
After being an SNSF professor for 6 years, Rico Sennrich has taken on a new role. This August, he became an assistant professor in computational linguistics at the University of Zurich. During his masters in English Literature, his minor in computational linguistic drew him into the field. “In computational linguistics, it’s easy to test your theories,” he says.
Making computers talk
At its core, computational linguistics is about trying to teach computers to understand and use human language. There are many different reasons for this, from automatic translations to language models.
One of the goals, for example, is to develop algorithms with which computers can process language. “The goal here is quality and efficiency, not that the process is human-like,” says Rico. “It’s interesting to think of the different ways humans and computers are constrained – we can feed computers with a lot more data than humans, but they are also less data-efficient. ” So, by definition, the algorithm will look different than the human language processing algorithm. “However, human processing can be an inspiration for the model in some fields,” he adds.
Professor Rico Sennrich
During his career, Rico Sennrich figured out a better way to map text into artificial neural networks, which then learn meaning representations. “Words are a poor choice, because the vocabulary grows too large and many don’t appear often enough, but the character-level is also problematic because longer sequences are harder to process efficiently,” explains the researcher. “So we developed an algorithm that separated words into subwords and used this for mapping.” This method allows for a great compromise between vocabulary size and sequence length, making it a practical choice for neural networks. It is now used in language models like ChatGPT.
Between models and tools
Rico Sennrich’s work focuses on multilingual language processing, a key discipline within the NCCR Evolving Language. “I am interested in how knowledge contained in linguistic models is transferred from one language to another,” he says. This allows models to learn facts in one language and then answer questions on that subject in any other language they have mastered. “This is a key factor in data efficiency, as models can become useful in a new language with much less data from that language than if we had to train a separate model for each language,” explains the researcher.
Within the NCCR Evolving Language, he is involved in a handful of tasks, in which he uses linguistic models to compare human behaviour or as a research tool. For example, in the ‘Silicon Meaning’ task, he is working on how linguistic models represent knowledge, also comparing them to human processing, while in the ‘Meaning Change’ task, he is developing tools to study changes in meaning over long periods of time, for example in Sanskrit. “In the NCCR field, there is a great need for new methods that go beyond the sentence level and focus on interactions; that’s what I hope to bring to the project,” concludes Rico Sennrich.
