My work pictured by AI – Kinkini Bhadra

What is this work about? Current successes of machine learning architectures are based on computationally expensive algorithms and prohibitively large amounts of data. We need to develop tasks and data to train networks to reach more complex and more compositional skills. In this paper, we illustrate Blackbird’s language matrices (BLMs), a novel grammatical task modelled on intelligence tests usually based on visual stimuli. The dataset is generatively constructed to support investigations of current models’ linguistic mastery and their ability to generalize them. We present the logic of the task, the method to automatically construct data on a large scale, and the architecture to learn them. Through error analysis and several experiments on variations of the dataset, we demonstrate that this language task and the data that instantiate it provide a new challenging testbed to understand generalization and abstraction.

The first word that came to mind when seeing the AI-generated picture? Goofy. 

What is this work about? Planning to speak is a challenge for the brain, and the challenge varies between and within languages. Yet, little is known about how neural processes react to these variable challenges beyond the planning of individual words. Here, we examine how fundamental differences in syntax shape the time course of sentence planning. Most languages treat alike (i.e., align with each other) the 2 uses of a word like “gardener” in “the gardener crouched” and in “the gardener planted trees.” A minority keeps these formally distinct by adding special marking in 1 case, and some languages display both aligned and nonaligned expressions. Exploiting such a contrast in Hindi, we used electroencephalography (EEG) and eye tracking to suggest that this difference is associated with distinct patterns of neural processing and gaze behavior during early planning stages, preceding phonological word form preparation. Planning sentences with aligned expressions induces larger synchronization in the theta frequency band, suggesting higher working memory engagement, and more visual attention to agents than planning nonaligned sentences, suggesting delayed commitment to the relational details of the event. Furthermore, plain, unmarked expressions are associated with larger desynchronization in the alpha band than expressions with special markers, suggesting more engagement in information processing to keep overlapping structures distinct during planning. Our findings contrast with the observation that the form of aligned expressions is simpler, and they suggest that the global preference for alignment is driven not by its neurophysiological effect on sentence planning but by other sources, possibly by aspects of production flexibility and fluency or by sentence comprehension. This challenges current theories on how production and comprehension may affect the evolution and distribution of syntactic variants in the world’s languages.

The first word that came to mind when seeing the AI-generated picture? Seeing into the mind.

What is this work about? Vocal learning plays an important role during speech development in human infants and is vital for language. However, the complex structure of language creates a colossal challenge in quantifying and tracking vocal changes in humans. Consequently, animals with simpler vocal communication systems are powerful tools for understanding the mechanisms underlying vocal learning. While human infants show the most drastic vocal changes, many adult animals, including humans, continue to show vocal learning in the form of a much-understudied phenomena called vocal accommodation. Vocal accommodation is often seen when people use similar words, pronunciations and speech rate to their conversing partner. Such a phenomena is also seen in common marmosets, a highly voluble Brazilian monkey species, with a simpler communication system compared to humans. In this project, I developed a mathematical model that explains the basic principles and rules underlying marmoset vocal accommodation. The model provides crucial insights into the mechanisms underlying vocal learning in adult animals and how they might differ from vocal learning in infant animals and humans.

The first word that came to mind when seeing the AI-generated picture? Monkey-learning.

What is this work about? Results revealed that vocal similarity between speakers increased with a larger group size which indicates a higher cooperative vocal behavior, with a negative impact on individual vocal recognizability. The results of this study inform about cross-species accommodative behavior, and human variability in cooperation in the lack of visual cues and have implications for voice processing and forensic voice comparison.

The first word that came to mind when seeing the AI-generated picture? Interaction.