My work pictured by AI – Kinkini Bhadra

What is this work about? Human language is characterized by a remarkable expressive capacity that allows us to convey complex thoughts, emotions, and ideas with a high degree of accuracy. At the core of this expressive capacity is the aptitude to manipulate syntactic configurations, which enables us to combine individual units (words) into larger sequences with a derived meaning (sentences and narratives). How syntactic structures are implemented in cortical circuits is still an open question in neuroscience and language evolution. In my project, I employ a combination of computational models and intracranial recordings (sEEG, ECoG) obtained from patients with intractable epilepsy to investigate how the human brain generates syntactic structures during spontaneous speech production.

The first word that came to mind when seeing the AI-generated picture? Complexity.

What is this work about? The speech signal carries hierarchically organized acoustic and linguistic information. Recent research suggests that the brain uses brain waves, called cortical oscillations, to process this information. Especially oscillations in the theta frequency range (4-8 Hz) have been found to be important: Theta oscillations process acoustic energy in the speech signal associated with the timing of syllables. However, there is also slower information in the speech signal that corresponds to stress and intonation patterns and are part of the prosody – the rhythm and melody – of a language.

To better understand how the brain processes these different levels at the same time, we conducted an experiment with 30 participants who listened to German sentences with manipulated stress and intonation patterns. We found that the brain is able to simultaneously process the syllable, stress and intonation patterns of speech. However, changes in stress patterns disrupted the brain’s ability to track syllables with theta oscillations. Conversely, the brain was able to compensate for changes in intonation patterns by using linguistic knowledge. Additionally, we found that individuals varied in their ability to process the prosodic structure of the speech signal, with some participants better able to compensate for acoustic changes than others. Overall, our results support the idea that the brain uses a hierarchical organization of cortical oscillations to process the speech signal.

The first word that came to mind when seeing the AI-generated picture? Nostalgia.

What is this work about? The paper explains how computers can be taught to recognize speech sounds in any language. In human language, there are sounds that carry meaning, and these sounds are called phonemes. The paper shows how a computer can learn to recognize these phonemes from raw speech data, without being told explicitly what the phonemes are. We use several mathematical techniques belonging to a family of methods called “unsupervised learning” to analyze the speech data and group similar sounds together. The resulting groups correspond to phonemes, which are the basic building blocks of language. This research helps us better understand how aspects of natural languages can be learnt by machines or by humans.

The first word that came to mind when seeing the AI-generated picture? Language and computer.