The way our senses work together to interpret speech can be changed by learning a second language, a new Northwestern University study suggests. Researchers found that bilingual people are better at integrating sight and hearing to make sense of speech.
When people hear a speech sound (e.g. “ba”) that conflicts with what they see (e.g. “ga”), they will often perceive a completely different sound (e.g. “da”). This illusion is called the McGurk Effect, and researchers found it is more likely to occur if you speak more than one language. This demonstrates that language experience can change the way we perceive the world around us.
“We find that language experience can change sensory perception. Our discovery is that bilinguals are more likely to integrate across auditory and visual senses,”
said Viorica Marian, a professor of communication sciences and disorders and psychology at Northwestern University.
Previous research demonstrated that multiple languages compete with each other in the brain, making it more difficult for bilinguals to process what they hear. As a result and out of necessity, they may rely more heavily on visual input to make sense of sound.
“A bilingual and monolingual listening to the same speaker can hear two completely different sounds, showing that language experience affects even the most basic cognitive processes,”
said Sayuri Hayakawa, study co-author and post-doctoral research scientist.
Bilingual experience can impact domains ranging from memory to decision making, to cognitive control, but these findings suggest that learning a second language can even change our basic sensory experiences.
Given that more than half of the world’s population is bilingual, educators and clinicians working with bilinguals should be aware of how language experience can change the way people process speech. This effect of bilingualism is also relevant for developers of technology related to speech recognition such as Siri and Alexa, as well as animators of CGI.
The research was funded by the National Institute of Child Health and Human Development and the National Institute on Deafness and Other Communication Disorders.
Viorica Marian, Sayuri Hayakawa, Tuan Q. Lam and Scott R. Schroeder
Language Experience Changes Audiovisual Perception
Brain Sci. 2018, 8(5), 85; doi:10.3390/brainsci8050085