Lipreading helps the brain tell similar sounds apart, not just hear better.

Watching a speaker's mouth move helps us understand what they say. Scientists wanted to know how the brain combines sights and sounds to do this. They studied brain activity in a hearing area called the superior temporal gyrus while people listened to words, watched silent faces, or experienced both together. The team tested how clearly computer models could tell words apart based on these brain signals. Adding visual mouth movements made the brain signals for distinct speech sounds, called phonemes, much clearer. It also sped up how quickly the brain figured out the words. This visual boost worked mainly for the starting consonants of words, rather than the word endings. However, it did not alter the brain's processing of finer sound details called phonetic features. Looking at a speaker's face mainly sharpens the brain's broader sound categories, which then speeds up how fast we recognize spoken words.

Why it matters

For anyone struggling to hear in a noisy room, watching the speaker's mouth can make words clearer. This study shows that the brain doesn't just amplify sound; it specifically sharpens the distinction between similar speech sounds, like 'bat' and 'gat'. That means lipreading helps at the level of individual sounds, not just overall volume. That could lead to better hearing aids or speech recognition systems that mimic this neural strategy.

Questions answered