When the sound of one syllable is dubbed over a face saying another, listeners often hear a third, so that what is seen changes what is heard.
McGurk and MacDonald reported in Nature in 1976, under the title Hearing Lips and Seeing Voices, that the audio ba played over video of a speaker saying ga was heard by most adults as da. The illusion persists when people know about it, and closing the eyes makes the true sound return at once.
In practice
- A badly dubbed film sounds subtly wrong even to viewers who cannot say why.
- In a noisy bar, a listener catches more of a conversation by watching the speaker's mouth.
- Possible explanations
- The standard explanation is that speech perception integrates sight and sound automatically, weighting each by how reliable it is. When the two conflict, the brain settles on the syllable most consistent with both.
- In its favour
- The effect has been replicated across many languages and laboratories, appears in infants, and is used widely as a laboratory measure of audiovisual integration.
- Against it
- Researchers such as Michael Beauchamp and colleagues have found that susceptibility varies enormously between people, from almost never to almost always, and some argue that the illusion is a poor measure of how sight helps speech in everyday conversation. Its strength also varies with language and the particular recordings used.
- Standing
- A cross-modal relative of the Kuleshov Effect: context from one channel rewrites what is perceived in another. Compare the Ventriloquism Effect and Synchresis.
- Further reading
- Wikipedia
- Subject
- Psychology & Behavior
- Essay
- A century and a half of measuring the mind