CNL Lunch

Virginie van Wassenhove

Temporal Window of Integration in the McGurk Effect

Thursday September 20th 2001, 12:30pm, 3416 Marie Mount Hall

Although speech has mainly been the object of investigation in the auditory domain, numerous studies have supported the existence of multisensory processes in speech perception. In both normal hearing and hearing impaired subjects, visual input has been shown to improve performance. The following study focuses on a specific instance of auditory visual integration in the speech domain: the McGurk effect (McGurk & McDonald, 1976).

The McGurk effect occurs when an auditory utterance /ba/ (auditory) is dubbed onto a face articulating /ga/ (visual). The resulting percept is a combination of the auditory and visual inputs (AV) biasing the subject to experience /da/. We tested two instances of the McGurk effect using an A /ba/ dubbed onto a V /ga/ and an A /pa/ dubbed onto a V /ka/. The robustness of the McGurk effect was investigated through a range of auditory visual asynchronies spanning from about 500ms of auditory lead to 500ms of auditory lag in 30ms steps. 43 students participated in the study. In the first task, participants were presented with one type of asynchronous and synchronous McGurk tokens and were asked to identify the signal. They were offered 3 choices to identify the AV stimuli: the corresponding auditory utterance (/ba/ or /pa/), the visual utterance (/ga/ or /ka/), and the fusion token (/da/, /tha/ or /ta/). In a second task, the same students were asked to determine whether the stimuli were synchronous ("simultaneous") or asynchronous ("successive") in time. Both McGurk pairs (AbVg and ApVk) and their congruent AV utterances -respective fusion responses (AV /da/ and AV /tha/)- were tested.

Results of these experiments demonstrated the existence of a temporal window of integration in bimodal speech perception. The window duration equals about 250 ms in both McGurk conditions and in both tasks. These results raise important questions on the neural coding of auditory and visual inputs in the speech domain as well as their processing at the cortical level. Implications of this study on models of speech perception will also be addressed.