Hearing Lips and Seeing Voices After Fifty Years: A Large-Scale McGurk Illusion Dataset for Audiovisual Speech Research

Efficient face-to-face communication relies on the integration of auditory speech and visual articulatory signals. Over the past five decades, the McGurk illusion has been widely used as an index of audiovisual speech integration. However, substantial variabilities in susceptibility to the illusion across participants and speakers limit its reliability as a stable measure of audiovisual integration ability. Here, we introduce the McGurk illusion dataset (MID), which, to our knowledge, is the largest publicly available McGurk stimulus dataset to date. The MID comprises auditory (N = 400), visual (N = 400), and audiovisual (N = 640) speech stimuli generated from 80 Mandarin speakers and validated through behavioral judgments across 360,900 trials. Using this dataset, we characterized the acoustic and facial articulatory properties of McGurk stimuli, replicated substantial inter-participant and inter-speaker variabilities in illusion susceptibility, and revealed the associations between variations in McGurk illusion rate and the variations in unisensory perception, audiovisual correspondences, and speakers characteristics. Furthermore, the stimulus set enabled systematic comparisons of the reliability of different McGurk illusion-based indices of audiovisual speech integration. Overall, the MID not only provides a standardized resource for investigating audiovisual speech integration and its alterations across populations, but also supports research on speaker normalization, lip-reading, and speech perception.