Beschreibung:
Neural processes related to audio–visual speech perception were investigated by measuring the mismatch magnetic fields (MMF), which reflect a neural activity detecting deviant stimuli randomly inserted in a stream of rapidly repeating frequent stimuli. Three audio–visual stimuli were used: AbVg (audio signal /ba/ with discrepant visual signal /ga/), AbVb, and AdVd. MMF were measured from the left hemisphere of 16 normal hearing subjects using a 37 ch SQUID magnetometer. The rate of non-/ba/ responses to AbVg (McGurk fusion effect) varied depending on the stimulus condition and subjects. It was 80% when AbVg was the frequent, but was 45% when AbVg was the deviant. Significant MMF-like fields were excited in the auditory cortex by deviant AbVg embedded in frequent AbVb. The subjects with a low-fusion rate had larger MMF-like fields than those with a high rate. These results suggest that the auditory mismatch detection process is affected by visual signal, and phonetic categorization is affected by a module which can either fuse or dissociate audio–visual information.