Here is the model in action: note how the activity follows the "active speaker", thereby predicting the ventriloquist illusion. This is the first perceptual model capable of predicting the illusion from raw audiovisual footage

Here's the model's response to McGurk stimuli with varying levels of AV sync. The regions of the face with the highest audiovisual correlation elicit strong model activity, and the overall population response is highly correlated to the probability of perceiving the illusion.

Correlation detection as a stimulus computable account for audiovisual perception, causal inference, and saliency maps in mammals Optimal cue integration, Bayesian Causal Inference, spatial orienting, speech illusions and other key phenomena in audiovisual perception naturally emerge from the collective behavior of a population ...

Correlation detection as a stimulus computable account for AV perception, causal inference & saliency maps in mammals elifesciences.org/articles/106... Image- & sound-computable population model for AV perception -> Used simulation to model psychophysical, eye-tracking & pharmacological experiments