In order to test the importance of causality in this paradigm, we offset the auditory and visual components of the gestures in order to explore the effect of temporal synchrony. The video on the left depicts a stimuli in which the sound leads the moment of visual impact, whereas the video on the right depicts when the sound lags. Because sound travels more slowly that light, the brain is far more experienced in dealing with audio lags (right), rather than audio leads. Our results indicate the illusion is weakened when sound leads the moment of visual impact, and vanishes when it follows. This is consistent with the idea that causality plays an integral role in the illusion, as the perceptual system refrains from integrating in situations where the seen gesture could not have caused the heard note (Schutz & Kubovy, 2009, JEP:HPP – experiment 2).
|
Audio Lead Condition 
|
Audio Lag Condition
|
|
Alternative sustained sound (clarinet)
|
Alternative percussive sound (piano)
|
Additionally, we have tested the role of causality by manipulating the timbre in ways that are either inconsistent with an impact (i.e. a sustained sound – left video) or consistent with an an impact event (i.e. piano sound – right video). Similar to the previous experiment, here we found that manipulations breaking the causal link eliminated vision’s influence, whereas manipulations preserving the link retained the audio-visual integration (Schutz & Kubovy, 2009, JEP:HPP – experiment 1).