BMI Seminar // The Multisensory Scaffold for Perception and Memory

I will summarise our efforts to identify the brain mechanisms and behavioural relevance of multisensory interactions in humans. This work in turn has wide-reaching consequences on our understanding of the organization of the brain, the functional selectivity of low-level cortices, and plasticity across the lifespan. Across studies we have used combinations of psychophysics, ERPs, fMRI and TMS, taking advantage of innovations in signal processing to yield greater mechanistic interpretability of the data. Several general conclusions are supported by the collective data. First, (near) primary cortices are loci of multisensory convergence and interactions. Second, these effects occur at early latencies (i.e. <100ms post-stimulus onset). Third, these effects directly impact behaviour and perception. Fourth, multisensory interactions are context-contingent. One the one hand, they affect not only current stimulus processing, but also later unisensory recognition. Current unisensory (auditory or visual) object recognition and brain activity are incidentally affected by prior single-trial multisensory experiences; the efficacy of which is predictable from an individual’s spatio-temporal dynamics of multisensory interactions. We then extend such findings across the lifespan to show how multisensory processes may be yoked together. Finally, examples of multisensory processes at the service of rehabilitation are presented. Together, these data underscore how multisensory research is changing long-held models of functional brain organization and perception in both health, across the lifespan, and in disease and its remediation.