Robert
Jacobs
University
of
Rochester
From Sensation to Conception: Theoretical Perspectives on Multisensory Perception and Cross-Modal Transfer
If a person is trained to recognize or categorize objects or events using one sensory modality, the person can often recognize or categorize those same (or similar) objects and events via a novel modality, an instance of cross-modal transfer of knowledge. How is this accomplished? The Multisensory Hypothesis states that people extract the intrinsic, modality-independent properties of objects and events, and represent these properties in multisensory representations. These representations mediate the transfer of knowledge across modality-specific representations. In this talk, I'll present two projects evaluating the Multisensory Hypothesis using experimental and computational methodologies. The first project examines visual-haptic transfer of object shape knowledge, and the second project examines a novel hidden (latent) variable model of multisensory perception. I'll also consider implications of an experiment demonstrating generalization from perception to motor production for our understanding of cross-modal transfer.