Multisensory Integration, Segregation, and Causal Inference in the Superior Colliculus

The environment is sampled by multiple senses, which are woven together to produce a unified perceptual state. However, unifying these senses requires assigning particular signals to the same or different underlying objects or events. Sensory signals originating from the same source should be integrated together, while signals originating from separate sources should be segregated from one another. Each of these computations is associated with different neural encoding strategies, and it is unknown how these strategies interact. Here, we begin to characterize how this problem is solved in the primate brain. First, we developed a behavioral paradigm and applied a computational modeling approach to demonstrate that monkeys, like humans, implement a form of Bayesian causal inference to decide whether two stimuli (one auditory and one visual) originated from the same source. We then recorded single unit neural activity from a representative multisensory brain region, the superior colliculus (SC), while monkeys performed this task. We found that SC neurons encoded either segregated unisensory or integrated multisensory target representations in separate sub-populations of neurons. These responses were well described by a weighted linear combination of unisensory responses which did not account for spatial separation between targets, suggesting that SC sensory responses did not immediately discriminate between common cause and separate cause conditions as predicted by Bayesian causal inference. These responses became less linear as the trial progressed, hinting that such a causal inference may evolve over time. Finally, we implemented a single trial analysis method to determine whether the observed linearity was indicative of true weighted combinations on each trial, or whether this observation was an artifact of pooling data across trials. We found that initial sensory responses (0-150 ms) were well described by linear models even at the single trial level, but that later sustained (150-600 ms) and saccade period responses were instead better described as fluctuating between encoding either the auditory or visual stimulus alone. We also found that these fluctuations were correlated with behavior, suggesting that they may reflect a convergence from the SC encoding all potential targets to preferentially encoding only a specific target on a given trial. Together, these results demonstrate that non-human primates (like humans) perform an idealized version of Bayesian causal inference, that this inference may depend on separate sub-populations of neurons maintaining either integrated or segregated stimulus representations, and that these responses then evolve over time to reflect more complex encoding rules.