Learning to Integrate an Artificial Sensory Device: From Decision Fusion to Sensory Integration

The present study examines how artificial tactile stimulation from a novel non-invasive sensory device is learned and integrated with information from another sensory system. Participants were trained to identify the direction of visual dot motion stimuli with a low, medium, and high signal-to-noise ratio. In bimodal trials, this visual direction information was paired with reliable symbolic tactile information. Over several training blocks, discrimination performance in unimodal tactile test trials and subjects confidence in their decision improved, indicating that participants were able to associate the visual and tactile information consciously and thus learned the meaning of the symbolic tactile cues. Formal analysis of the results in bimodal trials showed that both modalities are being integrated already in the early learning phases. Our modeling results revealed that this integration is consistent with a Bayesian model, which is an optimal integration of sensory information. Furthermore, we showed that a confidence-based Bayesian integration explains the observed behavioral data better than the classical variance-based Bayesian integration. Thus, the present study demonstrates that humans can consciously learn and integrate an artificial sensory device that delivers symbolic tactile information. This finding connects the field of multisensory integration research to the development of sensory substitution systems.