Decoding Continuous Character-based Language from Non-invasive Brain Recordings
Zhang, C.; Zheng, X.; Yin, R.; Geng, S.; Xu, J.; Gao, X.; Lv, C.; Ling, Z.; Huang, X.; Cao, M.; Feng, J.
Show abstract
Deciphering natural language from brain activity through non-invasive devices remains a formidable challenge. Previous non-invasive decoders either require multiple experiments with identical stimuli to pinpoint cortical regions and enhance signal-to-noise ratios in brain activity, or they are limited to discerning basic linguistic elements such as letters and words. We propose a novel approach to decoding continuous language from single-trial non-invasive fMRI recordings, in which a three-dimensional convolutional network augmented with information bottleneck is developed to automatically identify responsive voxels to stimuli, and a character-based decoder is designed for the semantic reconstruction of continuous language characterized by inherent character structures. The resulting decoder can produce intelligible textual sequences that faithfully capture the meaning of perceived speech both within and across subjects, while existing decoders exhibit significantly inferior performance in cross-subject contexts. The ability to decode continuous language from single trials across subjects demonstrates the promising applications of non-invasive language brain-computer interfaces in both healthcare and neuroscience.
Matching journals
The top 8 journals account for 50% of the predicted probability mass.
Similar papers in this journal
- Incremental Accumulation of Linguistic Context in Artificial and Biological Neural Networks 96%
- Higher-order connectomics of human brain function reveals local topological signatures of task decoding, individual identification, and behavior 94%
- Faster than thought: Detecting sub-second activation sequences with sequential fMRI pattern analysis 94%
Similar papers in this journal
- Searching through functional space reveals distributed visual, auditory, and semantic coding in the human brain 96%
- Variation in the geometry of concept manifolds acrosshuman visual cortex 95%
- Convolutional neural networks can identify brain interactions involved in decoding spatial auditory attention 94%
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.