Compositional decoding of neural activity enhances generalization in handwriting BCIs
Narasimha, S. M.; Huang, J.; Sristi, R. D.; Gilja, V.; Mishne, G.
Show abstract
Recent brain-computer interfaces (BCIs) have achieved state-of-the-art performance in decoding behavior from neural activity. These models are typically trained on a con-strained set of behaviors, which limits their ability to generalize to real-world settings where behavior is variable, complex, and context-dependent. However, many complex behaviors can be decomposed into a set of reusable behavioral motifs, indicating a compositional organization. Here, we analyze human intracortical neural activity underlying attempted handwriting and find signatures of neural compositionality at a finer resolution than individual letters. We further introduce a compositional temporal decoding model, MOtif-based Temporal Inference Framework (MOTIF), that jointly predicts the fine-scale behavioral motifs (e.g., strokes, phonemes) and the longer-timescale behavior class (e.g., characters, words). We show that the compositional structure leveraged by MOTIF enables improved generalization in few-shot learning. Our results demonstrate that explicitly incorporating compositionality into neural decoders can enhance generalization and sample efficiency, while providing a principled approach to designing more scalable, robust, and interpretable BCIs.
Matching journals
The top 2 journals account for 50% of the predicted probability mass.
Similar papers in this journal
- Learning neural decoders without labels using multiple data streams 95%
- Unsupervised, piecewise linear decoding enables an accurate prediction of muscle activity in a multi-task brain computer interface 95%
- Speech decoding from a small set of spatially segregated minimally invasive intracranial EEG electrodes with a compact and interpretable neural network 94%
Similar papers in this journal
- Behavioral Classification of Sequential Neural Activity Using Time Varying Recurrent Neural Networks 94%
- Sparse Ensemble Machine Learning to improve robustness of long-term decoding in iBMIs 93%
- Identification of Spared and Proportionally Controllable Hand Motor Dimensions in Motor Complete Spinal Cord Injuries Using Latent Manifold Analysis 93%
Similar papers in this journal
- Time-adaptive Unsupervised Auditory Attention Decoding Using EEG-based Stimulus Reconstruction 92%
- pathCLIP: Detection of Genes and Gene Relations from Biological Pathway Figures through Image-Text Contrastive Learning 92%
- Hyperbolic graph embedding of MEG brain networks to study brain alterations in individuals with subjective cognitive decline 91%
Similar papers in this journal
- Bridging Auditory Perception and Natural Language Processing with Semantically informed Deep Neural Networks 93%
- A novel interpretable deep transfer learning combining diverse learnable parameters for improved T2D prediction based on single-cell gene regulatory networks 93%
- Online speech synthesis using a chronically implanted brain-computer interface in an individual with ALS 93%
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.