Mora-ERP-based RNN-Transformer for decoding single-trial EEGs during silent Japanese speeches
Yamazaki, T.; Kudo, S.; Kamata, S.-i.; Fujii, S.; Tsukiyama, S.; Yata, T.; Aoki, S.
Show abstract
We developed a method for decoding single-trial electroencephalography (EEG) during silent Japanese speeches. In order to cope with problems that there would be always noises in single-trial EEGs, a recurrent neural network (RNN) was used which could reproduce signals under noises. Each of silent-mora-related potentials and the single-trial EEG minus the event-related potential (ERP) were assigned to the signal and the noise, respectively, with reference to the averaging principle. Next, in our Transformer, dot product between the RNN output and the single-trial EEG after positional encoding then Softmax with Loss yielded probabilities of moras, each of which consists of silent Japanese words, phrases or part of sentences. The present decoding was completed by tracing the maximal probability at each block representing time. Average mora error rates (MERs) on pretrained and validated performances for the patient was as low as 1.5 % and 0 %, respectively. The performance for the testing would be refined by many single-trial EEGs during silent Japanese speeches obtained by EEG Web interfaces. This method might be applied to other "mora" languages.
Matching journals
The top 4 journals account for 50% of the predicted probability mass.
Similar papers in this journal
- Feasibility of decoding covert speech in ECoG with aTransformer trained on overt speech 96%
- Sound perception in realistic surgery scenarios: Towards EEG-based auditory work strain measures for medical personnel. 94%
- Remembering what is relevant: how is goal-directed memory reactivation supported by attentional selection? 93%
Similar papers in this journal
- Direct Speech Reconstruction from Sensorimotor Brain Activity with Optimized Deep Learning Models 95%
- Robust and accurate decoding of hand kinematics from entire spiking activity using deep learning 94%
- Robust assessment of the cortical encoding of word-level expectations using the temporal response function 93%
Similar papers in this journal
- The Nested Hierarchy of Overt, Mouthed, and Imagined Speech Activity Evident in Intracranial Recordings 94%
- Post-hoc modification of linear models: combining machine learning with domain information to make solid inferences from noisy data 93%
- Neural representation of linguistic feature hierarchy reflects second-language proficiency 93%
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.