Optimizing MR-based gaze-decoding for eyes-closed eye-tracking in fMRI
Kling, S. M.; Lascombes, U.; Nau, M.; Masson, G. S.; Szinte, M.
Show abstract
Eye movements provide valuable insights into human cognition and are a critical variable in numerous functional magnetic resonance imaging (fMRI) studies. Yet, when the eyes are closed, camera-based eye-tracking is unavailable, making studies of eyes-closed states challenging. Here, we address this gap through MR-based gaze decoding with DeepMReye, a deep learning framework for camera-free reconstruction of gaze behavior from the MR-signal of the eyes. We first show that fine-tuning DeepMReye using visuomotor calibration data acquired when the eyes were open significantly improves gaze decoding, and that this fine-tuning does not require simultaneous camera-based data. We next assessed whether model performance could be further improved by incorporating data acquired while participants gazed at known positions with both eyes open and closed. Notably, while DeepMReye was originally trained exclusively on eyes-open data, the network successfully generalized eyes-closed periods, with performance improving significantly through fine-tuning on the eyes-closed data. These findings demonstrate that reliable gaze monitoring during eyes-closed periods is feasible, enabling a more effective integration of eye-tracking in fMRI research and, consequently, advancing our understanding of human cognition.
Matching journals
The top 7 journals account for 50% of the predicted probability mass.
Similar papers in this journal
Similar papers in this journal
- Previous motor actions outweigh sensory information in sensorimotor learning 94%
- Differential involvement of EEG oscillatory components in sameness vs. spatial-relation visual reasoning tasks 94%
- Neural representations of covert attention across saccades: comparing pattern similarity to shifting and holding attention during fixation 94%
Similar papers in this journal
- Encoding neural representations of time-continuous stimulus-response transformations in the human brain with advanced deep neural networks 94%
- The representational dynamics of the animal appearance bias in human visual cortex are indicative of fast feedforward processing 93%
- When spatial attention cannot be divided: Quadrantic enhancement of early visual processing across task-relevant and irrelevant locations 93%
Similar papers in this journal
Similar papers in this journal
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.