Self-Supervised Transformer Model Training for a Sleep-EEG Foundation Model
Ogg, M.; Coon, W. G.
Show abstract
The American Academy of Sleep Medicine (AASM) recognizes five sleep/wake states (Wake, N1, N2, N3, REM), yet this classification schema provides only a high-level summary of sleep and likely overlooks important neurological or health information. New, data-driven approaches are needed to more deeply probe the information content of sleep signals. Here we present a self-supervised approach that learns the structure embedded in large quantities of neurophysiological sleep data. This masked transformer training procedure is inspired by high performing self-supervised methods developed for speech transcription. We show that self-supervised pre-training matches or outperforms supervised sleep stage classification, especially when labeled data or compute-power is limited. Perhaps more importantly, we also show that our pre-trained model is flexible and can be fine-tuned to perform well on new EEG recording montages not seen in training, and for new tasks including distinguishing individuals or quantifying "brain age" (a potential health biomarker). This suggests that modern methods can automatically learn information that is potentially overlooked by the 5-class sleep staging schema, laying the groundwork for new sleep scoring schemas and further data-driven exploration of sleep.
Matching journals
The top 9 journals account for 50% of the predicted probability mass.
Similar papers in this journal
- An infection prediction model developed from inpatient data can predict out-of-hospital COVID-19 infections from wearable data when controlled for dataset shift 94%
- Novel Digital Markers of Sleep Dynamics: A Causal Inference Approach Revealing Age and Gender Phenotypes in Obstructive Sleep Apnea 94%
- Multi-channel recordings reveal age-related differences in the sleep of juvenile and adult zebra finches 92%
Similar papers in this journal
Similar papers in this journal
Similar papers in this journal
- A foundational transformer leveraging full night, multichannel sleep study data accurately classifies sleep stages 97%
- Corticothalamic modelling of sleep neurophysiology with applications to mobile EEG 93%
- The Aging Slow Wave: A Shifting Amalgam of Distinct Slow Wave and Spindle Coupling Subtypes Define Slow Wave Sleep Across the Human Lifespan 92%
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.