Vision-Based Automated Severity Rating of REM Sleep Behavior Disorder: From Heuristic Features to Foundation Models
Hwang, J.; van Pesch, N.; Raval, B.; Abdelfattah, M.; Gafsi, A.; Bertolaso, A.; Ryu, K. H.; Marwaha, S.; Sum-Ping, O.; Cesari, M.; Stefani, A.; Brink-Kjaer, A.; Mignot, E.; Alahi, A.; During, E.
Show abstract
Automated severity rating of rapid eye movement (REM) sleep behavior disorder (RBD) movements would enable multi-night monitoring to detect potentially injurious behaviors and provide objective endpoints for clinical trials. We compared a heuristic classifier using optical flow-derived features against V-JEPA2, a self-supervised video foundation model, for clip-level severity classification (3329 mild versus 284 moderate-to-severe) of in-laboratory video-polysomnography infrared recordings in 86 isolated RBD patients. V-JEPA2 with checkpoint fine-tuning and maximum optical flow-based frame sampling achieved the best performance across both evaluation conditions -- Macro F1 of 0.76 and 93% accuracy in the clip-level split, and 0.68 and 85% in the patient-level split -- outperforming heuristic and domain-specific pretrained models. Clip duration was the dominant heuristic predictor. Whole-night severity scores preserved patient-level ordering despite systematic overestimation, with V-JEPA2 achieving a mean absolute error of 25% versus 52% for the heuristic classifier. These findings establish a foundation for objective, home-deployable monitoring of RBD severity.
Matching journals
The top 3 journals account for 50% of the predicted probability mass.