Automatic sleep staging in patients with suspected sleep disorders: a comparison of existing methods on portable setups

Background: Automatic sleep staging algorithms are increasingly applied in clinical and home-based recordings. However, their performance may degrade when transferred to new montages and clinical populations. This is particularly relevant in reduced-channel portable PSG and in disorders such as REM sleep behaviour disorder (RBD), where altered sleep architecture may challenge pretrained models. Objective: To evaluate and compare multiple open-source sleep staging algorithms on a minimal portable PSG setup in controls and patients with and without RBD, and to assess the impact of fine-tuning on clinic-ascertained data. Methods: Six open-source models were applied to 76 subjects recruited from three clinical sleep medicine sites. Performance was assessed using accuracy, F1 scores, and Cohen's kappa, both overall and per sleep stage. Each model was evaluated out-of-the-box and after fine-tuning on clinical data. Results: Out-of-the-box performance varied substantially across models (Cohen's kappa 0.21-0.54). Fine-tuning consistently improved agreement, with the best-performing model (GSSC) reaching Cohen's kappa = 0.58 indicating moderate to good agreement. Performance was highest in controls and lower in patient groups. N3 was the most reliably classified stage across models, whereas N1 remained consistently challenging. REM classification improved after fine-tuning in several architectures but remained model, and subgroup-dependent, particularly in RBD subjects. Conclusion: Fine-tuning substantially mitigates domain shift, updating model parameters to align with new data distributions, when applying automatic sleep staging algorithms to portable clinical recordings. Model architecture influences robustness, with feature-learning approaches demonstrating greater adaptability than fixed-feature models. Despite moderate agreement after adaptation, performance, especially for REM and N1 remains insufficient for fully automated diagnostic use in clinical populations.