Comparative Analysis of State-of-the-Art Foundation Models for Sleep Analysis Under Channel Reduction
Authors: Hassan Mehdi, Riku Klen, Ayse Kosal Bulbul, Suzanne Timmons, Abdulhamit Subasi, Wei Chen, Zou Zhu, Muhammad Irfan
Organizations: Institute of Biomedicine, University of Turku, Finland · Turku PET Centre, University of Turku and Turku University Hospital, Finland · University College Cork, Ireland · AI4HEALTH Lab. University at Albany, SUNY, Albany, NY, USA · Fudan University, Shanghai, China · School of Biomedical Engineering, University of Sydney, Australia
Automatic sleep staging from polysomnography (PSG) is a well-studied task, but PSG itself is expensive, clinic-based, and burdensome to manually score, which limits its use for long-term or at-home monitoring. Most existing sleep-staging foundation models are evaluated using the full PSG montage. We instead ask how much of that montage is actually necessary. We evaluate six sleep staging models on the Multi-Ethnic Study of Atherosclerosis (MESA) PSG dataset across three signal conditions: electroencephalography (EEG), electrocardiography (ECG), and their combination (EEG+ECG). This is motivated by edge-cloud deployment, where EEG requires a clinic-grade scalp electrode, whereas ECG is already captured by consumer wearables. We test state-of-the-art foundation models such as SleepFM with an encoder trained from scratch on MESA, alongside BIOT, MOMENT, LaBraM, a base-scale Vision Transformer (ViT-B) reimplementation of SensorLM trained from scratch, and YASA, spanning EEG-pretrained, general-time-series, from-scratch, and classical non-learned approaches. No model architecture is modified from its original form; SensorLM's encoder is reimplemented only in PyTorch. For EEG-only staging, BIOT achieves the best result with a macroF1 of 0.7237, followed by LaBraM (0.6835) and SleepFM from scratch (0.6582). Across the five models capable of ECG-only staging, switching from EEG to ECG costs between 0.2798 (MOMENT) and 0.4151 (BIOT) macroF1, averaging 0.3531, while cutting the raw channel data rate to a third. Adding ECG to EEG provides no gain for most models. These results show that EEG carries most of the sleep-staging signal, quantify the consistent accuracy cost of the wearable-compatible alternative, and demonstrate that sleep-relevant pretraining transfers well to MESA.
Figures & tables
Fig. 1 : Clinical PSG montage compared to a single wearable device.
Fig. 2 : Overview of the evaluation pipeline.
Configuration
Channels
Rate (B/s)
% of EEG_ONLY
EEG+ECG
4
1024
133.3%
EEG_ONLY
3
768
100.0%
ECG_ONLY
1
256
33.3%
TABLE I : Raw data rate by signal condition, at the 128 Hz resample rate and 16-bit ADC resolution described in Section III-A . % is relative to EEG_ONLY.
Fig. 3 : Macro F1 by model across the three signal conditions. YASA is omitted from ECG-only, which it cannot run.
Model
F1
Acc
W
N1
N2
N3
REM
BIOT
0.7237
0.8007
0.9325
0.5327
0.7639
0.6594
0.7301
LaBraM
0.6835
0.7737
0.9203
0.4257
0.7511
0.6557
0.6647
SleepFM-FS
0.6582
0.7757
0.8941
0.2952
0.7653
0.6464
0.7201
SensorLM
0.6264
0.7235
0.9132
0.3549
0.6657
0.6344
0.5639
MOMENT
0.5894
0.6745
0.8163
0.3166
0.6900
0.6134
0.5105
YASA
0.4720
0.6687
0.7843
0.1519
0.6913
0.1795
0.5531
TABLE II : EEG-only staging, all models, held-out 50 test. Bold marks the best score in each column.
Model
F1
Acc
W
N1
N2
N3
REM
SleepFM-FS
0.3353
0.5268
0.7566
0.0000
0.4148
0.2188
0.3165
MOMENT
0.3096
0.4191
0.6239
0.1850
0.4467
0.1498
0.1427
BIOT
0.3086
0.4302
0.6815
0.1239
0.3933
0.1391
0.2051
SensorLM
0.2821
0.3695
0.6257
0.1330
0.3307
0.1296
0.1914
LaBraM
0.2803
0.4092
0.6211
0.1049
0.3920
0.1045
0.1792
YASA
N/A (requires EEG)
TABLE III : ECG-only staging, all models. Bold marks the best score in each column. YASA = N/A.
Model
F1
Acc
W
N1
N2
N3
REM
BIOT
0.7023
0.7975
0.9345
0.5001
0.7724
0.5733
0.7311
SleepFM-FS
0.6529
0.7765
0.9069
0.2411
0.7542
0.6309
0.7415
LaBraM
0.6524
0.7566
0.9203
0.3594
0.7431
0.6457
0.5935
SensorLM
0.6077
0.7062
0.8935
0.3337
0.6891
0.6155
0.5068
MOMENT
0.5953
0.6918
0.8395
0.3035
0.7053
0.6089
0.5194
YASA
0.4720
0.6687
0.7843
0.1519
0.6913
0.1795
0.5531
TABLE IV : EEG+ECG staging, all models. Bold marks the best score in each column. YASA uses EEG1 only.
Fig. 4 : Per-stage F1 by model, one radar per signal condition. Spokes are Wake/N1/N2/N3/REM F1; each line is one model. YASA cannot run ECG-only and is omitted from the centre panel.
Fig. 5 : Per-subject macro F1 distribution for BIOT, SleepFM from-scratch, LaBraM, SensorLM, MOMENT, and YASA, one panel per signal condition. YASA cannot run ECG-only and is omitted from that panel. Points are individual subjects.
Fig. 6 : The deployment cost of choosing ECG over EEG. (a) EEG-only vs. ECG-only macro F1 for the five models capable of ECG-only staging: direction is consistent, magnitude varies. (b) Macro F1 versus raw channel data rate (log scale), all models, all three signal conditions; marker shape denotes condition, marker size scales with channel count.
Department of Computer Science, The University of Warwick · School of Engineering, The University of Warwick · Department of Neurology, Huashan Hospital, Fudan University +2
Department of Medical Engineering and Technomathematics, FH Aachen University of Applied Sciences, 52428 Jülich, Germany · Department of Information and Computing Sciences, Utrecht University, Utrecht, The Netherlands · Institute for Data-Driven Technologies, FH Aachen University of Applied Sciences, 52428 Jülich, Germany
School of Electronic Information Engineering, Changchun University of Science and Technology, Changchun 130022, Jilin Province, China · Jilin University, Changchun 130012, Jilin Province, China