SleepVLM: A Rule-Grounded Vision-Language Model for Auditable Sleep Staging

Guifeng Deng; Pan Wang; Mengfan Niu; Jiquan Wang; Shuying Rao; Junyi Xie; Xi'ang Chen; Sha Zhao

cs.CVMar 22, 2026

SleepVLM: A Rule-Grounded Vision-Language Model for Auditable Sleep Staging

Authors: Guifeng Deng, Pan Wang, Mengfan Niu, Jiquan Wang, Shuying Rao, Junyi Xie, Xi'ang Chen, Sha Zhao,

Organizations: Affiliated Mental Health Center & Hangzhou Seventh People’s Hospital, School of Brain Science and Brain Medicine, and Liangzhu Laboratory, Zhejiang University School of Medicine, Hangzhou 310058, China · College of Biomedical Engineering & Instrument Science, Zhejiang University, Hangzhou 310058, China · Department of Psychiatry and Mental Health, Wenzhou Medical University, Wenzhou 325035, Zhejiang Province, China · MOE Frontier Science Center for Brain Science and Brain-machine Integration, State Key Laboratory of Brain-machine Intelligence, Zhejiang University, Hangzhou 311121, China ·

Abstract

Sleep staging is essential for sleep assessment and disorder diagnosis. In recent years, automatic sleep staging systems have achieved accuracy approaching that of human experts, but the black-box nature of their predictions hinders clinical adoption. Existing interpretability methods offer partial insight into model behavior, but their outputs still require expert reinterpretation and do not provide a direct basis for auditing individual predictions. To improve trustworthiness, we propose the task of auditable sleep staging. To solve this task, we present SleepVLM, a vision-language model that casts sleep staging as visual reasoning over rendered polysomnography (PSG) waveform images. For each epoch, SleepVLM outputs a stage together with the applicable American Academy of Sleep Medicine (AASM) rules and an auditable rationale. The model is trained using a two-stage framework: Waveform-Perceptual Pre-training followed by Rule-Grounded Supervised Fine-tuning over a mixture of fine-grained and coarse annotations. Experiments on four datasets show that SleepVLM outperforms state-of-the-art methods on average. An automated AASM-feature audit shows broad coverage of stage-defining evidence in the rationales, and independent experts validate their reasoning quality. To facilitate further research, we construct and release MASS-EX, an expert-annotated dataset for rule-grounded sleep staging with AASM rule annotations and expert-written rationales.

Figures & tables

Dataset	Subjects	Sampling rate (Hz)	Use
Development
MASS-SS2/SS4/SS5	85	256	WPT
MASS-EX	5	256	SFT (fine-grained)
MASS-EX	45	256	SFT (coarse)
MASS-EX	12	256	Validation
Evaluation

Table 1: Data used for model development and evaluation.

Method	Modality	MASS-SS1			ZUAMHCS			ABC			DCSM			Avg
Method	Modality	Acc	MF1	$\kappa$	Acc	MF1	$\kappa$	Acc	MF1	$\kappa$	Acc	MF1	$\kappa$	Acc	MF1	$\kappa$
AttnSleep	Signal	0.801	0.756	0.723	0.752	0.701	0.666	0.705	0.674	0.602	0.662	0.600	0.548	0.730	0.683	0.635
DeepSleepNet	Signal	0.803	0.751	0.725	0.749	0.701	0.658	0.652	0.606	0.534	0.660	0.594	0.544	0.716	0.663	0.615
LPSGM	Signal	0.831	0.794	0.763	0.818	0.775	0.750	0.683	0.663	0.580	0.657	0.616	0.552	0.747	0.712	0.661
ResnetMHA	Signal	0.784	0.761	0.709	0.751	0.712	0.667	0.716	0.715	0.627	0.667	0.633	0.567	0.730	0.705	0.643
RobustSleepNet	Signal	0.815	0.755	0.738	0.799	0.757	0.725	0.597	0.524	0.473	0.682	0.627	0.578	0.723	0.666	0.628

Table 2: Sleep staging performance on four datasets. Best value per column in bold.

Figure 2: Coverage of AASM stage-defining features in SleepVLM rationales. Error bars are 95% confidence intervals from 1,000 subject-level cluster bootstrap resamples.

Configuration	WPT	SFT data	Rule grounding	RFT	MASS-SS1			ZUAMHCS			ABC			DCSM
Configuration	WPT	SFT data	Rule grounding	RFT	Acc	MF1	$\kappa$	Acc	MF1	$\kappa$	Acc	MF1	$\kappa$	Acc	MF1	$\kappa$
Zero-shot baseline	$\times$	–	$\times$	$\times$	0.100	0.070	$-0.003$	0.201	0.104	$-0.013$	0.093	0.120	0.008	0.092	0.119	0.008
SleepVLM	✓	Fine + Coarse	✓	$\times$	0.835	0.793	0.767	0.812	0.766	0.743	0.739	0.706	0.640	0.789	0.733	0.717
w/o WPT	$\times$	Fine + Coarse	✓	$\times$	0.818	0.753	0.736	0.804	0.743	0.727	0.687	0.656	0.575	0.758	0.686	0.677
w/o Rule Grounding	✓	Fine + Coarse	$\times$	$\times$	0.829	0.785	0.758	0.795	0.746	0.720	0.704	0.651	0.581	0.770	0.700	0.686
+RFT	✓	Fine + Coarse	✓	✓	0.810	0.738	0.728	0.801	0.737	0.727	0.681	0.605	0.547	0.795	0.712	0.718

Table 3: Ablation study. ✓and

\times

mark whether a component is used. Best value per column in bold.

Figure 4: Representative SleepVLM outputs.

Base model		Qwen2.5-VL-3B			Qwen3.5-0.8B
Precision		BF16	W4A16	$\Delta$	BF16	W4A16	$\Delta$
Model size (GB)		7.1	3.2	-54.9%	1.7	1.2	-29.4%
Inference speed (epoch/s)		9.4	9.9	+5.3%	14.3	13.8	-3.5%
MASS-SS1	Acc	0.835	0.827	-0.008	0.832	0.832	0.000
	MF1	0.793	0.788	-0.005	0.773	0.774	+0.001
	$\kappa$	0.767	0.757	-0.010	0.761	0.760	-0.001

Table 4: Quantization comparison.

Appendix figures & tables21 assets

Supplementary material from the paper’s appendix.

Appendix

Main paper section	Reference	Here
MASS-EX Dataset	“The complete rule set is given in the Appendix.”	§ A
MASS-EX Dataset	“The full pipeline is given in the Appendix.”	§ B
Method	“The complete prompt and both output templates are given in the Appendix.”	§ C
Experiments	“Further details are given in the Appendix.”	§ J
Experiments	“The rating scale is given in the Appendix.”	§ K
Experiments	“The procedure is given in the Appendix.”	§ E

Appendix

Table S1: Where each reference in the main paper is answered.

Stage Category	Rule ID	Rule Type	Operationalized Visual Criteria	Assigned Stage
W	W.1	Onset	$>$ 50% of the epoch contains posterior dominant rhythm (alpha rhythm, 8–13 Hz), primarily measured in the occipital derivation (O2-M1).	W
	W.2	Onset	$>$ 50% of the epoch contains eye blinks (conjugate vertical movements at 0.5–2 Hz) in EOG channels.	W
	W.3	Onset	$>$ 50% of the epoch contains irregular, conjugate rapid eye movements (REMs) in EOG channels associated with normal or high chin muscle tone.	W
N1	N1.1	Onset	For alpha generators: Posterior dominant rhythm is attenuated and replaced by LAMF activity for $>$ 50% of the epoch.	N1
	N1.2	Onset	For non-alpha generators: Earliest appearance of EEG slowing by $\geq$ 1 Hz from stage W (into 4–7 Hz range), vertex sharp waves, or slow eye movements (SEMs).	N1
N2	N2.1	Onset	In the absence of criteria for stage N3, presence of $\geq$ 1 K complexes unassociated with arousals or $\geq$ 1 sleep spindles (11–16 Hz) occurring in the first half of the epoch or the last half of the preceding epoch.	N2

Appendix

Table S2: AASM sleep staging rules operationalized for visual reasoning.

Training	Phase 1 (WPT)	Phase 2 (SFT)	Phase 3 (RFT)
LoRA rank ( $r$ ) / $\alpha$ / dropout	16 / 32 / 0.05	16 / 32 / 0.05	16 / 32 / 0.05
LoRA applied layers	All nn.Linear excluding lm_head	nn.Linear in language model, excluding lm_head	nn.Linear in language model, excluding lm_head
Vision encoder	Unfrozen	Frozen	Frozen
Optimizer	AdamW ( $\beta_{1}$ =0.9, $\beta_{2}$ =0.95, $\epsilon$ = $10^{-8}$ )	AdamW ( $\beta_{1}$ =0.9, $\beta_{2}$ =0.95, $\epsilon$ = $10^{-8}$ )	AdamW ( $\beta_{1}$ =0.9, $\beta_{2}$ =0.95, $\epsilon$ = $10^{-8}$ )
Learning rate	$10^{-4}$	$10^{-4}$	$10^{-4}$
Learning rate schedule	Linear warmup (3%), linear decay	Linear warmup (3%), linear decay	Linear warmup (3%), linear decay

Appendix

Table S3: Training, quantization and inference hyperparameters.

Dataset	n	Age (mean $\pm$ SD)	Age range	Sex (M / F)	EEG/EOG/EMG electrodes	Fs (Hz)	Scoring standard	Epoch length (s)	Use in this study
MASS-SS1	53	63.0 $\pm$ 5.3	55–76	34 / 19	17–19/2/3	256/256/256	AASM	30	Held-out test set
MASS-SS2	19	23.6 $\pm$ 3.7	18–33	8 / 11	19/4/1	256/256/256	R&K	20	WPT
MASS-SS3	62	42.5 $\pm$ 18.9	20–69	29 / 33	20/2/3	256/256/128	AASM	30	SFT and validation
MASS-SS4	40	25.3 $\pm$ 4.3	18–35	14 / 26	4/4/1	256/256/256	R&K	20	WPT
MASS-SS5	26	25.0 $\pm$ 7.4	20–59	13 / 13	20/2/3	256/256/128	R&K	20	WPT
ZUAMHCS	100	28.4 $\pm$ 10.5	18–70	42 / 58	6/2/2–3	512/512/512	AASM	30	External test set

Appendix

Table S4: Characteristics of datasets used in this study.

Dataset	Total epochs	W	N1	N2	N3	R
MASS-SS1	51,165	12,152 (23.8%)	7,098 (13.9%)	22,152 (43.3%)	3,402 (6.6%)	6,361 (12.4%)
MASS-SS3	59,317	6,442 (10.9%)	4,839 (8.2%)	29,802 (50.2%)	7,653 (12.9%)	10,581 (17.8%)
ZUAMHCS	96,317	13,765 (14.3%)	7,059 (7.3%)	39,809 (41.3%)	19,549 (20.3%)	16,135 (16.8%)
ABC	133,000	30,938 (23.3%)	19,296 (14.5%)	52,334 (39.3%)	11,761 (8.8%)	18,671 (14.0%)
DCSM	304,266	79,636 (26.2%)	21,140 (6.9%)	113,027 (37.1%)	43,637 (14.3%)	46,826 (15.4%)

Appendix

Table S5: Sleep stage distribution across datasets used in this study.

Method	Acc	MF1	$\boldsymbol{\kappa}$	F1-W	F1-N1	F1-N2	F1-N3	F1-R
MASS-SS1 ( $n=53$ )
AttnSleep	0.801	0.756	0.723	0.912	0.514	0.841	0.712	0.801
	[0.779, 0.821]	[0.731, 0.777]	[0.693, 0.750]	[0.890, 0.929]	[0.483, 0.545]	[0.824, 0.857]	[0.657, 0.757]	[0.749, 0.840]
DeepSleepNet	0.803	0.751	0.725	0.907	0.558	0.840	0.606	0.844
	[0.783, 0.821]	[0.724, 0.773]	[0.696, 0.750]	[0.883, 0.926]	[0.528, 0.586]	[0.821, 0.857]	[0.522, 0.663]	[0.796, 0.883]
LPSGM	0.831	0.794	0.763	0.913	0.600	0.867	0.715	0.876

Appendix

Table S6: Baseline comparison on the four evaluation datasets.

Configuration	Acc	MF1	$\boldsymbol{\kappa}$	F1-W	F1-N1	F1-N2	F1-N3	F1-R
MASS-SS1 ( $n=53$ )
Zero-shot baseline	0.100	0.070	$-$ 0.003	0.005	0.077	0.140	0.122	0.005
	[0.087, 0.114]	[0.064, 0.076]	[ $-$ 0.006, $-$ 0.001]	[0.003, 0.007]	[0.069, 0.083]	[0.131, 0.149]	[0.093, 0.152]	[0.003, 0.007]
SleepVLM	0.835	0.793	0.767	0.917	0.527	0.872	0.784	0.863
	[0.824, 0.846]	[0.777, 0.806]	[0.750, 0.782]	[0.904, 0.929]	[0.498, 0.555]	[0.861, 0.884]	[0.722, 0.827]	[0.842, 0.880]
w/o WPT	0.818	0.753	0.736	0.910	0.380	0.858	0.766	0.853

Appendix

Table S7: Ablation study on the four evaluation datasets.

Model	Acc	MF1	$\boldsymbol{\kappa}$	F1-W	F1-N1	F1-N2	F1-N3	F1-R
MASS-SS1 ( $n=53$ )
Qwen2.5-VL-3B (BF16)	0.835	0.793	0.767	0.917	0.527	0.872	0.784	0.863
	[0.824, 0.846]	[0.777, 0.806]	[0.750, 0.782]	[0.904, 0.929]	[0.498, 0.555]	[0.861, 0.884]	[0.722, 0.827]	[0.842, 0.880]
Qwen2.5-VL-3B (W4A16)	0.827	0.788	0.757	0.912	0.518	0.865	0.786	0.860
	[0.815, 0.838]	[0.772, 0.801]	[0.740, 0.774]	[0.899, 0.924]	[0.490, 0.544]	[0.852, 0.877]	[0.724, 0.829]	[0.840, 0.876]
Qwen3.5-0.8B (BF16)	0.832	0.773	0.761	0.918	0.454	0.874	0.745	0.875

Appendix

Table S8: Quantization comparison on the four evaluation datasets.

ID	Stage	Proposition	Mention rate [95% CI]	Correct $-$ incorrect $\Delta$ [pp, 95% CI]
Required propositions
F1	W	Alpha rhythm (i.e., posterior dominant rhythm) over the occipital region for $>$ 50% of the epoch	81.5 [78.8, 83.8]	$+$ 90.8 [ $+$ 89.7, $+$ 91.9]
F2	W	0.5–2 Hz eye blinks for $>$ 50% of the epoch	5.5 [4.8, 6.3]	$+$ 5.8 [ $+$ 5.3, $+$ 6.4]
F3	N1	Attenuation or replacement of alpha rhythm by LAMF activity	89.6 [88.0, 91.0]	$+$ 23.0 [ $+$ 20.3, $+$ 25.8]
F4	N1	LAMF activity, or a 4–7 Hz background EEG	89.4 [87.8, 90.8]	$+$ 23.2 [ $+$ 20.3, $+$ 26.1]
F5	N1	Vertex sharp waves, or slow eye movements (SEMs)	53.0 [50.6, 55.5]	$+$ 92.1 [ $+$ 91.2, $+$ 93.1]

Appendix

Table S9: Per-proposition results of the AASM-feature audit.

Score	Description
Factual Accuracy & Perceptual Fidelity. Whether the model’s descriptions of waveform existence, frequency, amplitude, and temporal proportion faithfully reflect the rendered PSG image.
5 (Excellent)	All waveform descriptions match the image with zero factual errors in existence, frequency, amplitude, or epoch proportion.
4 (Good)	Core stage-defining features accurately described; only minor deviations in non-critical background signals.
3 (Acceptable)	Waveform existence correctly identified; localized attribute or temporal estimation errors that do not alter the overall staging determination.
2 (Marginal)	Significant attribute errors, or key waveform amplitude/frequency estimates severely violate the rendering scales.
1 (Poor)	Fabricated waveforms described (absent in image), or the majority of feature descriptions contradict image evidence.

Appendix

Table S10: Rating scale for expert evaluation of model reasoning.

Dimension	Rater	Mean	0	1	2	3	4	5
Factual accuracy	1	4.14	3	434	152	240	1,938	2,633
Factual accuracy	2	3.47	25	395	561	1,168	2,552	699
Evidence comprehensiveness	1	4.06	4	434	169	159	2,516	2,118
Evidence comprehensiveness	2	3.65	25	360	547	918	2,262	1,288
Logical coherence	1	4.08	4	568	159	120	1,976	2,573
Logical coherence	2	3.70	25	360	565	918	1,916	1,616

Appendix

Table S11: Per-rater score distributions over the 5,400 rated samples.

Figure S9: Ground-truth stage N2 (Rule N2.2); predicted stage R (Rule R.2).

Figure S10: Ground-truth stage W (Rules W.1, W.2); predicted stage N1 (Rule N1.2).

Sleep Stage Classification

Sleep stage classification assigns sleep epochs to physiological stages from signals such as EEG, helping quantify sleep architecture and support clinical monitoring.

See all 40 papers

Staging by the Book: Automatic Sleep Stage Classification Using Scoring Rules
May 19, 2026Emil Hardarson, Konstantin Popov, Sigridur Sigurdardottir +3Sleep Stage Classification Time Series Classification
Department of Computer Science, Reykjavik University, Reykjavik, Iceland · Reykjavik University Sleep Institute, Reykjavik University, Reykjavik, Iceland · Department of Engineering, Reykjavik University, Reykjavik, Iceland +1
StageGuard: Physiologically Constrained Sleep Staging
Jul 25, 2026Juntang Wang, Yihan Wang, Hao Wu +3Constrained Decoding Sleep Stage Classification
Zu Chongzhi Center, Duke Kunshan University Kunshan, China
SleepLM: Natural-Language Intelligence for Human Sleep
Feb 27, 2026Zongzhe Xu, Zitao Shuai, Eideen Mozaffari +3Multimodal Pretraining Cross-Modal Retrieval
University of California, Los Angeles

May 19, 2026J/K move · Enter open · S save

Staging by the Book: Automatic Sleep Stage Classification Using Scoring Rules

Emil Hardarson, Konstantin Popov, Sigridur Sigurdardottir, Anna Sigridur Islind+2

Department of Computer Science, Reykjavik University, Reykjavik, Iceland · Reykjavik University Sleep Institute, Reykjavik University, Reykjavik, Iceland · Department of Engineering, Reykjavik University, Reykjavik, Iceland+1

Sleep Stage Classification Time Series Classification Interpretable ML

Open paper AskPDF arXiv

Automated sleep staging is commonly approached as a supervised machine learning problem, with deep learning methods dominating recent research. While machine learning models achieve near-human level agreement with human-scored reference sleep stages, their decisions are typically opaque and not designed to follow clinical scoring rules. We propose a transparent alternative: a deterministic, rule-based sleep staging method that explicitly operationalizes the American Academy of Sleep Medicine's (AASM) scoring logic as executable code, coupled with epoch-level natural-language justifications derived from an explanation trace. We evaluate the approach on 50 polysomnography recordings with a 10-scorer majority-vote consensus as reference. Across all recordings, the method agreed with the majority-vote reference in 60.5% of epochs (

κ=0.42

), with substantially higher agreement on a dataset used during development (77.1%,

κ=0.61

). Agreement with the reference was highest for sleep stage N2 (recall 83.5%) and moderate for sleep stage R (recall 68.7%), while Wake and N1 recall were low. Despite lower agreement with the reference than contemporary deep learning models, the method provides deterministic decisions and natural language explanations aligned with AASM scoring rules, making it a complementary tool for auditing, debugging, and governing deep learning-based sleep staging.

Abstract

Sleep Stage Classification

Explore similar work

Staging by the Book: Automatic Sleep Stage Classification Using Scoring Rules

StageGuard: Physiologically Constrained Sleep Staging

SleepLM: Natural-Language Intelligence for Human Sleep