COVER: Learning to Accept More in Selective Sleep Staging

Yukai Song; Yangfan Deng; Jijun Yin; Zhi-Hong Mao; Jingtong Hu

cs.LGOct 2, 2026

COVER: Learning to Accept More in Selective Sleep Staging

Authors: Yukai Song, Yangfan Deng, Jijun Yin, Zhi-Hong Mao, Jingtong Hu

Organizations: Department of Electrical and Computer Engineering, University of Pittsburgh · Department of Electrical and Computer Engineering, University of Pittsburgh, Pittsburgh, PA, USA · Department of Electrical and Computer Engineering, University of Maryland, College Park · Department of Electrical and Computer Engineering, University of Maryland, College Park, MD, USA

Abstract

Traditional sleep-staging methods apply the same model to every EEG epoch. Such uniform deployment expends computation on epochs that a smaller model could handle reliably, motivating cascades in which a primary classifier accepts its reliable predictions and defers the remainder to a more capable model. In this paper, we study the first stage of such a cascade: maximizing the coverage of fixed primary predictions subject to a prescribed accepted-risk target. We propose COVER (COVerage-oriented Error Ranking), which integrates two key innovations: (i) auxiliary-informed primary-error learning, which replaces maximum softmax probability (MSP) with a learned error score while preserving the primary labels, and (ii) fixed-scale scorer refinement, which builds on this score to directly maximize coverage under an empirical accepted-risk constraint rather than error-prediction accuracy over all epochs. We evaluate COVER on Sleep-EDF-20 at a 5% accepted-risk target, with subjects held out from all fitting and selection. Auxiliary-informed error learning raises mean subject coverage from 31.5% for MSP to 48.8% at similar subject-equal risk. At equal acceptance volume, with MSP accepting the same number of epochs as the learned scorer in each subject (20,639 in total), errors fall from 1,467 to 867. Fixed-scale refinement then adds 1.7 percentage points of coverage over its initialization in nested development, and COVER attains the highest mean coverage among eight evaluated scorers, 50.4% at 4.5% subject-equal risk, above the selective-ranking baseline SELE (49.3%) and the probability-fusion comparator DuoF (43.6%). To the best of our knowledge, this is the first work to combine auxiliary-informed primary-error learning with fixed-scale coverage refinement for selective sleep staging, offering a basis for reliability-aware allocation of computation in cascaded sleep staging.

Figures & tables

Figure 1: Overview of COVER. (a) FIT component out-of-fold (OOF) outputs supply risk vectors

z

and primary-error targets

e

. Step (i) fits a subject-weighted BCE error score; step (ii) refines it toward coverage at the risk target; the refitted scorer is then Platt-calibrated and frozen. (b) CERT selects an empirical acceptance rule with the scorer frozen. (c) At inference, the frozen scorer and rule determine acceptance; the blue bypass releases the primary label unchanged. The waveform is schematic.

Scorer	Coverage	$R_{\rm SE}$	$R_{\rm pool}$	Accepted	Matched errors
Confidence
MSP	31.45	3.94	4.15	13,625	1,467
Step-(i) components
Forward-KL	30.56	5.17	5.17	12,935	2,284
Primary-only	36.24	3.59	3.67	15,603	1,198
DuoF	43.62	4.29	4.36	18,598	1,031

Table 1: Main evaluation of eight scorers. Coverage and risks (%) use each own CERT-locked rule; Accepted is its epoch count. Matched errors use Linear-BCE’s counts within every subject, totaling 20,639 epochs per method.

Figure 2: Locked operating points. (a) Main evaluation; Table 1 also includes BCEAnchor. (b) Nested development within FIT; the arrow marks Linear-BCE to FixedScale-1 refinement. Axes show mean subject coverage and subject-equal accepted risk. Dashed lines mark the 5% target; evaluation labels do not select these rules.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Item	Setting
Cohort and input	Sleep-EDF-20; 20 subjects, 39 recordings, 42,308 epochs. Fpz–Cz at 100 Hz, 30-s epochs; five stages; no other channels or neighboring epochs.
Outer roles	20 folds, each with 15 FIT / 4 CERT / 1 OUTER subjects.
Component fits	15 component OOF fits and one final fit per outer fold; roles in Table 3 .
Primary structure	EEG Conformer; embedding 40; depth 6; $1\times 3000$ samples.
Primary optimization	Adam; learning rate $10^{-3}$ ; batch 64; cosine schedule; at most 100 epochs; patience 20.
Checkpoint	Mean accuracy on two selection subjects; earliest epoch on ties. Class weights use component-training subjects only.

Appendix

Table 2: Experimental settings, combining component and evaluation configuration. Every method shares primary predictions and the risk target.

	Leave-one-subject-out fits (15 per outer fold)	Final fit (one per outer fold)
Purpose	Outputs for one held-out FIT subject	Outputs for CERT and OUTER subjects
Subject pool	The 14 FIT subjects other than the target	14 of 15 FIT subjects; one left unused
Roles	10 training / 2 checkpoint selection / 2 RF calibration	Same 10/2/2 structure, fresh assignment
Scored subjects	The held-out FIT subject	All CERT and OUTER subjects
Label use	Target labels form $e$ after outputs are fixed	None at this stage; CERT labels are read for rule locking, OUTER labels only for evaluation
Unused subject	—	Its risk-training rows come from its own leave-one-subject-out fit

Appendix

Table 3: Component fits within each outer fold. CERT and OUTER subjects never fit, select, or calibrate components.

Indices	Features in stored order
1–11	Standard deviation; interquartile range; skewness; excess kurtosis; sign-change rate; Hjorth mobility and complexity; mean; root mean square; sample entropy; normalized permutation entropy
12–18	Absolute band powers: slow delta (0.4–1 Hz), fast delta (1–4), theta (4–8), alpha (8–12), sigma (12–16), beta (16–30); their sum
19–24	The same six band powers divided by their sum plus $\epsilon$
25–29	Total delta/theta, delta/sigma, delta/beta, alpha/theta, theta/sigma, with $\epsilon$ added to each denominator
30–33	Spectral centroid, spectral entropy, 95% spectral edge, spectral slope
34–35	Mean squared db4 wavelet detail coefficients at levels D4 and D5

Appendix

Table 4: RF input features, distinct from the risk vector in Table 5 . Total delta is slow plus fast delta;

\epsilon=10^{-10}

Feature block	Count	Contents
Primary	14	$p_{T}$ (5); primary class indicators (5); MSP, runner-up, margin, normalized entropy (4)
Auxiliary	14	$p_{F}$ (5); RF class indicators (5); MSP, runner-up, margin, normalized entropy (4)
Disagreement	9	Forward, reverse and symmetric KL; Jensen–Shannon divergence; probability L1/L2 distances; label disagreement; $p_{T}(\hat{y}_{F})$ and $p_{F}(\hat{y}_{T})$

Appendix

Table 5: Risk-vector blocks. Primary-only learning uses the first block; the heterogeneous vector uses all

d=37

features.

Model	Settings
Logistic regression (4)	L2; $C=\kappa\in\{0.01,0.1,1,10\}$ ; lbfgs; 2,000 iterations; tolerance $10^{-8}$ ; training-only StandardScaler.
Histogram boosting (4)	Log loss; 7/15 leaves $\times$ L2 penalties 0/1; learning rate 0.05; 200 iterations; minimum leaf size 100; 64 bins; no early stopping.
Subject weights	Inverse subject epoch counts, normalized to mean one; no risk-model SMOTE or class weights.

Appendix

Table 6: Risk-model candidates. The primary-only search uses all eight; the linear group uses the logistic configurations with its shared calibration.

Component	Aggregation	Accuracy	Macro P	Macro R	Macro-F1
EEG Conformer	Pooled epochs	76.01	67.80	71.07	69.02
EEG Conformer	Mean subject	75.86	69.25	71.56	67.76
RF (isotonic)	Pooled epochs	77.52	68.23	67.77	67.35
RF (isotonic)	Mean subject	77.58	68.74	68.07	65.99

Appendix

Table 7: Component aggregate metrics (%). Pooled epochs use one confusion matrix; mean subject averages 20 subject summaries.

Score	Errors	Risk	Pass	$\Delta R$	Descriptive 95% interval
COVER	817	4.11	12/20	–	reference
MSP	1,369	6.88	9/20	-2.77	[-5.11, -1.13]
DuoF	946	4.76	14/20	-0.65	[-2.24, 0.36]
Primary-only	1,133	5.70	10/20	-1.59	[-3.39, -0.34]
Forward-KL	2,171	10.91	3/20	-6.81	[-9.64, -4.46]

Appendix

Table 8: Prespecified matched comparisons at 19,892 accepted epochs, with identical counts within every subject. COVER denotes Original COVER;

\Delta R

is its pooled risk minus the comparator (pp).

Comparison	Endpoint	Difference [95% interval]
Linear-BCE – MSP	Own-rule $C$	+17.34 [+9.74, +25.54]
Linear-BCE – MSP	Own-rule $R_{\rm SE}$	+0.21 [-1.29, +1.81]
Linear-BCE – MSP	Matched $R_{\rm pool}$	-2.91 [-5.37, -1.09]
Linear-BCE – Primary-only	Own-rule $C$	+12.55 [+6.81, +19.25]
Linear-BCE – Primary-only	Matched $R_{\rm pool}$	-1.60 [-3.07, -0.63]
Linear-BCE – DuoF	Own-rule $C$	+5.17 [+0.98, +9.83]

Appendix

Table 9: Linear-BCE comparisons at the main-text matching budget. Values are saved differences and descriptive 95% intervals, in pp; directions follow the first column.

Figure 3: Ranking diagnostics for the eight main-text scorers from saved scores. Left: full within-subject quota range; right: the 30–60% region. Risks are subject-equal; the dashed line marks 5%. OUTER outcomes diagnose ranking and never select deployed rules. Own-rule points need not lie on these curves because per-subject acceptance allocations differ.

Scorer	Locked $C$	Locked $R_{\rm SE}$	Quota $R_{\rm SE}$	Zero	$\Delta C$ vs. FixedScale-1 [95% CI], pp
FixedScale-1	50.21	4.45	4.69	3	–
BCE	48.56	4.84	5.03	9	-1.65 [-2.52, -0.47]
SELE	49.62	4.96	5.07	12	-0.60 [-1.45, +0.50]
BCEAnchor-1	49.55	4.64	4.88	6	-0.66 [-1.27, +0.15]
BCEAnchor-selected	49.83	4.73	4.92	6	-0.39 [-1.05, +0.54]

Appendix

Table 10: Nested development within FIT. BCE denotes Linear-BCE. Locked columns apply frozen inner rules; quota risk fixes 50% acceptance per subject. Values are percentages;

\Delta C

is row minus FixedScale-1 in pp. Zero counts no-acceptance records among 300; aggregation uses 20 subjects.

Scope / comparison	Rule	Metric	Difference	Lower	Upper
Nested: FixedScale-1 – BCE	Locked	$C$	+1.65	+0.47	+2.52
Nested: FixedScale-1 – BCE	Locked	$R_{\rm SE}$	-0.39	-1.41	+0.23
Nested: FixedScale-1 – BCE	50%-quota	$R_{\rm SE}$	-0.34	-1.30	+0.24
Nested: FixedScale-1 – SELE	Locked	$C$	+0.60	-0.50	+1.45
Nested: FixedScale-1 – SELE	Locked	$R_{\rm SE}$	-0.51	-1.41	+0.05
Nested: FixedScale-1 – SELE	50%-quota	$R_{\rm SE}$	-0.37	-1.45	+0.27

Appendix

Table 11: Core paired differences (pp), with descriptive 95% intervals. BCE denotes Linear-BCE. Locked and fixed-quota outcomes are distinct evaluations; all rows are copied from the saved comparison table.

Comparison	Metric	Delete-one min.	Max.	Without S11
FixedScale-1 – BCE	$C$	+1.50	+2.12	+2.12
FixedScale-1 – BCE	Locked $R_{\rm SE}$	-0.47	+0.08	+0.08
FixedScale-1 – BCE	Quota $R_{\rm SE}$	-0.43	+0.09	+0.09
FixedScale-1 – SELE	$C$	+0.44	+1.01	+1.01
FixedScale-1 – SELE	Locked $R_{\rm SE}$	-0.58	-0.10	-0.10
FixedScale-1 – SELE	Quota $R_{\rm SE}$	-0.47	+0.12	+0.12

Appendix

Table 12: Delete-one sensitivity of nested differences (pp). Min./max. span all 20 deletions; the identified column is a sensitivity analysis, not a subject exclusion from primary results.

True stage	$N$	$B$	$A$	$Q$	$K$	$A/N$	$Q/A$	$Q/B$
Wake	8,285	6,727	4,827	4,786	41	58.26	99.15	71.15
N1	2,804	831	288	19	269	10.27	6.60	2.29
N2	17,799	13,437	10,899	10,594	305	61.23	97.20	78.84
N3	5,703	5,320	3,690	3,579	111	64.70	96.99	67.27
REM	7,717	5,844	1,751	1,495	256	22.69	85.38	25.58

Appendix

Table 13: FixedScale own-rule stage counts and rates.

N,B,A,Q,K

use the definitions above; the last three columns are percentages.

True stage	Epochs	Errors	Epochs	Errors	Epochs	Errors
	FixedScale only		BCE only		Common
Wake	653	12	287	14	4,174	29
N1	85	76	41	34	203	193
N2	1,065	62	678	50	9,834	243
N3	232	25	180	17	3,458	86
REM	286	69	319	14	1,465	187

Appendix

Table 14: FixedScale versus Linear-BCE accepted-set exchange, grouped by true stage. Common means accepted by both.

Explore similar work

Jul 25, 2026cs.LG

StageGuard: Physiologically Constrained Sleep Staging

Automated sleep staging is increasingly used in large-scale studies to derive sleep-architecture endpoints: total sleep time, REM latency, sleep efficiency, and bout-duration statistics. Deep learning models achieve epoch-level accuracy approaching inter-rater agreement, yet often produce hypnograms that violate physiological invariants, such as rare transitions (e.g., direct Wake -> REM) or excessively fragmented sequences. Such violations can bias downstream sleep metrics, regardless of overall accuracy. We propose StageGuard, a plug-and-play, backbone-agnostic structured-inference framework that wraps any neural sleep-staging backbone with physiology-informed priors. StageGuard combines (1) a differentiable soft transition penalty that discourages physiologically rare transitions during training, and (2) a semi-Markov constrained decoder with a duration-augmented state space that jointly enforces transition penalties and minimum bout durations at inference. Unlike hard-prohibition methods, it admits rare transitions when emission evidence is overwhelming, leaving informative pathological events recoverable rather than blocked. StageGuard constrains staging outputs to satisfy known physiological priors rather than modeling sleep generatively. We quantify the validity gap using transition-violation rate (TVR) and fragmentation index (FI) and demonstrate that, across six backbones and four datasets, StageGuard reduces TVR to physiologically plausible levels and lowers FI by 56-62%, while maintaining or slightly improving classification accuracy. Crucially, improved constraint satisfaction translates into 59-79% lower error on derived sleep-architecture statistics not directly optimized by the method, and recovers the direction and effect size of expert-defined subgroup differences (OSA severity, age) more faithfully than the unconstrained baseline.

Juntang Wang, Yihan Wang, Hao Wu +3

Zu Chongzhi Center, Duke Kunshan University Kunshan, China

Constrained Decoding Sleep Stage Classification Sequence Modeling

Ask PDF

Aug 12, 2026cs.LG

Personalized Scorer Modeling: A Learning-Based Framework for Deriving Robust Sleep Stage Labels from Multiple Experts

Sleep stage classification is important for the diagnosis and management of sleep disorders, yet most automatic staging studies evaluate models against a single reference hypnogram despite known inter-scorer variability. This study investigates whether multi-scored datasets can be used to construct more reliable reference labels from the collective behavior of multiple experts. We use the publicly available DOD-H and DOD-O datasets. EEG (C3-M2) and chin EMG signals were segmented into 30-s epochs, and 30 features were extracted from each modality, yielding 60 features for EEG+EMG. We propose a learning-based hypnogram (LBH) that models the stage-specific behavior of each scorer using confusion matrices derived from machine-learning models. After column normalization, these matrices estimate the probability of each true sleep stage given each scorer's label; probabilities are aggregated across scorers to assign the final label for each epoch. LBH was evaluated with random forest, support vector machine, and multilayer perceptron classifiers under EEG-only and EEG+EMG settings, and compared with the dataset hypnogram (DH) and best-scorer hypnogram (BSH). LBH consistently improved overall performance. The best results were obtained with random forest and EEG+EMG, reaching 86.07% accuracy, 85.46% precision, and 85.29% F1-score on DOD-H, and 86.04% accuracy, 85.21% precision, and 84.70% F1-score on DOD-O. These findings suggest that personalized scorer modeling can improve reference hypnogram construction without discarding information from individual experts.

Seyyed Ali Hoseini, Javad Baseri, Hamid Saadatfar +2

Department of Electrical and Computer Engineering, University of Birjand, Birjand, South Khorasan, Iran

Sleep Stage Classification Human-in-the-Loop Annotation Learning with Noisy Labels

Ask PDF

May 19, 2026eess.SP

Staging by the Book: Automatic Sleep Stage Classification Using Scoring Rules

Automated sleep staging is commonly approached as a supervised machine learning problem, with deep learning methods dominating recent research. While machine learning models achieve near-human level agreement with human-scored reference sleep stages, their decisions are typically opaque and not designed to follow clinical scoring rules. We propose a transparent alternative: a deterministic, rule-based sleep staging method that explicitly operationalizes the American Academy of Sleep Medicine's (AASM) scoring logic as executable code, coupled with epoch-level natural-language justifications derived from an explanation trace. We evaluate the approach on 50 polysomnography recordings with a 10-scorer majority-vote consensus as reference. Across all recordings, the method agreed with the majority-vote reference in 60.5% of epochs (

κ=0.42

), with substantially higher agreement on a dataset used during development (77.1%,

κ=0.61

). Agreement with the reference was highest for sleep stage N2 (recall 83.5%) and moderate for sleep stage R (recall 68.7%), while Wake and N1 recall were low. Despite lower agreement with the reference than contemporary deep learning models, the method provides deterministic decisions and natural language explanations aligned with AASM scoring rules, making it a complementary tool for auditing, debugging, and governing deep learning-based sleep staging.

Emil Hardarson, Konstantin Popov, Sigridur Sigurdardottir +3

Department of Computer Science, Reykjavik University, Reykjavik, Iceland · Reykjavik University Sleep Institute, Reykjavik University, Reykjavik, Iceland · Department of Engineering, Reykjavik University, Reykjavik, Iceland +1

Sleep Stage Classification Time Series Classification Interpretable ML

Ask PDF