Conformal Margin Risk Minimization: An Envelope Framework for Robust Learning under Label Noise

Yuanjie Shi; Peihong Li; Zijian Zhang; Janardhan Rao Doppa; Yan Yan

cs.LGApr 7, 2026

Conformal Margin Risk Minimization: An Envelope Framework for Robust Learning under Label Noise

Authors: Yuanjie Shi, Peihong Li, Zijian Zhang, Janardhan Rao Doppa, Yan Yan

Organizations: School of EECS, Washington State University, Pullman, WA, USA

Abstract

Most methods for learning with noisy labels require privileged knowledge such as noise transition matrices, clean subsets or pretrained feature extractors, resources typically unavailable when robustness is most needed. We propose Conformal Margin Risk Minimization (CMRM), a plug-and-play envelope framework that improves any classification loss under label noise by adding a single quantile-calibrated regularization term, with no privileged knowledge or training pipeline modification. CMRM measures the confidence margin between the observed label and competing labels, and thresholds it with a conformal quantile estimated per batch to focus training on high-margin samples while suppressing likely mislabeled ones. We derive a learning bound for CMRM under arbitrary label noise requiring only mild regularity of the margin distribution. Across five base methods and six benchmarks with synthetic and real-world noise, CMRM consistently improves accuracy (up to +3.39%), reduces conformal prediction set size (up to -20.44%) and does not hurt under 0% noise, showing that CMRM captures a method-agnostic uncertainty signal that existing mechanisms did not exploit.

Figures & tables

Methods	Assumptions
LNL ( Natarajan et al., 2013 )	Symmetric noise
Forward ( Patrini et al., 2017 )	Known label noise transition matrix
GLC ( Hendrycks et al., 2018 )	Small clean data subsets
Co-teaching ( Han et al., 2018 )	Two peer nets
CORES ( Cheng et al., 2020 )	Instance-dependent noise
VolMinNet ( Li et al., 2021b )	Class-conditional noise

Table 1: Classical and recent methods for learning from noisy labels and their assumptions . CMRM only assumes mild regularity conditions, a smooth CDF with positive density at the target quantile. Details in Section 3 . CMRM’s assumptions are automatically satisfied in standard training (see Figure 2 (d)).

Panel of Figure 1 — Figure 1: Confidence margin distributions for clean (blue) and noisy (orange) samples on CIFAR-100. (a) Class-conditional noise at $20\%$ and (b) human annotation noise at $40\%$ . In both cases, clean samples concentrate on positive margins while noisy samples shift to negative, showing that confidence margins can distinguish clean from noisy labels without assumptions on the noise process.

Dataset	Metric	CE		Focal		LDAM		GCE
Dataset	Metric	Base	+CMRM	Base	+CMRM	Base	+CMRM	Base	+CMRM
CIFAR-100	ACC (%) $\uparrow$	65.16	66.32 ( $+1.16$ )	64.42	65.39 ( $+0.97$ )	59.63	61.12 ( $+1.49$ )	62.17	63.65 ( $+1.48$ )
CIFAR-100	M.APSS $\downarrow$	6.67	6.52 ( $-2.25\%$ )	6.89	6.61 ( $-4.06\%$ )	17.85	17.67 ( $-0.78\%$ )	7.70	6.28 ( $-18.44\%$ )
mini-ImageNet	ACC (%) $\uparrow$	57.42	59.41 ( $+1.99$ )	55.54	57.93 ( $+2.39$ )	56.60	56.62 ( $+0.02$ )	55.16	55.51 ( $+0.35$ )
mini-ImageNet	M.APSS $\downarrow$	7.40	7.04 ( $-4.86\%$ )	7.67	7.28 ( $-5.08\%$ )	13.07	12.88 ( $-1.45\%$ )	9.90	9.59 ( $-3.13\%$ )
Food-101	ACC (%) $\uparrow$	56.21	58.48 ( $+2.27$ )	56.35	58.92 ( $+2.57$ )	55.49	56.42 ( $+0.93$ )	55.66	59.05 ( $+3.39$ )

Table 2: Top-1 accuracy (%) and marginal average prediction set size (M.APSS

\downarrow

) on multi-class datasets corrupted by synthetic noise with noise rate

20\%

. Each Base objective is paired with its +CMRM counterpart; the better value within each pair is in bold . Numbers in parentheses indicate the relative change (%):

+

denotes accuracy improvement, and

-

denotes M.APSS reduction compared to the corresponding Base objective. On average across all datasets and objectives, CMRM improves accuracy by

1.58

and reduces M.APSS by

7.28\%

Panel of Figure 2 — Figure 2: Justification experiments for multi-class classification on CIFAR-100 with $20\%$ synthetic label noise. Subfigure (a) shows the training dynamics of total loss (Total), classification loss (cl), and CMRM loss (cr) over epochs. CMRM exhibits stable and monotonic convergence alongside standard loss components. Subfigure (b) reports the ratio of noisy samples among those filtered (with soft weight $<0.5$ ) out by CMRM ( $\alpha=0.15$ ) at each epoch, demonstrating that CMRM consistently suppresses noisy examples by excluding low-margin samples during training. Subfigure (c) examines the sensitivity of $\alpha$ , showing that CMRM maintains higher accuracy than CE across a range of $\alpha$ values, indicating robustness to hyperparameter $\alpha$ . Subfigure (d) depicts the kernel density estimate (KDE) of the margin distribution, with the vertical dashed line indicating the estimated $\tau_{\alpha}(f)$ with $\alpha=0.15$ . The density curve is smooth and strictly positive around $\tau_{\alpha}(f)$ , supporting the differentiability and positive-density assumption in Proposition 1 .

Method	CIFAR-10N (worst)		CIFAR-100N
Method	ACC(%)	M.APSS	ACC(%)	M.APSS
NI-ERM	95.71	0.93	83.17	1.49
NI-ERM+CMRM	97.19	0.91	83.95	1.29
	( $+1.48$ )	( $-2.15\%$ )	( $+0.78$ )	( $-13.42\%$ )

Table 3: Top-1 accuracy (%) and marginal average prediction set size (M.APSS

\downarrow

) on CIFAR-10N and CIFAR-100N corrupted by human annotation noise. Numbers in parentheses indicate the relative change:

+

denotes accuracy improvement and

-

denotes M.APSS reduction. CMRM consistently improves accuracy and reduces uncertainty across CIFAR-N variants, with the largest gains observed on CIFAR-10N Worst and CIFAR-100N. Full results on all CIFAR-10N variants (Aggre, Rand1–3, Worst) and CIFAR-100N are provided in Appendix B .

Method	Evaluation Metric
	AUROC ( $\uparrow$ )	AUPRC ( $\uparrow$ )	FNR ( $\downarrow$ )	FPR ( $\downarrow$ )	ACC ( $\uparrow$ )	M.APSS ( $\downarrow$ )	PC APSS ( $\downarrow$ )	NC APSS ( $\downarrow$ )
LR	0.784	0.885	0.073	0.571	0.802	1.223	1.154	1.432
LR + CMRM	0.852 ( $+0.068$ )	0.925 ( $+0.04$ )	0.082 ( $+0.009$ )	0.422 ( $-0.149$ )	0.833 ( $0.031$ )	1.209 ( $-1.15\%$ )	1.109 ( $-3.9\%$ )	1.308 ( $-8.66\%$ )
Focal	0.809	0.890	0.136	0.388	0.801	1.257	1.224	1.356
Focal + CMRM	0.872 ( $+0.063$ )	0.942 ( $+0.052$ )	0.128 ( $-0.008$ )	0.324 ( $-0.064$ )	0.823 ( $0.022$ )	1.221 $-2.87\%$	1.148 $-6.21\%$	1.295 $-4.5\%$
SVM	0.808	0.925	0.029	0.807	0.776	1.276	1.370	1.512

Table 4: Results for binary classification on Adult with

20\%

label noise. We report ranking (AUROC, AUPRC), error rates (FNR, FPR), accuracy (ACC), and uncertainty (M.APSS, PC APSS, NC APSS).

\uparrow

indicates higher is better;

\downarrow

lower is better. Best results for each base method (LR, Focal, SVM, GCE) are in bold . Numbers in parentheses indicate absolute and relative changes (%) of CMRM, where

+

denotes increase and

-

denotes decrease. On average, CMRM improves AUROC by

0.049

, AUPRC by

0.032

, and ACC by

0.023

, while reducing FPR by

0.143

and slightly increasing FNR by

0.018

. For uncertainty-aware metrics, CMRM reduces M.APSS, PC APSS, and NC APSS by

2.77\%

2.75\%

, and

7.09\%

on average, respectively. Results on Email and Credit show similar trends and are provided in Appendix C.2 .

Panel of Figure 3 — Figure 3: Training dynamics of LR+CMRM for binary classification on the Email dataset. Subfigure (a) Training dynamics of total loss (all loss), classification loss (cl loss), and CMRM loss (cr loss) over epochs. CMRM exhibits stable and monotonic convergence alongside standard loss components. Subfigure (b) $\tau^{-}$ (negative class threshold) and $\tau^{+}$ (positive class threshold) of LR+CMRM during training. The separation between the thresholds increases, indicating that CMRM actively maximizes the margin between positive and negative classes.

Panel of Figure 4 — Figure 4: Histograms of positive confidence distributions for clean (blue) and noisy (orange) samples on the Credit dataset with $20\%$ label noise. The top and bottom rows correspond to samples with observed labels $\widetilde{Y}=0$ (negative) and $\widetilde{Y}=1$ (positive), respectively. Distributions are obtained using LR (left) and LR+CMRM (right). CMRM induces a clearer separation between clean and noisy confidence distributions for both classes.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Data	Loss	Batch size	Epochs	$\eta$	lr schedule	Momentum	$\gamma$	weight decay	$\lambda$	$\alpha$
CIFAR-100	CE+CMRM	128	50	0.05	[10]	0.9	0.01	0.0002	0.1	0.15
	Focal+CMRM	128	50	0.05	[10]	0.9	0.01	0.0002	0.15	0.1
	LDAM+CMRM	128	100	0.05	[60, 80]	0.9	0.01	0.0002	0.1	0.15
	GCE+CMRM	128	50	0.05	[10]	0.9	0.01	0.0002	0.005	0.05
mini-ImageNet	CE+CMRM	512	50	0.05	[10]	0.9	0.01	0.0002	0.15	0.2
	Focal+CMRM	512	50	0.05	[10]	0.9	0.01	0.0002	0.15	0.15

Appendix

Table 5: The details we used to train our models for multi-class classification corrupted by synthetic noise with noise rate

20\%

. We reported the hyperparameters that give the best accuracy. We employed SGD optimizer for all training unless specified.

Noise Rates	Metric	CE		Focal		LDAM		GCE
Noise Rates	Metric	Base	+CMRM	Base	+CMRM	Base	+CMRM	Base	+CMRM
$0\%$	ACC (%) $\uparrow$	68.82	69.38 ( $+0.56$ )	68.32	69.27 ( $+0.95$ )	62.94	64.15 ( $+1.19$ )	62.80	62.80 ( $+0$ )
$0\%$	M.APSS $\downarrow$	3.22	3.37 ( $+4.52\%$ )	3.04	3.26 ( $+7.24\%$ )	10.89	11.06 ( $+1.56\%$ )	5.95	5.95 ( $+0\%$ )
$5\%$	ACC (%) $\uparrow$	67.48	68.02 ( $+0.54$ )	66.76	67.66 ( $+0.90$ )	61.63	63.00 ( $+1.37$ )	63.88	65.29 ( $+1.41$ )
$5\%$	M.APSS $\downarrow$	3.79	4.13 ( $+8.97\%$ )	3.66	3.82 ( $+4.37\%$ )	13.03	13.28 ( $+1.92\%$ )	6.27	6.30 ( $+0.48\%$ )
$10\%$	ACC (%) $\uparrow$	66.29	67.32 ( $+1.03$ )	66.52	66.82 ( $+0.30$ )	61.06	62.28 ( $+1.22$ )	62.80	64.73 ( $+1.93$ )

Appendix

Table 6: Top-1 accuracy (%) and marginal average prediction set size (M.APSS

\downarrow

) on CIFAR-100 datasets corrupted by synthetic noise with noise rates

\{0\%,5\%,10\%,20\%,30\%,40\%\}

. Each Base objective is paired with its +CMRM counterpart; the better value within each pair is in bold . Numbers in parentheses indicate the relative change (%):

+

denotes accuracy improvement, and

-

denotes M.APSS reduction compared to the corresponding Base objective. On average across all datasets and objectives, CMRM improves accuracy by

1.34

and reduces M.APSS by

3.89\%

Metric	CE		Focal		LDAM		GCE
Metric	Base	+CMRM	Base	+CMRM	Base	+CMRM	Base	+CMRM
ACC (%) $\uparrow$	64.74 $\pm$ 0.50	65.95 $\pm$ 0.30 (+1.21)	64.33 $\pm$ 0.15	65.26 $\pm$ 0.21 (+0.93)	59.64 $\pm$ 0.34	60.76 $\pm$ 0.26 (+1.12)	62.46 $\pm$ 0.70	63.60 $\pm$ 0.24 (+1.14)

Appendix

Table 7: Mean

\pm

standard deviation (std) of Top-1 accuracy (%) across multiple random seeds on CIFAR-100 with synthetic label noise (noise rate

20\%

). Each base objective is paired with its +CMRM variant. The better value within each pair is shown in bold . Numbers in parentheses denote the absolute accuracy improvement over the corresponding base objective.

Method	CIFAR-10N (Aggre)		CIFAR-10N (Rand1)		CIFAR-10N (Rand2)		CIFAR-10N (Rand3)		CIFAR-10N (Worst)		CIFAR-100N
Method	ACC(%)	M.APSS	ACC(%)	M.APSS	ACC(%)	M.APSS	ACC(%)	M.APSS	ACC(%)	M.APSS	ACC(%)	M.APSS
NI-ERM	98.69	0.904	98.80	0.902	98.65	0.903	98.67	0.904	95.71	0.93	83.17	1.49
NI-ERM+CMRM	98.81	0.903	99.03	0.901	98.95	0.903	98.88	0.899	97.19	0.91	83.95	1.29
	( $+0.12$ )	( $-0.11\%$ )	( $+0.23$ )	( $-0.11\%$ )	( $+0.30$ )	( $0\%$ )	( $+0.21$ )	( $-0.55\%$ )	( $+1.48$ )	( $-2.15\%$ )	( $+0.78$ )	( $-13.42\%$ )

Appendix

Table 8: Top-1 accuracy (%) and marginal average prediction set size (M.APSS

\downarrow

) on CIFAR-10N and CIFAR-100N corrupted by human annotation noise. Numbers in parentheses indicate the relative change:

+

denotes accuracy improvement and

-\%

denotes M.APSS reduction. CMRM consistently improves accuracy and reduces uncertainty across CIFAR-N variants, with the largest gains observed on CIFAR-10N and CIFAR-100N. On average across all datasets, CMRM improves accuracy by

0.52

and reduces M.APSS by

2.72\%

Panel of Figure 5 — Figure 5: training dynamics of total loss (Total), classification loss (cl), and CMRM loss (cr) for all base losses over epochs on CIFAR-100 with $20\%$ synthetical label noise. Subfigure (a) CE+CERM; Subfigure (b) Focal+CERM; Subfigure (c) LDAM+CERM; Subfigure (d) GCE+CERM; CMRM exhibits stable and monotonic convergence alongside standard loss components.

Panel of Figure 6 — Figure 6: The ratio of noisy samples among those filtered out by CMRM at each epoch on CIFAR-100 with $20\%$ synthetical label noise. Subfigure (a) CE+CMRM ( $\alpha=0.15$ ); Subfigure (b) Focal+CMRM ( $\alpha=0.1$ ); Subfigure (c) LDAM+CMRM ( $\alpha=0.15$ ); Subfigure (d) GCE+CMRM ( $\alpha=0.05$ ); CMRM consistently suppresses noisy examples by excluding low-margin samples during training.

Panel of Figure 7 — Figure 7: The sensitivity of $\alpha$ of different base losses on CIFAR-100 with $20\%$ synthetical label noise. Subfigure (a) CE+CERM; Subfigure (b) Focal+CERM; Subfigure (c) LDAM+CERM; Subfigure (d) GCE+CERM; CMRM maintains higher accuracy than CE across a range of $\alpha$ values, indicating robustness to hyperparameter $\alpha$ .

Panel of Figure 8 — Figure 8: The sensitivity of $\lambda$ of different base losses on CIFAR-100 with $20\%$ synthetical label noise. Subfigure (a) CE+CERM; Subfigure (b) Focal+CERM; Subfigure (c) LDAM+CERM; Subfigure (d) GCE+CERM; CMRM maintains higher accuracy than CE across a range of $\lambda$ values, indicating robustness to hyperparameter $\lambda$ .

Panel of Figure 9 — Figure 9: the kernel density estimate (KDE) of the margin distribution of different base losses on CIFAR-100 with $20\%$ synthetical label noise. Subfigure (a) CE+CMRM ( $\alpha=0.15$ ); Subfigure (b) Focal+CMRM ( $\alpha=0.1$ ); Subfigure (c) LDAM+CMRM ( $\alpha=0.15$ ); Subfigure (d) GCE+CMRM ( $\alpha=0.05$ ); The vertical dashed line indicating the estimated $\tau_{\alpha}(f)$ . The density curve is smooth and strictly positive around $\tau_{\alpha}(f)$ , supporting the differentiability and positive-density assumption in Proposition 1 .

Dataset	Method	Measure
		AUROC ( $\uparrow$ )	AUPRC ( $\uparrow$ )	FNR ( $\downarrow$ )	FPR ( $\downarrow$ )	ACC ( $\uparrow$ )	M.APSS ( $\downarrow$ )	PC APSS ( $\downarrow$ )	NC APSS ( $\downarrow$ )
Adult	LR	0.784	0.885	0.073	0.571	0.802	1.223	1.154	1.432
	LR + CMRM	0.852	0.925	0.082	0.422	0.833	1.209	1.109	1.308
	Focal	0.809	0.890	0.136	0.388	0.801	1.257	1.224	1.356
	Focal + CMRM	0.872	0.942	0.128	0.324	0.823	1.221	1.148	1.295
	SVM	0.808	0.925	0.029	0.807	0.776	1.276	1.370	1.512

Appendix

Table 9: Performance comparison on binary classification. We evaluate three datasets (Adult, Email, and Credit) under a

20\%

label noise setting. Models are assessed across accuracy (ACC), ranking (AUROC, AUPRC), calibration (FNR, FPR), and uncertainty metrics: marginal average prediction set size (M. APSS), positive-class prediction set size (PC APSS), and negative-class prediction set size (NC APSS).

\uparrow

means higher is better;

\downarrow

lower is better. Best results for each base model (LR, Focal, SVM, GCE) are in bold . Our method consistently improves uncertainty estimation (M.APSS, PC APSS, NC APSS), ranking metrics (AUROC, AUPRC), and accuracy (ACC, FPR), while slightly increasing FNR.

Data	loss	Architecture	Epochs	$\eta$	$\lambda^{+}$	$\lambda^{-}$	$\alpha^{+}$	$\alpha^{-}$
Adult	LR+CMRM	MLP	150	0.01	0.4	0.5	0.3	0.1
	Focal+CMRM	MLP	150	0.01	0.1	0.4	0.3	0.5
	SVM+CMRM	SVM	150	0.01	0.15	1.5	0.2	0.5
	GCE+CMRM	MLP	150	0.001	0.0	0.2	0.1	0.5
Email	LR+CMRM	MLP	150	0.01	0.7	0.2	0.2	0.2
	Focal+CMRM	MLP	150	0.01	0.4	0.6	0.1	0.1

Appendix

Table 10: The details we used to train our models for binary classification . We reported the hyperparameters that give the best combination os metrics.

Panel of Figure 10 — Figure 10: Dynamics of losses (Top Row) and two thresholds ( $\tau^{+}$ and $\tau^{-}$ , Bottom Row) for binary classification of all base loss (LR, Focal, SVM, GCE) with CMRM on Email datasets. Top row shows the training dynamics of total loss (all loss), classification loss (cl loss), and CMRM regularization loss (CMRM loss) over epochs. CMRM exhibits stable and monotonic convergence alongside standard loss components. Bottom row shows $\tau^{+}$ (negative class threshold) and $\tau^{-}$ (positive class threshold) of LR+CMRM during training. The separation between the thresholds increases, indicating that CMRM actively maximizes the margin between favorable and unfavorable classes.

Explore similar work

Apr 26, 2026

Risk-Aware Robust Learning: Reducing Clinical Risk under Label Noise in Medical Image Classification

Noisy labels are a pervasive challenge in medical image classification, where annotation errors arise from inter-observer variability and diagnostic ambiguity. Although several noise-robust learning methods have been proposed, their evaluation predominantly relies on accuracy-oriented metrics, overlooking the clinical implications of asymmetric error costs. In medical diagnosis, a false negative (missed disease) carries substantially higher consequences than a false positive (false alarm), as delayed treatment can directly impact patient outcomes. In this work, we investigate whether noise-robust training methods preserve clinical safety under label noise. We conduct a systematic risk-aware evaluation of the state-of-the-art noise-robust methods Coteaching, DivideMix, UNICON, and a GMM-based filtering approach on binarized DermaMNIST and PathMNIST datasets under clean and label noise rates of 20%, and 40%. Beyond balanced accuracy, we adopt a cost-sensitive Global Risk formulation that explicitly penalizes false negatives. Our analysis reveals that the robustness of state-of-the-art methods does not guarantee clinical safety. Furthermore, we demonstrate that integrating cost-sensitive optimization into noise-robust training significantly reduces clinical risk, while mantaining model utility. These findings demonstrate that noise-robust learning must be evaluated through a clinical risk lens, and that combining robust training with cost-sensitive optimization can meaningfully reduce risk in noisy-label medical imaging scenarios.

Maycon R. S. Pereira, Filipe R. Cordeiro

Visual Computing Lab, Departamento de Computac¸˜ao, Universidade Federal Rural de Pernambuco (UFRPE), Brasil

Cost-Sensitive Learning Medical Image Classification Learning with Noisy Labels

Ask PDF

Jul 18, 2026

Robust Losses from Univariate Base Functions for Noisy-Label Learning

Learning with noisy labels is a fundamental problem in training reliable deep neural networks. Robust loss functions provide a direct and effective way to mitigate the adverse effects of label noise. However, most existing robust losses are designed directly at the level of the final multiclass objective, which makes it difficult to systematically characterize and extend their robustness properties. In this paper, we propose a general framework that constructs robust multiclass losses from univariate base functions. By defining mapping operators from base functions to multiclass losses, the robustness of the induced losses can be characterized through simple properties of the base functions. We develop two complementary construction schemes, Target Separation and Binary Reduction, corresponding to inter-class independent and inter-class dependent formulations, respectively. For both schemes, we analyze their symmetry and asymmetry properties and derive corresponding sufficient conditions, which provide theoretical criteria for noise-robust loss design. The proposed framework also provides a new route to constructing symmetric losses, serving as a complement to normalization-based symmetric loss designs. Extensive experiments on synthetic and real-world noisy-label benchmarks demonstrate that the proposed losses achieve competitive or superior performance under various noise settings.

Peng Hu, Jianwei Ma

School of Mathematics, Harbin Institute of Technology, Harbin, China · Institute for Artificial Intelligence and the School of Earth and Space Sciences, Peking University, Beijing, China · Institute for Artificial Intelligence and the School of Mathematics, Harbin Institute of Technology, Harbin, China

Noisy-Label Learning Robust Loss Functions Multiclass Classification Statistical Learning Theory

Ask PDF

May 27, 2026

Unification and Optimization of Robust Supervised Learning

Robust learning methods such as distributionally robust optimization, vicinal risk minimization, Mixup, and label smoothing address different sources of uncertainty, including distribution shift, finite-sample effects, and label uncertainty. In practice, however, several such sources may occur simultaneously, and it remains unclear which robust method or combination of methods to use. We introduce a unified framework that organizes a broad class of robustness methods along explicit design axes and makes their relations and combinations explicit within a shared formulation. Within this framework, we analyze how robustness methods interact: some are redundant under particular modeling assumptions, whereas others yield genuinely distinct solution families, and these relationships can change with the model class or loss. We further derive a tractable training procedure that exposes the design choices and their strengths as tunable hyperparameters. Across vision and reward-modeling tasks, the preferred robustness method depends on the task, target metric, and validation data, while suitably selected combinations often outperform the best single method.

Jonas Hanselle, Valentin Margraf, Clemens Damke +1

LMU Munich, MCML · LMU Munich, MCML, DFKI

Distribution Shift Robustness Robust Optimization Learning with Noisy Labels

Ask PDF