Learning with class-conditional label noise often relies on a transition model from latent clean classes to observed annotations. Forward correction embeds this transition in the likelihood, yet finite-sample networks may still memorize corrupted labels. The corrected likelihood also induces a reverse posterior over the clean classes that could explain each annotation. When its leading class differs from the annotation, the model and transition matrix provide evidence against that annotation, but the leading alternatives can remain nearly tied. We introduce Selective Posterior Margin Regularization (SPMR), which preserves the Forward objective and converts this disagreement into a graded update on the clean classifier. SPMR selects the leading reverse-posterior class, scales a detached pairwise margin by the separation between the two leading posterior classes, and assigns correspondingly little influence to diffuse conflicts. The gap factorizes into transition- adjusted pairwise separation and the posterior mass carried by the leading pair. The active margin follows the locally minimum-norm logit direction that enlarges the selected pairwise margin. Across five known-transition benchmarks, SPMR improves full-length Forward by 2.5-7.0 percentage points and remains 0.7-2.5 percentage points above Forward with Mixup and early stopping. Matched interventions support distinct gains from the posterior-space coefficient, transition-adjusted target, and pairwise action. The same design transfers to estimated transitions, human annotations, architectural changes, and stronger Forward recipes. The formulation uses latent-class evidence already available inside Forward correction without promoting every posterior conflict to a corrected label.
Figures & tables
Figure 1: Posterior disagreement requires a second decision. Both examples conflict with the observed annotation. The displayed posterior values illustrate how a small separation yields a weak intervention, whereas a clearly separated alternative receives a stronger clean-logit margin. Distances and distribution shapes are schematic; the annotated posterior values and gaps are exact.
Figure 2: SPMR separates posterior evidence, target choice, and classifier action while preserving the Forward likelihood. The compact lower row gives the two structural identities used in the analysis. Transition stability is developed in Eq. ( 8 ).
Figure 3: Summary of performance, attribution, and transition reliability. (a) Paired gains over full-length Forward (filled circles) and Forward+Mixup-ES (open squares) across known-transition settings. (b) Five-seed gains over the strongest matched alternative for posterior evidence, target choice, and norm-matched action. Open circles are individual seeds, and bars are paired 95% bootstrap intervals. (c) Cumulative auxiliary-update mass assigned to incorrect targets under known and estimated transitions.
Method
C10-S50
C10-S80
C10-A40
C100-S50
C100-S80
Forward
89.1 ± 0.4
68.4 ± 0.8
87.5 ± 0.5
54.7 ± 0.6
24.9 ± 0.7
Forward-ES
90.0 ± 0.3
72.0 ± 0.6
88.0 ± 0.4
56.0 ± 0.5
27.8 ± 0.6
Forward+Mixup-ES
90.9 ± 0.3
73.7 ± 0.6
88.8 ± 0.4
57.1 ± 0.5
29.3 ± 0.6
Noise-Bounded Forward
90.4 ± 0.3
71.7 ± 0.7
–
55.8 ± 0.5
27.4 ± 0.6
SPMR
91.6 ± 0.2
75.4 ± 0.4
90.2 ± 0.2
59.0 ± 0.3
31.8 ± 0.3
SPMR − Forward (pp)
+2.50
+7.00
+2.72
+4.26
+6.91
Table 1: Accuracy under known transitions and across transfer settings. Panel (a) uses five paired seeds; Figure 3 (a) shows the paired intervals. Panel (b) reports mean ± SD for three seeds, with positive gains on all three. Accuracy is in percent and differences are in percentage points (pp).
Role
Variant
Accuracy (%)
SPMR advantage (pp)
Evidence
fixed coefficient
72.6 ± 0.3
–
p top-two gap
74.1 ± 0.3
1.34 [1.19, 1.50]
q top-two gap
75.4 ± 0.4
–
Practical action
Forward reweighting
73.3 ± 0.3
–
hard-target CE
73.8 ± 0.3
1.61 [1.14, 2.11]
pairwise margin
75.4 ± 0.4
–
Table 2: Controlled comparisons of posterior evidence, target, and action. Evidence and action use C10-S80 with five paired seeds; target choice uses C10-A40. Intervals are paired 95% bootstrap intervals against the strongest matched alternative.
Most methods for learning with noisy labels require privileged knowledge such as noise transition matrices, clean subsets or pretrained feature extractors, resources typically unavailable when robustness is most needed. We propose Conformal Margin Risk Minimization (CMRM), a plug-and-play envelope framework that improves any classification loss under label noise by adding a single quantile-calibrated regularization term, with no privileged knowledge or training pipeline modification. CMRM measures the confidence margin between the observed label and competing labels, and thresholds it with a conformal quantile estimated per batch to focus training on high-margin samples while suppressing likely mislabeled ones. We derive a learning bound for CMRM under arbitrary label noise requiring only mild regularity of the margin distribution. Across five base methods and six benchmarks with synthetic and real-world noise, CMRM consistently improves accuracy (up to +3.39%), reduces conformal prediction set size (up to -20.44%) and does not hurt under 0% noise, showing that CMRM captures a method-agnostic uncertainty signal that existing mechanisms did not exploit.
Yuanjie Shi, Peihong Li, Zijian Zhang +2
School of EECS, Washington State University, Pullman, WA, USA
Strictly proper scoring rules identify the true conditional class distribution at population level, but their curvature can alter optimization and finite-sample behavior. We study three multiclass objectives: a class-aware quadratic Bregman score (CAPM), a strongly convex generator with constrained log-cosh ridges (HPG), and an HPG objective with an annealed probability-margin penalty (APMS). CAPM is treated as a structured instance of established quadratic scoring-rule theory. We derive conditional-regret, curvature, range, and logit-gradient bounds for CAPM and HPG, and prove exact penalty-range and conditional-target displacement bounds for APMS. Controlled five-seed experiments use Digits, Wisconsin breast cancer, and synthetic confusion and long-tail problems under clean labels, symmetric and pair-flip corruption, class imbalance, calibration evaluation, input corruption, and first-order adversarial perturbations. The candidates are close to cross-entropy on clean data and show descriptive gains in some noisy-label cells, but the five-seed comparisons are interpreted descriptively rather than as significance evidence. The selected noisy-label baselines perform better on Digits with 40% symmetric label noise, and explicit prior-adjustment methods perform better in the 30:1 synthetic long-tail experiment. Ablations do not show a consistent benefit from the candidate-specific graph, ridge, or margin components. The mathematical analysis establishes the stated properties, and the experiments delimit the empirical evidence; together they do not support a claim of general superiority.
Machine learning models often suffer performance degradation under subpopulation shift, particularly when spurious correlations cause models to rely on shortcut features that fail to generalize across subgroups. A recent line of work mitigates this issue by using loss-based signals to identify informative samples, but these signals can become severely distorted under label noise: mislabeled samples may also incur large losses and contaminate subsequent reweighting or retraining. Despite its practical importance, this intersection remains largely underexplored. We propose POTER, a reweighting framework based on optimal transport that derives sample importance from the transport geometry between the training distribution and a reference distribution constructed from limited validation group annotations. By measuring alignment at the individual-sample level rather than relying on loss, POTER downweights mislabeled or strongly bias-aligned samples while assigning higher importance to samples better aligned with the reference distribution. In addition, POTER requires only a single ERM training stage, moving beyond the retraining paradigm common in recent work. Across standard benchmarks and noisy-label settings, POTER achieves state-of-the-art worst-group accuracy, including cases where label corruption is concentrated within minority subgroups.
Sung Ho Jo, Seonghwi Kim, Wonsang Yun +1
Pohang University of Science and Technology (POSTECH)