Deconstructing the Failure of Ideal Noise Correction: A Three-Pillar Diagnosis

Chen Feng; Zhuo Zhi; Zhao Huang; Jiawei Ge; Ling Xiao; Nicu Sebe; Georgios Tzimiropoulos; Ioannis Patras

cs.LGMar 13, 2026

Deconstructing the Failure of Ideal Noise Correction: A Three-Pillar Diagnosis

Authors: Chen Feng, Zhuo Zhi, Zhao Huang, Jiawei Ge, Ling Xiao, Nicu Sebe, Georgios Tzimiropoulos, Ioannis Patras

Organizations: Queen’s University Belfast · University College London · University of Aberdeen · Queen Mary University of London ·

Abstract

Statistically consistent methods based on the noise transition matrix ( $T$ ) offer a theoretically grounded solution to Learning with Noisy Labels (LNL), with guarantees of convergence to the optimal clean-data classifier. In practice, however, these methods are often outperformed by empirical approaches such as sample selection, and this gap is usually attributed to the difficulty of accurately estimating $T$ . The common assumption is that, given a perfect $T$ , noise-correction methods would recover their theoretical advantage. In this work, we put this longstanding hypothesis to a decisive test. We conduct experiments under idealized conditions, providing correction methods with a perfect, oracle transition matrix. Even under these ideal conditions, we observe that these methods still suffer from performance collapse during training. This compellingly demonstrates that the failure is not fundamentally a $T$ -estimation problem, but stems from a more deeply rooted flaw. To explain this behaviour, we provide a unified analysis that links three levels: macroscopic convergence states, microscopic optimisation dynamics, and information-theoretic limits on what can be learned from noisy labels. Together, these results give a formal account of why ideal noise correction fails and offer concrete guidance for designing more reliable methods for learning with noisy labels.

Figures & tables

Panel of Figure 1 — Figure 1: Comparison of test accuracy across different training durations. Even when provided with the ground-truth transition matrix (Oracle $T$ ), the model’s performance collapses after an initial peak. Prolonged training sees the correction method’s efficacy degrade, ultimately offering no benefit over the uncorrected baseline.

Panel of Figure 2 — Figure 2: ACC and ECE comparison on CIFAR datasets under Ideal Fitted Case.

Panel of Figure 3 — Figure 3: Comparison of Accuracy and ECE for CIFAR-10 on multi-labeled dataset.

Table 1: Experimental results across datasets.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Figure 4: Gradient vector field of the FC loss on a 3-class simplex. We denote the clean label vertex as $A$ ( $\mathbf{e}_{y^{*}}$ ), the noisy label as $B$ ( $\mathbf{e}_{y^{n}}$ ), and the theoretical FC optimum as $C$ ( $\mathbf{e}_{k_{\mathtt{FC}}^{*}}$ ). The vector field confirms that the global minimum is at $C$ . However, the noisy vertex $B$ acts as a strong, non-optimal attractor. The vanishing gradient magnitude (“dead zone”) near $B$ traps SGD, leading to the ’pseudo-convergence’ analyzed in Appendix E .

Learning with Noisy Labels

Learning with noisy labels studies how to train reliable models when annotations are incorrect, incomplete, subjective, or weakly supervised.

See all 150 papers

Robust Self-Training with Closed-loop Label Correction for Learning from Noisy Labels
Mar 14, 2026Zhanhui Lin, Yanlin Liu, Sanping ZhouNoisy-Label Learning Learning with Noisy Labels
Stop Replacing Noise with Noise: Two-Source Reliability Assessment for Label Correction and Sample Reweighting in Label-Noise Learning
Aug 4, 2026Wenxiao Fan, Kan LiInstance Reweighting Noisy-Label Learning
School of Computer Science, Beijing Institute of Technology
LiNC: Lightweight Noise Correction via Per-Sample Trust and Gaussian Mixture Modeling
Aug 4, 2026Abhishek Moturu, Babak Taati, Anna GoldenbergMedical Image Classification Noisy-Label Learning
Department of Computer Science University of Toronto The Hospital for Sick Children UHN KITE Research Institute T-CAIREM Vector Institute · Department of Computer Science Institute of Biomedical Engineering University of Toronto Rehabilitation Sciences Institute UHN KITE Research Institute Vector Institute · Department of Computer Science Department of Laboratory Medicine and Pathobiology University of Toronto The Hospital for Sick Children T-CAIREM Vector Institute

Mar 14, 2026J/K move · Enter open · S save

Robust Self-Training with Closed-loop Label Correction for Learning from Noisy Labels

Zhanhui Lin, Yanlin Liu, Sanping Zhou

Noisy-Label Learning Learning with Noisy Labels

Open paper AskPDF arXiv

Training deep neural networks with noisy labels remains a significant challenge, often leading to degraded performance. Existing methods for handling label noise typically rely on either transition matrix, noise detection, or meta-learning techniques, but they often exhibit low utilization efficiency of noisy samples and incur high computational costs. In this paper, we propose a self-training label correction framework using decoupled bilevel optimization, where a classifier and neural correction function co-evolve. Leveraging a small clean dataset, our method employs noisy posterior simulation and intermediate features to transfer ground-truth knowledge, forming a closed-loop feedback system that prevents error amplification. Theoretical guarantees underpin the stability of our approach, and extensive experiments on benchmark datasets like CIFAR and Clothing1M confirm state-of-the-art performance with reduced training time, highlighting its practical applicability for learning from noisy labels.

Abstract

Learning with Noisy Labels

Explore similar work

Robust Self-Training with Closed-loop Label Correction for Learning from Noisy Labels

Stop Replacing Noise with Noise: Two-Source Reliability Assessment for Label Correction and Sample Reweighting in Label-Noise Learning

LiNC: Lightweight Noise Correction via Per-Sample Trust and Gaussian Mixture Modeling