Robust Self-Training with Closed-loop Label Correction for Learning from Noisy Labels

Zhanhui Lin; Yanlin Liu; Sanping Zhou

cs.LGMar 14, 2026

Robust Self-Training with Closed-loop Label Correction for Learning from Noisy Labels

Authors: Zhanhui Lin, Yanlin Liu, Sanping Zhou

Abstract

Training deep neural networks with noisy labels remains a significant challenge, often leading to degraded performance. Existing methods for handling label noise typically rely on either transition matrix, noise detection, or meta-learning techniques, but they often exhibit low utilization efficiency of noisy samples and incur high computational costs. In this paper, we propose a self-training label correction framework using decoupled bilevel optimization, where a classifier and neural correction function co-evolve. Leveraging a small clean dataset, our method employs noisy posterior simulation and intermediate features to transfer ground-truth knowledge, forming a closed-loop feedback system that prevents error amplification. Theoretical guarantees underpin the stability of our approach, and extensive experiments on benchmark datasets like CIFAR and Clothing1M confirm state-of-the-art performance with reduced training time, highlighting its practical applicability for learning from noisy labels.

Figures & tables

Figure 1: This framework provides a general overview of the approach. (a) Training the classifier on the corrected labels. (b) Feature collection and simulation of the clean-data noisy posterior. (c) Optimization of the correction function. (d) Noisy label correction.

Dataset	Method		Symmetric			Asymmetric			Instance-dependent
			$\eta$			$\eta$			$\eta$
			0.2	0.4	0.6	0.2	0.4	0.6	0.2	0.4	0.6
CIFAR-10	Cross-Entropy	Best	91.13	87.93	82.54	92.87	90.00	71.99	90.75	82.74	68.52
	Cross-Entropy	Last	82.84	71.30	59.93	88.45	79.92	70.19	78.15	54.54	39.72
	FasTEN	Best	91.46	88.42	82.88	92.87	92.68	91.39	91.19	85.44	77.09
	FasTEN	Last	86.19	77.62	70.04	91.63	88.82	85.71	86.82	81.16	61.88

Table 1: Test accuracy (%) of all competing methods on CIFAR-10 and CIFAR-100 under different noise types with different noise levels. The best results are highlighted in bold.

Setting	Ablation	Symmetric			Asymmetric			Instance-dependent
		$\eta$			$\eta$			$\eta$
		0.2	0.4	0.6	0.2	0.4	0.6	0.2	0.4	0.6
(a)	Final-Layer Logits	66.76	60.00	45.47	67.41	62.13	54.72	67.98	59.41	46.65
	Pseudo-Soft-Labels	64.81	51.70	28.32	64.60	51.61	32.51	66.81	58.91	40.71
	No Convex Combination	66.13	59.56	47.12	68.57	65.51	60.40	69.21	62.73	55.15
	Ours	68.47	62.36	50.74	69.76	66.76	63.22	70.64	63.58	54.97

Table 2: Test accuracy (%) of our method and its ablations on CIFAR-100. Setting (a) uses 5,000 clean samples (50 samples per class), and Setting (b) uses 1,000 clean samples (10 samples per class). The last recorded performances in the training procedure are reported.

Method	Extra data	Accuracy (%)
Cross-entropy	✗	69.21
FasTEN	✓	77.83
MW-Net	✦	73.72
EMLC ⧫	✦	79.35
DMLP ⧫ (10% clean)	✦	75.50
DMLP ⧫ (50% clean)	✦	77.30

Table 3: Comparison of test accuracy (%) on Clothing1M with advanced methods utilizing the clean data (or its subset). The symbol ✦denotes the use of meta-knowledge, ✓ denotes that the method is directly trained on clean instances with full forward and back propagation on the classifier parameters, and

\blacklozenge

denotes methods that are trained with additional self-supervised learning procedures.

Method	GPU Memory Cost (MiB)	Training Time per Epoch (hours)
L2B	24120	3.47
EMLC	22190 + 21813	3.00
Ours	16915	0.55
Ours	(no additional cost)	(avg)

Table 4: Comparison of average memory usage and training time per epoch for our method, L2B, and EMLC on Clothing1M under the specified training configurations. EMLC’s memory usage denotes the sum from its 2-GPU setup. We calculate our training time as the average (avg) by dividing the total training time by the complete 15 epochs of our experiments.

Phase	Classifier Training	Feature Extraction & Simulation	Train Neural Label Correction	Train Convex Combination	Noisy Label Update	Total
Total Time	0.327	0.014	0.004	0.001	0.002	0.348
(Execution Times)	(100×)	(12×)	(12×)	(12×)	(12×)	0.348

Table 5: Time consumption breakdown (in hours) of our method’s different phases on CIFAR datasets using a single RTX 4090.

Dataset	Method		Symmetric			Asymmetric			Instance-dependent
			$\eta$			$\eta$			$\eta$
			0.2	0.4	0.6	0.2	0.4	0.6	0.2	0.4	0.6
CIFAR-100	Cross-Entropy	Best	67.96	63.70	53.92	70.57	66.48	57.17	63.09	47.32	27.30
	Cross-Entropy	Last	62.88	48.12	29.72	63.60	48.58	32.32	59.87	41.41	22.59
	FasTEN	Best	67.03	55.87	51.47	64.79	60.77	60.79	66.98	57.78	34.60
	FasTEN	Last	61.56	41.95	28.71	58.10	46.09	45.32	60.85	43.07	13.03

Table 6: Test accuracy (%) of all competing methods on CIFAR-100 with 1000 clean samples and 49000 noisy samples under different noise types with different noise levels. The best results are highlighted in bold.

	Symmetric			Asymmetric			Instance-dependent
$\lambda$ value	$\eta$			$\eta$			$\eta$
	0.2	0.4	0.6	0.2	0.4	0.6	0.2	0.4	0.6
$0.2$	68.95	63.54	52.69	71.30	68.18	63.78	70.01	62.84	55.21
$0.5$	68.47	62.36	50.74	69.76	66.76	63.22	70.64	63.58	54.97
$1.0$	66.35	59.30	48.74	69.43	66.41	62.61	68.83	60.83	53.96

Table 7: Test accuracy (%) of our method using different values of loss balancing factor

\lambda

on CIFAR-100 under different noise types with different noise levels. The last recorded performances in the training procedure are reported.

	Symmetric			Asymmetric			Instance-dependent
Label Correction Frequency	$\eta$			$\eta$			$\eta$
	0.2	0.4	0.6	0.2	0.4	0.6	0.2	0.4	0.6
$5$	68.47	62.36	50.74	69.76	66.76	63.22	70.64	63.58	54.97
$8$	68.09	61.82	50.20	69.94	67.36	62.99	69.87	62.71	53.87
$10$	68.09	60.94	49.36	69.76	66.36	61.66	69.10	61.68	52.99

Table 8: Test accuracy (%) of our method using different label correction frequencies on CIFAR-100 under different noise types with different noise levels. The frequency value indicates how often (in epochs) the feature sets and correction function are updated.

Dataset	CIFAR-10	CIFAR-100	Clothing1M
Architecture	ResNet-34	ResNet-34	ResNet-50
Pretraining	None	None	ImageNet
Training Epochs	100	100	15
Training Augmentations	Horizontal Flip Random Crop	Horizontal Flip Random Crop	Resized Crop Horizontal Flip Random ColorJitter Random Grayscale
Optimizer	SGD	SGD	SGD
Batch Size	$128$	$128$	$256$

Table 9: Experimental details for our method across different datasets.

Figure 6: Training dynamics of all methods on CIFAR-10 under different noise scenarios.

Figure 7: Training dynamics of all methods on CIFAR-100 under different noise scenarios.

Explore similar work

Noise-Aware Framework for Correcting Corrupted Labels
Jun 10, 2026Ha-Linh Nguyen, Hong-Anh Nguyen, Minh-Duc La +4Training Data Curation Noisy-Label Learning
Faculty of Information Technology, VNU University of Engineering and Technology, Hanoi, Vietnam
Pre-train to Gain: Robust Learning Without Clean Labels
Nov 25, 2025David Szczecina, Nicholas Pellegrino, Paul FieguthSelf-Supervised Pre-Training Learning with Noisy Labels
Systems Design Engineering University of Waterloo Waterloo, Ontario, Canada
An Adaptive Data cleaning Framework for Noisy Label Detection
Jun 5, 2026Chen-Hsuan Fang, Wei-Hsinag Chen, Pin-Hsuan Yu +2Noisy-Label Learning Learning with Noisy Labels
Department of Electrical Eng. National Taiwan Ocean University, Keelung City 20224, Taiwan · AI Research Center, National Taiwan Ocean University, Keelung City 20224, Taiwan

Jun 10, 2026J/K move · Enter open · S save

Noise-Aware Framework for Correcting Corrupted Labels

Ha-Linh Nguyen, Hong-Anh Nguyen, Minh-Duc La, Phong Lam+3

Faculty of Information Technology, VNU University of Engineering and Technology, Hanoi, Vietnam

Training Data Curation Noisy-Label Learning Learning with Noisy Labels

Open paper AskPDF arXiv

High-quality labeled data is essential for training reliable ML/DL models. However, real-world datasets often contain a considerable proportion of corrupted labels, which can severely degrade model performance. To address this problem, we propose CANOLA, a novel framework for correcting corrupted labels through noise-aware learning and iterative label refinement. CANOLA explicitly estimates the underlying noise distribution of the dataset and incorporates this information into the training of a noise-aware Deep Neural Network. By incorporating noise characteristics during learning, CANOLA enables the model to down-weight unreliable supervision signals and focus on trustworthy patterns, thereby improving robustness and generalization. Label correction is performed via cautious, iterative soft label refinement, in which model predictions are blended with observed labels to prevent premature or erroneous updates. This progressive refinement allows the dataset to be repaired in a stable and controlled manner. We evaluate CANOLA on six widely used datasets under realistic noisy labeling scenarios. Experimental results show that CANOLA consistently outperforms SOTA label correction methods, achieving relative improvements ranging from 19% to 52% in error reduction. Moreover, models trained on datasets corrected by CANOLA obtain substantial downstream performance gains. Even simple classifiers trained on CANOLA's corrected data can outperform complex model-centric approaches by margins of up to 67%.

Abstract

Explore similar work

Noise-Aware Framework for Correcting Corrupted Labels

Pre-train to Gain: Robust Learning Without Clean Labels

An Adaptive Data cleaning Framework for Noisy Label Detection