ACD-U: Asymmetric co-teaching with machine unlearning for robust learning with noisy labels

Reo Fukunaga; Soh Yoshida; Mitsuji Muneyasu

cs.CVMar 7, 2026

ACD-U: Asymmetric co-teaching with machine unlearning for robust learning with noisy labels

Authors: Reo Fukunaga, Soh Yoshida, Mitsuji Muneyasu

Organizations: Kansai University 3-3-35 Yamate-cho, Suita, Osaka 564-8680, Japan

Abstract

Deep neural networks are prone to memorizing incorrect labels during training, which degrades their generalizability. Although recent methods have combined sample selection with semi-supervised learning (SSL) to exploit the memorization effect -- where networks learn from clean data before noisy data -- they cannot correct selection errors once a sample is misclassified. To overcome this, we propose asymmetric co-teaching with different architectures (ACD)-U, an asymmetric co-teaching framework that uses different model architectures and incorporates machine unlearning. ACD-U addresses this limitation through two core mechanisms. First, its asymmetric co-teaching pairs a contrastive language-image pretraining (CLIP)-pretrained vision Transformer with a convolutional neural network (CNN), leveraging their complementary learning behaviors: the pretrained model provides stable predictions, whereas the CNN adapts throughout training. This asymmetry, where the vision Transformer is trained only on clean samples and the CNN is trained through SSL, effectively mitigates confirmation bias. Second, selective unlearning enables post-hoc error correction by identifying incorrectly memorized samples through loss trajectory analysis and CLIP consistency checks, and then removing their influence via Kullback--Leibler divergence-based forgetting. This approach shifts the learning paradigm from passive error avoidance to active error correction. Experiments on synthetic and real-world noisy datasets, including CIFAR-10/100, CIFAR-N, WebVision, Clothing1M, and Red Mini-ImageNet, demonstrate state-of-the-art performance, particularly in high-noise regimes and under instance-dependent noise. The code is publicly available at https://github.com/meruemon/ACD-U.

Figures & tables

Figure 1: Overview of the ACD-U framework, comprising three main components: (1) unlearning sample selection, (2) sample forgetting via unlearning, and (3) asymmetric co-teaching between different model architectures. Unlearning sample selection is performed every $E_{UP}$ epochs. Subsequently, the sample forgetting process is executed for $E_{UD}$ epochs. The training dataset, $D_{t}$ , used for ACD is constructed by excluding samples from the unlearning target datasets, $D^{(A)}_{u}$ and $D^{(V)}_{u}$ .

Figure 2: Architecture of asymmetric Co-teaching with different architectures (ACD) framework, which employs an asymmetric training strategy where the pretrained Vision Transformer (net

V

) trains only on labeled samples, whereas the CNN (net

A

) utilizes both labeled and unlabeled samples in a semi-supervised learning pipeline.

Technical Gap	Problem in Existing Methods	Representative Methods	ACD-U Solution
Irreversible Error Embedding	When both networks misclassify a noisy sample as clean, error becomes permanent with no correction mechanism	Co-teaching [ 14 ] , DivideMix [ 16 ] , ProMix [ 20 ] , RankMatch [ 19 ]	Selective unlearning enables post-hoc removal of memorized errors through loss trajectory analysis
Uniform Training of Heterogeneous Models	Different architectures trained identically, ignoring their distinct memorization patterns	CLIPCleaner [ 23 ] (static validator), TURN [ 27 ] (frozen ViT extractor)	Asymmetric training: ViT uses clean samples only, CNN uses SSL on all data
Unknown Forgetting Targets	Standard unlearning assumes prior knowledge of problematic samples, incompatible with noisy-label discovery during training	SCRUB [ 29 ] , SAP [ 46 ] (pretrained only)	Dynamic identification via loss changes + CLIP consistency checks (Eqs. 2 - 5 )
Independent Component Operation	Techniques applied sequentially without feedback between components	NoiseBox [ 22 ] (sequential), CLIPCleaner [ 23 ] (fixed CLIP)	Interdependent system: ViT guides CNN, CNN informs ViT, unlearning corrects both

Table 1: Summary of key technical gaps in existing LNL methods and the corresponding solutions provided by the ACD-U framework.

Method	CIFAR-10/-100	CIFAR-N	Red Mini-ImageNet	Clothing1M	WebVision
DivideMix [ 16 ]	✓	✓	✓	✓	✓
LongReMix [ 17 ]	✓		✓	✓	✓
ProMix [ 20 ]	✓	✓		✓
RankMatch [ 19 ]	✓			✓	✓
SV-Learner [ 21 ]	✓			✓	✓
NoiseBox+SS-KNN [ 22 ]	✓		✓	✓	✓

Table 2: Summary of the comparison methods evaluated on each dataset.

Parameter	Dataset
	CIFAR-10	CIFAR-100	CIFAR-10N	CIFAR-100N	Red Mini-ImageNet	WebVision	Clothing1M
Architecture (net $A$ )	PreAct ResNet18	PreAct ResNet18	PreAct ResNet18	PreAct ResNet18	PreAct ResNet18	ResNet-50	ResNet-50
Architecture (net $V$ )	ViT	ViT	ViT	ViT	ViT	ViT	ViT
Optimizer	SGD	SGD	SGD	SGD	SGD	SGD	SGD
Momentum	0.9	0.9	0.9	0.9	0.9	0.9	0.9
Weight Decay	0.0005	0.0005	0.0005	0.0005	0.0005	0.001	0.001

Table 3: Hyperparameter settings for the proposed method across all experimental datasets.

Dataset		CIFAR-10					CIFAR-100
Noise type		Sym.				Asym.	Sym.
Method/Noise ratio		20%	50%	80%	90%	40%	20%	50%	80%	90%
DivideMix [ 16 ]	Best	96.1	94.6	93.2	76.0	93.4	77.3	74.6	60.2	31.5
DivideMix [ 16 ]	Last	95.7	94.4	92.9	75.4	92.1	76.9	74.2	59.6	31.0
LongReMix [ 17 ]	Best	96.3	95.1	93.8	79.9	94.7	77.9	75.5	62.3	34.7
LongReMix [ 17 ]	Last	96.0	94.8	93.3	79.1	94.3	77.5	74.9	61.7	30.7

Table 4: Test accuracies (%) on the CIFAR-10 and CIFAR-100 datasets with synthetic symmetric and asymmetric noise. “Best” indicates the highest accuracy achieved across all training epochs, and “Last” is the average accuracy over the final 10 epochs, based on [ 16 , 20 , 19 ] .

Dataset	CIFAR-10N			CIFAR-100N
Method	aggre	random	worst	noisy
DivideMix [ 16 ]	95.01	95.16	92.56	71.13
ProMix [ 20 ]	97.65	97.39	96.34	73.79
LSL [ 24 ]	-	-	94.57	74.46
Semi-RML++ [ 25 ]	96.84	96.67	94.87	73.68
ACD-U	96.46	96.28	94.64	75.98

Table 5: Test accuracies (%) on the real-world CIFAR-10N and CIFAR-100N benchmarks.

Method	WebVision		ImageNet
	Top1	Top5	Top1	Top5
DivideMix [ 16 ]	77.3	91.6	75.2	90.8
LongReMix [ 17 ]	78.9	92.3	_	_
RankMatch [ 19 ]	79.9	93.6	77.4	94.3
SV-Learner [ 21 ]	78.9	93.6	76.9	93.9
NoiseBox+SS-KNN [ 22 ]	81.4	93.0	77.0	91.0

Table 6: Top-1 and Top-5 accuracies (%) on the WebVision and ImageNet validation sets.

Method	Top1 (%)
DivideMix [ 16 ]	74.8
LongReMix [ 17 ]	74.4
ProMix [ 20 ]	74.9
RankMatch [ 19 ]	75.2
SV-Learner [ 21 ]	75.2
NoiseBox+SS-KNN [ 22 ]	74.7

Table 7: Top-1 accuracies (%) comparison on the Clothing1M test set.

Method/ noise ratio	20%	40%	60%	80%
DivideMix [ 16 ]	50.96	46.72	43.14	34.50
LongReMix [ 17 ]	56.03	50.69	46.81	38.24
NoiseBox+SS-KNN [ 22 ]	60.94	59.08	50.30	44.44
LSL [ 24 ]	54.68	49.80	45.46	36.78
CLIPCleaner [ 23 ]	61.44	58.42	53.18	43.82
ACD-U	61.52	60.66	57.00	48.94

Table 8: Top-1 accuracies (%) on the Red Mini-ImageNet test set with varying noise rates.

Panel of Figure 3 — Figure 3: Accuracy curves on CIFAR-100 with (a) 50% and (b) 80% symmetric noise, showing the effects of the ACD and unlearning components, respectively.

Method		CIFAR-100 (Sym.)
		50%	80%
ACD-U	Best	81.6	74.4
ACD-U	Last	81.2	73.3
ACD-U w/o unlearning	Best	81.6 ( $\pm 0.0$ )	72.7 ( $-1.7$ )
ACD-U w/o unlearning	Last	81.0 ( $-0.2$ )	72.3 ( $-1.0$ )
ACD-U w/o ACD	Best	79.7 ( $-1.9$ )	74.4 ( $\pm 0.0$ )

Table 9: Ablation analysis of the unlearning and ACD components on CIFAR-100 (Sym.50%, 80%). Values in parentheses denote the change in accuracy compared with the full ACD-U model.

Figure 4: Comparison of how noisy samples misidentified as clean (HN) are treated over time on CIFAR-100 (Sym. 80%). Left: In DivideMix, initial errors persist. Right: In ACD-U, initial errors are corrected.

Method	HN	LN	CS
DivideMix	6444	33167	10389
ACD-U	1127	38484	10389

Table 10: Number of misclassified samples in each category during early training on CIFAR-100 (Sym.80%). HN: noisy samples incorrectly judged as clean; LN: noisy samples correctly judged as noisy; CS: clean samples.

Method		CIFAR-100 (Sym.)
		80%
ACD-U	Best	74.4
ACD-U	Last	73.3
ACD-U w/o Low-loss	Best	73.9 (-0.5)
ACD-U w/o Low-loss	Last	73.5 (+0.2)
ACD-U w/o Loss-drop	Best	72.8 (-1.6)

Table 11: Ablation analysis of the three conditions used for unlearning sample selection on CIFAR-100 (Sym.80%). Numbers in parentheses represent differences from ACD-U.

Figure 5: Sensitivity analysis for the unlearning batch size and forgetting intensity parameter

T_{unl}

. Panels (a) and (b) show the results for the CIFAR datasets, whereas (c) and (d) show those for Clothing1M. For the CIFAR-10/-100 datasets, the “Best” accuracy is shown, whereas for CIFAR-100N and Clothing1M, the reported accuracies are based on the same evaluation criteria as those detailed in Tables 5 and 7 .

Figure 6: Sensitivity analysis for the unlearning sample-selection hyperparameters (left column: low-loss sample proportion (

p_{low}

), right column: loss-drop sample proportion (

p_{drop}

)). For the CIFAR-10/-100 datasets, the “Best” accuracy is shown, whereas for CIFAR-100N and Clothing1M, the reported accuracies are based on the same evaluation criteria as those detailed in Tables 5 and 7 .

Dataset		CIFAR-10					CIFAR-100
Noise type		Sym.				Asym.	Sym.
Method/ Noise rate		20%	50%	80%	90%	40%	20%	50%	80%	90%
ACD-U	best	97.2	96.8	95.6	93.9	95.7	83.3	81.6	74.4	66.5
ACD-U	last	97.0	96.6	95.4	93.3	94.4	83.0	81.2	73.3	63.4
ACD-U w/ ULdW	best	97.3	96.7	95.6	94.0	95.9	83.3	81.5	74.0	61.6
ACD-U w/ ULdW	last	97.0	96.6	95.3	93.5	94.9	82.9	81.0	73.7	60.9

Table 12: Ablation analysis comparing two warmup strategies for net

V

on CIFAR-10/100: self-supervised learning without labels (ACD-U) versus supervised learning with the original noisy labels (ACD-U w/ ULdW).

Dataset		CIFAR-100
Noise type		Sym.
Method/ Noise rate		20%	50%	80%	90%
ACD-U	best	83.3	81.6	74.4	66.5
ACD-U	last	83.0	81.2	73.3	63.4
ACD-U w/ stop-grad	best	81.0	80.5	75.1	70.9
ACD-U w/ stop-grad	last	80.5	79.7	74.6	70.5

Table 13: Ablation analysis comparing the effect of fine-tuning versus freezing the ViT encoder for net

V

on CIFAR-100

Noisy-Label Learning

Noisy-label learning develops ways to train and evaluate models when some supervision is incorrect, ambiguous, or unreliable.

See all 93 papers

Forget or Fine-tune? A Comparative Study of Machine Unlearning Strategies for Noisy Label Correction
Aug 30, 2026João L. P. Santana, Filipe R. CordeiroNoisy-Label Learning Classification
Department of Computing Universidade Federal Rural de Pernambuco, Brazil
Pre-train to Gain: Robust Learning Without Clean Labels
Nov 25, 2025David Szczecina, Nicholas Pellegrino, Paul FieguthSelf-Supervised Pre-Training Learning with Noisy Labels
Systems Design Engineering University of Waterloo Waterloo, Ontario, Canada
Robust Self-Training with Closed-loop Label Correction for Learning from Noisy Labels
Mar 14, 2026Zhanhui Lin, Yanlin Liu, Sanping ZhouNoisy-Label Learning Learning with Noisy Labels

Aug 30, 2026J/K move · Enter open · S save

Forget or Fine-tune? A Comparative Study of Machine Unlearning Strategies for Noisy Label Correction

João L. P. Santana, Filipe R. Cordeiro

Department of Computing Universidade Federal Rural de Pernambuco, Brazil

Noisy-Label Learning Classification Learning with Noisy Labels Machine Unlearning Machine Unlearning Evaluation

Open paper AskPDF arXiv

Noisy labels remain a critical challenge for training deep neural networks, since memorizing incorrect labels degrades generalization. Once noisy samples are identified after training, the standard solution is to retrain the model from scratch on the cleaned dataset, which is increasingly expensive as datasets and models grow. Machine Unlearning (MU) has recently emerged as a computationally efficient alternative, but the relative effectiveness of different MU strategies for noisy-label correction remains poorly understood. In this work, we conduct a comparative empirical study of five MU methods (NegGrad, Fine-Tuning (FT), Random Labeling (RL), SalUn, and MUNBa) across symmetric, asymmetric, instance-dependent, and open-set noise on CIFAR-10, CIFAR-100, and the real-world noisy dataset Food-101N. Our central finding is that the appropriate unlearning strategy is conditioned on the noise structure. Simple FT is a strong baseline across most closed-set scenarios; RL and SalUn are the most consistently robust methods and, under instance-dependent noise, approach retraining accuracy at a fraction of the computational cost; MUNBa shows advantages mainly under extreme symmetric noise. Under open-set noise, in contrast, we show that retraining on the cleaned subset degrades accuracy relative to the noisy baseline, so approximating the retrained model is not an adequate objective in this regime. On Food-101N, all MU methods remain competitive and achieve accuracies close to retraining despite reducing runtime by an order of magnitude. These findings provide practical guidelines for selecting MU strategies for post-training noisy-label correction.

Abstract

Noisy-Label Learning

Explore similar work

Forget or Fine-tune? A Comparative Study of Machine Unlearning Strategies for Noisy Label Correction

Pre-train to Gain: Robust Learning Without Clean Labels

Robust Self-Training with Closed-loop Label Correction for Learning from Noisy Labels