cs.CVMar 7, 2026

ACD-U: Asymmetric co-teaching with machine unlearning for robust learning with noisy labels

Authors: Reo Fukunaga, Soh Yoshida, Mitsuji Muneyasu

Organizations: Kansai University 3-3-35 Yamate-cho, Suita, Osaka 564-8680, Japan

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Deep neural networks are prone to memorizing incorrect labels during training, which degrades their generalizability. Although recent methods have combined sample selection with semi-supervised learning (SSL) to exploit the memorization effect -- where networks learn from clean data before noisy data -- they cannot correct selection errors once a sample is misclassified. To overcome this, we propose asymmetric co-teaching with different architectures (ACD)-U, an asymmetric co-teaching framework that uses different model architectures and incorporates machine unlearning. ACD-U addresses this limitation through two core mechanisms. First, its asymmetric co-teaching pairs a contrastive language-image pretraining (CLIP)-pretrained vision Transformer with a convolutional neural network (CNN), leveraging their complementary learning behaviors: the pretrained model provides stable predictions, whereas the CNN adapts throughout training. This asymmetry, where the vision Transformer is trained only on clean samples and the CNN is trained through SSL, effectively mitigates confirmation bias. Second, selective unlearning enables post-hoc error correction by identifying incorrectly memorized samples through loss trajectory analysis and CLIP consistency checks, and then removing their influence via Kullback--Leibler divergence-based forgetting. This approach shifts the learning paradigm from passive error avoidance to active error correction. Experiments on synthetic and real-world noisy datasets, including CIFAR-10/100, CIFAR-N, WebVision, Clothing1M, and Red Mini-ImageNet, demonstrate state-of-the-art performance, particularly in high-noise regimes and under instance-dependent noise. The code is publicly available at https://github.com/meruemon/ACD-U.

Figures & tables

Figure 1: Overview of the ACD-U framework, comprising three main components: (1) unlearning sample selection, (2) sample forgetting via unlearning, and (3) asymmetric co-teaching between different model architectures. Unlearning sample selection is performed every EUPE_{UP} epochs. Subsequently, the sample forgetting process is executed for EUDE_{UD} epochs. The training dataset, DtD_{t} , used for ACD is constructed by excluding samples from the unlearning target datasets, Du(A)D^{(A)}_{u} and Du(V)D^{(V)}_{u} .
Figure 2: Architecture of asymmetric Co-teaching with different architectures (ACD) framework, which employs an asymmetric training strategy where the pretrained Vision Transformer (net VV ) trains only on labeled samples, whereas the CNN (net AA ) utilizes both labeled and unlabeled samples in a semi-supervised learning pipeline.
Technical GapProblem in Existing MethodsRepresentative MethodsACD-U Solution
Irreversible Error EmbeddingWhen both networks misclassify a noisy sample as clean, error becomes permanent with no correction mechanismCo-teaching [ 14 ] , DivideMix [ 16 ] , ProMix [ 20 ] , RankMatch [ 19 ]Selective unlearning enables post-hoc removal of memorized errors through loss trajectory analysis
Uniform Training of Heterogeneous ModelsDifferent architectures trained identically, ignoring their distinct memorization patternsCLIPCleaner [ 23 ] (static validator), TURN [ 27 ] (frozen ViT extractor)Asymmetric training: ViT uses clean samples only, CNN uses SSL on all data
Unknown Forgetting TargetsStandard unlearning assumes prior knowledge of problematic samples, incompatible with noisy-label discovery during trainingSCRUB [ 29 ] , SAP [ 46 ] (pretrained only)Dynamic identification via loss changes + CLIP consistency checks (Eqs. 2 - 5 )
Independent Component OperationTechniques applied sequentially without feedback between componentsNoiseBox [ 22 ] (sequential), CLIPCleaner [ 23 ] (fixed CLIP)Interdependent system: ViT guides CNN, CNN informs ViT, unlearning corrects both
Table 1: Summary of key technical gaps in existing LNL methods and the corresponding solutions provided by the ACD-U framework.
MethodCIFAR-10/-100CIFAR-NRed Mini-ImageNetClothing1MWebVision
DivideMix [ 16 ]✓✓✓✓✓
LongReMix [ 17 ]✓ ✓✓✓
ProMix [ 20 ]✓✓ ✓ 
RankMatch [ 19 ]✓  ✓✓
SV-Learner [ 21 ]✓  ✓✓
NoiseBox+SS-KNN [ 22 ]✓ ✓✓✓
Table 2: Summary of the comparison methods evaluated on each dataset.
ParameterDataset
 CIFAR-10CIFAR-100CIFAR-10NCIFAR-100NRed Mini-ImageNetWebVisionClothing1M
Architecture (net AA )PreAct ResNet18PreAct ResNet18PreAct ResNet18PreAct ResNet18PreAct ResNet18ResNet-50ResNet-50
Architecture (net VV )ViTViTViTViTViTViTViT
OptimizerSGDSGDSGDSGDSGDSGDSGD
Momentum0.90.90.90.90.90.90.9
Weight Decay0.00050.00050.00050.00050.00050.0010.001
Table 3: Hyperparameter settings for the proposed method across all experimental datasets.
DatasetCIFAR-10CIFAR-100
Noise typeSym.Asym.Sym.
Method/Noise ratio20%50%80%90%40%20%50%80%90%
DivideMix [ 16 ]Best96.194.693.276.093.477.374.660.231.5
Last95.794.492.975.492.176.974.259.631.0
LongReMix [ 17 ]Best96.395.193.879.994.777.975.562.334.7
Last96.094.893.379.194.377.574.961.730.7
Table 4: Test accuracies (%) on the CIFAR-10 and CIFAR-100 datasets with synthetic symmetric and asymmetric noise. “Best” indicates the highest accuracy achieved across all training epochs, and “Last” is the average accuracy over the final 10 epochs, based on [ 16 , 20 , 19 ] .
DatasetCIFAR-10NCIFAR-100N
Methodaggrerandomworstnoisy
DivideMix [ 16 ]95.0195.1692.5671.13
ProMix [ 20 ]97.6597.3996.3473.79
LSL [ 24 ]--94.5774.46
Semi-RML++ [ 25 ]96.8496.6794.8773.68
ACD-U96.4696.2894.6475.98
Table 5: Test accuracies (%) on the real-world CIFAR-10N and CIFAR-100N benchmarks.
MethodWebVisionImageNet
 Top1Top5Top1Top5
DivideMix [ 16 ]77.391.675.290.8
LongReMix [ 17 ]78.992.3__
RankMatch [ 19 ]79.993.677.494.3
SV-Learner [ 21 ]78.993.676.993.9
NoiseBox+SS-KNN [ 22 ]81.493.077.091.0
Table 6: Top-1 and Top-5 accuracies (%) on the WebVision and ImageNet validation sets.
MethodTop1 (%)
DivideMix [ 16 ]74.8
LongReMix [ 17 ]74.4
ProMix [ 20 ]74.9
RankMatch [ 19 ]75.2
SV-Learner [ 21 ]75.2
NoiseBox+SS-KNN [ 22 ]74.7
Table 7: Top-1 accuracies (%) comparison on the Clothing1M test set.
Method/ noise ratio20%40%60%80%
DivideMix [ 16 ]50.9646.7243.1434.50
LongReMix [ 17 ]56.0350.6946.8138.24
NoiseBox+SS-KNN [ 22 ]60.9459.0850.3044.44
LSL [ 24 ]54.6849.8045.4636.78
CLIPCleaner [ 23 ]61.4458.4253.1843.82
ACD-U61.5260.6657.0048.94
Table 8: Top-1 accuracies (%) on the Red Mini-ImageNet test set with varying noise rates.
Figure 3: Accuracy curves on CIFAR-100 with (a) 50% and (b) 80% symmetric noise, showing the effects of the ACD and unlearning components, respectively.
MethodCIFAR-100 (Sym.)
 50%80%
ACD-UBest81.674.4
Last81.273.3
ACD-U w/o unlearningBest81.6 ( ±0.0\pm 0.0 )72.7 ( −1.7-1.7 )
Last81.0 ( −0.2-0.2 )72.3 ( −1.0-1.0 )
ACD-U w/o ACDBest79.7 ( −1.9-1.9 )74.4 ( ±0.0\pm 0.0 )
Table 9: Ablation analysis of the unlearning and ACD components on CIFAR-100 (Sym.50%, 80%). Values in parentheses denote the change in accuracy compared with the full ACD-U model.
Figure 4: Comparison of how noisy samples misidentified as clean (HN) are treated over time on CIFAR-100 (Sym. 80%). Left: In DivideMix, initial errors persist. Right: In ACD-U, initial errors are corrected.
MethodHNLNCS
DivideMix64443316710389
ACD-U11273848410389
Table 10: Number of misclassified samples in each category during early training on CIFAR-100 (Sym.80%). HN: noisy samples incorrectly judged as clean; LN: noisy samples correctly judged as noisy; CS: clean samples.
MethodCIFAR-100 (Sym.)
 80%
ACD-UBest74.4
Last73.3
ACD-U w/o Low-lossBest73.9 (-0.5)
Last73.5 (+0.2)
ACD-U w/o Loss-dropBest72.8 (-1.6)
Table 11: Ablation analysis of the three conditions used for unlearning sample selection on CIFAR-100 (Sym.80%). Numbers in parentheses represent differences from ACD-U.
Figure 5: Sensitivity analysis for the unlearning batch size and forgetting intensity parameter TunlT_{unl} . Panels (a) and (b) show the results for the CIFAR datasets, whereas (c) and (d) show those for Clothing1M. For the CIFAR-10/-100 datasets, the “Best” accuracy is shown, whereas for CIFAR-100N and Clothing1M, the reported accuracies are based on the same evaluation criteria as those detailed in Tables 5 and 7 .
Figure 6: Sensitivity analysis for the unlearning sample-selection hyperparameters (left column: low-loss sample proportion ( plowp_{low} ), right column: loss-drop sample proportion ( pdropp_{drop} )). For the CIFAR-10/-100 datasets, the “Best” accuracy is shown, whereas for CIFAR-100N and Clothing1M, the reported accuracies are based on the same evaluation criteria as those detailed in Tables 5 and 7 .
DatasetCIFAR-10CIFAR-100
Noise typeSym.Asym.Sym.
Method/ Noise rate20%50%80%90%40%20%50%80%90%
ACD-Ubest97.296.895.693.995.783.381.674.466.5
last97.096.695.493.394.483.081.273.363.4
ACD-U w/ ULdWbest97.396.795.694.095.983.381.574.061.6
last97.096.695.393.594.982.981.073.760.9
Table 12: Ablation analysis comparing two warmup strategies for net VV on CIFAR-10/100: self-supervised learning without labels (ACD-U) versus supervised learning with the original noisy labels (ACD-U w/ ULdW).
DatasetCIFAR-100
Noise typeSym.
Method/ Noise rate20%50%80%90%
ACD-Ubest83.381.674.466.5
last83.081.273.363.4
ACD-U w/ stop-gradbest81.080.575.170.9
last80.579.774.670.5
Table 13: Ablation analysis comparing the effect of fine-tuning versus freezing the ViT encoder for net VV on CIFAR-100
More in this topic

Noisy-Label Learning

Noisy-label learning develops ways to train and evaluate models when some supervision is incorrect, ambiguous, or unreliable.

See all 93 papers

Explore similar work

CardsList
Aug 30, 2026

Forget or Fine-tune? A Comparative Study of Machine Unlearning Strategies for Noisy Label Correction

Noisy labels remain a critical challenge for training deep neural networks, since memorizing incorrect labels degrades generalization. Once noisy samples are identified after training, the standard solution is to retrain the model from scratch on the cleaned dataset, which is increasingly expensive as datasets and models grow. Machine Unlearning (MU) has recently emerged as a computationally efficient alternative, but the relative effectiveness of different MU strategies for noisy-label correction remains poorly understood. In this work, we conduct a comparative empirical study of five MU methods (NegGrad, Fine-Tuning (FT), Random Labeling (RL), SalUn, and MUNBa) across symmetric, asymmetric, instance-dependent, and open-set noise on CIFAR-10, CIFAR-100, and the real-world noisy dataset Food-101N. Our central finding is that the appropriate unlearning strategy is conditioned on the noise structure. Simple FT is a strong baseline across most closed-set scenarios; RL and SalUn are the most consistently robust methods and, under instance-dependent noise, approach retraining accuracy at a fraction of the computational cost; MUNBa shows advantages mainly under extreme symmetric noise. Under open-set noise, in contrast, we show that retraining on the cleaned subset degrades accuracy relative to the noisy baseline, so approximating the retrained model is not an adequate objective in this regime. On Food-101N, all MU methods remain competitive and achieve accuracies close to retraining despite reducing runtime by an order of magnitude. These findings provide practical guidelines for selecting MU strategies for post-training noisy-label correction.

João L. P. Santana, Filipe R. Cordeiro

Department of Computing Universidade Federal Rural de Pernambuco, Brazil

Noisy-Label LearningClassificationLearning with Noisy LabelsMachine UnlearningMachine Unlearning Evaluation
AskPDF
Nov 25, 2025

Pre-train to Gain: Robust Learning Without Clean Labels

Training deep networks with noisy labels leads to poor generalization and degraded accuracy due to overfitting to label noise. Existing approaches for learning with noisy labels often rely on the availability of a clean subset of data. By pre-training a feature extractor on the target dataset without labels using in-domain self-supervised learning (SSL), followed by standard supervised training on the same noisy dataset, we can train a more noise robust model without requiring a subset with clean labels. We evaluate both contrastive and non-contrastive SSL pre-training methods across datasets with synthetic and real-world label noise, demonstrating the broad applicability of our approach across large-scale datasets, diverse downstream tasks, and model architectures. Across all noise rates, in-domain self-supervised pre-training consistently improves classification accuracy and downstream label-error detection (F1 and Balanced Accuracy) compared with supervised training from scratch. The performance gap widens as the noise rate increases, demonstrating improved robustness. Notably, our approach achieves comparable results to ImageNet and DinoV2 pre-trained models at low noise levels, while substantially outperforming them under high noise conditions.

David Szczecina, Nicholas Pellegrino, Paul Fieguth

Systems Design Engineering University of Waterloo Waterloo, Ontario, Canada

Self-Supervised Pre-TrainingLearning with Noisy Labels
AskPDF
Mar 14, 2026

Robust Self-Training with Closed-loop Label Correction for Learning from Noisy Labels

Training deep neural networks with noisy labels remains a significant challenge, often leading to degraded performance. Existing methods for handling label noise typically rely on either transition matrix, noise detection, or meta-learning techniques, but they often exhibit low utilization efficiency of noisy samples and incur high computational costs. In this paper, we propose a self-training label correction framework using decoupled bilevel optimization, where a classifier and neural correction function co-evolve. Leveraging a small clean dataset, our method employs noisy posterior simulation and intermediate features to transfer ground-truth knowledge, forming a closed-loop feedback system that prevents error amplification. Theoretical guarantees underpin the stability of our approach, and extensive experiments on benchmark datasets like CIFAR and Clothing1M confirm state-of-the-art performance with reduced training time, highlighting its practical applicability for learning from noisy labels.

Zhanhui Lin, Yanlin Liu, Sanping Zhou

Noisy-Label LearningLearning with Noisy Labels
AskPDF