Leveraging Dissimilarity Invariance as a Robust Anchor for Learning with Noisy Labels

Wenxiao Fan; Kan Li

cs.CVJul 20, 2026

Leveraging Dissimilarity Invariance as a Robust Anchor for Learning with Noisy Labels

Authors: Wenxiao Fan, Kan Li

Organizations: School of Computer Science, Beijing Institute of Technology

Abstract

Deep learning models excel in visual recognition but suffer severe performance drops when training labels are corrupted by noise. Under label noise prior work cannot learn accurate similarities and thus misguide the learning process. In this paper, we uncover a complementary and novel phenomenon, Dissimilarity Invariance, whereby semantic dissimilarity between unrelated samples remains stable despite label noise. Leveraging this insight, we propose NegScale, a plug-and-play framework that shifts focus from fragile similarity to robust dissimilarity. NegScale integrates: (1) Structured Negative Orthogonality Penalty (SNOP), enforcing subspace orthogonality for unrelated samples; and (2) Dissimilarity-Calibrated Similarity Adjustment (DCSA), suppressing spurious similarity using dissimilarity anchors. We also give theoretical analysis that proves Dissimilarity Invariance and the effectiveness of NegScale. Empirical results demonstrate that NegScale consistently outperforms state-of-the-art baselines, establishing new benchmarks on CIFAR with synthetic noise and real-world datasets.

Explore similar work

Pre-train to Gain: Robust Learning Without Clean Labels
Nov 25, 2025David Szczecina, Nicholas Pellegrino, Paul FieguthNoisy Labels Self-Supervised
MIND: Decoupling Model-Induced Label Noise via Latent Manifold Disentanglement
May 15, 2026Dayong RenNoisy Labels Wireless Foundation Models
Learning from Imperfect Text Guidance: Robust Long-Tail Visual Recognition with High-Noise Label
Apr 25, 2026Mengke Li, Haiquan Ling, Yiqun Zhang +2Noisy Labels Large Vision Language Models

Nov 25, 2025 · cs.LGJ/K move · Enter open · S save

Pre-train to Gain: Robust Learning Without Clean Labels

David Szczecina, Nicholas Pellegrino, Paul Fieguth

Systems Design Engineering University of Waterloo Waterloo, Ontario, Canada

Training deep networks with noisy labels leads to poor generalization and degraded accuracy due to overfitting to label noise. Existing approaches for learning with noisy labels often rely on the availability of a clean subset of data. By pre-training a feature extractor on the target dataset without labels using in-domain self-supervised learning (SSL), followed by standard supervised training on the same noisy dataset, we can train a more noise robust model without requiring a subset with clean labels. We evaluate both contrastive and non-contrastive SSL pre-training methods across datasets with synthetic and real-world label noise, demonstrating the broad applicability of our approach across large-scale datasets, diverse downstream tasks, and model architectures. Across all noise rates, in-domain self-supervised pre-training consistently improves classification accuracy and downstream label-error detection (F1 and Balanced Accuracy) compared with supervised training from scratch. The performance gap widens as the noise rate increases, demonstrating improved robustness. Notably, our approach achieves comparable results to ImageNet and DinoV2 pre-trained models at low noise levels, while substantially outperforming them under high noise conditions.

Noisy Labels Self-Supervised Model Pretraining Contrastive Learning Deep Network

Open paper Ask about it PDF

Abstract

Explore similar work

Pre-train to Gain: Robust Learning Without Clean Labels

MIND: Decoupling Model-Induced Label Noise via Latent Manifold Disentanglement

Learning from Imperfect Text Guidance: Robust Long-Tail Visual Recognition with High-Noise Label