rSDNet: Unified Robust Neural Learning against Label Noise and Adversarial Attacks

Suryasis Jana; Abhik Ghosh

stat.MLMar 18, 2026

rSDNet: Unified Robust Neural Learning against Label Noise and Adversarial Attacks

Authors: Suryasis Jana, Abhik Ghosh

Organizations: Indian Statistical Institute, Kolkata, India

Abstract

Neural networks are central to modern artificial intelligence, yet their training remains highly sensitive to data contamination. Standard neural classifiers are trained by minimizing the categorical cross-entropy loss, corresponding to maximum likelihood estimation under a multinomial model. While statistically efficient under ideal conditions, this approach is highly vulnerable to contaminated observations including label noises corrupting supervision in the output space, and adversarial perturbations inducing worst-case deviations in the input space. In this paper, we propose a unified and statistically grounded framework for robust neural classification that addresses both forms of contamination within a single learning objective. We formulate neural network training as a minimum-divergence estimation problem and introduce rSDNet, a robust learning algorithm based on the general class of $S$ -divergences. The resulting training objective inherits robustness properties from classical statistical estimation, automatically down-weighting aberrant observations through model probabilities. We establish essential population-level properties of rSDNet, including Fisher consistency, classification calibration implying Bayes optimality, and robustness guarantees under uniform label noise and infinitesimal feature contamination. Experiments on three benchmark image classification datasets show that rSDNet improves robustness to label corruption and adversarial attacks while maintaining competitive accuracy on clean data, Our results highlight minimum-divergence learning as a principled and effective framework for robust neural classification under heterogeneous data contamination.

Figures & tables

Panel of Figure 1 — Figure 1: Plots of the bound $M_{\beta,\lambda}^{\eta}$ on the excess risk of rSDNet, as a function of tuning parameters $(\beta,\lambda)$ , for different contamination proportion $\eta$ and $J=10$ .

Panel of Figure 2 — Figure 2: IFs for the MSDFs of parameters under the NN model (M1) in Example 3.1 , for different tuning parameters $(\beta,\lambda)$ .

Panel of Figure 3 — Figure 3: IFs for the MSDFs of parameters under the NN model (M2) in Example 3.1 , for different tuning parameters $(\beta,\lambda)$ .

Panel of Figure 4 — Figure 4: IFs for the MSDFs of parameters under the NN model (M3) in Example 3.1 , for different tuning parameters $(\beta,\lambda)$ .

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 5 — Figure 5: Test accuracies obtained by different NN learning methods trained with varying numbers of epochs for the MNIST dataset

Panel of Figure 6 — Figure 6: Test accuracies obtained by different NN learning methods trained with varying numbers of epochs for the Fashion-MNIST dataset

Dataset $\rightarrow$	MNIST	Fashion-MNIST	CIFAR-10
Existing losses for neural classification
CCE	0.9816	0.8891	0.6782
MAE	0.9770	0.7940	0.1004
TCCE(0.1)	0.9354	0.8815	0.6733
TCCE(0.2)	0.8665	0.3094	0.6687
TCCE(0.3)	0.7787	0.2939	0.6275

Appendix

Table 1: Average

k

-fold CV accuracies of the NN classifiers trained on the three selected datasets using the proposed rSDNet with different

(\beta,\lambda)

and the benchmark losses/classifiers under clean data [

k=7

for MNIST and Fashion-MNIST, and

k=6

for CIFAR-10]

$\eta\rightarrow$	0.1	0.2	0.3	0.4	0.5
Existing losses for neural classification
CCE	0.8691	0.7528	0.6461	0.5789	0.5201
MAE	0.9750	0.9720	0.9691	0.9662	0.9588
TCCE(0.1)	0.9585	0.8917	0.7654	0.6535	0.5555
TCCE(0.2)	0.9500	0.9456	0.8678	0.7448	0.6196
TCCE(0.3)	0.8764	0.9476	0.9282	0.8408	0.7057

Appendix

Table 2: Average 7-fold CV accuracies of the NN classifiers trained on the MNIST data using the proposed rSDNet with different

(\beta,\lambda)

and the benchmark losses/classifiers under uniform label noise with contamination proportion

\eta

[Highest accuracy in each case is highlighted with bold font]

$\eta\rightarrow$	0.1	0.2	0.3	0.4	0.5
Existing losses for neural classification
CCE	0.8015	0.7401	0.6955	0.6549	0.6205
MAE	0.7910	0.7907	0.7224	0.7491	0.7179
TCCE(0.1)	0.8771	0.8070	0.7387	0.6669	0.6098
TCCE(0.2)	0.8769	0.8630	0.7923	0.7137	0.6327
TCCE(0.3)	0.6118	0.8715	0.8509	0.7794	0.6848

Appendix

Table 3: Average 7-fold CV accuracies of the NN classifiers trained on the Fashion-MNIST data using the proposed rSDNet with different

(\beta,\lambda)

and the benchmark losses/classifiers under uniform label noise with contamination level

\eta

[Highest accuracy in each case is highlighted with bold font]

$\eta\rightarrow$	0.1	0.2	0.3	0.4	0.5
Existing losses for neural classification
CCE	0.5884	0.5117	0.4330	0.3696	0.3004
MAE	0.0997	0.0988	0.1006	0.0987	0.0996
TCCE(0.1)	0.6440	0.5816	0.5075	0.4243	0.3425
TCCE(0.2)	0.6508	0.6137	0.5476	0.4665	0.3822
TCCE(0.3)	0.6341	0.6120	0.5576	0.4896	0.3938

Appendix

Table 4: Average 6-fold CV accuracies of the NN classifiers trained on the CIFAR-10 data using the proposed rSDNet with different

(\beta,\lambda)

and the benchmark losses/classifiers under uniform label noise with contamination proportion

\eta

[Highest accuracy in each case is highlighted with bold font]

Attack type $\rightarrow$	FGSM		PGD		CW		Deepfool
	Clean test	Adv. test	Clean test	Adv. test	Clean test	Adv. test	Clean test	Adv. test
Existing losses for neural classification
CCE	0.9073	0.9776	0.6516	0.9949	0.9815	0.9815	0.9731	0.9833
MAE	0.9000	0.9744	0.6005	0.9910	0.9770	0.9770	0.9671	0.9807
TCCE(0.1)	0.7947	0.9383	0.4653	0.9870	0.9549	0.9549	0.9466	0.9574
TCCE(0.2)	0.7387	0.9117	0.4493	0.9434	0.9184	0.9184	0.9086	0.9147

Appendix

Table 5: Average test accuracies of the NN classifiers trained on adversarially perturbed MNIST images using the proposed rSDNet with different

(\beta,\lambda)

and the benchmark losses/classifiers [Highest accuracy in each case is highlighted with bold font]

Neural Network Robustness

Neural network robustness studies how models withstand adversarial inputs, data shifts, noise, and hardware faults while maintaining reliable predictions.

See all 226 papers

Aug 9, 2026

No Unique Minimizer, No Problem: On the Consistency of Robust Neural Classifiers

Neural network classifiers trained by cross-entropy minimization are highly sensitive to label noise and adversarial contamination. While robust alternatives offer bounded influence and resistance to corruption, their statistical foundations in the deep learning setting are insufficient due to a fundamental difficulty: neural parameterizations are non-identifiable, so the population loss minimizer is an equivalence class of parameters, not a unique point. We develop a consistency theory for robust neural classifiers based on the S-divergence family that requires no identifiability assumption. Casting training as stochastic optimization over a non-identifiable parameter space, we prove that empirical S-divergence minimizers converge to the population-optimal equivalence class under mild regularity conditions, and verify these conditions for three architecture choices. We further establish that limit points of the robust training algorithm are stationary points of the empirical objective. Experiments on vision and language benchmark datasets confirm that S-divergence training maintains clean-data accuracy while exhibiting performance competitive with existing robust methods.

Subhabrata Majumdar, Anand Deo, Partha Pratim Saha +1

Indian Institute of Management Bangalore · Independent Researcher · Indian Statistical Institute, Kolkata

Learning with Noisy Labels Neural Network Robustness Robust Loss Functions Neural Network Generalization Statistical Learning Theory Robust Estimation+1

Ask PDF

Jun 15, 2026

Neural Bayesian Anomaly Mitigation: A Robust Loss that Doubles as an Unsupervised Contamination Classifier

Engineered robust losses such as Huber, Student-

t

, and generalised cross-entropy make supervised models tolerant of contamination but cannot answer which observations are corrupted. We introduce Neural Bayesian Anomaly Mitigation (NBAM), a general-purpose drop-in loss derived from a Bayesian latent-switch mixture model: the marginal likelihood defines a robust supervised loss, and the associated posterior defines an unsupervised contamination classifier. Like Huber or Student-

t

, NBAM can replace the standard training loss in any supervised pipeline; unlike them, it additionally learns a structured contamination model and returns a calibrated per-sample contamination posterior. A learned input-dependent prior

π_φ(x)

captures the spatial locality of contamination, so that samples near known corruptions are more likely to be flagged, while an Occam penalty emerges automatically and regularises against over-flagging. On CIFAR-10 with asymmetric label contamination, NBAM recovers the structure of the corruption process without supervision: the contamination posterior separates clean from corrupted samples, and the learned anomaly head identifies the direction of every label-flip pair. Alongside these capabilities, NBAM outperforms the four robust-loss baselines considered here at contamination rates 0.2-0.6.

S. A. K. Leeney, W. J. Handley, H. T. J. Bevins +1

Astrophysics Group, Cavendish Laboratory University of Cambridge · Institute of Astronomy University of Cambridge

Neural Network Robustness Noisy-Label Learning Robust Loss Functions Corruption Robustness Unsupervised Anomaly Detection Classification+2

Ask PDF

Jun 9, 2026

Learning from almost nothing: How neural networks survive heavy input corruption

Learning from imperfect data is a central theme in machine learning, connecting practical questions of robustness to fundamental questions of learnability. Here we examine attribute noise: learning from corrupted inputs while keeping the labels intact, a setting that has received considerably less analytical attention than its label-noise counterpart. We consider two types of corruption models: additive noise and replacement noise. Through experiments with multi-layer perceptrons (MLPs) on corrupted classification datasets, we find that neural networks remain robust, maintaining well-above-chance accuracy even when inputs are >90% corrupted -- far beyond human recognition. To understand this robustness, we analyze infinite-width networks in the heavy-corruption regime using a mean-field-inspired approach and derive a leading-order decision rule for the classification outcome: the network implements a prototype rule, the nearest-class-mean, assigning each test point to the class whose training-set average it most closely resembles. This leading-order decision rule is universal across a broad range of MLP architectures, holding for any depth, as well as a wide class of activation functions and noise distributions. The same centroid mechanism closely matches finite-width network behavior in our experiments and provides an interpretable and analytically tractable account of why learning can succeed even when individual training examples carry almost no signal.

Justin Tahmassebpur, Asadullah Bhuiyan, Hyejin Kim +1

Department of Applied and Engineering Physics, Cornell University, Ithaca, NY 14853, USA · Department of Physics, Cornell University, Ithaca, NY 14853, USA

Neural Network Generalization Multilayer Perceptrons Prototype-Based Classification Neural Network Robustness Corruption Robustness Classification+1

Ask PDF

Abstract

Neural Network Robustness

Explore similar work

No Unique Minimizer, No Problem: On the Consistency of Robust Neural Classifiers

Neural Bayesian Anomaly Mitigation: A Robust Loss that Doubles as an Unsupervised Contamination Classifier

Learning from almost nothing: How neural networks survive heavy input corruption