rSDNet: Unified Robust Neural Learning against Label Noise and Adversarial Attacks

Suryasis Jana; Abhik Ghosh

stat.MLMar 18, 2026

rSDNet: Unified Robust Neural Learning against Label Noise and Adversarial Attacks

Authors: Suryasis Jana, Abhik Ghosh

Organizations: Indian Statistical Institute, Kolkata, India

Abstract

Neural networks are central to modern artificial intelligence, yet their training remains highly sensitive to data contamination. Standard neural classifiers are trained by minimizing the categorical cross-entropy loss, corresponding to maximum likelihood estimation under a multinomial model. While statistically efficient under ideal conditions, this approach is highly vulnerable to contaminated observations including label noises corrupting supervision in the output space, and adversarial perturbations inducing worst-case deviations in the input space. In this paper, we propose a unified and statistically grounded framework for robust neural classification that addresses both forms of contamination within a single learning objective. We formulate neural network training as a minimum-divergence estimation problem and introduce rSDNet, a robust learning algorithm based on the general class of $S$ -divergences. The resulting training objective inherits robustness properties from classical statistical estimation, automatically down-weighting aberrant observations through model probabilities. We establish essential population-level properties of rSDNet, including Fisher consistency, classification calibration implying Bayes optimality, and robustness guarantees under uniform label noise and infinitesimal feature contamination. Experiments on three benchmark image classification datasets show that rSDNet improves robustness to label corruption and adversarial attacks while maintaining competitive accuracy on clean data, Our results highlight minimum-divergence learning as a principled and effective framework for robust neural classification under heterogeneous data contamination.

Figures & tables

Panel of Figure 1 — Figure 1: Plots of the bound $M_{\beta,\lambda}^{\eta}$ on the excess risk of rSDNet, as a function of tuning parameters $(\beta,\lambda)$ , for different contamination proportion $\eta$ and $J=10$ .

Panel of Figure 2 — Figure 2: IFs for the MSDFs of parameters under the NN model (M1) in Example 3.1 , for different tuning parameters $(\beta,\lambda)$ .

Panel of Figure 3 — Figure 3: IFs for the MSDFs of parameters under the NN model (M2) in Example 3.1 , for different tuning parameters $(\beta,\lambda)$ .

Panel of Figure 4 — Figure 4: IFs for the MSDFs of parameters under the NN model (M3) in Example 3.1 , for different tuning parameters $(\beta,\lambda)$ .

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 5 — Figure 5: Test accuracies obtained by different NN learning methods trained with varying numbers of epochs for the MNIST dataset

Panel of Figure 6 — Figure 6: Test accuracies obtained by different NN learning methods trained with varying numbers of epochs for the Fashion-MNIST dataset

Dataset $\rightarrow$	MNIST	Fashion-MNIST	CIFAR-10
Existing losses for neural classification
CCE	0.9816	0.8891	0.6782
MAE	0.9770	0.7940	0.1004
TCCE(0.1)	0.9354	0.8815	0.6733
TCCE(0.2)	0.8665	0.3094	0.6687
TCCE(0.3)	0.7787	0.2939	0.6275

Appendix

Table 1: Average

k

-fold CV accuracies of the NN classifiers trained on the three selected datasets using the proposed rSDNet with different

(\beta,\lambda)

and the benchmark losses/classifiers under clean data [

k=7

for MNIST and Fashion-MNIST, and

k=6

for CIFAR-10]

$\eta\rightarrow$	0.1	0.2	0.3	0.4	0.5
Existing losses for neural classification
CCE	0.8691	0.7528	0.6461	0.5789	0.5201
MAE	0.9750	0.9720	0.9691	0.9662	0.9588
TCCE(0.1)	0.9585	0.8917	0.7654	0.6535	0.5555
TCCE(0.2)	0.9500	0.9456	0.8678	0.7448	0.6196
TCCE(0.3)	0.8764	0.9476	0.9282	0.8408	0.7057

Appendix

Table 2: Average 7-fold CV accuracies of the NN classifiers trained on the MNIST data using the proposed rSDNet with different

(\beta,\lambda)

and the benchmark losses/classifiers under uniform label noise with contamination proportion

\eta

[Highest accuracy in each case is highlighted with bold font]

$\eta\rightarrow$	0.1	0.2	0.3	0.4	0.5
Existing losses for neural classification
CCE	0.8015	0.7401	0.6955	0.6549	0.6205
MAE	0.7910	0.7907	0.7224	0.7491	0.7179
TCCE(0.1)	0.8771	0.8070	0.7387	0.6669	0.6098
TCCE(0.2)	0.8769	0.8630	0.7923	0.7137	0.6327
TCCE(0.3)	0.6118	0.8715	0.8509	0.7794	0.6848

Appendix

Table 3: Average 7-fold CV accuracies of the NN classifiers trained on the Fashion-MNIST data using the proposed rSDNet with different

(\beta,\lambda)

and the benchmark losses/classifiers under uniform label noise with contamination level

\eta

[Highest accuracy in each case is highlighted with bold font]

$\eta\rightarrow$	0.1	0.2	0.3	0.4	0.5
Existing losses for neural classification
CCE	0.5884	0.5117	0.4330	0.3696	0.3004
MAE	0.0997	0.0988	0.1006	0.0987	0.0996
TCCE(0.1)	0.6440	0.5816	0.5075	0.4243	0.3425
TCCE(0.2)	0.6508	0.6137	0.5476	0.4665	0.3822
TCCE(0.3)	0.6341	0.6120	0.5576	0.4896	0.3938

Appendix

Table 4: Average 6-fold CV accuracies of the NN classifiers trained on the CIFAR-10 data using the proposed rSDNet with different

(\beta,\lambda)

and the benchmark losses/classifiers under uniform label noise with contamination proportion

\eta

[Highest accuracy in each case is highlighted with bold font]

Attack type $\rightarrow$	FGSM		PGD		CW		Deepfool
	Clean test	Adv. test	Clean test	Adv. test	Clean test	Adv. test	Clean test	Adv. test
Existing losses for neural classification
CCE	0.9073	0.9776	0.6516	0.9949	0.9815	0.9815	0.9731	0.9833
MAE	0.9000	0.9744	0.6005	0.9910	0.9770	0.9770	0.9671	0.9807
TCCE(0.1)	0.7947	0.9383	0.4653	0.9870	0.9549	0.9549	0.9466	0.9574
TCCE(0.2)	0.7387	0.9117	0.4493	0.9434	0.9184	0.9184	0.9086	0.9147

Appendix

Table 5: Average test accuracies of the NN classifiers trained on adversarially perturbed MNIST images using the proposed rSDNet with different

(\beta,\lambda)

and the benchmark losses/classifiers [Highest accuracy in each case is highlighted with bold font]

Neural Network Robustness

Neural network robustness studies how models withstand adversarial inputs, data shifts, noise, and hardware faults while maintaining reliable predictions.

See all 226 papers

No Unique Minimizer, No Problem: On the Consistency of Robust Neural Classifiers
Aug 9, 2026Subhabrata Majumdar, Anand Deo, Partha Pratim Saha +1Learning with Noisy Labels Neural Network Robustness
Indian Institute of Management Bangalore · Independent Researcher · Indian Statistical Institute, Kolkata
Neural Bayesian Anomaly Mitigation: A Robust Loss that Doubles as an Unsupervised Contamination Classifier
Jun 15, 2026S. A. K. Leeney, W. J. Handley, H. T. J. Bevins +1Neural Network Robustness Noisy-Label Learning
Astrophysics Group, Cavendish Laboratory University of Cambridge · Institute of Astronomy University of Cambridge
Learning from almost nothing: How neural networks survive heavy input corruption
Jun 9, 2026Justin Tahmassebpur, Asadullah Bhuiyan, Hyejin Kim +1Neural Network Generalization Multilayer Perceptrons
Department of Applied and Engineering Physics, Cornell University, Ithaca, NY 14853, USA · Department of Physics, Cornell University, Ithaca, NY 14853, USA

Aug 9, 2026J/K move · Enter open · S save

No Unique Minimizer, No Problem: On the Consistency of Robust Neural Classifiers

Subhabrata Majumdar, Anand Deo, Partha Pratim Saha, Abhik Ghosh

Indian Institute of Management Bangalore · Independent Researcher · Indian Statistical Institute, Kolkata

Learning with Noisy Labels Neural Network Robustness Robust Loss Functions Neural Network Generalization Statistical Learning Theory Robust Estimation+1

Open paper AskPDF arXiv

Neural network classifiers trained by cross-entropy minimization are highly sensitive to label noise and adversarial contamination. While robust alternatives offer bounded influence and resistance to corruption, their statistical foundations in the deep learning setting are insufficient due to a fundamental difficulty: neural parameterizations are non-identifiable, so the population loss minimizer is an equivalence class of parameters, not a unique point. We develop a consistency theory for robust neural classifiers based on the S-divergence family that requires no identifiability assumption. Casting training as stochastic optimization over a non-identifiable parameter space, we prove that empirical S-divergence minimizers converge to the population-optimal equivalence class under mild regularity conditions, and verify these conditions for three architecture choices. We further establish that limit points of the robust training algorithm are stationary points of the empirical objective. Experiments on vision and language benchmark datasets confirm that S-divergence training maintains clean-data accuracy while exhibiting performance competitive with existing robust methods.

Abstract

Neural Network Robustness

Explore similar work

No Unique Minimizer, No Problem: On the Consistency of Robust Neural Classifiers

Neural Bayesian Anomaly Mitigation: A Robust Loss that Doubles as an Unsupervised Contamination Classifier

Learning from almost nothing: How neural networks survive heavy input corruption