Robust Classification

Follow this topic Ask about this topic

Latest papers

37 papers4 papers in the last four weeks, down 33% on the four weeks before. 0.0% of all new papers.

Oct 17, 2026

Contrastive Adversarial Training for Robust Graph Neural Networks under Label Poisoning

Graph Neural Networks (GNNs) are effective for modeling relational data but are vulnerable to label poisoning, where a small number of corrupted training labels can propagate errors across the graph via message passing. Despite this risk, defenses against label poisoning remain underexplored: existing methods are primarily designed for label noise and often rely on robust losses or heuristic data cleaning that fail to distinguish adversarial poisoning from naturally hard examples. In this paper, we propose CoLAT (Contrastive Label-flipping for Adversarial Training), a novel contrastive adversarial training framework tailored for robust node classification on graphs. The core of our approach is a structure-aware detection model that uses contrastive learning to identify label-structure inconsistencies. Unlike prior contrastive methods that focus on representation learning, CoLAT leverages contrastive embeddings to select high-risk nodes that guide adversarial label-flipping during training. This alternating optimization not only performs structure-aware adversarial training but also implicitly sanitizes some corrupted labels, improving robustness against diverse attacks from the literature. Extensive experiments on multiple benchmarks show that CoLAT consistently outperforms existing defenses under various poisoning intensities while scaling efficiently to large graphs.

Manshika Charvi Bissessur, Melis Ilayda Bal, Michael Muehlebach

University of Oxford · Max Planck Institute for Intelligent Systems, Tübingen, Germany · LIONS – École Polytechnique Fédérale de Lausanne +1

Graph Neural Networks Robust Classification Adversarial Training

Oct 16, 2026

On the tradeoff between robustness and geometric accuracy in adversarial training

For non-parametric binary classification in the presence of an adversary, we prove an optimal quantitative estimate for adversarial classifiers controlling geometric accuracy, as measured by the Hausdorff distance to the Bayes classifier, in terms or their robustness, as encoded by the adversarial budget

\varepsilon

. This rigorously shows that robust classifiers sacrifice at most

O(\sqrt{\varepsilon})

or

O(\varepsilon)

in accuracy, depending on the regularity of the underlying Bayes classifier. In contrast to prior work on the incompatibility of robustness and accuracy over low-complexity hypothesis classes, our work shows that high-complexity hypothesis classes allow for robust classifiers with precisely quantified accuracy. Our results apply for adversaries determined by any norm on Euclidean space and in both the infinite and finite data settings.

Rachel Morris, Ryan Murray, Kerrek Stinson

Adversarial Robustness Robust Classification Classification Adversarial Training

Oct 6, 2026

Adversarially Trained Linear Transformers Are Optimal Robust In-Context Learners for Gaussian Mixtures

Adversarial training is one of the most reliable defenses against adversarial attacks, but its high computational cost must generally be paid anew for each task. Robust foundation models offer a promising alternative: adversarially pretrain a model once and then transfer its robustness to downstream tasks through lightweight adaptation. However, a fundamental question remains open: can robustness acquired during pretraining transfer to unseen tasks without further adversarial training? In this study, we answer this question affirmatively. A single model adversarially pretrained at scale can achieve optimal robustness on new tasks without additional task-specific training. Specifically, we show that, for a family of Gaussian-mixture classification tasks, a sufficiently deep linear transformer adversarially trained across tasks can asymptotically attain the robust Bayes error on previously unseen tasks through in-context learning from clean demonstrations. By contrast, a standardly trained model cannot. We further analyze convergence under gradient flow, an accuracy--robustness trade-off, and demonstration complexity.

Soichiro Kumano

LY Corporation

Adversarial Training Transformer Adversarial Robustness In-Context Learning Robust Classification

Sep 21, 2026

Adversarially Robust PAC Learning with Optimal VC Rates

We study the problem of adversarially robust PAC learning. In this framework, the learner observes independent samples from an unknown distribution over

\mathcal{X} \times \{0,1\}

, as in classical PAC learning. However, given a perturbation map

\mathcal{U} : \mathcal{X} \to 2^{\mathcal{X}}

known to the learner, the goal is to output, with high probability, a predictor that correctly classifies every perturbation

z \in \mathcal{U}(x)

of most future examples

(x,y)

drawn from the same underlying distribution. We determine the optimal

\mathcal{U}

-independent sample complexity of this problem in both the realizable and agnostic settings. More specifically, for every concept class

\mathcal{H}

of

\operatorname{VC}

dimension

d

, we prove upper bounds of

\mathcal{O} \big( d/ε+ \log(1/δ)/ε\big)

in the realizable setting and

\mathcal{O} \big( d/ε^2 + \log(1/δ)/ε^2 \big)

in the agnostic setting, together with an optimal first-order refinement of the latter. These bounds match the corresponding lower bounds for classical PAC learning. Consequently, and perhaps surprisingly, adversarial robustness incurs no additional distribution-free statistical cost, uniformly over all perturbation maps. Our bounds improve exponentially on those of [Montasser, Hanneke, and Srebro; COLT '19]. On the technical side, we present short and elementary proofs based on a new algorithmic principle that we call binomial-bagging. We believe that binomial-bagging and its analysis may be of independent interest.

Steve Hanneke, Amirreza Shaeiri

Department of Computer Sciences, Purdue University, USA.

Sample Complexity Adversarial Robustness PAC Learning Statistical Learning Theory Robust Classification

Sep 15, 2026

Efficient Robust Learning at the Information-Theoretic Limit

In an important recent work, Blanc (2026) gave an algorithm for robustly learning Boolean concept classes with respect to a fixed distribution that outputs a (randomized) classifier achieving the optimal error of

η+ \varepsilon

where

η

is the noise rate. In contrast, it is well known that deterministic hypotheses cannot achieve error less than

2η+ \varepsilon.

Blanc's algorithm is computationally inefficient, and the main problem left open in his work is to find a polynomial-time algorithm given access to an oracle for empirical risk minimization (ERM). In this paper, we resolve this problem and give such an algorithm. Perhaps surprisingly, our techniques make crucial use of various types of no-regret learners. Additionally, we give an efficient algorithm (no ERM oracle required) for robustly learning any function class that admits sandwiching polynomials with respect to hypercontractive distributions. As one consequence, we give the first polynomial-time algorithm for robustly learning a halfspace with respect to Gaussian marginals that achieves error

η+ \varepsilon

for any constant

\varepsilon

.

Adam R. Klivans, Konstantinos Stavropoulos, Sergei Tikhonov +1

UT Austin · Institute for Advanced Study · Aarhus University

Boolean Function Learning Learning with Noisy Labels Halfspace Learning Robust Classification

Sep 11, 2026

A family of spectral conjugate gradient algorithms derived by least-squares approximations based on a modified quasi--Newton update with application to a revised robust binary classification model

We develop a spectral three-term modification of the classic Hestenes--Stiefel conjugate gradient algorithm, preserving its anti-jamming characteristic and, simultaneously, taking care of the sufficient descent property. We discuss how a modified secant equation can be extracted from our modification scheme, yielding a memoryless BFGS updating formula. Then, the spectral parameter of our method is obtained by steering its direction toward the given BFGS direction within a least-squares context. Using our technical improvements, we outline the general framework of our algorithm and discuss its theoretical features, including the descent and convergence properties, without the convexity assumption. We put our algorithm to the test in comparison with the three other conjugate gradient algorithms on a set of CUTEr unconstrained optimization test models, comparing the outputs using the Dolan--More measure. Next, we provide a concise evaluation of the results, highlighting the practical advantages of our algorithm. As a real-world case study, we introduce a reduced quadratic surface SVM with the rescaled loss for robust binary classification and apply the proposed algorithm to assess its accuracy and training time against several other SVMs.

Saman Babaie-Kafaki, Maryam Khoshsimaye-Bargard, Ahmad Mousavi

Faculty of Engineering, Free University of Bozen–Bolzano, 39100 Bolzano, Italy · Department of Mathematics, Faculty of Mathematics, Statistics and Computer Science, Semnan University, Semnan, Iran · Department of Mathematics and Statistics, American University, Washington, D.C., USA

Robust Classification Nonconvex Optimization Support Vector Machines Quasi-Newton Methods

Sep 10, 2026

Local Robustness Quantification for Naive Bayes Classifiers and Generative Forests: a General Approach

We provide methods for calculating the robustness of the predictions of two types of generative classifiers whose underlying distribution is a Probabilistic Graphical Model (PGM): naive Bayes classifiers and generative forests (a probabilistic extension of random forests). Following the paradigm of robustness quantification, we define the robustness of a prediction as the extent to which the distribution of the classifier can be perturbed without changing this prediction. We consider perturbations obtained by varying the local models of the PGMs within general neighborhoods and focus in particular on epsilon-contamination, total variation distance and chi-squared divergence balls. We test our methods on benchmark datasets, demonstrate that the robustness value of a prediction serves as an indicator for its trustworthiness and compare our approach with other such indicators.

Adrián Detavernier, Jasper De Bock

Foundations Lab for imprecise probabilities Ghent University Ghent, Belgium

Distribution Shift Robustness Adversarial Robustness Probabilistic Graphical Models Robust Classification

Sep 9, 2026

Counterfactual Marginalisation: Framework for Evaluating Robustness to Nuisance Variables

Machine learning models can achieve strong test performance while relying on demographic or acquisition-related shortcuts. We propose counterfactual (CF) marginalisation as a test-time evaluation procedure for assessing robustness of classification models to such variables. Given a CF image generator, we intervene on nuisance parent variables such as age or sex, generate CF versions of each test image, and average predictions over a target intervention distribution. This produces intervention-aware predictions that marginalise demographic effects while preserving patient-specific latent information. We use these predictions to define metrics for CF risk, calibration, stability and worst-case sensitivity. We demonstrate this framework's utility for quantitative robustness evaluation.

Yasin Ibrahim, Hermione Warr, Robin J. Evans +1

Department of Engineering Science, University of Oxford, Oxford, UK · Department of Statistics, University of Oxford, Oxford, UK

Counterfactual Evaluation Classification Spurious Correlation Robustness Robust Classification

Sep 2, 2026

IFW-BLS: Dual-Robust Broad Learning System with Intuitionistic Fuzzy Wave Loss

Broad Learning System is an efficient randomized learning model that expands network width through feature and enhancement nodes and estimates the output weights without deep backpropagation. Its standard least-squares training, however, is vulnerable in two different ways: (i) large residuals caused by noise, outliers, or corrupted labels can dominate the objective, and (ii) all samples are treated as equally reliable even when some lie in ambiguous or locally conflicting regions. This paper proposes IFW-BLS, an Intuitionistic Fuzzy Wave Broad Learning System that addresses these two sources of fragility within one optimization model. The first robustness mechanism is residual-level protection, obtained by replacing the squared loss with the bounded, smooth, and asymmetric wave loss. Boundedness prevents extreme residuals from receiving unbounded influence, while asymmetry allows positive and negative deviations to be penalized differently when the dominant error direction varies. The second mechanism is sample-level credibility control, obtained through intuitionistic fuzzy scores that combine global class-center consistency with local neighborhood conflict. The resulting model evaluates the wave loss on credibility-weighted residuals, so unreliable samples are down-weighted before the bounded loss further limits the effect of extreme errors. A Nesterov accelerated gradient based optimizer is used to solve the proposed objective, avoiding the explicit matrix inversion used in conventional BLS. Experiments on UCI benchmark datasets validate the superiority of the proposed IFW-BLS model over the baseline models; additional corruption experiments also show more stable performance than BLS under noise and outlier contamination.

Mushir Akhtar, M. Tanveer

Indian Institute of Technology Indore, Simrol, Indore, India

Robust Classification Robust Loss Functions

Aug 30, 2026

Robust Broad Learning System with Wave Loss for Classification under Data Uncertainty

Broad Learning System (BLS) offers an efficient alternative to deep architectures by enabling fast learning through randomized feature mapping and closed-form solutions. However, its reliance on squared error loss makes it highly sensitive to noise, outliers, and corrupted labels, limiting its reliability in real-world scenarios. To address this limitation, we propose Wave-BLS, a robust broad learning framework that integrates the wave loss function, which is asymmetric, bounded, and smooth, enabling controlled penalization of large errors. The proposed formulation replaces the standard least-squares objective with a wave-loss-based optimization problem, solved efficiently using a Nesterov accelerated gradient (NAG)-based scheme without requiring matrix inversion, thereby improving scalability. Extensive experiments on 30 UCI benchmark datasets demonstrate that Wave-BLS consistently outperforms classical BLS and several robust variants. Statistical validation using Friedman and Nemenyi post-hoc tests confirms the significance of the observed improvements. Furthermore, robustness evaluations under controlled noise and outlier injection reveal that Wave-BLS exhibits substantially slower performance degradation compared to BLS, even in challenging contamination settings. These results establish Wave-BLS as a stable and robust alternative to existing broad learning models for learning under data uncertainty.

Mushir Akhtar, A. Varshney, A. Quadir +3

Department of Mathematics Indian Institute of Technology Indore

Robust Loss Functions Learning with Noisy Labels Robust Classification

Aug 13, 2026

Bagging Robustly Learns VC Classes with Linear Sample Complexity

We revisit the problem of learning predictors robust to adversarial examples at test-time. We prove that VC classes are adversarially robustly learnable with sample complexity linear in the VC dimension

d

, providing an exponential improvement over the previous upper bound of Montasser, Hanneke, and Srebro (2019). Remarkably, this result is achieved with a simple improper algorithm that combines the classic heuristic bagging (bootstrap aggregation) of Breiman (1996) with robust empirical risk minimization (RERM). Our algorithm computes RERMs on

O(d^\star)

independent bootstrap samples and outputs their majority vote, where

d^\star

denotes the dual VC dimension. We complement this result with a lower bound showing that this is unavoidable: in general, any learner in this oracle model requires

Ω(d^\star)

calls to an RERM oracle, even when given arbitrarily many training examples.

Omar Montasser

Yale University

Adversarial Robustness Classification Statistical Learning Theory Robust Classification

Aug 13, 2026

ProME: Prototype-Margin Environments with Repair-Aware Selection for Group-Robust Learning

Group-robust learning is crucial for maintaining accuracy on rare subpopulations when training-group labels are unavailable. However, existing methods often infer environments from a separate reference model and select representations before fitting the classifier used at deployment, leaving both decisions misaligned with the deployed predictor. In this work, we formulate group robustness without training-group labels as the endogenous environments with repair-aware selection (ERAS) problem, and propose ProME (Prototype-Margin Environments) to align both decisions with the deployed predictor. ProME splits prototype margins at their median to construct approximately balanced environments along the training trajectory, and fits a group-balanced linear head on group-annotated validation data to rank the resulting predictors by validation worst-group accuracy. We theoretically bound the worst risk across the inferred environments for a fixed predictor and partition, showing that this bound transfers to the oracle groups under an explicit alignment condition. Extensive experiments show that prototype margins enrich shortcut-conflicting examples, classifier repair reshapes candidate evaluation, and ProME achieves the highest average worst-group accuracy among the compared methods with the same group-label access.

Qianqian Wang, Yunshan Li, Dawei Huang +2

aShenzhen Key Laboratory of Safety and Security for Next Generation of Industrial Internet, Shenzhen, China · bDepartment of Statistics and Data Science, Southern University of Science and Technology, Shenzhen, China

Distribution Shift Robustness Spurious Correlation Robustness Robust Classification

Aug 9, 2026

No Unique Minimizer, No Problem: On the Consistency of Robust Neural Classifiers

Neural network classifiers trained by cross-entropy minimization are highly sensitive to label noise and adversarial contamination. While robust alternatives offer bounded influence and resistance to corruption, their statistical foundations in the deep learning setting are insufficient due to a fundamental difficulty: neural parameterizations are non-identifiable, so the population loss minimizer is an equivalence class of parameters, not a unique point. We develop a consistency theory for robust neural classifiers based on the S-divergence family that requires no identifiability assumption. Casting training as stochastic optimization over a non-identifiable parameter space, we prove that empirical S-divergence minimizers converge to the population-optimal equivalence class under mild regularity conditions, and verify these conditions for three architecture choices. We further establish that limit points of the robust training algorithm are stationary points of the empirical objective. Experiments on vision and language benchmark datasets confirm that S-divergence training maintains clean-data accuracy while exhibiting performance competitive with existing robust methods.

Subhabrata Majumdar, Anand Deo, Partha Pratim Saha +1

Indian Institute of Management Bangalore · Independent Researcher · Indian Statistical Institute, Kolkata

Learning with Noisy Labels Neural Network Robustness Robust Loss Functions Neural Network Generalization Statistical Learning Theory Robust Estimation+1

Jul 11, 2026

When Data Imbalance Helps: Robust Generalization Through Shortcut Saturation

We study robust generalization under spurious correlations: tasks where a shortcut feature is correlated with the true label in training but anti-correlated in an adversarial held-out split. Varying the spurious ratio

r

(the fraction of training examples where shortcut = true label) and model capacity, we find a counterintuitive result: data imbalance promotes generalization in sufficiently capable models. On a synthetic task where the true label is sum parity of an integer sequence and the shortcut is the parity of the maximum-valued element, a 2-layer, 2-head transformer generalized (reached

100\%

adversarial accuracy) in 0% of seeds at

r{=}0.50

but 77% of seeds at

r{=}0.90

. The effect is absent in 1-layer models, where imbalance instead traps the model on the shortcut. Through mechanistic analysis -- gradient conflict dynamics, circuit evolution, and QK/OV circuit ablations -- we characterize a mechanistic pathway consistent with imbalance promoting generalization.

Cheng-Ting Chou, Duc Binh Hoang

University of California, Los Angeles

Class-Imbalanced Learning Transformer Neural Network Robustness Classification Spurious Correlation Robustness Shortcut Learning+3

Jul 7, 2026

Optimized Instance Alteration for Explaining and Assessing Robustness of Classifiers

In this work, we propose a unified approach for diagnosing misclassification and assessing the robustness of black-box classifiers. Central to our method is an optimization framework that modifies an instance so that the classifier predicts a specified target label, while ensuring that the modification remains easily explainable. The objective function contains two components: an explainability-aware

L_0

(XA-

L_0

) penalty that promotes sparse and interpretable modifications, and a classifier loss objective that steers the perturbed instance toward the desired output. This integrated optimization formulation is used both to identify the underlying causes of misclassification and to evaluate robustness by determining how an instance can change within a tolerance region before being reassigned to another class. To quantify robustness, we introduce the Tolerance Region Confusion Matrix (TOR-Confusion Matrix), which measures a classifier's susceptibility by modeling the class-to-class transition probabilities induced by tolerance-bounded perturbations. We validate the proposed method on both image and tabular datasets, demonstrating its ability to jointly deliver interpretability and robustness assessment.

Evgenii Kuriabov, David Miller, Jia Li

Department of Statistics, The Pennsylvania State University, University Park, PA 16802, USA · Department of Electrical Engineering, The Pennsylvania State University, University Park, PA 16802, USA

Counterfactual Explanations Adversarial Robustness Black-Box Adversarial Attacks Explainability Evaluation Robust Classification

Jun 22, 2026

Discovering Latent Groups for Robust Classification

Machine learning models exploit spurious correlations, achieving high average accuracy but failing disproportionately on underrepresented subgroups. Existing methods address this by adjusting network parameters, guided either by subgroup annotations or inferred pseudo-group labels. Yet at inference, these methods produce only a class prediction, with no insight into a sample's latent subgroup. We propose neural classification trees (NCT), a framework that achieves robustness by encoding subgroup structure in its tree-shaped architecture. By routing each sample to an "easy" or "hard" node of this tree -- based on prediction correctness -- and reusing these routes as pseudo-labels for the next iteration, NCT disentangles conflicting subgroups, without requiring subgroup supervision. We evaluate NCT on five benchmarks spanning binary and multi-class spurious correlations. Our experiments show that the learned tree topology provides strong interpretability by consistently isolating minority subgroups, which provides a transparent mapping between the model architecture and the data's latent group structure, while yielding competitive robustness with state-of-the-art methods.

Ankur Garg, Ulrich Aïvodji, Samira Ebrahimi Kahou +1

Interpretable ML Neural Network Robustness Classification Spurious Correlation Robustness Robust Classification

Jun 15, 2026

Neural Bayesian Anomaly Mitigation: A Robust Loss that Doubles as an Unsupervised Contamination Classifier

Engineered robust losses such as Huber, Student-

t

, and generalised cross-entropy make supervised models tolerant of contamination but cannot answer which observations are corrupted. We introduce Neural Bayesian Anomaly Mitigation (NBAM), a general-purpose drop-in loss derived from a Bayesian latent-switch mixture model: the marginal likelihood defines a robust supervised loss, and the associated posterior defines an unsupervised contamination classifier. Like Huber or Student-

t

, NBAM can replace the standard training loss in any supervised pipeline; unlike them, it additionally learns a structured contamination model and returns a calibrated per-sample contamination posterior. A learned input-dependent prior

π_φ(x)

captures the spatial locality of contamination, so that samples near known corruptions are more likely to be flagged, while an Occam penalty emerges automatically and regularises against over-flagging. On CIFAR-10 with asymmetric label contamination, NBAM recovers the structure of the corruption process without supervision: the contamination posterior separates clean from corrupted samples, and the learned anomaly head identifies the direction of every label-flip pair. Alongside these capabilities, NBAM outperforms the four robust-loss baselines considered here at contamination rates 0.2-0.6.

S. A. K. Leeney, W. J. Handley, H. T. J. Bevins +1

Astrophysics Group, Cavendish Laboratory University of Cambridge · Institute of Astronomy University of Cambridge

Neural Network Robustness Noisy-Label Learning Robust Loss Functions Corruption Robustness Unsupervised Anomaly Detection Classification+2

Jun 11, 2026

Distributional Loss for Robust Classification

This paper proposes a novel loss concept for supervised classification tasks. Rather than enforcing a direct mapping from each input sample to a single assigned label, we define an optimization objective over all classifier outputs as a bimodal Gaussian distribution. This softer target formulation implicitly captures class ambiguity, mitigates overfitting, and encourages the learning of more robust decision boundaries, all without requiring additional label information. Experimental results demonstrate consistent improvements in robustness, with particularly pronounced gains in low-data regimes, while requiring only minimal modifications to standard training pipelines.

Kathleen Anderson, Thomas Martinetz

Institute for Neuro- and Bioinformatics, University of Lübeck, Germany

Robust Loss Functions Classification Robust Classification

May 20, 2026

Polynomial-Time Robust Multiclass Linear Classification under Gaussian Marginals

We study the task of agnostic learning of multiclass linear classifiers under the Gaussian distribution. Given labeled examples

(x, y)

from a distribution over

\mathbb{R}^d \times [k]

, with Gaussian

x

-marginal, the goal is to output a hypothesis whose error is comparable to that of the best

k

-class linear classifier. While the binary case

k=2

has a well-developed algorithmic theory, much less is known for

k \ge 3

. Even for

k=3

, prior robust algorithms incur exponential dependence on the inverse of the desired accuracy in both complexity and representation size. In this work, we develop new structural results for multiclass linear classifiers and use them to design fully polynomial-time robust learners with dimension-independent error guarantees. Our first result shows that the standard multiclass perceptron algorithm requires super-polynomially many samples and updates, even with clean labels and Gaussian marginals, revealing a basic obstruction absent in the binary case. Our main positive result is a pairwise improper-learning framework which yields an efficient learner with error

\widetilde O(k^{3/2}\sqrt{\mathrm{opt}})+ε

for general

k

. Additionally, we develop a sharper localization-based framework which leads to error

O(\mathrm{opt})+ε

for

k=3

, and error

\mathrm{poly}(k)\mathrm{opt}+ε

for geometrically regular

k

-class linear classifiers.

Ilias Diakonikolas, Giannis Iakovidis, Mingchen Ma

University of Wisconsin-Madison

Agnostic Learning Multiclass Classification Classification Statistical Learning Theory Linear Classification Robust Classification

May 20, 2026

Holistic Reliability Propagation: Decoupling Annotation and Prediction for Robust Noisy-Label

Learning with noisy labels in multimedia classification often combines external annotations and model predictions into a single reliability weight, even though the two sources can fail for different reasons. We instead estimate disentangled reliabilities: bilevel meta-learning produces two batch-normalized scalars per sample, alpha for the given label and beta for the pseudo-label, without constraining them to sum to one. Holistic Reliability Propagation (HRP) then routes them to different objectives, using reliability-aware Mixup with global gating on the input branch and beta-gated pseudo-label positives on the contrastive branch. On synthetic and real-world benchmarks, HRP improves average accuracy over strong baselines and remains competitive at the highest noise rates.

Jingyang Mao, Ningkang Peng, Yanhui Gu

Nanjing Normal University Nanjing, China

Classification Learning with Noisy Labels Noise Robustness Mixup Robust Classification

May 19, 2026

Symmetrization of Loss Functions for Robust Training of Neural Networks in the Presence of Noisy Labels

Labeling a training set is often expensive and susceptible to errors, making the design of robust loss functions for label noise an important problem. The symmetry condition provides theoretical guarantees for robustness to such noise. In this work, we study a symmetrization method arising from the unique decomposition of any multi-class loss function into a symmetric component and a class-insensitive term. In particular, symmetrizing the cross-entropy loss leads to a linear multi-class extension of the unhinged loss. Unlike in the binary case, the multi-class version must have specific coefficients in order to satisfy the symmetry condition. Under suitable assumptions, we show that this multi-class unhinged loss is the unique convex multi-class symmetric loss. We also show that it has a fundamental local role: the linear approximation of any symmetric loss around score vectors with equal components is equivalent to the multi-class unhinged loss. We then introduce SGCE and alpha-MAE, two loss functions that interpolate between the multi-class unhinged loss and the Mean Absolute Error while allowing control of the beta-smoothness of the loss. Experiments on standard noisy-label benchmarks show competitive performance compared with existing robust loss functions.

Alexandre Lemire Paquin, Brahim Chaib-Draa, Philippe Giguère

Department of Computer Science and Software Engineering Université Laval, Québec, Canada

Neural Network Robustness Noisy-Label Learning Robust Loss Functions Multiclass Classification Robust Classification

May 13, 2026

Shortcut Mitigation via Spurious-Positive Samples

Shortcut mitigation strategies commonly rely on training data annotations, group-balanced held-out data or the presence of all groups, i.e., all combinations of (spurious) attributes and classes, in the training data. However, these requirements are rarely met in practice. We instead propose a method for targeted model analysis to identify a small set of instances in which the model relies on spurious attributes. Using that set and following ``this feature should not be used for prediction'' reasoning, we identify highly relevant neurons in an intermediate layer and regularize their impact. This ensures that models learn to depend on informative features rather than being right for the wrong reasons, thereby improving robustness without requiring additional balanced held-out data or annotations.

Phuong Quynh Le, Jörg Schlötterer, Sari Sadiya +2

University of Marburg · Goethe University Frankfurt

Neural Network Robustness Spurious Correlation Robustness Shortcut Learning Spurious Correlation Robust Classification

May 12, 2026

Robust Biomedical Publication Type and Study Design Classification with Knowledge-Guided Perturbations

Accurately and consistently indexing biomedical literature by publication type and study design is essential for supporting evidence synthesis and knowledge discovery. Prior work on automated publication type and study design indexing has primarily focused on expanding label coverage, enriching feature representations, and improving in-domain accuracy, with evaluation typically conducted on data drawn from the same distribution as training. Although pretrained biomedical language models achieve strong performance under these settings, models optimized for in-domain accuracy may rely on superficial lexical or dataset-specific cues, resulting in reduced robustness under distributional shift. In this study, we introduce an evaluation framework based on controlled semantic perturbations to assess the robustness of a publication type classifier and investigate robustness-oriented training strategies that combine entity masking and domain-adversarial training to mitigate reliance on spurious topical correlations. Our results show that the commonly observed trade-off between robustness and in-domain accuracy can be mitigated when robustness objectives are designed to selectively suppress non-task-defining features while preserving salient methodological signals. We find that these improvements arise from two complementary mechanisms: (1) increased reliance on explicit methodological cues when such cues are present in the input, and (2) reduced reliance on spurious domain-specific topical features. These findings highlight the importance of feature-level robustness analysis for publication type and study design classification and suggest that refining masking and adversarial objectives to more selectively suppress topical information may further improve robustness. Data, code, and models are available at: https://github.com/ScienceNLP-Lab/MultiTagger-v2/tree/main/ICHI

Shufan Ming, Joe D. Menke, Neil R. Smalheiser +1

School of Information Sciences University of Illinois Urbana-Champaign Champaign, USA · Department of Psychiatry University of Illinois Chicago Chicago, USA

Adversarial Training Distribution Shift Robustness Text Classification Spurious Correlation Robustness Robust Classification

May 3, 2026

Large margin classifier with graph-based adaptive regularization

This paper introduces the use of per-class regularization hyperparameters in Gabriel graph-based binary classifiers. We demonstrate how the quality index used for regularization behaves both in the margin region and in the presence of outliers, and how incorporating this regularization flexibility can lead to solutions that effectively eliminate outliers while training the classifier. We also show how it can address class imbalance by generating higher and lower thresholds for the majority and minority classes, respectively. Thus, rather than having a single solution based on fixed thresholds, flexible thresholds expand the solution space and can be optimized through hyperparameter tuning algorithms. Friedman test shows that flexible thresholds are capable of improving Gabriel graph-based classifiers.

Vítor M. Hanriot, Turíbio T. Salis, Luiz C. B. Torres +2

Graduate Program in Electrical Engineering - Universidade Federal de Minas Gerais - Av. Antônio Carlos 6627, 31270-901, Belo Horizonte, MG, Brazil · Department of Computer and Systems - Universidade Federal de Ouro Preto - João Monlevade 35931-022, Brazil.

Class-Imbalanced Learning Binary Classification Robust Classification

Apr 24, 2026

Useful nonrobust features are ubiquitous in biomedical images

We study whether deep networks for medical imaging learn useful nonrobust features - predictive input patterns that are not human interpretable and highly susceptible to small adversarial perturbations - and how these features impact test performance. We show that models trained only on nonrobust features achieve well above chance accuracy across five MedMNIST classification tasks, confirming their predictive value in-distribution. Conversely, adversarially trained models that primarily rely on robust features sacrifice in-distribution accuracy but yield markedly better performance under controlled distribution shifts (MedMNIST-C). Overall, nonrobust features boost standard accuracy yet degrade out-of-distribution performance, revealing a practical robustness-accuracy trade-off in medical imaging classification tasks that should be tailored to the requirements of the deployment setting.

Coenraad Mouton, Randle Rabe, Niklas C. Koser +4

Section Biomedical Imaging, Department of Radiology and Neuroradiology, University Hospital Schleswig-Holstein, Kiel University · Faculty of Engineering, North-West University, South Africa

Healthcare Neural Network Robustness Adversarial Robustness OOD Generalization Medical Image Classification Robust Classification

Apr 18, 2026

CCAR: Intrinsic Robustness as an Emergent Geometric Property

Standard supervised learning optimizes for predictive accuracy but remains agnostic to the internal geometry of learned features, often yielding representations that are entangled and brittle. We propose Class-Conditional Activation Regularization (CCAR) to explicitly engineer the feature space, imposing a block-diagonal structure via a soft inductive bias. By shaping the latent representation to confine class energy to orthogonal subspaces, we create an intrinsic geometric scaffold that naturally filters noise and adversarial perturbations. We provide theoretical analysis linking this structural constraint to the maximization of the Fisher Discriminant Ratio, establishing a formal connection between geometric disentanglement and algorithmic stability. Empirically, this approach demonstrates that robustness is an emergent property of a well-engineered feature space, significantly outperforming baselines on label noise and input corruption benchmarks.

Akash Samanta, Manish Pratap Singh, Debasis Chaudhuri

Techno India University, Salt Lake, Kolkata, India · DRDO Young Scientist Laboratory - CT, Chennai, India

Disentangled Representation Learning Representation Learning Algorithmic Stability Neural Network Robustness Adversarial Robustness Orthogonal Representation Learning+3

Apr 14, 2026

GF-Score: Certified Class-Conditional Robustness Evaluation with Fairness Guarantees

Adversarial robustness is essential for deploying neural networks in safety-critical applications, yet standard evaluation methods either require expensive adversarial attacks or report only a single aggregate score that obscures how robustness is distributed across classes. We introduce the GF-Score (GREAT-Fairness Score), a framework that decomposes the certified GREAT Score into per-class robustness profiles and quantifies their disparity through four metrics grounded in welfare economics: the Robustness Disparity Index (RDI), the Normalized Robustness Gini Coefficient (NRGC), Worst-Case Class Robustness (WCR), and a Fairness-Penalized GREAT Score (FP-GREAT). The framework further eliminates the original method's dependence on adversarial attacks through a self-calibration procedure that tunes the temperature parameter using only clean accuracy correlations. Evaluating 22 models from RobustBench across CIFAR-10 and ImageNet, we find that the decomposition is exact, that per-class scores reveal consistent vulnerability patterns (e.g., ``cat'' is the weakest class in 76% of CIFAR-10 models), and that more robust models tend to exhibit greater class-level disparity. These results establish a practical, attack-free auditing pipeline for diagnosing where certified robustness guarantees fail to protect all classes equally. We release our code on GitHub.

Arya Shah, Kaveri Visavadiya, Manisha Padala

IIT Gandhinagar

Adversarial Robustness Certified Robustness Robust Classification Algorithmic Fairness Image Classification

Apr 7, 2026

Conformal Margin Risk Minimization: An Envelope Framework for Robust Learning under Label Noise

Most methods for learning with noisy labels require privileged knowledge such as noise transition matrices, clean subsets or pretrained feature extractors, resources typically unavailable when robustness is most needed. We propose Conformal Margin Risk Minimization (CMRM), a plug-and-play envelope framework that improves any classification loss under label noise by adding a single quantile-calibrated regularization term, with no privileged knowledge or training pipeline modification. CMRM measures the confidence margin between the observed label and competing labels, and thresholds it with a conformal quantile estimated per batch to focus training on high-margin samples while suppressing likely mislabeled ones. We derive a learning bound for CMRM under arbitrary label noise requiring only mild regularity of the margin distribution. Across five base methods and six benchmarks with synthetic and real-world noise, CMRM consistently improves accuracy (up to +3.39%), reduces conformal prediction set size (up to -20.44%) and does not hurt under 0% noise, showing that CMRM captures a method-agnostic uncertainty signal that existing mechanisms did not exploit.

Yuanjie Shi, Peihong Li, Zijian Zhang +2

School of EECS, Washington State University, Pullman, WA, USA

Margin-Based Learning Learning with Noisy Labels Robust Classification Robust Loss Functions Statistical Learning Theory

Apr 2, 2026

Robust Learning with Optimal Error

We construct algorithms with optimal error for learning with adversarial noise. The overarching theme of this work is that the use of randomized hypotheses can substantially improve upon the best error rates achievable with deterministic hypotheses. - For

η

-rate malicious noise, we show the optimal error is

\frac{1}{2} \cdot η/(1-η)

, improving on the optimal error of deterministic hypotheses by a factor of

1/2

. This answers an open question of Cesa-Bianchi et al. (JACM 1999) who showed randomness can improve error by a factor of

6/7

. - For

η

-rate nasty noise, we show the optimal error is

\frac{3}{2} \cdot η

for distribution-independent learners and

η

for fixed-distribution learners, both improving upon the optimal

2 η

error of deterministic hypotheses. This closes a gap first noted by Bshouty et al. (Theoretical Computer Science 2002) when they introduced nasty noise and reiterated in the recent works of Klivans et al. (NeurIPS 2025) and Blanc et al. (SODA 2026). - For

η

-rate agnostic noise and the closely related nasty classification noise model, we show the optimal error is

η

, improving upon the optimal

2η

error of deterministic hypotheses. All of our learners have sample complexity linear in the VC-dimension of the concept class and polynomial in the inverse excess error. All except for the fixed-distribution nasty noise learner are time efficient given access to an oracle for empirical risk minimization.

Guy Blanc

Stanford

Agnostic Learning Robust Classification Classification

Mar 24, 2026

Robustness Quantification for Discriminative Models: a New Robustness Metric and its Application to Dynamic Classifier Selection

Among the different possible strategies for evaluating the reliability of individual predictions of classifiers, robustness quantification stands out as a method that evaluates how much uncertainty a classifier could cope with before changing its prediction. However, its applicability is more limited than some of its alternatives, since it requires the use of generative models and restricts the analyses either to specific model architectures or discrete features. In this work, we propose a new robustness metric applicable to any probabilistic discriminative classifier and any type of features. We demonstrate that this new metric is capable of distinguishing between reliable and unreliable predictions, and use this observation to develop new strategies for dynamic classifier selection.

Rodrigo F. L. Lassance, Jasper De Bock

Foundations Lab for imprecise probabilities (FLip), Ghent University, Ghent, Belgium · Statistics Dept., Federal University of São Carlos, São Carlos, São Paulo, Brazil · Institute of Mathematic and Computer Sciences, University of São Paulo, São Carlos, São Paulo, Brazil

Adversarial Robustness Classification Robust Classification