Neural Network Robustness

Follow this topic Ask about this topic

Momentum

9 papers in the last four weeks, up 125% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 173

Mar 10, 2026eess.AS

Attack-Dependent Robustness of Neural Audio Codecs for Adversarial ASR

Neural audio codecs impose a discrete bottleneck through residual vector quantization (RVQ), making them a useful class of inference-time transformations for reducing adversarial perturbations before ASR inference. We study how codec quantization depth affects defended ASR under non-adaptive, standard adaptive, and quantization-aware adaptive untargeted

\ell_\infty

attacks. Under non-adaptive attacks, intermediate RVQ depths yield the lowest word error rates and outperform traditional compression at comparable bitrates. However, this apparent optimum is not stable under adaptive evaluation. The standard identity-gradient adaptive baseline (BPDA+EOT) can overestimate robustness, while an implementation of an RVQ-relaxed adaptive attack (SoftVQ-PGD) substantially changes the observed depth trend and largely removes the intermediate-depth advantage. Overall, neural codecs can improve defended ASR under specific threat models. However, the relationship between robustness and RVQ depth depends on the attack used for evaluation, rather than on the codec architecture alone.

Jordan Prescott, Thanathai Lertpetchpun, Shrikanth Narayanan

Signal Analysis and Interpretation Laboratory, University of Southern California, USA

ASR Evaluation Neural Audio Codecs Residual VQ Neural Network Robustness Automatic Speech Recognition

Mar 6, 2026cs.LG

Lipschitz-Based Robustness Certification Under Floating-Point Execution

Lipschitz-based robustness certification bounds a network's sensitivity through concrete numerical computation rather than symbolic reasoning, and so scales efficiently. It is increasingly used even where verifiable guarantees matter. Yet, as with most prior work on robustness certification and verification, soundness is typically proved against a semantic model assuming exact real arithmetic. Deployed networks instead execute in floating-point, creating a gap between certified properties and executed behaviour. As motivating evidence, we give counterexamples showing that real arithmetic robustness guarantees can fail under floating-point execution, even for previously verified certifiers. We then develop a formal, compositional theory relating real arithmetic Lipschitz-based sensitivity bounds to floating-point execution under standard rounding-error models for feed-forward ReLU networks. We derive sound conditions for floating-point robustness, including bounds on certificate degradation and sufficient conditions for the absence of overflow. We also give an efficient floating-point Gram iteration algorithm for Lipschitz bounds and prove that it never under-estimates the true norm. Separately, when a model is certified pre-deployment, we show how measuring its actual deviation against a high-precision execution can substantially reduce certificate degradation. We formalise the theory and its soundness, and implement an executable certifier, evaluated across dense networks spanning image, tabular, and many-class classification. To our knowledge, ours is the first method for soundly accounting for floating-point effects in Lipschitz-based robustness certification, and, done efficiently, the first floating-point-sound robustness checking procedure of any kind to certify models' entire test sets -- even those with 500,000 examples -- while retaining enough precision to be practical.

Toby Murray

University of Melbourne, Australia

Neural Network Verification Neural Network Robustness Formal Verification Certified Robustness

Feb 6, 2026cs.AI

Same Answer, Different Representations: Hidden instability in VLMs

The robustness of Vision Language Models (VLMs) is commonly assessed through output-level invariance, implicitly assuming that stable predictions reflect stable multimodal processing. In this work, we argue that this assumption is insufficient. We introduce a representation-aware and frequency-aware evaluation framework that measures internal embedding drift, spectral sensitivity, and structural smoothness (spatial consistency of vision tokens), alongside standard label-based metrics. Applying this framework to modern VLMs across the SEEDBench, MMMU, and POPE datasets reveals three distinct failure modes. First, models frequently preserve predicted answers while undergoing substantial internal representation drift; for perturbations such as text overlays, this drift approaches the magnitude of inter-image variability, indicating that representations move to regions typically occupied by unrelated inputs despite unchanged outputs. Second, robustness does not improve with scale; larger models achieve higher accuracy but exhibit equal or greater sensitivity, consistent with sharper yet more fragile decision boundaries. Third, we find that perturbations affect tasks differently: they harm reasoning when they disrupt how models combine coarse and fine visual cues, but on the hallucination benchmarks, they can reduce false positives by making models generate more conservative answers.

Farooq Ahmad Wani, Alessandro Suglia, Rohit Saxena +6

Sapienza University of Rome · University of Edinburgh · University of Oxford +5

VLM Evaluation VLM Robustness Vision-Language Models Neural Network Robustness Adversarial Robustness

Feb 6, 2026cs.LG

Perturbing the Phase: Analyzing Adversarial Robustness of Complex-Valued Neural Networks

Complex-valued neural networks (CVNNs) are rising in popularity for all kinds of applications. To safely use CVNNs in practice, analyzing their robustness against outliers is crucial. One well known technique to understand the behavior of deep neural networks is to investigate their behavior under adversarial attacks, which can be seen as worst case minimal perturbations. We design Phase Attacks, a kind of attack specifically targeting the phase information of complex-valued inputs. Additionally, we derive complex-valued versions of commonly used adversarial attacks. We show that in some scenarios CVNNs are more robust than RVNNs and that both are very susceptible to phase changes with the Phase Attacks decreasing the model performance more, than equally strong regular attacks, which can attack both phase and magnitude.

Florian Eilers, Christof Duhme, Xiaoyi Jiang

Department of Computer Science University of Münster Münster, Germany

Adversarial Attacks Neural Network Robustness Adversarial Robustness Complex-Valued Neural Networks

Feb 3, 2026cs.LG

Robustness as an Emergent Property of Task Performance

Robustness is widely viewed as a key challenge for real-world applications. However, because current research focuses only on difficult tasks, it partially captures real-world readiness. In this paper, we argue and verify that robustness, defined as consistency across semantically equivalent inputs, closely follows task difficulty: once models master a task, robustness emerges naturally. Through an empirical analysis of multiple models across diverse datasets and configurations (e.g., paraphrases, temperature changes), we observe a strong positive correlation between task performance and robustness. Furthermore, our findings indicate that robustness is driven primarily by task-specific competence rather than inherent model attributes, challenging the common view of robustness as an independent capability. This perspective implies that as tasks mature and model performance saturates, robustness on those tasks will similarly emerge. For researchers, this suggests that explicit efforts to measure robustness may deserve reduced emphasis, as robustness is likely to improve alongside performance. For practitioners, it signals that while many existing benchmarks are still unstable, models are already reliable on earlier tasks and suitable for deployment.

Shir Ashury-Tahan, Ariel Gera, Elron Bandel +2

IBM Research · MIT · Weizmann Institute of Science

Neural Network Robustness Language Model Robustness

Dec 1, 2025cs.LG

Dual Randomized Smoothing: Beyond Global Noise Variance

Randomized Smoothing (RS) is a prominent technique for certifying the robustness of neural networks against adversarial perturbations. With RS, achieving high accuracy at small radii requires a small noise variance, while achieving high accuracy at large radii requires a large noise variance. However, the global noise variance used in the standard RS formulation leads to a fundamental limitation: there exists no global noise variance that simultaneously achieves strong performance at both small and large radii. To break through the global variance limitation, we propose a dual RS framework which enables input-dependent noise variances. To achieve that, we first prove that RS remains valid with input-dependent noise variances, provided the variance is locally constant around each input. Building on this result, we introduce two components: (i) a variance estimator predicts an optimal noise variance for each input, (ii) this estimated variance is then used by a standard RS classifier. The variance estimator is independently smoothed via RS to ensure local constancy, enabling flexible design. We also introduce training strategies to iteratively optimize the two components. Experiments on CIFAR-10 demonstrate that our dual RS method provides strong performance for both small and large radii-unattainable with global noise variance-while incurring only a 60% computational overhead at inference. Moreover, it outperforms prior input-dependent noise approaches across most radii, with gains at radii 0.5, 0.75, and 1.0 of 15.6%, 20.0%, and 15.7%. On ImageNet, dual RS remains effective across all radii, with advantages of 8.6%, 17.1%, and 9.1% at radii 0.5, 1.0, and 1.5. Additionally, the dual RS framework provides a routing perspective for certified robustness, improving the accuracy-robustness trade-off with off-the-shelf expert RS models.

Chenhao Sun, Yuhao Mao, Martin Vechev

Department of Computer Science, ETH Zürich, Switzerland

Randomized Smoothing Neural Network Robustness Adversarial Robustness Certified Robustness Noise Robustness

Nov 24, 2025cs.CV

Diagnosing Corruption-Induced Reliability Failures in Vision-Language Models

Visual corruptions can change vision--language model (VLM) behavior in ways that top-1 accuracy does not capture. A model may keep the same answer while losing distributional support, or improve accuracy through unstable wrong-to-correct changes. We introduce Bench-C, a controlled multiple-choice testbed for studying these effects. It selects semantically diverse samples whose predictions respond to corruption, and evaluates them under 19 corruption types and five severity levels. To measure how corruption changes the option distribution, we introduce the Robustness Alignment Score (RAS), which combines confidence-correctness alignment with uncertainty direction. We further separate originally correct samples from originally wrong samples, and track whether changes are temporary or persistent across severity. Experiments across 13 VLMs reveal a counterintuitive pattern: mild corruptions can improve top-1 accuracy while degrading prediction structure. These failures include silent degradation, erroneous overconfidence, and severity-dependent persistence. Bench-C therefore supports robustness evaluation that goes beyond final answers and attributes where reliability changes occur. Code and data are available at https://github.com/xiangjieSui/Bench-C.

Xiangjie Sui, Songyang Li, Hanwei Zhu +3

City University of Macau, Macau SAR, China · Jiangxi University of Finance and Economics, Nanchang, China · Nanyang Technological University, Singapore

VLM Evaluation VLM Robustness Probability Calibration Image Corruption Robustness Neural Network Robustness Corruption Robustness

Nov 23, 2025cs.SD

DHAuDS: A Dynamic and Heterogeneous Audio Benchmark for Test-Time Adaptation

Existing Test-time Adaptation (TTA) studies rely heavily on static and homogeneous corruption protocols, such as ImageNet-C and CIFAR-10-C/100-C, leading to inconsistent evaluation settings and potentially inflated robustness estimates that are compared with real-world situations. TTA lacks a standardized evaluation infrastructure capable of modeling realistic heterogeneous acoustic degradation. We introduce DHAuDS, a standardized benchmark suite for evaluating audio classification TTA robustness under dynamic corruption severity and heterogeneous noise mixtures. Rather than proposing a new TTA algorithm, DHAuDS focuses on exposing robustness limitations that remain hidden under conventional fixed-noise evaluation protocols.

Weichuang Shao, Iman Yi Liao, Tomas Henrique Bode Maul +1

School of Computer and Mathematical Sciences, University of Nottingham Malaysia, Semenyih, Malaysia

Test-Time Adaptation Neural Network Robustness Noise Robustness Audio Classification

Nov 13, 2025cs.CL

destroR: A Benchmark and Adversarial-Training Defense for Bangla Transfer Models under Meaning-Preserving Attacks

Transformer-based transfer models now dominate Bangla sentiment classification, yet their adversarial robustness remains largely unexamined, and no prior study pairs a Bangla attack suite with a defense that measurably recovers robustness. We address this gap with destroR, a unified pipeline for evaluating and hardening Bangla text classifiers. First, we introduce three meaning-preserving Bangla attack recipes a paraphrase attack, a back-translation attack, and a one-hot word-swap attack that perturb inputs while regenerating fluent, semantically faithful sentences, inducing model prediction perplexity rather than input noise. Second, we construct a robustness benchmark that evaluates five transfer models (BanglaBERT, BanglishBERT, XLM-RoBERTa, MuRIL, and IndicBERTv2) across four datasets against five attacks, placing our recipes against two strong word-substitution baselines, TextFooler and BAE, under an identical protocol. Third, we harden every model through adversarial training and report a full robustness matrix. Our analysis yields three findings: word-substitution baselines are more potent than semantically constrained recipes (BAE reaches a 54.2% attack success rate); adversarial training on the union of all attack families lowers residual attack success for every attack; and, contrary to expectation, the Indic-multilingual MuRIL backbone is markedly more robust than the Bangla-dedicated models. All models, adversarial data, and code are released for full reproducibility.

Saadat Rafid Ahmed, Rubayet Shareen, Radoan Sharkar +3

Dept. of Computer Science and Engineering, BRAC University

Adversarial Training Sentiment Analysis Neural Network Robustness Adversarial Robustness Adversarial Robustness of Language Models Adversarial Text Attacks

Oct 21, 2025cs.LG

Solver-Integrated Adversarial Attacking and Training of Neural Operators

Neural operators are widely used as fast surrogates for numerical PDE solvers, mapping input functions to solution functions. However, their generalizability and robustness are not yet clearly defined in the operator-learning setting, which differs from traditional adversarial robustness definitions. This paper studies the generalizability and robustness of a learned neural operator from a solver-integrated perspective, addressing the challenge that the output of a learned operator and a numerical solver tends to change in tandem under input perturbation. First, we formalize the definition of generalization and robustness through a model-solver error operator, identifying fixed-input model-solver loss as generalization metric, and norm-bounded adversarial attack loss increase and Jacobian-error function norm as robustness metric. Second, we identify the solver-integrated adversarial attack as appropriate for PDE operator learning and show why model-only or fixed-ground-truth attacks can be insufficient when the solver output also changes with the input. Third, we develop solver-integrated adversarial training methods for neural operators. Experiments on representative PDE benchmarks show that this solver-integrated adversarial training clearly improves both generalizability and robustness. Deeper solver integration yields more effective attacks, more informative samples, and more efficient training than less integrated alternatives. These results provide a general framework for robust operator training and automatic sample selection without heavy manual intervention. More broadly, the formulation applies to adversarial regression whenever a ground-truth oracle can evaluate, and ideally differentiate, the true input-output map; PDE operator learning is one such case.

Yifei Sun

Adversarial Training Adversarial Attacks Neural Network Robustness Adversarial Robustness PDE Operator Learning Neural Operators

Sep 29, 2025cs.CV

Improved Robustness from Biologically Inspired Sparse Contrast Representations

Deep neural networks surpass humans on many vision benchmarks, yet remain far less robust to distribution shifts such as illumination and weather changes. Existing approaches address this challenge by additional training data, extensive augmentation, architectural modifications, or test-time adaptation. In this work, we explore a complementary direction: inspired by the human retina, we propose a fixed, model-agnostic preprocessing module that extracts signals that are more stable with respect to variations of illumination. Our method combines color remapping with local contrast extraction, producing sparse representations that emphasize structural features. We study its impact on semantic segmentation by training on Cityscapes and evaluating generalization under adverse conditions on Dark Zurich and ACDC. Our results show that the biologically inspired preprocessing preserves in-distribution performance while consistently improving robustness in challenging lighting scenarios, such as nighttime, where annotated training data are scarce. Moreover, the segmentation accuracy remains stable even when the contrast-based representation is sparsified by up to 70%. These gains suggest that rethinking the input representation itself can improve robustness while also opening opportunities for lower-latency, transmission-aware imaging sensors when sparsity can be exploited close to acquisition.

Lorena Stracke, Lia Nimmermann, Shashank Agnihotri +3

Media Systems, University of Siegen, Germany · Data and Web Science Group, University of Mannheim, Germany · Analogue Circuits and Image Sensors, University of Siegen, Germany +1

Distribution Shift Robustness Image Segmentation Image Corruption Robustness Visual Representation Learning Neural Network Robustness Semantic Segmentation

Sep 28, 2025cs.NE

Quantifying How Training Gradient Sparsity Affect Spiking Neural Network Accuracy And Robustness

Spiking Neural Networks (SNNs) have recently received increasing attention in both computational neuroscience and artificial intelligence owing to their potential for energy-efficient computation and reduced memory requirements. Despite these advantages, improving adversarial robustness in SNNs (particularly for vision-based applications) remains an emerging and relatively underexplored research problem. Recent work has suggested that encouraging sparse gradients can act as a regularization mechanism to improve resistance against adversarial perturbations. In this study, we report an unexpected observation: under certain architectural configurations, SNNs inherently exhibit sparse gradients and can attain state-of-the-art adversarial defense performance without requiring any explicit regularization strategy. Further investigation reveals an inherent trade-off between robustness and generalization. Specifically, increased gradient sparsity enhances resistance to adversarial attacks but may reduce the model's generalization capability, whereas denser gradients tend to improve generalization while simultaneously increasing susceptibility to adversarial perturbations. These findings provide new perspectives on the role of gradient sparsity in the training dynamics of SNNs.

Nhan Trong Luu, Duong Trung Luu

College of Information and Communication Technology, Can Tho University, Vietnam · College of Engineering and Computer Science, VinUniversity, Hanoi, Vietnam · Center of Digital Transformation and Communication, Can Tho University, Vietnam

Neural Network Generalization Neural Network Robustness Adversarial Robustness Spiking Neural Networks Neural Network Training Dynamics

Jul 23, 2025cs.LG

On the Interaction of Compressibility and Adversarial Robustness

As demands for resource efficiency and safety in modern neural networks intensify, substantial research effort has gone into model compression and adversarial robustness. Yet despite progress on each in isolation, a systematic understanding of how compressibility shapes robustness remains elusive. In this paper, we develop a principled framework to analyze how different forms of structured compressibility - such as neuron-level and spectral compressibility - affect adversarial robustness. We show that structured compressibility can induce a small number of highly sensitive directions in the representation space, which adversaries can exploit to construct effective perturbations. Our analysis yields a robustness bound that reveals how neuron and spectral compressibility impact

\ell_\infty

and

\ell_2

robustness via their effects on the learned representations. Crucially, the vulnerabilities we identify arise irrespective of how compressibility is achieved - whether via regularization, architectural bias, or learning dynamics. Through empirical evaluations across synthetic and realistic tasks, we confirm our theoretical predictions, and further demonstrate that these vulnerabilities persist under adversarial training and transfer learning, and contribute to the emergence of universal adversarial examples. Our findings show a fundamental tension between structured compressibility and robustness and highlight new pathways for designing models that are efficient and safe.

Melih Barsbey, Antônio H. Ribeiro, Umut Şimşekli +1

Department of Computing, Imperial College London, UK · Department of Information Technology, Uppsala University, Sweden · INRIA, CNRS, Département d’Informatique de l’Ecole Normale Supérieure / PSL, France

Neural Network Robustness Adversarial Robustness Adversarial Perturbations Neural Network Compression

Apr 1, 2025cs.LG

R2DN: Scalable Parameterization of Contracting and Lipschitz Recurrent Deep Networks

This paper presents the Robust Recurrent Deep Network (R2DN), a scalable parameterization of stable and robust recurrent neural networks for machine learning and data-driven control. We construct R2DNs as the feedback interconnection of a linear time-invariant system and a 1-Lipschitz deep feedforward network, and directly parameterize the weights so that our models are stable (contracting) and robust to input perturbations (Lipschitz) by design. Our parameterization uses a structure similar to the recurrent equilibrium network (REN), but without having to iteratively solve an equilibrium layer at each time-step. This speeds up model inference and training on GPUs, and makes it computationally feasible to scale up the network size and input sequence length in comparison to RENs. We compare R2DNs to RENs on representative problems in nonlinear system identification, observer design, learning-based feedback control, and sequential image classification. We find that training and inference are up to an order of magnitude faster with similar performance, and that they scale more favorably with respect to model expressivity.

Nicholas H. Barbara, Ruigang Wang, Ian R. Manchester

Australian Centre for Robotics (ACFR) and the School of Aerospace, Mechanical and Mechatronic Engineering, The University of Sydney, Australia

Neural Network Robustness Recurrent Neural Networks Lipschitz Neural Networks

Mar 19, 2025cs.CL

Unified Enhancement of the Generalization and Robustness of Language Models via Bi-Stage Optimization

Neural network language models (LMs) are confronted with significant challenges in generalization and robustness. Currently, many studies focus on improving either generalization or robustness in isolation, without methods addressing both aspects simultaneously, which presents a significant challenge in developing LMs that are both robust and generalized. In this paper, we propose a bi-stage optimization framework to uniformly enhance both the generalization and robustness of LMs, termed UEGR. Specifically, during the forward propagation stage, we enrich the output probability distributions of adversarial samples by adaptive dropout to generate diverse sub models, and incorporate JS divergence and adversarial losses of these output distributions to reinforce output stability. During backward propagation stage, we compute parameter saliency scores and selectively update only the most critical parameters to minimize unnecessary deviations and consolidate the model's resilience. Theoretical analysis shows that our framework includes gradient regularization to limit the model's sensitivity to input perturbations and selective parameter updates to flatten the loss landscape, thus improving both generalization and robustness. The experimental results show that our method significantly improves the generalization and robustness of LMs compared to other existing methods across 13 publicly available language datasets, achieving state-of-the-art (SOTA) performance.

Yudao Sun, Juan Yin, Juan Zhao +3

Department of New Networks, Peng Cheng Laboratory, Shenzhen 518055, China · Department of Industrial Systems Engineering and Management, National University of Singapore, Singapore 117576 · School of Data Science and Artificial Intelligence, Chang’an University, Shaanxi, 710064, China

Adversarial Training Neural Network Generalization Neural Network Robustness Adversarial Robustness of Language Models Language Model Robustness

Oct 28, 2024cs.CV

Large Pretraining Datasets Don't Guarantee Robustness after Fine-Tuning in Image Classification

Large-scale pretrained models are widely leveraged as foundations for learning new specialized tasks via fine-tuning, with the goal of maintaining the general performance of the model while allowing it to gain new skills. A valuable goal for all such models is robustness: the ability to perform well on out-of-distribution (OOD) tasks. We assess whether fine-tuning preserves the overall robustness of the pretrained model in image classification, and observed that models pretrained on large datasets exhibited strong catastrophic forgetting and loss of OOD generalization. To systematically assess robustness preservation in fine-tuned models, we propose the Robustness Inheritance Benchmark (ImageNet-RIB). The benchmark, which can be applied to any pretrained model, consists of a set of related but distinct OOD (downstream) tasks and involves fine-tuning on one of the OOD tasks in the set then testing on the rest. We find that though continual learning methods help, fine-tuning reduces robustness across pretrained models. Surprisingly, models pretrained on the largest and most diverse datasets (e.g., LAION-2B) exhibit both larger robustness losses and lower absolute robustness after fine-tuning on small datasets, relative to models pretrained on smaller datasets. We observe this collapse in contrastively pretrained (CLIP) models and their fine-tuned variants, where it grows with pretraining scale; the supervised models we test do not exhibit it. These findings suggest that starting with the strongest foundation model is not necessarily the best approach for performance on specialist tasks. https://jd730.github.io/projects/ImageNet-RIB

Jaedong Hwang, Brian Cheung, Zhang-Wei Hong +3

Massachusetts Institute of Technology

Neural Network Robustness OOD Generalization Vision Foundation Model Adaptation Image Classification Catastrophic Forgetting

Oct 10, 2024cs.LG

How Learning Dynamics Drive Adversarially Robust Generalization?

Despite being widely adopted as a canonical framework for learning robust models, adversarial training suffers from robust overfitting. Existing empirical and theoretical explorations fail to provide a satisfactory mechanistic interpretation of the phenomenon. By modeling adversarial training with momentum SGD as a discrete-time dynamical system, we propose a PAC-Bayesian analytical framework that proves time-resolved robust generalization bounds. Specifically, our framework tracks the closed-form evolution of the posterior mean and covariance under both stationary and non-stationary transient regimes, connecting the model's robust generalization performance to learning rate, local loss geometry, and mini-batch stochastic gradients. By estimating the key quantities associated with the bound, we illustrate the underlying mechanism of robust overfitting. Our framework also shows how adversarial weight perturbation reduces robust generalization gaps by suppressing dominant loss-curvature modes, while suggesting that excessive penalization can be sub-optimal for optimization.

Yuelin Xu, Xiao Zhang

CISPA Helmholtz Center for Information Security, Saarbrücken, Germany

Adversarial Training PAC-Bayesian Generalization Bounds Neural Network Robustness Adversarial Robustness Neural Network Training Dynamics

Aug 17, 2024cs.LG

Training Verifiably Robust Agents Using Set-Based Reinforcement Learning

Reinforcement learning policies parametrized by deep neural networks have achieved strong performance for continuous control, yet even small input perturbations may lead to unpredictable behavior. This sensitivity limits their use in safety-critical domains, where robustness guarantees are required. Our work addresses this gap between state-of-the-art adversarial training methods and formal verification to train verifiably robust agents. Previous works train networks with individual adversarial perturbations, making them only robust against the specific adversarial attacks used. In contrast, our approach propagates entire perturbed input sets, enclosing all possible adversarial attacks within a single network pass. We leverage this to explicitly penalize the size of the output set (minimizing closed-loop uncertainty) and thereby make the actor robust against all possible attacks. This is realized by the use of set-based policy gradients, where each output within the set has a different gradient, thereby balancing the accuracy and robustness of the network. Doing so, we achieve formal verifiability across different verification frameworks for up to 9 times larger input perturbations compared to standard reinforcement learning and improve certified worst-case performance.

Manuel Wendl, Lukas Koller, Tobias Ladner +1

Technical University of Munich, Germany

RL Control Reinforcement Learning Neural Network Robustness Certified Robustness Robust RL

Aug 2, 2024cs.AI

Interpreting Global Perturbation Robustness of Image Models using Axiomatic Spectral Importance Decomposition

Perturbation robustness evaluates the vulnerabilities of models, arising from a variety of perturbations, such as data corruptions and adversarial attacks. Understanding the mechanisms of perturbation robustness is critical for global interpretability. We present a model-agnostic, global mechanistic interpretability method to interpret the perturbation robustness of image models. This research is motivated by two key aspects. First, previous global interpretability works, in tandem with robustness benchmarks, e.g. mean corruption error (mCE), are not designed to directly interpret the mechanisms of perturbation robustness within image models. Second, we notice that the spectral signal-to-noise ratios (SNR) of perturbed natural images exponentially decay over the frequency. This power-law-like decay implies that: Low-frequency signals are generally more robust than high-frequency signals -- yet high classification accuracy can not be achieved by low-frequency signals alone. By applying Shapley value theory, our method axiomatically quantifies the predictive powers of robust features and non-robust features within an information theory framework. Our method, dubbed as \textbf{I-ASIDE} (\textbf{I}mage \textbf{A}xiomatic \textbf{S}pectral \textbf{I}mportance \textbf{D}ecomposition \textbf{E}xplanation), provides a unique insight into model robustness mechanisms. We conduct extensive experiments over a variety of vision models pre-trained on ImageNet to show that \textbf{I-ASIDE} can not only \textbf{measure} the perturbation robustness but also \textbf{provide interpretations} of its mechanisms.

Róisín Luo, James McDermott, Colm O'Riordan

SFI Centre for Research Training in Artificial Intelligence · School of Computer Science, University of Galway

Image Corruption Robustness Neural Network Robustness Adversarial Robustness Mechanistic Interpretability

Jun 14, 2024cs.LG

Over-parameterization and Adversarial Robustness in Neural Networks: An Overview and Empirical Analysis

Thanks to their extensive capacity, over-parameterized neural networks exhibit superior predictive capabilities and generalization. However, having a large parameter space is considered one of the main suspects of the neural networks' vulnerability to adversarial example -- input samples crafted ad-hoc to induce a desired misclassification. Relevant literature has claimed contradictory remarks in support of and against the robustness of over-parameterized networks. These contradictory findings might be due to the failure of the attack employed to evaluate the networks' robustness. Previous research has demonstrated that depending on the considered model, the algorithm employed to generate adversarial examples may not function properly, leading to overestimating the model's robustness. In this work, we empirically study the robustness of over-parameterized networks against adversarial examples. However, unlike the previous works, we also evaluate the considered attack's reliability to support the results' veracity. Our results show that over-parameterized networks are robust against adversarial attacks as opposed to their under-parameterized counterparts.

Srishti Gupta, Zhang Chen, Luca Demetrio +9

Sapienza University of Rome, Rome, Italy. · University of Cagliari, Cagliari, Italy. · Northwestern Polytechnical University, Xi’An, China. +1

Adversarial Attacks Neural Network Robustness Adversarial Robustness

Apr 1, 2024cs.LG

Perturbation Effects on Robustness and Individual Fairness

Deep neural networks are vulnerable to adversarial perturbations that can simultaneously degrade prediction robustness and individual fairness across diverse application settings. However, existing evaluation protocols typically assess these dimensions in isolation, thereby obscuring critical failure modes. To bridge this gap, we formalize Robust Individual Fairness (RIF): under semantic-preserving (truth-condition-preserving) perturbations, predictions should remain both correct with respect to the ground truth and invariant across semantically equivalent individuals. To surface RIF violations in practice, we introduce RIFair, a black-box adversarial framework that leverages a decoupled perturbation strategy to construct semantically preserved yet unrobust and/or unfair instance pairs. Experiments across multiple model architectures and real-world textual datasets show that robustness-only or fairness-only metrics often miss Robust Biased and Unrobust Fair behaviors. RIFair}reliably exposes these hidden vulnerabilities, supporting RIF as a necessary criterion for trustworthy model assessment. The experimental code is publicly available at https://github.com/Xuran-LI/RIFair.

Xuran Li, Hao Xue, Peng Wu +4

University of New South Wales Sydney, Australia · The Hong Kong University of Science and Technology Guangzhou, China · Key Laboratory of System Software, Institute of Software, Chinese Academy of Sciences Beijing, China +3

Neural Network Robustness Algorithmic Fairness Adversarial Robustness Black-Box Adversarial Attacks Adversarial Text Attacks

Jun 17, 2023cs.LG

Understanding Certified Training with Interval Bound Propagation

As robustness verification methods are becoming more precise, training certifiably robust neural networks is becoming ever more relevant. To this end, certified training methods compute and then optimize an upper bound on the worst-case loss over a robustness specification. Curiously, training methods based on the imprecise interval bound propagation (IBP) consistently outperform those leveraging more precise bounding methods. Still, we lack an understanding of the mechanisms making IBP so successful. In this work, we thoroughly investigate these mechanisms by leveraging a novel metric measuring the tightness of IBP bounds. We first show theoretically that, for deep linear models, tightness decreases with width and depth at initialization, but improves with IBP training, given sufficient network width. We, then, derive sufficient and necessary conditions on weight matrices for IBP bounds to become exact and demonstrate that these impose strong regularization, explaining the empirically observed trade-off between robustness and accuracy in certified training. Our extensive experimental evaluation validates our theoretical predictions for ReLU networks, including that wider networks improve performance, yielding state-of-the-art results. Interestingly, we observe that while all IBP-based training methods lead to high tightness, this is neither sufficient nor necessary to achieve high certifiable robustness. This hints at the existence of new training methods that do not induce the strong regularization required for tight IBP bounds, leading to improved robustness and standard accuracy.

Yuhao Mao, Mark Niklas Müller, Marc Fischer +1

Department of Computer Science, ETH Zürich, Swizterland

Neural Network Verification Neural Network Robustness Certified Robustness

Nov 27, 2022cs.LG

Adversarial Rademacher Complexity of Deep Neural Networks

Deep neural networks (DNNs) are highly vulnerable to adversarial attacks. Ideally, a robust model should perform well on both perturbed training data and unseen perturbed test data. While DNNs can fit perturbed training data, generalizing to perturbed test data remains a significant challenge. This motivates the study of generalization guarantees from a learning theory perspective. This paper focuses on adversarial Rademacher complexity (ARC), first introduced by Khim and Loh (2018) and Yin et al. (2019). Their work primarily addressed linear functions and highlighted the open question of how to bound ARC for neural networks. Since then, several attempts have been made, with the latest results applying ARC only to two-layer neural networks. The main challenge arises from the dynamic nature and unknown closed-form solution of adversarial examples. In this paper, we resolve this issue and provide the first bound on ARC for deep neural networks. Our bound is qualitatively comparable to Rademacher complexity bounds in similar settings. The key ingredient is a new concept we introduce, termed intermediate adversarial examples, along with a framework for calculating the covering number that is compatible with them. Finally, we present experiments to analyze poor robust generalization, demonstrating that the weight norm is a crucial factor influencing the robust generalization gap.

Jiancong Xiao, Yanbo Fan, Ruoyu Sun +1

The Chinese University of Hong Kong, Shenzhen Shenzhen, China · Nanjing University Suzhou, China

Neural Network Generalization Neural Network Robustness Adversarial Robustness Generalization Bounds