Neural Network Robustness

Follow this topic Ask about this topic

Neural network robustness asks whether a model keeps making reliable predictions when inputs, training data, or computing hardware differ from expectations. Common approaches include adversarial training, corruption and shift testing, Lipschitz constraints, randomized smoothing, and formal verification that certifies behavior within a defined perturbation range. Work spans image, language, audio, scientific, and control systems, with evaluations increasingly examining confidence, internal representations, adaptive attacks, and deployment conditions rather than accuracy alone. A central tension is that improving worst-case or certified robustness can reduce clean accuracy, increase computational cost, or leave other kinds of failures untouched.

Latest papers

238 papers24 papers in the last four weeks, up 85% on the four weeks before. 0.1% of all new papers.

Apr 16, 2026

Robustness of Vision Foundation Models to Common Perturbations

A vision foundation model outputs an embedding vector for an image, which can be affected by common editing operations (e.g., JPEG compression, brightness, contrast adjustments). These common perturbations alter embedding vectors and may impact the performance of downstream tasks using these embeddings. In this work, we present the first systematic study on foundation models' robustness to such perturbations. We propose three robustness metrics and formulate five desired mathematical properties for these metrics, analyzing which properties they satisfy or violate. Using these metrics, we evaluate six industry-scale foundation models (OpenAI, Meta) across nine common perturbation categories, finding them generally non-robust. We also show that common perturbations degrade downstream application performance (e.g., classification accuracy) and that robustness values can predict performance impacts. Finally, we propose a fine-tuning approach to improve robustness without sacrificing utility.

Hongbin Liu, Zhengyuan Jiang, Cheng Hong +1

Duke University · Ant Group

Image Corruption Robustness Vision Foundation Models Neural Network Robustness Vision Foundation Model Adaptation

Apr 15, 2026

General aspects of internal noise in spiking neural networks

This study examines the impact of additive and multiplicative noise on both a single leaky integrate-and-fire (LIF) neuron and a trained spiking neural network (SNN). Noise was introduced at different stages of neural processing, including the input current, membrane potential, and output spike generation. The results show that multiplicative noise applied to the membrane potential has the most detrimental effect on network performance, leading to a significant degradation in accuracy. This is primarily due to its tendency to suppress membrane potentials toward large negative values, effectively silencing neuronal activity. To address this issue, input pre-filtering strategies were evaluated, with a sigmoid-based filter demonstrating the best performance by shifting inputs to a strictly positive range. Under these conditions, additive noise in the input current becomes the dominant source of performance degradation, while other noise configurations reduce accuracy by no more than 1%, even at high noise intensity. Additionally, the study compares the effects of common and uncommon noise across neuron populations in hidden layer, revealing that SNNs exhibit greater robustness to common noise. Overall, the findings identify the most critical noise mechanisms affecting SNNs and provide practical approaches for improving their robustness.

I. D. Kolesnikov, D. A. Maksimov, V. M. Moskvitin +1

Saratov State University, Astrakhanskaya str. 83, Saratov 410012, Russia

LIF Neurons Spiking Neural Networks Neural Network Robustness

Apr 14, 2026

Algorithmic Analysis of Dense Associative Memory: Finite-Size Guarantees and Adversarial Robustness

Dense Associative Memory (DAM) generalizes Hopfield networks through higher-order interactions and achieves storage capacity that scales as

O(N^{n-1})

under suitable pattern separation conditions. Existing dynamical analyses primarily study the thermodynamic limit

N\to\infty

with randomly sampled patterns and therefore do not provide finite-size guarantees or explicit convergence rates. We develop an algorithmic analysis of DAM retrieval dynamics that yields finite-

N

guarantees under explicit, verifiable pattern conditions. Under a separation assumption and a bounded-interference condition at high loading, we prove geometric convergence of asynchronous retrieval dynamics, which implies

O(\log N)

convergence time once the trajectory enters the basin of attraction. We further establish adversarial robustness bounds expressed through an explicit margin condition that quantifies the number of corrupted bits tolerable per sweep, and derive capacity guarantees that scale as

Θ(N^{n-1})

up to polylogarithmic factors in the worst case, while recovering the classical

Θ(N^{n-1})

scaling for random pattern ensembles. Finally, we show that DAM retrieval dynamics admit a potential-game interpretation that ensures convergence to pure Nash equilibria under asynchronous updates. Complete proofs are provided in the appendices, together with preliminary experiments that illustrate the predicted convergence, robustness, and capacity scaling behavior.

Madhava Gaikwad

Independent Researcher

Associative Memory Neural Network Robustness

Apr 14, 2026

Stability and Geometry of Attractors in Neural Cellular Automata

Throughout the literature on Neural Cellular Automata (NCAs), it is often taken for granted that the systems learn attractors. This is shown through evolving the system for many timesteps and noting visual similarity to the goal state. There remain many questions after such an analysis. Namely, what kind of attractors do we have? Is their behavior ordered or chaotic? Can we estimate stability over very long time horizons? What really happens in the attractor when perturbations are applied? In this paper, we present a case study to help answer these questions, with methods drawn from the literature on dynamical systems theory. We use the growing gecko NCA of Mordvintsev et al. (2020) with deterministic cell updates as a case study. To the best of the authors' knowledge, we present the first visualizations of NCA attractor dynamics. We also analyze them using the Lyapunov and Fourier spectra, to reveal that the NCA displays oscillatory, periodic and quasi-periodic behavior, and that these behaviors arise early during training. This challenges the belief that NCAs learn fixed point attractors. Finally, we show that large perturbations to the attractor states can throw the NCAs into a secondary mode separate from the original attractor. We hope that this initial foray into NCA attractor dynamics expands the toolkit for NCA researchers to analyze the robustness and stability of their systems.

Mia-Katrin Kvalsund, James Stovold

Department of Physics, University of Oslo, Norway · Department of Computer Science, University of York, UK

Neural Cellular Automata Neural Network Robustness Dynamical Systems

Apr 9, 2026

Internal noise in deep neural networks: interplay of depth, neuron number, and noise injection step

This paper examines the influence of internal Gaussian noise on the performance of deep feedforward neural networks, focusing on the role of the noise injection stage relative to the activation function. Two scenarios are analyzed: noise introduced before and after the activation function, for both additive and multiplicative noise influence. The case of noise before activation function is similar to perturbations in the input channel of neuron, while the noise introduced after activation function is analogous to noise occurring either within the neuron itself or in its output channel. The types of noise and the method of their introduction were inspired by analog neural networks. The results show that the activation function acts as an effective nonlinear filter of noise. Networks with noise introduced before the activation function consistently achieve higher accuracy than those with noise applied after it, with additive noise being more effectively suppressed in this case. For noise introduced after the activation function, multiplicative noise is less detrimental than additive noise, and earlier hidden layers contribute more significantly to performance degradation due to cumulative noise amplification governed by the statistical properties of subsequent weight matrices. The study also demonstrates that pooling-based noise reduction is effective in both cases when noise is introduced before and after the activation function, consistently improving network performance.

D. A. Maksimov, V. M. Moskvitin, N. Semenova

Saratov State University, Astrakhanskaya str. 83, Saratov 410012, Russia

Feedforward Neural Networks Neural Network Robustness

Apr 7, 2026

Bridging Theory and Practice in Crafting Robust Spiking Reservoirs

Spiking reservoir computing provides an energy-efficient approach to temporal processing, but reliably tuning reservoirs to operate at the edge-of-chaos is challenging due to experimental uncertainty. This work bridges abstract notions of criticality and practical stability by introducing and exploiting the robustness interval, an operational measure of the hyperparameter range over which a reservoir maintains performance above task-dependent thresholds. Through systematic evaluations of Leaky Integrate-and-Fire (LIF) architectures on both static (MNIST) and temporal (synthetic Ball Trajectories) tasks, we identify consistent monotonic trends in the robustness interval across a broad spectrum of network configurations: the robustness-interval width decreases with presynaptic connection density

β

(i.e., directly with sparsity) and directly with the firing threshold

θ

. We further identify specific

(β, θ)

pairs that preserve the analytical mean-field critical point

w_{\text{crit}}

, revealing iso-performance manifolds in the hyperparameter space. Control experiments on Erdős-Rényi graphs show the phenomena persist beyond small-world topologies. Finally, our results show that

w_{\text{crit}}

consistently falls within empirical high-performance regions, validating

w_{\text{crit}}

as a robust starting coordinate for parameter search and fine-tuning. To ensure reproducibility, the full Python code is publicly available.

Ruggero Freddi, Nicolas Seseri, Diana Nigrisoli +1

Research and Development Department, Manava.plus, Milan, Italy · Department of Engineering and Geology, “G’. d’Annunzio” University of Chieti-Pescara, Pescara, Italy

Reservoir Computing Spiking Neural Networks Neural Network Robustness

Apr 6, 2026

Context is All You Need

Artificial Neural Networks (ANNs) are increasingly deployed across diverse real-world settings, where they must operate under data distributions that differ from those seen during training. This challenge is central to Domain Generalization (DG), which trains models to generalize to unseen domains without target data, and Test-Time Adaptation (TTA), which improves robustness by adapting to unlabeled test data at deployment. Existing approaches to address these challenges are often complex, resource-intensive, and difficult to scale. We introduce CONTXT (Contextual augmentatiOn for Neural feaTure X Transforms), a simple and intuitive method for contextual adaptation. CONTXT modulates internal representations using simple additive and multiplicative feature transforms. Within a TTA setting, it yields consistent gains across discriminative tasks (e.g., ANN/CNN classification) and generative models (e.g., LLMs). The method is lightweight, easy to integrate, and incurs minimal overhead, enabling robust performance under domain shift without added complexity. More broadly, CONTXT provides a compact way to steer information flow and neural processing without retraining.

Jean Erik Delanois, Shruti Joshi, Ryan Golden +2

Department of Computer Science & Engineering, University of California, San Diego, La Jolla, California, USA · Department of Medicine, University of California, San Diego, La Jolla, California, USA · Microsoft Corporation, Redmond, WA, USA

Neural Network Robustness Test-Time Adaptation Domain Generalization

Apr 3, 2026

WGFINNs: Weak formulation-based GENERIC formalism informed neural networks

Data-driven discovery of governing equations from noisy observations remains a fundamental challenge in scientific machine learning. While GENERIC formalism informed neural networks (GFINNs) provide a principled framework that enforces the laws of thermodynamics by construction, their reliance on strong-form loss formulations makes them highly sensitive to measurement noise. To address this limitation, we propose weak formulation-based GENERIC formalism informed neural networks (WGFINNs), which integrate the weak formulation of dynamical systems with the structure-preserving architecture of GFINNs. WGFINNs significantly enhance robustness to noisy data while retaining exact satisfaction of GENERIC degeneracy and symmetry conditions. We further incorporate a state-wise weighted loss and a residual-based attention mechanism to mitigate scale imbalance across state variables. Theoretical analysis contrasts quantitative differences between the strong-form and the weak-form estimators. Mainly, the strong-form estimator diverges as the time step decreases in the presence of noise, while the weak-form estimator can be accurate even with noisy data if test functions satisfy certain conditions. Numerical experiments demonstrate that WGFINNs consistently outperform GFINNs at varying noise levels, achieving more accurate predictions and reliable recovery of physical quantities.

Jun Sur Richard Park, Auroni Huque Hashim, Siu Wun Cheung +2

Division of Big Data Science, Korea University, Sejong, 30019, Republic of Korea · Department of Mathematics, North Carolina State University, Raleigh, NC 27695, USA · Center for Applied Scientific Computing, Lawrence Livermore National Laboratory, Livermore, CA 94550, USA.

Physics-Informed ML AI-Assisted Scientific Research Neural Network Robustness PDE Solving

Apr 2, 2026

Where Does Robustness Live? Neuron-Guided Adaptation for Retrieval-Augmented Language Models

Retrieval-Augmented Language Models (RALMs) have shown strong potential in knowledge-intensive tasks, yet they remain vulnerable when retrieved contexts are noisy or irrelevant. Robustness against such contexts requires two distinct capabilities: abstention when contexts are uninformative, and selective extraction when relevant evidence is buried in noise. Yet existing methods face two key limitations: they do not train separately for these two capabilities, and they adapt the model at a coarse layer- or module-level granularity, overlooking that only a small subset of neurons is strongly activated for a given input. We propose NeuRIT, a Neuron-guided Robust Instruction-Tuning framework built on a localization-first perspective. NeuRIT mines context-aware neurons associated with relevant and irrelevant context processing, and uses them as anchors to selectively adapt both the identified neuron groups and the layers in which they concentrate. NeuRIT then performs two-stage instruction tuning that teaches complementary behaviors: suppress generation when there is nothing to extract, and extract relevant evidence when there is. NeuRIT consistently outperforms strong baselines across diverse QA benchmarks and generator backbones. Our code is available at https://github.com/HYU-ARK-Lab/NeuRIT.

Jae O Lee, Jaemin Kim, Sumyeong Ahn +1

1Hanyang University · 2KENTECH

Retrieval-Augmented Generation Fine-Tuning Knowledge Augmentation for Language Models LLM Fine-Tuning Neural Network Robustness Retrieval Robustness+2

Apr 2, 2026

BraiNCA: brain-inspired neural cellular automata and applications to morphogenesis and motor control

Most of the Neural Cellular Automata (NCAs) defined in the literature have a common theme: they are based on regular grids with a Moore neighborhood (one-hop neighbour). They do not take into account long-range connections and more complex topologies as we can find in the brain. In this paper, we introduce BraiNCA, a brain-inspired NCA with an attention layer, long-range connections and complex topology. BraiNCAs shows better results in terms of robustness and speed of learning on the two tasks compared to Vanilla NCAs establishing that incorporating attention-based message selection together with explicit long-range edges can yield more sample-efficient and damage-tolerant self-organization than purely local, grid-based update rules. These results support the hypothesis that, for tasks requiring distributed coordination over extended spatial and temporal scales, the choice of interaction topology and the ability to dynamically route information will impact the robustness and speed of learning of an NCA. More broadly, BraiNCA provides brain-inspired NCA formulation that preserves the decentralized local update principle while better reflecting non-local connectivity patterns, making it a promising substrate for studying collective computation under biologically-realistic network structure and evolving cognitive substrates.

Léo Pio-Lopez, Benedikt Hartl, Michael Levin

Allen Discovery Center at Tufts University, Medford, MA, USA · Wyss Institute for Biologically Inspired Engineering at Harvard University, Boston, MA, USA

Neural Cellular Automata Neural Network Robustness

Mar 26, 2026

Fast Preemptive Robustification: High-Frequency Response Anti-Aligns Shared Vulnerability

Adversarial attacks can readily compromise deep neural networks (DNNs). In particular, transferable attacks (TAs) exploit the shared vulnerabilities among DNNs, enabling perturbations crafted on surrogates to transfer to unseen models. Training-time and post-attack defenses have been extensively studied for combating TAs. Orthogonal to these approaches, preemptive robustification (PR) has emerged as a pre-attack defense that enhances the robustness of benign samples by superimposing protective variations before attacks. Despite its promise, PR remains underexplored and faces several important limitations. First, dependence on well-trained surrogate classifiers limits applicability, as surrogates are task-specific and may even be unavailable in some practical settings. Second, the required iterative optimization or dedicated PR generator training incurs substantial costs. Third, the generated variations are opaque to humans. To address these, we seek an efficient PR that is surrogate-free, optimization-free, training-free, and human-interpretable. Intriguingly, we discover a numerical correlation between the shared vulnerabilities of DNNs and Laplacian responses, with their cosine similarity being significantly negative. This indicates that negated high-frequency response constitutes an important component of shared vulnerabilities. Consequently, strengthening Laplacian responses counteracts this component, improving resistance to TAs. Building upon this insight, we propose Fast Preemptive Robustification (FPR), which performs Laplacian sharpening via a single channel-wise convolution with a 3\times3 kernel. FPR is simple yet effective, as demonstrated by extensive experiments. Specifically, FPR reduces the attack success rate (ASR) of untargeted TAs by 12.7% and that of targeted TAs from 10.7% to 4.1%. Code will be released publicly.

Jiaming Liang, Chi-Man Pun

University of Macau, Macau, China

Neural Network Robustness Adversarial Robustness Transfer-Based Adversarial Attacks Adversarial Transferability

Mar 25, 2026

The Luna Bound Propagator for Formal Analysis of Neural Networks

The parameterized CROWN analysis, a.k.a., alpha-CROWN has emerged as a practically successful abstract interpretation method for neural network verification. However, existing implementations of alpha-CROWN are limited to Python, which complicates integration into existing DNN verifiers and long-term production-level systems. We introduce Luna, a new abstract-interpretation-based bound propagator implemented in C++. Luna supports Interval Bound Propagation, the DeepPoly/CROWN analysis, and the alpha-CROWN analysis over a general computational graph. We describe the architecture of Luna and show that it outperforms the state-of-the-art alpha-CROWN implementation in terms of both bound tightness and computational efficiency on supported benchmarks from VNN-COMP 2025. Luna is publicly available at https://github.com/ai-ar-research/luna.

Henry LeCates, Haoze Wu

Amherst College, USA

Abstract Interpretation Neural Network Verification Neural Network Robustness

Mar 24, 2026

From Arithmetic to Logic: The Resilience of Logic and Lookup-Based Neural Networks Under Parameter Bit-Flips

The deployment of deep neural networks (DNNs) in safety-critical edge environments necessitates robustness against hardware-induced bit-flip errors. While empirical studies indicate that reducing numerical precision can improve fault tolerance, the theoretical basis of this phenomenon remains underexplored. In this work, we study resilience as a structural property of neural architectures rather than solely as a property of a dataset-specific trained solution. By deriving the expected squared error (MSE) under independent parameter bit flips across multiple numerical formats and layer primitives, we show that lower precision, higher sparsity, bounded activations, and shallow depth are consistently favored under this corruption model. We then argue that logic and lookup-based neural networks realize the joint limit of these design trends. Through ablation studies on the MLPerf Tiny benchmark suite, we show that the observed empirical trends are consistent with the theoretical predictions, and that LUT-based models remain highly stable in corruption regimes where standard floating-point models fail sharply. Furthermore, we identify a novel even-layer recovery effect unique to logic-based architectures and analyze the structural conditions under which it emerges. Overall, our results suggest that shifting from continuous arithmetic weights to discrete Boolean lookups can provide a favorable accuracy-resilience trade-off for hardware fault tolerance.

Alan T. L. Bacellar, Sathvik Chemudupati, Shashank Nag +4

The University of Texas at Austin · Federal University of Rio de Janeiro · Instituto de Telecomunicações, Porto, Portugal

Neural Network Robustness Neural Network Quantization

Mar 21, 2026

Neuronal Self-Adaptation Enhances Capacity and Robustness of Representation in Spiking Neural Networks

Spiking Neural Networks (SNNs) are promising for energy-efficient, real-time edge computing, yet their performance is often constrained by the limited adaptability of conventional leaky integrate-and-fire (LIF) neurons. Existing LIF models struggle with restricted information capacity and susceptibility to noise, leading to degraded accuracy and compromised robustness. Inspired by the dynamic self-regulation of biological potassium channels, we propose the Potassium-regulated LIF (KvLIF) neuron model. KvLIF introduces an auxiliary conductance state that integrates membrane potential and spiking history to adaptively modulate neuronal excitability and reset dynamics. This design extends the dynamic response range of neurons to varying input intensities and effectively suppresses noise-induced spikes. We extensively evaluate KvLIF on both static image and neuromorphic datasets, demonstrating consistent improvements in classification accuracy and superior robustness compared to existing LIF models. Our work bridges biological plausibility with computational efficiency, offering a neuron model that enhances SNN performance while maintaining suitability for low-power neuromorphic deployment.

Zhuobin Yang, Yeyao Bao, Liangfu Lv +3

Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin, China.

Spiking Neural Networks Neural Network Robustness

Mar 21, 2026

IBCapsNet: Information Bottleneck Capsule Network for Noise-Robust Representation Learning

Capsule networks (CapsNets) are superior at modeling hierarchical spatial relationships but suffer from two critical limitations: high computational cost due to iterative dynamic routing and poor robustness under input corruptions. To address these issues, we propose IBCapsNet, a novel capsule architecture grounded in the Information Bottleneck (IB) principle. Instead of iterative routing, IBCapsNet employs a one-pass variational aggregation mechanism, where primary capsules are first compressed into a global context representation and then processed by class-specific variational autoencoders (VAEs) to infer latent capsules regularized by the KL divergence. This design enables efficient inference while inherently filtering out noise. Experiments on MNIST, Fashion-MNIST, SVHN and CIFAR-10 show that IBCapsNet matches CapsNet in clean-data accuracy (achieving 99.41% on MNIST and 92.01% on SVHN), yet significantly outperforms it under four types of synthetic noise - demonstrating average improvements of +17.10% and +14.54% for clamped additive and multiplicative noise, respectively. Moreover, IBCapsNet achieves 2.54x faster training and 3.64x higher inference throughput compared to CapsNet, while reducing model parameters by 4.66%. Our work bridges information-theoretic representation learning with capsule networks, offering a principled path toward robust, efficient, and interpretable deep models. Code is available at https://github.com/cxiang26/IBCapsnet

Canqun Xiang, Chen Yang, Jiaoyan Zhao

an independent researcher · Institute of Applied Artificial Intelligence of the Guangdong-Hong Kong-Macao Greater Bay Area, Shenzhen Polytechnic University, Shenzhen, China

Neural Network Robustness Representation Learning Efficient Neural Network Inference Variational Information Bottleneck

Mar 19, 2026

Beyond Weighted Summation: Learnable Nonlinear Aggregation Functions for Robust Artificial Neurons

Weighted summation has remained the default input aggregation mechanism in artificial neurons since the earliest neural network models. While computationally efficient, this design implicitly behaves like a mean-based estimator and is therefore sensitive to noisy or extreme inputs. This paper investigates whether replacing fixed linear aggregation with learnable nonlinear alternatives can improve neural network robustness without sacrificing trainability. Two differentiable aggregation mechanisms are introduced: an F-Mean neuron based on a learnable power-weighted aggregation rule, and a Gaussian Support neuron based on distance-aware affinity weighting. To preserve the optimisation stability of standard neurons, hybrid neurons are proposed that interpolate between linear and nonlinear aggregation through a learnable blending parameter. Evaluated in multilayer perceptrons and convolutional neural networks on CIFAR-10 and a noisy CIFAR-10 variant with additive Gaussian corruption, hybrid neurons consistently improve robustness under noise while F-Mean hybrids also yield modest gains on clean data. The three-way hybrid achieves robustness scores of up to 0.991 compared to 0.890 for the standard baseline, and learned parameters converge consistently to sub-linear aggregation (p

\approx

0.43--0.50) and high novelty utilisation (

α

\approx

0.69--0.79). These findings suggest that neuron-level aggregation is a meaningful and underexplored design dimension for building more noise-tolerant neural networks.

Berke Deniz Bozyigit

Independent Researcher

Neural Network Robustness

Mar 18, 2026

rSDNet: Unified Robust Neural Learning against Label Noise and Adversarial Attacks

Neural networks are central to modern artificial intelligence, yet their training remains highly sensitive to data contamination. Standard neural classifiers are trained by minimizing the categorical cross-entropy loss, corresponding to maximum likelihood estimation under a multinomial model. While statistically efficient under ideal conditions, this approach is highly vulnerable to contaminated observations including label noises corrupting supervision in the output space, and adversarial perturbations inducing worst-case deviations in the input space. In this paper, we propose a unified and statistically grounded framework for robust neural classification that addresses both forms of contamination within a single learning objective. We formulate neural network training as a minimum-divergence estimation problem and introduce rSDNet, a robust learning algorithm based on the general class of

S

-divergences. The resulting training objective inherits robustness properties from classical statistical estimation, automatically down-weighting aberrant observations through model probabilities. We establish essential population-level properties of rSDNet, including Fisher consistency, classification calibration implying Bayes optimality, and robustness guarantees under uniform label noise and infinitesimal feature contamination. Experiments on three benchmark image classification datasets show that rSDNet improves robustness to label corruption and adversarial attacks while maintaining competitive accuracy on clean data, Our results highlight minimum-divergence learning as a principled and effective framework for robust neural classification under heterogeneous data contamination.

Suryasis Jana, Abhik Ghosh

Indian Statistical Institute, Kolkata, India

Neural Network Robustness Robust Classification Learning with Noisy Labels Adversarial Robustness Robust Loss Functions

Mar 12, 2026

Incremental Neural Network Verification via Learned Conflicts

Neural network verification is often used as a core component within larger analysis procedures, which generate sequences of closely related verification queries over the same network. In existing neural network verifiers, each query is typically solved independently, and information learned during previous runs is discarded, leading to repeated exploration of the same infeasible regions of the search space. In this work, we aim to expedite verification by reducing this redundancy. We propose an incremental verification technique that reuses learned conflicts across related verification queries. The technique can be added on top of any branch-and-bound-based neural network verifier. During verification, the verifier records conflicts corresponding to learned infeasible combinations of activation phases, and retains them across runs. We formalize a refinement relation between verification queries and show that conflicts learned for a query remain valid under refinement, enabling sound conflict inheritance. Inherited conflicts are handled using a SAT solver to perform consistency checks and propagation, allowing infeasible subproblems to be detected and pruned early during search. We implement the proposed technique in the Marabou verifier and evaluate it on three verification tasks: local robustness radius determination, verification with input splitting, and minimal sufficient feature set extraction. Our experiments show that incremental conflict reuse reduces verification effort and yields speedups of up to

1.9\times

over a non-incremental baseline.

Raya Elsaleh, Liam Davis, Haoze Wu +1

Hebrew University of Jerusalem, Jerusalem, Israel · Amherst College, Amherst, USA

Neural Network Verification Neural Network Robustness

Mar 10, 2026

Intrinsic Numerical Robustness and Fault Tolerance in a Neuromorphic Algorithm for Scientific Computing

The potential for neuromorphic computing to provide intrinsic fault tolerance has long been speculated, but the brain's robustness in neuromorphic applications has yet to be demonstrated. Here, we show that a previously described, natively spiking neuromorphic algorithm for solving partial differential equations is intrinsically tolerant to structural perturbations in the form of ablated neurons and dropped spikes. The tolerance band for these perturbations is large: we find that as many as 32 percent of the neurons and up to 90 percent of the spikes may be entirely dropped before a significant degradation in the accuracy results. Furthermore, this robustness is tunable through structural hyperparameters. This work demonstrates that the specific brain-like inspiration behind the algorithm contributes to a significant degree of robustness expected from brain-like neuromorphic algorithms.

Bradley H. Theilman, James B. Aimone

Neural Exploration & Research Laboratory, Sandia National Laboratories, Albuquerque, NM

Neuromorphic Computing Neural Network Robustness Neural PDE Solvers Spiking Neural Networks

Mar 10, 2026

Rotation Equivariant Mamba for Vision Tasks

Rotation equivariance is an important structural prior for a broad range of vision tasks, particularly in domains such as remote sensing and medical imaging where objects have no canonical orientation, yet it remains notably absent from current Mamba-based vision architectures. Despite the success of Mamba in natural language processing and its growing adoption in computer vision, existing visual Mamba models fail to account for rotational symmetry in their design. This omission renders them sensitive to image rotations, thereby constraining their robustness and cross-task generalization. To address this limitation, we incorporate rotation symmetry, an important geometric prior in many visual domains, into Mamba-based architectures. Specifically, we introduce EQ-VMamba, the first rotation equivariant visual Mamba architecture for vision tasks. The core components of EQ-VMamba include a carefully designed rotation equivariant cross-scan strategy and group Mamba blocks. Moreover, we provide a rigorous theoretical analysis of the intrinsic equivariance error, demonstrating that the proposed architecture enforces end-to-end rotation equivariance under the p4 rotation group. Extensive experiments across multiple benchmarks---including high-level image classification, mid-level semantic segmentation, and low-level image super-resolution---demonstrate that EQ-VMamba consistently improves rotation robustness and achieves superior or competitive performance compared to non-equivariant baselines on most benchmarks, while requiring 40-70% fewer parameters. These results indicate that embedding rotation equivariance not only effectively bolsters the robustness of visual Mamba models against rotation transformations, but also improves accuracy on data exhibiting rotational symmetry, with substantially improved parameter efficiency. Code is available at https://github.com/zhongchenzhao/EQ-VMamba.

Zhongchen Zhao, Qi Xie, Keyu Huang +3

School of Mathematics and Statistics, Xi’an Jiaotong University, Shaanxi, P.R. China · Department of Computing, the Hong Kong Polytechnic University, Hong Kong, P.R. China

Equivariant Neural Networks Neural Network Robustness

Mar 10, 2026

Attack-Dependent Robustness of Neural Audio Codecs for Adversarial ASR

Neural audio codecs impose a discrete bottleneck through residual vector quantization (RVQ), making them a useful class of inference-time transformations for reducing adversarial perturbations before ASR inference. We study how codec quantization depth affects defended ASR under non-adaptive, standard adaptive, and quantization-aware adaptive untargeted

\ell_\infty

attacks. Under non-adaptive attacks, intermediate RVQ depths yield the lowest word error rates and outperform traditional compression at comparable bitrates. However, this apparent optimum is not stable under adaptive evaluation. The standard identity-gradient adaptive baseline (BPDA+EOT) can overestimate robustness, while an implementation of an RVQ-relaxed adaptive attack (SoftVQ-PGD) substantially changes the observed depth trend and largely removes the intermediate-depth advantage. Overall, neural codecs can improve defended ASR under specific threat models. However, the relationship between robustness and RVQ depth depends on the attack used for evaluation, rather than on the codec architecture alone.

Jordan Prescott, Thanathai Lertpetchpun, Shrikanth Narayanan

Signal Analysis and Interpretation Laboratory, University of Southern California, USA

ASR Evaluation Neural Audio Codecs Residual VQ Neural Network Robustness Automatic Speech Recognition

Mar 9, 2026

SYNAPSE: Framework for Neuron Analysis and Perturbation in Sequence Encoding

In recent years, Artificial Intelligence has become a powerful partner for complex tasks such as data analysis, prediction, and problem-solving, yet its lack of transparency raises concerns about its reliability. In sensitive domains such as healthcare or cybersecurity, ensuring transparency, trustworthiness, and robustness is essential, since the consequences of wrong decisions or successful attacks can be severe. To overcome these limitations, this work proposes SYNAPSE, a systematic, training-free framework (requiring no retraining of the base model) for understanding and stress-testing the internal behavior of Transformer models across domains. SYNAPSE extracts per-layer [CLS] representations, trains a lightweight linear probe to obtain global and per-class neuron rankings, and applies forward-hook interventions during inference. This design enables controlled, repeatable experiments on internal representations without altering the original model, thereby allowing weaknesses, stability patterns, and label-specific sensitivities to be measured and compared directly across tasks and architectures. SYNAPSE reveals a consistent organization of internal representations across the two evaluated domains, in which task-relevant information is encoded in broad, overlapping neuron subsets. This redundancy provides a strong degree of functional stability, while class-wise asymmetries expose heterogeneous specialization patterns and enable fine-grained, label-aware analysis. In contrast, small structured manipulations in weight or logit space are sufficient to redirect predictions, highlighting complementary vulnerability profiles and illustrating how SYNAPSE can guide the development of more robust and interpretable Transformer models.

Jesús Sánchez Ochoa, Enrique Tomás Martínez Beltrán, Alberto Huertas Celdrán

Department of Information and Communications Engineering, University of Murcia, Murcia, 30100, Spain

Neuron-Level Interpretability Transformer Interpretability Neural Network Robustness Mechanistic Interpretability

Mar 6, 2026

Polarized Direct Cross-Attention Message Passing in GNNs for Machinery Fault Diagnosis

The reliability of safety-critical industrial systems hinges on accurate and robust fault diagnosis in rotating machinery. Conventional graph neural networks (GNNs) for machinery fault diagnosis face limitations in modeling complex dynamic interactions due to their reliance on predefined static graph structures and homogeneous aggregation schemes. To overcome these challenges, this paper introduces polarized direct cross-attention (PolaDCA), a novel relational learning framework that enables adaptive message passing through data-driven graph construction. Our approach builds upon a direct cross-attention (DCA) mechanism that dynamically infers attention weights from three semantically distinct node features (such as individual characteristics, neighborhood consensus, and neighborhood diversity) without requiring fixed adjacency matrices. Theoretical analysis establishes PolaDCA's superior noise robustness over conventional GNNs. Extensive experiments on industrial datasets (i.e., XJTUSuprgear, CWRUBearing and Three-Phase Flow Facility datasets) demonstrate state-of-the-art diagnostic accuracy and enhanced generalization under varying noise conditions, outperforming seven competitive baseline methods. The proposed framework provides an effective solution for safety-critical industrial applications.

Zongyu Shi, Laibin Zhang, Maoyin Chen

Department of Automation, College of Artificial Intelligence, China University of Petroleum (Beijing), Beijing 102299, China · Department of Safety Engineering, College of Safety and Ocean Engineering, China University of Petroleum (Beijing), Beijing 102299, China

Graph Neural Networks Fault Diagnosis Dynamic Graph Learning Graph Structure Learning Neural Network Robustness

Mar 6, 2026

Lipschitz-Based Robustness Certification Under Floating-Point Execution

Lipschitz-based robustness certification bounds a network's sensitivity through concrete numerical computation rather than symbolic reasoning, and so scales efficiently. It is increasingly used even where verifiable guarantees matter. Yet, as with most prior work on robustness certification and verification, soundness is typically proved against a semantic model assuming exact real arithmetic. Deployed networks instead execute in floating-point, creating a gap between certified properties and executed behaviour. As motivating evidence, we give counterexamples showing that real arithmetic robustness guarantees can fail under floating-point execution, even for previously verified certifiers. We then develop a formal, compositional theory relating real arithmetic Lipschitz-based sensitivity bounds to floating-point execution under standard rounding-error models for feed-forward ReLU networks. We derive sound conditions for floating-point robustness, including bounds on certificate degradation and sufficient conditions for the absence of overflow. We also give an efficient floating-point Gram iteration algorithm for Lipschitz bounds and prove that it never under-estimates the true norm. Separately, when a model is certified pre-deployment, we show how measuring its actual deviation against a high-precision execution can substantially reduce certificate degradation. We formalise the theory and its soundness, and implement an executable certifier, evaluated across dense networks spanning image, tabular, and many-class classification. To our knowledge, ours is the first method for soundly accounting for floating-point effects in Lipschitz-based robustness certification, and, done efficiently, the first floating-point-sound robustness checking procedure of any kind to certify models' entire test set -- seven those with 500,000 examples -- while retaining enough precision to be practical.

Toby Murray

University of Melbourne, Melbourne, Australia

Neural Network Verification Neural Network Robustness Formal Verification Certified Robustness

Mar 5, 2026

MCEL: Margin-Based Cross-Entropy Loss for Error-Tolerant Quantized Neural Networks

Robustness to bit errors is a key requirement for the reliable use of neural networks (NNs) on emerging approximate computing platforms and error-prone memory technologies. A common approach to achieve bit error tolerance in NNs is injecting bit flips during training according to a predefined error model. While effective in certain scenarios, training-time bit flip injection introduces substantial computational overhead, often degrades inference accuracy at high error rates, and scales poorly for larger NN architectures. These limitations make error injection an increasingly impractical solution for ensuring robustness on future approximate computing platforms and error-prone memory technologies. In this work, we investigate the mechanisms that enable NNs to tolerate bit errors without relying on error-aware training. We establish a direct connection between bit error tolerance and classification margins at the output layer. Building on this insight, we propose a novel loss function, the Margin Cross-Entropy Loss (MCEL), which explicitly promotes logit-level margin separation while preserving the favorable optimization properties of the standard cross-entropy loss. Furthermore, MCEL introduces an interpretable margin parameter that allows robustness to be tuned in a principled manner. Extensive experimental evaluations across multiple datasets of varying complexity, diverse NN architectures, and a range of quantization schemes demonstrate that MCEL substantially improves bit error tolerance, up to 15 % in accuracy for an error rate of 1 %. Our proposed MCEL method is simple to implement, efficient, and can be integrated as a drop-in replacement for standard CEL. It provides a scalable and principled alternative to training-time bit flip injection, offering new insights into the origins of NN robustness and enabling more efficient deployment on approximate computing and memory systems.

Mikail Yayla, Akash Kumar

Ruhr-Universität Bochum, Germany

Neural Network Robustness Robust Loss Functions

Mar 3, 2026

When Small Variations Become Big Failures: Reliability Challenges in Compute-in-Memory Neural Accelerators

Compute-in-memory (CiM) architectures promise significant improvements in energy efficiency and throughput for deep neural network acceleration by alleviating the von Neumann bottleneck. However, their reliance on emerging non-volatile memory devices introduces device-level non-idealities-such as write variability, conductance drift, and stochastic noise-that fundamentally challenge reliability, predictability, and safety, especially in safety-critical applications. This talk examines the reliability limits of CiM-based neural accelerators and presents a series of techniques that bridge device physics, architecture, and learning algorithms to address these challenges. We first demonstrate that even small device variations can lead to disproportionately large accuracy degradation and catastrophic failures in safety-critical inference workloads, revealing a critical gap between average-case evaluations and worst-case behavior. Building on this insight, we introduce SWIM, a selective write-verify mechanism that strategically applies verification only where it is most impactful, significantly improving reliability while maintaining CiM's efficiency advantages. Finally, we explore a learning-centric solution that improves realistic worst-case performance by training neural networks with right-censored Gaussian noise, aligning training assumptions with hardware-induced variability and enabling robust deployment without excessive hardware overhead. Together, these works highlight the necessity of cross-layer co-design for CiM accelerators and provide a principled path toward dependable, efficient neural inference on emerging memory technologies-paving the way for their adoption in safety- and reliability-critical systems.

Yifan Qin, Jiahao Zheng, Zheyu Yan +3

University of Notre Dame · NC State University

Compute-in-Memory Neural Network Robustness Noise Robustness Efficient Neural Network Inference

Mar 3, 2026

Guiding Sparse Neural Networks with Neurobiological Principles to Elicit Biologically Plausible Representations

While deep neural networks (DNNs) have achieved remarkable performance in tasks such as image recognition, they often struggle with generalization, learning from few examples, and continuous adaptation - abilities inherent in biological neural systems. These challenges arise due to DNNs' failure to emulate the efficient, adaptive learning mechanisms of biological networks. To address these issues, we explore the integration of neurobiologically inspired assumptions in neural network learning. This study introduces a biologically inspired learning rule that naturally integrates neurobiological principles, including sparsity, lognormal weight distributions, and adherence to Dale's law, without requiring explicit enforcement. By aligning with these core neurobiological principles, our model enhances robustness against adversarial attacks and demonstrates superior generalization, particularly in few-shot learning scenarios. Notably, integrating these constraints leads to the emergence of biologically plausible neural representations, underscoring the efficacy of incorporating neurobiological assumptions into neural network design. Preliminary results suggest that this approach could extend from feature-specific to task-specific encoding, potentially offering insights into neural resource allocation for complex tasks.

Patrick Inoue, Florian Röhrbein, Andreas Knoblauch

KEIM Institute, Albstadt-Sigmaringen University, Germany · Department of Computer Science, Chemnitz University of Technology, Germany

Biologically Plausible Learning Sparse Neural Networks Neural Network Generalization Neural Network Robustness

Mar 3, 2026

The Price of Robustness: Stable Classifiers Need Overparameterization

The relationship between overparameterization, stability, and generalization remains incompletely understood in the setting of discontinuous classifiers. We address this gap by establishing a generalization bound for finite function classes that improves inversely with class stability, defined as the expected distance to the decision boundary in the input domain (margin). Interpreting class stability as a quantifiable notion of robustness, we derive as a corollary a law of robustness for classification that extends the results of Bubeck and Sellke beyond smoothness assumptions to discontinuous functions. In particular, any interpolating model with

p \approx n

parameters on

n

data points must be unstable, implying that substantial overparameterization is necessary to achieve high stability. We obtain analogous results for parameterized infinite function classes by analyzing a stronger robustness measure derived from the margin in the codomain, which we refer to as the normalized co-stability. Experiments support our theory: stability increases with model size and correlates with test performance, while traditional norm-based measures remain largely uninformative.

Jonas von Berg, Adalbert Fono, Massimiliano Datres +2

Ludwig-Maximilians-Universität München Munich, Germany Munich Center for Machine Learning (MCML) · ∗now at Aleph Alpha Research · †University of Tromso, DLR-German Aerospace Center

Overparameterization Robust Classification Classification Neural Network Robustness Generalization Bounds

Feb 28, 2026

Exact and Asymptotically Complete Robust Verifications of Neural Networks via Ising Solvers

We present an Ising-compatible framework for formal neural-network robustness verification under bounded input perturbations. For piecewise activations (both continuous piecewise-linear and discrete piecewise-constant), the Exact Logarithmic Piecewise Model (Log-PW) provides an exact, sound, and complete formulation with a state-optimal logarithmic encoding, reducing the binary variables per neuron from linear

O(κ)

to information-theoretically minimal logarithmic complexity

O(\lceil \log_2 κ\rceil)

via logarithmic SOS2 (Log-PWL) and SOS1 (Log-PWC) constraints. For general element-wise activations, the Asymptotic Step-Envelope Model (Step-Env) leverages Log-PWC to construct sound, refinable endpoint enclosures. An exact monotonic reduction based on Jordan decomposition further extends this framework to arbitrary non-monotonic activations. We further develop a hybrid Benders solver with output-sensitive iteration bounds, leveraging a Helly-based conflict compression theory to avoid generic exponential cut explosion. In particular, for shallow networks with fixed input dimension, this establishes an unconditionally polynomial Benders iteration complexity in network size. Interval pruning, certificate transfer for pruned networks, and layerwise classical--Ising partitioning further reduce spin requirements. Experiments demonstrate exact certification fidelity for piecewise-linear networks and near-reference accuracy for non-linear activations under compact spin budgets. On MNIST benchmarks, Log-PW halves binary decision spins over One-Hot formulations, while Helly-based cuts achieve up to a

13.41\times

speedup and an

87.6\%

reduction in master queries over standard baselines.

Wenxin Li, Wenchao Liu, Anzhe Jiang +6

Beijing QBoson Quantum Technology Co., Ltd. · Tsinghua University · Beijing Normal University

Neural Network Verification Neural Network Robustness Certified Robustness

Feb 25, 2026

Pulse-Driven Neural Architecture: Learnable Oscillatory Dynamics for Robust Continuous-Time Sequence Processing

We introduce PDNA (Pulse-Driven Neural Architecture), a method for augmenting continuous-time recurrent networks with learnable oscillatory dynamics that maintain internal state evolution independently of external input. Built on Closed-form Continuous-time (CfC) networks, PDNA adds two components: (1) a pulse module that generates structured oscillations

A \cdot \sin(ωt + \varphi(h))

with learnable frequencies and state-dependent phase, and (2) a self-attend module that applies recurrent self-attention to the hidden state. Through a controlled ablation study on sequential MNIST (sMNIST) with five random seeds, we evaluate gap robustness -- the ability to maintain performance when portions of the input sequence are removed at test time. Our key finding is that structured oscillatory dynamics significantly improve robustness to input interruptions: the self-attend variant achieves a statistically significant 2.78 percentage point multi-gap advantage over baseline (

p = 0.041

), while the pulse variant shows a 4.62 pp advantage with large effect size (Cohen's

d = 0.87

). A noise control (random perturbation of equal magnitude) provides no benefit, confirming that the advantage is structural rather than merely dynamic. These results provide evidence that continuous-time models can benefit from biologically-inspired internal oscillatory mechanisms for temporal robustness.

Paras Sharma

Recurrent Neural Networks Oscillatory Neural Networks Neural Network Robustness