Adversarial Perturbations

Ask about this topic

Overview

Adversarial perturbation research studies how small, structured, or physically realizable changes to inputs alter model predictions and behavior. Common approaches include gradient-based attacks such as FGSM and PGD, universal and transfer attacks, query-efficient black-box search, and attacks designed for text, audio, 3D data, or physical settings. Defenses include adversarial training, input purification, detection, and robustness certification, while protective perturbations also aim to preserve privacy or limit unauthorized use of images and videos. A central challenge is that success on fixed benchmarks may not predict performance after compression, resizing, environmental change, or closed-loop deployment, and stronger protection often trades off against visual quality, utility, or attack transferability.

Latest papers

CardsList

105 papers10 papers in the last four weeks, up 100% on the four weeks before. 0.1% of all new papers.

Feb 22, 2026

DefenseSplat: Enhancing the Robustness of 3D Gaussian Splatting via Frequency-Aware Filtering

3D Gaussian Splatting (3DGS) has emerged as a powerful paradigm for real-time and high-fidelity 3D reconstruction from posed images. However, recent studies reveal its vulnerability to adversarial corruptions in input views, where imperceptible yet consistent perturbations can drastically degrade rendering quality, increase training and rendering time, and inflate memory usage, even leading to server denial-of-service. In our work, to mitigate this issue, we begin by analyzing the distinct behaviors of adversarial perturbations in the low- and high-frequency components of input images using wavelet transforms. Based on this observation, we design a simple yet effective frequency-aware defense strategy that reconstructs training views by filtering high-frequency noise while preserving low-frequency content. This approach effectively suppresses adversarial artifacts while maintaining the authenticity of the original scene. Notably, it does not significantly impair training on clean data, achieving a desirable trade-off between robustness and performance on clean inputs. Through extensive experiments under a wide range of attack intensities on multiple benchmarks, we demonstrate that our method substantially enhances the robustness of 3DGS without access to clean ground-truth supervision. By highlighting and addressing the overlooked vulnerabilities of 3D Gaussian Splatting, our work paves the way for more robust and secure 3D reconstructions.

Yiran Qiao, Yiren Lu, Yunlai Zhou +4

Case Western Reserve University, Cleveland OH 44106, USA

3D Gaussian Splatting3D ReconstructionAdversarial RobustnessAdversarial Perturbations
AskPDF
Feb 19, 2026

Same Meaning, Different Scores: Lexical and Syntactic Sensitivity in LLM Evaluation

The rapid advancement of Large Language Models (LLMs) has established standardized evaluation benchmarks as the primary instrument for model comparison. Yet, their reliability is increasingly questioned due to sensitivity to shallow variations in input prompts. This paper examines how controlled, truth-conditionally equivalent lexical and syntactic perturbations affect the absolute performance and relative ranking of 23 contemporary LLMs across three benchmarks: MMLU, SQuAD, and AMEGA. We employ two linguistically principled pipelines to generate meaning-preserving variations: one performing synonym substitution for lexical changes, and another using dependency parsing to determine applicable syntactic transformations. Results show that lexical perturbations consistently induce substantial, statistically significant performance degradation across nearly all models and tasks, while syntactic perturbations have more heterogeneous effects, occasionally improving results. Both perturbation types destabilize model leaderboards on complex tasks. Furthermore, model robustness did not consistently scale with model size, revealing strong task dependence. Overall, the findings suggest that LLMs rely more on surface-level lexical patterns than on abstract linguistic competence, underscoring the need for robustness testing as a standard component of LLM evaluation.

Bogdan Kostić, Conor Fallon, Julian Risch +1

Berliner Hochschule für Technik (BHT) - Luxemburger Straße 10, 13467 Berlin · deepset GmbH - Zinnowitzer Straße 1, 10115 Berlin

Prompt SensitivityLLM EvaluationLanguage Model RobustnessAdversarial Perturbations
AskPDF
Feb 16, 2026

Universal Image Immunization against Diffusion-based Image Editing via Semantic Injection

Diffusion model advances have enabled powerful text-guided image editing, but also raise ethical and legal risks such as deepfakes and unauthorized use. To prevent these risks, adversarial attack-based image immunization has emerged as a promising defense against AI-driven semantic manipulation. Yet, most existing approaches require image-specific optimization or additional neural networks at inference time, hindering scalability and practicality. In this paper, we propose the first universal adversarial perturbation-based image immunization framework that generates a single, image-agnostic adversarial perturbation specifically designed for diffusion-based editing pipelines. Inspired by UAP used in targeted attacks, our method aims to generate a UAP that induces diffusion models to misinterpret the input image as a specific semantic target. Simultaneously, it suppresses original content to misdirect the model's attention during editing, thereby effectively blocking unauthorized edits by overwriting the image's original semantics via the UAP. Extensive experiments show that our method, as the first universal immunization approach, significantly outperforms several baselines in the UAP setting. Notably, despite the inherent difficulty of universal perturbations, our method achieves competitive or superior performance compared to image-specific methods under a more restricted perturbation budget, while also exhibiting strong black-box transferability across diverse diffusion models.

Chanhui Lee, Donggyu Choi, Seunghyun Shin +2

POSTECH · GIST · Yonsei University

Adversarial RobustnessAdversarial PerturbationsDiffusion-Based Image Editing
AskPDF
Feb 6, 2026

Exploring Sparsity and Smoothness of Arbitrary Lp Norms in Adversarial Attacks

Adversarial attacks against deep neural networks are commonly constructed under ℓp\ell_p norm constraints, most often using p=1p=1, p=2p=2 or p=∞p=\infty, and potentially regularized for specific demands such as sparsity or smoothness. These choices are typically made without a systematic investigation of how the norm parameter pp influences the structural and perceptual properties of adversarial perturbations. In this work, we study how the choice of pp affects sparsity and smoothness of adversarial attacks generated under ℓp\ell_p norm constraints for values of p∈[1,2]p \in [1,2]. To enable a quantitative analysis, we adopt two established sparsity measures from the literature and introduce three smoothness measures. In particular, we propose a general framework for deriving smoothness measures based on smoothing operations and additionally introduce a smoothness measure based on first-order Taylor approximations. Using these measures, we conduct a comprehensive empirical evaluation across multiple real-world image datasets and a diverse set of model architectures, including both convolutional and transformer-based networks. We show that the choice of ℓ1\ell_1 or ℓ2\ell_2 is suboptimal in most cases and the optimal pp value is dependent on the specific task. In our experiments, using ℓp\ell_p norms with p∈[1.3,1.5]p\in [1.3, 1.5] yields the best trade-off between sparse and smooth attacks. These findings highlight the importance of principled norm selection when designing and evaluating adversarial attacks.

Christof Duhme, Florian Eilers, Xiaoyi Jiang

Department of Computer Science University of Münster Münster, Germany

Adversarial AttacksAdversarial Perturbations
AskPDF
Jan 18, 2026

DDSA: Dual-Domain Strategic Attack for Spatial-Temporal Efficiency in Adversarial Robustness Testing

Image transmission and processing systems in resource-critical applications face significant challenges from adversarial perturbations that compromise mission-specific object classification. Current robustness testing methods require excessive computational resources through exhaustive frame-by-frame processing and full-image perturbations, proving impractical for large-scale deployments where massive image streams demand immediate processing. This paper presents DDSA (Dual-Domain Strategic Attack), a resource-efficient adversarial robustness testing framework that optimizes testing through temporal selectivity and spatial precision. We introduce a scenario-aware trigger function that identifies critical frames requiring robustness evaluation based on class priority and model uncertainty, and employ explainable AI techniques to locate influential pixel regions for targeted perturbation. Our dual-domain approach achieves substantial temporal-spatial resource conservation while maintaining attack effectiveness. The framework enables practical deployment of comprehensive adversarial robustness testing in resource-constrained real-time applications where computational efficiency directly impacts mission success.

Jinwei Hu, Shiyuan Meng, Yi Dong +1

University of Liverpool, Liverpool, UK · Shanghai Artificial Intelligence Laboratory, Shanghai, China

Adversarial AttacksAdversarial RobustnessAdversarial Perturbations
AskPDF
Dec 4, 2025

Counterfeit Answers: Adversarial Forgery against OCR-Free Document Visual Question Answering

Document Visual Question Answering (DocVQA) enables end-to-end reasoning grounded on information present in a document input. While recent models have shown impressive capabilities, they remain vulnerable to adversarial attacks. In this work, we introduce a novel attack scenario that aims to forge document content in a visually imperceptible yet semantically targeted manner, allowing an adversary to induce specific or generally incorrect answers from a DocVQA model. We develop specialized attack algorithms that can produce adversarially forged documents tailored to different attackers' goals, ranging from targeted misinformation to systematic model failure scenarios. We demonstrate the effectiveness of our approach against two end-to-end state-of-the-art models: Pix2Struct, a vision-language transformer that jointly processes image and text through sequence-to-sequence modeling, and Donut, a transformer-based model that directly extracts text and answers questions from document images. Our findings highlight critical vulnerabilities in current DocVQA systems and call for the development of more robust defenses. We release our open source code at https://github.com/pralab/adv-docVQA.

Marco Pintore, Maura Pintor, Dimosthenis Karatzas +1

University of Cagliari, Italy · CINI, Italy · Computer Vision Center, UAB, Spain

Visual Question AnsweringVLM RobustnessAdversarial Attacks on VLMsAdversarial PerturbationsDocument Image Understanding
AskPDF
Nov 21, 2025

Enhancing Adversarial Transferability through Block Stretch and Shrink

Input transformation-based attacks improve adversarial transferability by aggregating gradients over transformed inputs. Existing analyses mainly explain their efficacy from image diversity, semantic preservation, attention variance or hypothesis space augmentation, yet overlook the critical role of model frontend responses. In this paper, we revisit transformation-based attacks from an implicit ensemble perspective: each transformation can be viewed as a pre-processing operator before the surrogate model, inducing a distinct frontend response for gradient aggregation. Based on this view, we propose FRO, a Frontend Response-Oriented input transformation method that enriches such responses through two complementary operators. The Local Scaling Operator perturbs local content sampling via block-wise stretch-and-shrink operations, while the Projection Operator modifies global spatial organization through coherent perspective deformation. Together, they produce structured transformed views to optimize transferable adversarial perturbations. Experiments on an ImageNet subset show that FRO consistently improves black-box transferability across diverse CNN and Vision Transformer models. We further analyze the effect of implicit ensemble size and evaluate different transformation-based methods under a unified ensemble scale, demonstrating the superiority of designing input transformations from the perspective of front-end response ensembles.

Quan Liu, Feng Ye, Chenhao Lu +4

Adversarial AttacksAdversarial PerturbationsTransfer-Based Adversarial AttacksBlack-Box Adversarial AttacksAdversarial Transferability
AskPDF
Nov 12, 2025

SEBA: Sample-Efficient Black-Box Attacks on Visual Reinforcement Learning

Visual reinforcement learning has achieved remarkable progress in visual control and robotics, but its vulnerability to adversarial perturbations remains underexplored. Most existing black-box attacks focus on vector-based or discrete-action RL, and their effectiveness on image-based continuous control is limited by the large action space and excessive environment queries. We propose SEBA, a sample-efficient framework for black-box adversarial attacks on visual RL agents. SEBA integrates a shadow Q model that estimates cumulative rewards under adversarial conditions, a generative adversarial network that produces visually imperceptible perturbations, and a world model that simulates environment dynamics to reduce real-world queries. Through a two-stage iterative training procedure that alternates between learning the shadow model and refining the generator, SEBA achieves strong attack performance while maintaining efficiency. Experiments on MuJoCo and Atari benchmarks show that SEBA significantly reduces cumulative rewards, preserves visual fidelity, and greatly decreases environment interactions compared to prior black-box and white-box methods. The code is available at https://github.com/tairanhuang/seba online.

Tairan Huang, Yulin Jin, Junxu Liu +2

The Hong Kong Polytechnic University Hong Kong, China

Adversarial AttacksAdversarial PerturbationsBlack-Box Adversarial AttacksVisual RL
AskPDF
Nov 12, 2025

Unveiling Hidden Threats: Using Fractal Triggers to Boost Stealthiness of Distributed Backdoor Attacks in Federated Learning

Traditional distributed backdoor attacks (DBA) in federated learning improve stealthiness by decomposing global triggers into sub-triggers, which however requires more poisoned data to maintian the attck strength and hence increases the exposure risk. To overcome this defect, This paper proposes a novel method, namely Fractal-Triggerred Distributed Backdoor Attack (FTDBA), which leverages the self-similarity of fractals to enhance the feature strength of sub-triggers and hence significantly reduce the required poisoning volume for the same attack strength. To address the detectability of fractal structures in the frequency and gradient domains, we introduce a dynamic angular perturbation mechanism that adaptively adjusts perturbation intensity across the training phases to balance efficiency and stealthiness. Experiments show that FTDBA achieves a 92.3% attack success rate with only 62.4% of the poisoning volume required by traditional DBA methods, while reducing the detection rate by 22.8% and KL divergence by 41.2%. This study presents a low-exposure, high-efficiency paradigm for federated backdoor attacks and expands the application of fractal features in adversarial sample generation.

Jian Wang, Hong Shen, Chan-Tong Lam

Faculty of Applied Sciences, Macao Polytechnic University, Macao SAR, China · School of Cyberspace Security, Guangzhou University of Software, Guangzhou, China · School of Engineering and Technology, Central Queensland University, Australia

Adversarial AttacksBackdoor Attacks in Federated LearningAdversarial PerturbationsFederated Learning
AskPDF
Oct 19, 2025

UNDREAM: Bridging Differentiable Rendering and Photorealistic Simulation for End-to-end Adversarial Attacks

Deep learning models deployed in safety critical applications like autonomous driving use simulations to test their robustness against adversarial attacks in realistic conditions. However, these simulations are non-differentiable, forcing researchers to create attacks that do not integrate simulation environmental factors, reducing attack success. To address this limitation, we introduce UNDREAM, the first software framework that bridges the gap between photorealistic simulators and differentiable renderers to enable end-to-end optimization of adversarial perturbations on any 3D objects. UNDREAM enables manipulation of the environment by offering complete control over weather, lighting, backgrounds, camera angles, trajectories, and realistic human and object movements, thereby allowing the creation of diverse scenes. We showcase a wide array of distinct physically plausible adversarial objects that UNDREAM enables researchers to swiftly explore in different configurable environments. This combination of photorealistic simulation and differentiable optimization opens new avenues for advancing research of physical adversarial attacks.

Mansi Phute, Matthew Hull, Haoran Wang +6

Georgia Institute of Technology, USA · Technology Innovation Institute, UAE

Adversarial AttacksAdversarial PerturbationsPhysical Adversarial Attacks
AskPDF
Jul 23, 2025

On the Interaction of Compressibility and Adversarial Robustness

As demands for resource efficiency and safety in modern neural networks intensify, substantial research effort has gone into model compression and adversarial robustness. Yet despite progress on each in isolation, a systematic understanding of how compressibility shapes robustness remains elusive. In this paper, we develop a principled framework to analyze how different forms of structured compressibility - such as neuron-level and spectral compressibility - affect adversarial robustness. We show that structured compressibility can induce a small number of highly sensitive directions in the representation space, which adversaries can exploit to construct effective perturbations. Our analysis yields a robustness bound that reveals how neuron and spectral compressibility impact ℓ∞\ell_\infty and ℓ2\ell_2 robustness via their effects on the learned representations. Crucially, the vulnerabilities we identify arise irrespective of how compressibility is achieved - whether via regularization, architectural bias, or learning dynamics. Through empirical evaluations across synthetic and realistic tasks, we confirm our theoretical predictions, and further demonstrate that these vulnerabilities persist under adversarial training and transfer learning, and contribute to the emergence of universal adversarial examples. Our findings show a fundamental tension between structured compressibility and robustness and highlight new pathways for designing models that are efficient and safe.

Melih Barsbey, Antônio H. Ribeiro, Umut Şimşekli +1

Department of Computing, Imperial College London, UK · Department of Information Technology, Uppsala University, Sweden · INRIA, CNRS, Département d’Informatique de l’Ecole Normale Supérieure / PSL, France

Neural Network RobustnessAdversarial RobustnessAdversarial PerturbationsNeural Network Compression
AskPDF
Jun 14, 2025

On the Existence of Consistent Adversarial Attacks in High-Dimensional Linear Classification

What fundamentally distinguishes an adversarial attack from a misclassification due to limited model expressivity or finite data? In this work, we investigate this question in the setting of high-dimensional binary classification, where statistical effects due to limited data availability play a central role. We introduce a new error metric that precisely capture this distinction, quantifying model vulnerability to consistent adversarial attacks -- perturbations that preserve the ground-truth labels. Our main technical contribution is an exact and rigorous asymptotic characterization of these metrics in both well-specified models and latent space models, revealing different vulnerability patterns compared to standard robust error measures. The theoretical results demonstrate that as models become more overparameterized, their vulnerability to label-preserving perturbations grows, offering theoretical insight into the mechanisms underlying model sensitivity to adversarial attacks.

Matteo Vilucchio, Lenka Zdeborová, Bruno Loureiro

Information Learning and Physics Laboratory, École Polytechnique Fédérale de Lausanne (EPFL) · Statistical Physics of Computation Laboratory, École Polytechnique Fédérale de Lausanne (EPFL) · Département d’Informatique, École Normale Supérieure - PSL & CNRS, France

Adversarial AttacksAdversarial RobustnessAdversarial PerturbationsConsistent Adversarial AttacksStatistical Learning Theory
AskPDF
Feb 6, 2025

How Vulnerable Is My Learned Policy? Universal Adversarial Perturbation Attacks On Modern Behavior Cloning Policies

Imitation learning, also known as learning from demonstrations, is a popular approach to train AI models; however, the vulnerability of these models to adversarial attacks remains underexplored. We present the first systematic study of adversarial attacks, across a range of both classic and recently proposed imitation learning algorithms, including Vanilla Behavior Cloning (Vanilla BC), LSTM-GMM, Implicit Behavior Cloning (IBC), Diffusion Policy (DP), and Vector-Quantized Behavior Transformer (VQ-BET). We study the vulnerability of these methods to white-box, grey-box and black-box adversarial perturbations. Our experiments reveal that most existing methods are highly vulnerable to these attacks, including black-box transfer attacks that transfer across algorithms. White-box attacks cause at least a 65% reduction in average task success across all evaluated tasks and algorithms, while the black-box transfer attacks reduce task success by up to 88% on Lift, 99% on Can, and 100% on Square. To the best of our knowledge, we are the first to study and compare the vulnerabilities of different popular imitation learning algorithms to both white-box and black-box attacks. Our findings highlight the vulnerabilities of modern imitation learning algorithms, paving the way for future work in addressing such limitations. Videos and code are available at https://sites.google.com/view/uap-attacks-on-bc.

Akansha Kalra, Basavasagar Patil, Guanhong Tao +1

University of Utah

Adversarial RobustnessAdversarial Imitation LearningRobot Imitation LearningBehavior CloningAdversarial PerturbationsTransfer-Based Adversarial Attacks+1
AskPDF
Dec 11, 2024

Exploiting Vision Encoder Vulnerabilities for Universal Adversarial Perturbations on Large Vision-Language Models

Large Vision-Language Models (LVLMs) have achieved remarkable performance on multimodal tasks but remain highly vulnerable to small adversarial perturbations in input images. Existing attacks typically target the vision encoder's final output embeddings, implicitly treating the encoder as a uniform attack surface, while a systematic analysis of which internal components are most vulnerable has remained largely unexplored. We show such analysis is essential, as adversarial vulnerability in LVLM vision encoders is structurally concentrated rather than uniformly distributed. Building on this, we propose Vision Encoder Vulnerable-Component-Targeted Universal Adversarial Perturbation (VEV-UAP), a task-agnostic and cost-efficient attack framework. Through a component- and layer-wise analysis of attention mechanisms, we identify the value components in middle layers as critical vulnerabilities that strongly influence downstream language model behavior. VEV-UAP selectively targets these components to generate a single universal perturbation shared across images, without involving textual inputs or the language model during optimization. Experiments across multiple LVLMs and tasks show VEV-UAP achieves state-of-the-art attack success rates with reduced computational overhead. Moreover, a single VEV-UAP transfers across LVLMs sharing the same vision encoder, even when paired with different language models, making it a practical framework for scalable robustness evaluation.

Hee-Seon Kim, Minbeom Kim, Seokil Ham +1

School of Electrical Engineering, KAIST, Daejeon 34141, Republic of Korea

VLM RobustnessAdversarial Attacks on VLMsLarge Vision-Language ModelsAdversarial PerturbationsTransfer-Based Adversarial Attacks
AskPDF
Nov 26, 2024

Stealthy Multi-Task Adversarial Attacks

Deep neural networks are highly vulnerable to adversarial perturbations, raising serious safety concerns in the real-world systems. While prior work mainly explores single-task attacks or jointly degrading all tasks in multi-task models, practical scenarios often demand more selective and stealthy attack strategies. To address this challenge, we propose Stealthy Multi-Task Adversarial Attack (SMTA2^{2}), a novel framework that selectively degrades a targeted task while strictly preserving the performance of non-targeted tasks. We formulate this objective as a constrained multi-objective optimization problem and design task-aware adversarial perturbations that maximize degradation on the targeted task without causing collateral damage on non-targeted tasks. To enhance practicality, we further introduce an automated loss-weight tuning strategy that dynamically balances attack and preservation objectives. Experiments on two multi-task benchmarks NYUv2 and Cityscapes demonstrate that SMTA2^{2} achieves strong attack performance on targeted tasks while maintaining non-targeted tasks intact on both undefended and adversarially trained models, establishing the first systematic framework for stealthy and selective multi-task attack framework.

Jiacheng Guo, Tianyun Zhang, Lei Li +3

Cleveland State University, Cleveland, OH 44115, USA · University of Wisconsin-Madison, Madison, WI 53706, USA · Western Digital Research, San Jose, CA 95119, USA

Adversarial AttacksAdversarial Perturbations
AskPDF