Adversarial Attacks

Follow this topic Ask about this topic

Momentum

28 papers in the last four weeks, up 155% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 292

May 1, 2026cs.LG

Watch Your Step: Information Injection in Diffusion Models via Shadow Timestep Embedding

Diffusion models have become the foundation of modern generative systems, with most research focusing primarily on improving generation efficiency and output quality. The timestep embedding component is a crucial part of the diffusion pipeline, which provides a temporal conditioning signal to the denoising network, enabling it to adapt its predictions across different noise levels throughout the process. Despite their potential to contain substantial information, timestep embeddings remain underexplored in current research, especially for security risks and reliable provenance. To fill this gap, we introduce Shadow Timestep Embedding (STE), a novel mechanism that investigates the underutilized temporal space for malicious information injection into diffusion models. In particular, when zooming in on the timestep embedding space, we find that different timesteps exhibit distinct representational capabilities that can encode side-channel information. Moreover, such encoded information can be utilized for attack and defense purposes through the scheduler interface. We present a theoretical analysis of timestep embeddings as position-encoding mappings and derive a mutual coherence evaluation that explains the separability of disjoint timestep intervals. Our findings reveal the diffusion model's timestep as a powerful side channel for carrying dedicated information, motivating new directions for adversarial generative modeling by understanding the temporal dimension.

An Huang, Junggab Son, Zuobin Xiong

Department of Computer Science, University of Nevada Las Vegas, Las Vegas, USA.

Adversarial Attacks Diffusion Models

Apr 30, 2026cs.CR

Attention Is Where You Attack

Safety-aligned large language models rely on RLHF and instruction tuning to refuse harmful requests, yet the internal mechanisms implementing safety behavior remain poorly understood. We introduce the Attention Redistribution Attack (ARA), a white-box adversarial attack that identifies safety-critical attention heads and crafts nonsemantic adversarial tokens that redirect attention away from safety-relevant positions. Unlike prior jailbreak methods operating at the semantic or output-logit level, ARA targets the geometry of softmax attention on the probability simplex using Gumbel-softmax optimization over targeted heads. Across LLaMA-3-8B-Instruct, Mistral-7B-Instruct-v0.1, and Gemma-2-9B-it, ARA bypasses safety alignment with as few as 5 tokens and 500 optimization steps, achieving 36% ASR on Mistral-7B and 30% on LLaMA-3 against 200 HarmBench prompts, while Gemma-2 remains at 1%. Our principal mechanistic finding is a dissociation between ablation and redistribution: zeroing out the top-ranked safety heads produces at most 1 flip among 39 to 50 baseline refusals, while ARA targeting the corresponding safety-heavy layers flips 72/200 prompts on Mistral-7B and 60/200 on LLaMA-3. This suggests that safety is not localized in these heads as removable components, but emerges from the attention routing they perform. Removing a head allows compensation through the residual stream, while redirecting its attention propagates a corrupted signal downstream.

Aviral Srivastava, Sourav Panda

Amazon · The Pennsylvania State University

Self-Attention Adversarial Attacks Adversarial Attacks on LLMs LLM Safety Alignment Attention Mechanisms Causal Interventions in Language Models+1

Apr 30, 2026cs.LG

Low Rank Adaptation for Adversarial Perturbation

Low-Rank Adaptation (LoRA), which leverages the insight that model updates typically reside in a low-dimensional space, has significantly improved the training efficiency of Large Language Models (LLMs) by updating neural network layers using low-rank matrices. Since the generation of adversarial examples is an optimization process analogous to model training, this naturally raises the question: Do adversarial perturbations exhibit a similar low-rank structure? In this paper, we provide both theoretical analysis and extensive empirical investigation across various attack methods, model architectures, and datasets to show that adversarial perturbations indeed possess an inherently low-rank structure. This insight opens up new opportunities for improving both adversarial attacks and defenses. We mainly focus on leveraging this low-rank property to improve the efficiency and effectiveness of black-box adversarial attacks, which often suffer from excessive query requirements. Our method follows a two-step approach. First, we use a reference model and auxiliary data to guide the projection of gradients into a low-dimensional subspace. Next, we confine the perturbation search in black-box attacks to this low-rank subspace, significantly improving the efficiency and effectiveness of the adversarial attacks. We evaluated our approach across a range of attack methods, benchmark models, datasets, and threat models. The results demonstrate substantial and consistent improvements in the performance of our low-rank adversarial attacks compared to conventional methods.

Han Liu, Shanghao Shi, Yevgeniy Vorobeychik +2

Washington University in St. Louis

Adversarial Attacks Low-Rank Approximation Adversarial Perturbations Black-Box Adversarial Attacks

Apr 28, 2026cs.CV

One Perturbation, Two Failure Modes: Probing VLM Safety via Embedding-Guided Typographic Perturbations

Typographic prompt injection exploits vision language models' (VLMs) ability to read text rendered in images, posing a growing threat as VLMs power autonomous agents. Prior work typically focus on maximizing attack success rate (ASR) but does not explain \emph{why} certain renderings bypass safety alignment. We make two contributions. First, an empirical study across four VLMs including GPT-4o and Claude, twelve font sizes, and ten transformations reveals that multimodal embedding distance strongly predicts ASR (

r{=}{-}0.71

to

{-}0.93

,

p{<}0.01

), providing an interpretable, model agnostic proxy. Since embedding distance predicts ASR, reducing it should improve attack success, but the relationship is mediated by two factors: perceptual readability (whether the VLM can parse the text) and safety alignment (whether it refuses to comply). Second, we use this as a red teaming tool: we directly maximize image text embedding similarity under bounded

\ell_\infty

perturbations via CWA-SSA across four surrogate embedding models, stress testing both factors without access to the target model. Experiments across five degradation settings on GPT-4o, Claude Sonnet 4.5, Mistral-Large-3, and Qwen3-VL confirm that optimization recovers readability and reduces safety aligned refusals as two co-occurring effects, with the dominant mechanism depending on the model's safety filter strength and the degree of visual degradation.

Ravikumar Balakrishnan, Sanket Mendapara

Cisco Systems

VLM Evaluation VLM Robustness Adversarial Attacks on VLMs Adversarial Attacks

Apr 26, 2026cs.CV

Adversarial Flow Matching for Imperceptible Attacks on End-to-End Autonomous Driving

Autonomous driving (AD) is evolving towards end-to-end (E2E) frameworks through two primary paradigms: monolithic models exemplified by Vision-Language-Action (VLA), and specialized modular architectures. Despite their divergent designs, both paradigms increasingly rely on Transformer backbones for complex reasoning, potentially causing a shared vulnerability: visually imperceptible perturbations can manipulate E2E AD models into hazardous maneuvers by targeting the Transformer module. Most existing adversarial attack approaches against AD systems operate under white-box or black-box settings; yet, they typically necessitate full model transparency, or suffer from either prohibitive query latency or limited attack transferability. In this paper, we propose Adversarial Flow Matching (AFM), a novel gray-box attack framework that exploits Transformer structural vulnerabilities in E2E AD models. AFM enables efficient one-step generation of adversarial examples via a neural average velocity field. Additionally, the proposed technique yields effective and visually imperceptible attacks by synergistically perturbing the generative latent space and the neural average velocity field. Extensive experiments demonstrate that AFM achieves a superior trade-off between attack effectiveness and imperceptibility: it substantially degrades the performance of both VLA and modular AD agents across various scenarios compared to baselines, while maintaining state-of-the-art visual imperceptibility. Furthermore, adversarial examples generated by AFM exhibit robust cross-model transferability, indicating that AFM closely approximates a black-box attack setting while requiring only the prior knowledge that the target AD model incorporates a Transformer-based module.

Xinyu Zeng, Xiangkun He, Lei Tao +2

Shenzhen Institute for Advanced Study, University of Electronic Science and Technology of China, Shenzhen 518110, China · School of Mechanical and Aerospace Engineering, Nanyang Technological University, Singapore 639798 · School of Mechanical and Electrical Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China

Flow Matching End-to-End Autonomous Driving Adversarial Attacks Adversarial Transferability

Apr 25, 2026cs.SE

UniAda: Universal Adaptive Multi-objective Adversarial Attack for End-to-End Autonomous Driving Systems

Adversarial attacks play a pivotal role in testing and improving the reliability of deep learning (DL) systems. Existing literature has demonstrated that subtle perturbations to the input can elicit erroneous outcomes, thereby substantially compromising the security of DL systems. This has emerged as a critical concern in the development of DL-based safety-critical systems like Autonomous Driving Systems (ADSs). The focus of existing adversarial attack methods on End-to-End (E2E) ADSs has predominantly centered on misbehaviors of steering angle, which overlooks speed-related controls or imperceptible perturbations. To address these challenges, we introduce UniAda, a multi-objective white-box attack technique with a core function that revolves around crafting an image-agnostic adversarial perturbation capable of simultaneously influencing both steering and speed controls. UniAda capitalizes on an intricately designed multi-objective optimization function with the Adaptive Weighting Scheme (AWS), enabling the concurrent optimization of diverse objectives. Validated with both simulated and real-world driving data, UniAda outperforms five benchmarks across two metrics, inducing steering and speed deviations from 3.54 degrees to 29 degrees and 11 km per hour to 22 km per hour on average. This systematic approach establishes UniAda as a proven technique for adversarial attacks on modern DL-based E2E ADSs.

Jingyu Zhang, Jacky Wai Keung, Yan Xiao +3

Department of Computer Science, City University of Hong Kong, Hong Kong SAR · School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-sen University, Shenzhen, China

End-to-End Autonomous Driving Adversarial Attacks Adversarial Perturbations

Apr 24, 2026cs.CR

Adversarial Malware Generation in Linux ELF Binaries via Semantic-Preserving Transformations

Malware development and detection have undergone significant changes in recent years as modern concepts, such as machine learning, have been used for both adversarial attacks and defense. Despite intensive research on Windows Portable Executable (PE) files, there is minimal work on Linux Executable and Linkable Format (ELF). In this work, we summarize the academic papers submitted in this field and develop a new adversarial malware generator for the ELF format. Using a variety of metrics, we thoroughly evaluated our generator and achieved an Evasion Rate of 67.74 % while changing the confidence of the malware detector by -0.50 in the mean case for the dataset used. In our approach, we chose MalConv as the target classifier. Using this classifier, we found that the most successful modifications used strings typical of benign files as a data source. We conducted a variety of experiments and concluded that the target classifier appears sensitive to strings at any location within the executable file.

Lukáš Hrdonka, Martin Jureček

Faculty of Information Technology, Czech Technical University in Prague, Thákurova 9, Prague, Czech Republic

Adversarial Attacks Malware Classification Malware Analysis Malware Detection

Apr 24, 2026cs.CR

Adversarial Co-Evolution of Malware and Detection Models: A Bilevel Optimization Perspective

Machine learning-based malware detectors are increasingly vulnerable to adversarial examples. Traditional defenses, such as one-shot adversarial training, often fail against adaptive attackers who use reinforcement learning to bypass detection. This paper proposes a robust defense framework based on bilevel optimization, explicitly modeling the strategic interaction between a defender and an attacker as an adversarial co-evolutionary process. We evaluate our approach using the MAB-malware framework against three distinct malware families: Mokes, Strab, and DCRat. Our experimental results demonstrate that while standard classifiers and basic adversarial retraining often remain vulnerable, showing evasion rates as high as 90 %, the proposed bilevel optimization approach consistently achieves near-total immunity, reducing evasion rates to 0 - 1.89 %. Furthermore, the iterative framework significantly increases the attacker's query complexity, raising the average cost of successful evasion by up to two orders of magnitude. These findings suggest that modeling the iterative cycle of attack and defense through bilevel optimization is essential for developing resilient malware detection systems capable of withstanding evolving adversarial threats.

Olha Jurečková, Martin Jureček, Matouš Kozák +1

Faculty of Information Technology, Czech Technical University in Prague, Th´akurova 9, Prague, 16000, Czech Republic

Adversarial Attacks Adversarial Robustness Bilevel Optimization Malware Analysis Malware Detection

Apr 24, 2026cs.LG

Sharpness-Aware Poisoning: Enhancing Transferability of Injective Attacks on Recommender Systems

Recommender Systems~(RS) have been shown to be vulnerable to injective attacks, where attackers inject limited fake user profiles to promote the exposure of target items to real users for unethical gains (e.g., economic or political advantages). Since attackers typically lack knowledge of the victim model deployed in the target RS, existing methods resort to using a fixed surrogate model to mimic the potential victim model. Despite considerable progress, we argue that the assumption that \textit{poisoned data generated for the surrogate model can be used to attack other victim models} is wishful. When there are significant structural discrepancies between the surrogate and victim models, the attack transferability inevitably suffers. Intuitively, if we can identify the worst-case victim model and iteratively optimize the poisoning effect specifically against it, then the generated poisoned data would be better transferred to other victim models. However, exactly identifying the worst-case victim model during the attack process is challenging due to the large space of victim models. To this end, in this work, we propose a novel attack method called Sharpness-Aware Poisoning (\textit{SharpAP}). Specifically, it employs the sharpness-aware minimization principle to seek the approximately worst-case victim model and optimizes the poisoned data specifically for this worst-case model. The poisoning attack with SharpAP is formulated as a min-max-min tri-level optimization problem. By integrating SharpAP into the iterative process for attacks, our method can generate more robust poisoned data which is less sensitive to the shift of model structure, mitigating the overfitting to the surrogate model. Comprehensive experimental comparisons on three real-world datasets demonstrate that \name~can significantly enhance the attack transferability.

Junsong Xie, Yonghui Yang, Pengyang Shao +1

Key Laboratory of Knowledge Engineering with Big Data, Hefei University of Technology, Hefei 230009, China · Intelligent Interconnected Systems Laboratory of Anhui Province, Hefei University of Technology, Hefei 230009, China · Laboratory of NExT++, National University of Singapore, Singapore

Sharpness-Aware Minimization Adversarial Attacks Recommender Systems Data Poisoning Attacks

Apr 23, 2026cs.CR

Adversarial Evasion in Non-Stationary Malware Detection: Minimizing Drift Signals through Similarity-Constrained Perturbations

Deep learning has emerged as a powerful approach for malware detection, demonstrating impressive accuracy across various data representations. However, these models face critical limitations in real-world, non-stationary environments where both malware characteristics and detection systems continuously evolve. Our research investigates a fundamental security question: Can an attacker generate adversarial malware samples that simultaneously evade classification and remain inconspicuous to drift monitoring mechanisms? We propose a novel approach that generates targeted adversarial examples in the classifier's standardized feature space, augmented with sophisticated similarity regularizers. By carefully constraining perturbations to maintain distributional similarity with clean malware, we create an optimization objective that balances targeted misclassification with drift signal minimization. We quantify the effectiveness of this approach by comprehensively comparing classifier output probabilities using multiple drift metrics. Our experiments demonstrate that similarity constraints can reduce output drift signals, with

\ell_2

regularization showing the most promising results. We observe that perturbation budget significantly influences the evasion-detectability trade-off, with increased budget leading to higher attack success rates and more substantial drift indicators.

Pawan Acharya, Lan Zhang

Northern Arizona University, Flagstaff, AZ, USA

Concept Drift Adversarial Attacks Adversarial Perturbations Malware Analysis Malware Detection

Apr 22, 2026cs.CR

Towards Certified Malware Detection: Provable Guarantees Against Evasion Attacks

Machine learning-based static malware detectors remain vulnerable to adversarial evasion techniques, such as metamorphic engine mutations. To address this vulnerability, we propose a certifiably robust malware detection framework based on randomized smoothing through feature ablation and targeted noise injection. During evaluation, our system analyzes an executable by generating multiple ablated variants, classifies them by using a smoothed classifier, and identifies the final label based on the majority vote. By analyzing the top-class voting distribution and the Wilson score interval, we derive a formal certificate that guarantees robustness within a specific radius against feature-space perturbations. We evaluate our approach by comparing the performance of the base classifier and the smoothed classifier on both clean executables and ablated variants generated using PyMetaEngine. Our results demonstrate that the proposed smoothed classifier successfully provides certifiable robustness against metamorphic evasion attacks without requiring modifications to the underlying machine learning architecture.

Nandakrishna Giri, Asmitha K. A., Serena Nicolazzo +2

Department of Computer Applications, Cochin University of Science and Technology, India · Department of Electrical, Computer and Biomedical Engineering, University of Pavia, Via Adolfo Ferrata 5, 27100, Pavia, Italy

Randomized Smoothing Adversarial Attacks Malware Classification Certified Robustness Malware Analysis Malware Detection

Apr 21, 2026cs.CV

PASTA: A Patch-Agnostic Twofold-Stealthy Backdoor Attack on Vision Transformers

Vision Transformers (ViTs) have achieved remarkable success across vision tasks, yet recent studies show they remain vulnerable to backdoor attacks. Existing patch-wise attacks typically assume a single fixed trigger location during inference to maximize trigger attention. However, they overlook the self-attention mechanism in ViTs, which captures long-range dependencies across patches. In this work, we observe that a patch-wise trigger can achieve high attack effectiveness when activating backdoors across neighboring patches, a phenomenon we term the Trigger Radiating Effect (TRE). We further find that inter-patch trigger insertion during training can synergistically enhance TRE compared to single-patch insertion. Prior ViT-specific attacks that maximize trigger attention often sacrifice visual and attention stealthiness, making them detectable. Based on these insights, we propose PASTA, a twofold stealthy patch-wise backdoor attack in both pixel and attention domains. PASTA enables backdoor activation when the trigger is placed at arbitrary patches during inference. To achieve this, we introduce a multi-location trigger insertion strategy to enhance TRE. However, preserving stealthiness while maintaining strong TRE is challenging, as TRE is weakened under stealthy constraints. We therefore formulate a bi-level optimization problem and propose an adaptive backdoor learning framework, where the model and trigger iteratively adapt to each other to avoid local optima. Extensive experiments show that PASTA achieves 99.13% attack success rate across arbitrary patches on average, while significantly improving visual and attention stealthiness (144.43x and 18.68x) and robustness (2.79x) against state-of-the-art ViT defenses across four datasets, outperforming CNN- and ViT-based baselines.

Dazhuang Liu, Yanqi Qiao, Rui Wang +2

Delft University of Technology, Delft, the Netherlands. · University of Turku, Turku, Finland

Adversarial Attacks Vision Transformer Backdoor Attacks

Apr 20, 2026cs.CR

Temporal UI State Inconsistency in Desktop GUI Agents: Formalizing and Defending Against TOCTOU Attacks on Computer-Use Agents

GUI agents that control desktop computers via screenshot-and-click loops introduce a new class of vulnerability: the observation-to-action gap (mean 6.51 s on real OSWorld workloads) creates a Time-Of-Check, Time-Of-Use (TOCTOU) window during which an unprivileged attacker can manipulate the UI state. We formalize this as a Visual Atomicity Violation and characterize three concrete attack primitives: (A) Notification Overlay Hijack, (B) Window Focus Manipulation, and (C) Web DOM Injection. Primitive B, the closest desktop analog to Android Action Rebinding, achieves 100% action-redirection success rate with zero visual evidence at the observation time. We propose Pre-execution UI State Verification (PUSV), a lightweight three-layer defense that re-verifies the UI state immediately before each action dispatch: masked pixel SSIM at the click target (L1), global screenshot diff (L2a), and X Window snapshot diff (L2b). PUSV achieves 100% Action Interception Rate across 180 adversarial trials (135 Primitive A + 45 Primitive B) with zero false positives and < 0.1 s overhead. Against Primitive C (zero-visual-footprint DOM injection), PUSV reveals a structural blind spot (~0% AIR), motivating future OS+DOM defense-in-depth architectures. No single PUSV layer alone achieves full coverage; different primitives require different detection signals, validating the layered design.

Wenpeng Xu

University of California, San Diego

Adversarial Attacks Computer-Use Agents AI Agent Security Runtime Verification GUI Automation

Apr 20, 2026cs.LG

Perturbation Sensitivity of Maximum-Likelihood Pairwise Ranking in Computational Decision Systems

Maximum-likelihood pairwise ranking is a com- mon computational mechanism for prioritization, reputation estimation, and comparison-driven decision support. Despite its broad use, the perturbation sensitivity of this estimator under structured changes in comparison data remains insufficiently characterized. We study this question as an applied-mathematics and computational-science problem in stability analysis. We for- mulate coordinated perturbation as a budgeted subset-selection problem over pairwise observations and introduce an Adaptive Subset Selection Attack (ASSA) as a scalable search heuristic for probing high-impact perturbation sets. Through experiments on synthetic and observed preference datasets, we show that MLE-based ranking can exhibit pronounced regime-dependent sensitivity: relatively small but coordinated perturbations may in- duce meaningful changes in output orderings, while the response profile varies across budgets and data conditions. By comparing ASSA with random, greedy, and randomized subset baselines under repeated trials, we characterize both the magnitude and the variability of perturbation-induced ranking shifts. These results position pairwise ranking sensitivity as a problem in computational reliability, numerical stability, and robustness auditing for engineering systems built on comparison-driven inference.

Junyi Yao, Zihao Zheng, Jiayu Long

Washington University in St. Louis USA

Algorithmic Auditing Learning to Rank Adversarial Attacks Pairwise Ranking Adversarial Perturbations

Apr 16, 2026cs.LG

No More Guessing: a Verifiable Gradient Inversion Attack in Federated Learning

Gradient inversion attacks threaten client privacy in federated learning by reconstructing training samples from clients' shared gradients. Gradients aggregate contributions from multiple records and existing attacks may fail to disentangle them, yielding incorrect reconstructions with no intrinsic way to certify success. In vision and language, attackers may fall back on human inspection to judge reconstruction plausibility, but this is far less feasible for numerical tabular records, fueling the impression that tabular data is less vulnerable. We challenge this perception by proposing a verifiable gradient inversion attack (VGIA) that provides an explicit certificate of correctness for reconstructed samples. Our method adopts a geometric view of ReLU leakage: the activation boundary of a fully connected layer defines a hyperplane in input space. VGIA introduces an algebraic, subspace-based verification test that detects when a hyperplane-delimited region contains exactly one record. Once isolation is certified, VGIA recovers the corresponding feature vector analytically and reconstructs the target via a lightweight optimization step. Experiments on tabular benchmarks with large batch sizes demonstrate exact record and target recovery in regimes where existing state-of-the-art attacks either fail or cannot assess reconstruction fidelity. Compared to prior geometric approaches, VGIA allocates hyperplane queries more effectively, yielding faster reconstructions with fewer attack rounds.

Francesco Diana, Chuan Xu, André Nusser +1

Université Côte d’Azur, France · Inria, France · CNRS, France +1

Gradient Inversion Attacks Adversarial Attacks Federated Learning

Apr 11, 2026cs.LG

When Can You Poison Rewards? A Tight Characterization of Reward Poisoning in Linear MDPs

We study reward poisoning attacks in reinforcement learning (RL), where an adversary manipulates rewards within constrained budgets to force the target RL agent to adopt a policy that aligns with the attacker's objectives. Prior works on reward poisoning mainly focused on sufficient conditions to design a successful attacker, while only a few studies discussed the infeasibility of targeted attacks. This paper provides the first precise necessity and sufficiency characterization of the attackability of a linear MDP under reward poisoning attacks. Our characterization draws a bright line between the vulnerable RL instances, and the intrinsically robust ones which cannot be attacked without large costs even running vanilla non-robust RL algorithms. Our theory extends beyond linear MDPs -- by approximating deep RL environments as linear MDPs, we show that our theoretical framework effectively distinguishes the attackability and efficiently attacks the vulnerable ones, demonstrating both the theoretical and practical significance of our characterization.

Jose Efraim Aguilar Escamilla, Haoyang Hong, Jiawei Li +4

Oregon State University, OR, USA · University of Illinois Urbana–Champaign, IL, USA · Princeton University, NJ, USA +1

Markov Decision Processes Adversarial Attacks Linear Markov Decision Processes Robust RL

Mar 19, 2026cs.LG

On Optimizing Multimodal Jailbreaks for Spoken Language Models

As Spoken Language Models (SLMs) integrate speech and text modalities, they inherit the safety vulnerabilities of their LLM backbone while introducing an expanded attack surface. SLMs have been previously shown to be susceptible to jailbreaking, where adversarial prompts induce harmful responses. Yet existing attacks largely remain unimodal, optimizing either text or audio in isolation. We explore gradient-based multimodal jailbreaks by introducing JAMA (Joint Audio-text Multimodal Attack), a joint multimodal optimization framework combining Greedy Coordinate Gradient (GCG) for text and Projected Gradient Descent (PGD) for audio, to simultaneously perturb both modalities. Evaluations across four state-of-the-art SLMs and four audio types demonstrate that JAMA surpasses unimodal jailbreak rate by 1.5x to 20x. We analyze the operational dynamics of this joint attack and show that a sequential approximation method makes it 4x to 6x faster. Our findings suggest that unimodal safety is insufficient for robust SLMs. The code and data are available at https://repos.lsv.uni-saarland.de/akrishnan/multimodal-jailbreak-slm.

Aravind Krishnan, Karolina Stańczak, Dietrich Klakow

Saarland University, Germany · DFKI GmbH, Saarbrücken, Germany · ETH AI Center, Switzerland

Adversarial Attacks LLM Jailbreak Attacks Multimodal Jailbreak Attacks Speech Language Models

Mar 4, 2026cs.LG

Dual-Modality Multi-Stage Adversarial Safety Training: Robustifying Multimodal Web Agents Against Cross-Modal Attacks

Multimodal web agents that process both screenshots and accessibility trees are increasingly deployed to interact with web interfaces, yet their dual-stream architecture opens an underexplored attack surface: an adversary who injects content into the webpage DOM simultaneously corrupts both observation channels with a consistent deceptive narrative. Our vulnerability analysis on MiniWob++ reveals that attacks including a visual component far outperform text-only injections, exposing critical gaps in text-centric VLM safety training. Motivated by this finding, we propose Dual-Modality Multi-Stage Adversarial Safety Training (DMAST), a framework that formalizes the agent-attacker interaction as a two-player general-sum Markov game and co-trains both players through a three-stage pipeline: (1) imitation learning from a strong teacher model, (2) oracle-guided supervised fine-tuning that uses a novel zero-acknowledgment strategy to instill task-focused reasoning under adversarial noise, and (3) adversarial reinforcement learning via Group Relative Policy Optimization (GRPO) self-play. On out-of-distribution tasks, DMAST nearly halves the attack success rate (41.2%

\rightarrow

21.4%) while raising task completion by over 60% relative (6.2%

\rightarrow

10.2%). Our approach outperforms established training-based defenses and complements prompt-based defenses, demonstrating genuine co-evolutionary progress and robust generalization to complex, unseen environments. Code is available at https://github.com/huajianduzhuo-code/DMAST_official.

Haoyu Liu, Dingcheng Li, Lukas Rutishauser +1

UC Berkeley, IEOR & BAIR · Google · Google Deepmind

Adversarial Training Multimodal Robustness Adversarial Attacks Multimodal Agents Web Agents Multimodal Understanding

Feb 20, 2026cs.LG

Generating adversarial inputs for a graph neural network model of AC power flow

This work formulates and solves optimization problems to generate input points that yield high errors between a neural network's predicted AC power flow solution and solutions to the AC power flow equations. We demonstrate this capability on an instance of the CANOS-PF graph neural network model, as implemented by the PF

Δ

benchmark library, operating on a 14-bus test grid. Generated adversarial points yield errors as large as 3.7 per-unit in reactive power and 0.08 per-unit in voltage magnitude. When minimizing the perturbation from a training point necessary to satisfy adversarial constraints, we find that the constraints can be met with as little as an 0.04 per-unit perturbation in voltage magnitude on a single bus. This work motivates the development of rigorous verification and robust training methods for neural network surrogate models of AC power flow.

Robert Parker

Los Alamos National Laboratory, Los Alamos, NM, USA

Neural Surrogate Modeling Graph Neural Networks Adversarial Attacks Power Systems

Feb 6, 2026cs.LG

Temperature Scaling Attack Disrupting Model Confidence in Federated Learning

Predictive confidence serves as a foundational control signal in mission-critical systems, directly governing risk-aware logic such as escalation, abstention, and conservative fallback. While prior federated learning attacks predominantly target accuracy or implant backdoors, we identify confidence calibration as a distinct attack objective. We present the Temperature Scaling Attack (TSA), a training-time attack that degrades calibration while preserving accuracy. By injecting temperature scaling with learning rate-temperature coupling during local training, TSA shifts model confidence while keeping predictive accuracy and common optimization signals close to benign training. We provide a convergence analysis under non-IID settings, showing that the coupling controls the primary update scale while leaving a bounded temperature-induced residual, yielding the standard non-convex FL convergence structure with an additional residual term. Across three benchmarks, TSA substantially shifts calibration (e.g., 145% error increase on CIFAR-100) with <2% accuracy change, and remains effective under robust aggregation and post-hoc calibration defenses. Case studies further show up to a 7.2x increase in missed verifications in healthcare and severe confidence-gating failures in autonomous driving, even when accuracy is unchanged. Overall, our results establish calibration integrity as a critical attack surface in federated learning.

Kichang Lee, Jaeho Jin, JaeYeon Park +2

Yonsei University · Dankook University

Adversarial Attacks Federated Learning

Feb 6, 2026cs.LG

Exploring Sparsity and Smoothness of Arbitrary Lp Norms in Adversarial Attacks

Adversarial attacks against deep neural networks are commonly constructed under

\ell_p

norm constraints, most often using

p=1

,

p=2

or

p=\infty

, and potentially regularized for specific demands such as sparsity or smoothness. These choices are typically made without a systematic investigation of how the norm parameter

p

influences the structural and perceptual properties of adversarial perturbations. In this work, we study how the choice of

p

affects sparsity and smoothness of adversarial attacks generated under

\ell_p

norm constraints for values of

p \in [1,2]

. To enable a quantitative analysis, we adopt two established sparsity measures from the literature and introduce three smoothness measures. In particular, we propose a general framework for deriving smoothness measures based on smoothing operations and additionally introduce a smoothness measure based on first-order Taylor approximations. Using these measures, we conduct a comprehensive empirical evaluation across multiple real-world image datasets and a diverse set of model architectures, including both convolutional and transformer-based networks. We show that the choice of

\ell_1

or

\ell_2

is suboptimal in most cases and the optimal

p

value is dependent on the specific task. In our experiments, using

\ell_p

norms with

p\in [1.3, 1.5]

yields the best trade-off between sparse and smooth attacks. These findings highlight the importance of principled norm selection when designing and evaluating adversarial attacks.

Christof Duhme, Florian Eilers, Xiaoyi Jiang

Department of Computer Science University of Münster Münster, Germany

Adversarial Attacks Adversarial Perturbations

Feb 6, 2026cs.LG

Perturbing the Phase: Analyzing Adversarial Robustness of Complex-Valued Neural Networks

Complex-valued neural networks (CVNNs) are rising in popularity for all kinds of applications. To safely use CVNNs in practice, analyzing their robustness against outliers is crucial. One well known technique to understand the behavior of deep neural networks is to investigate their behavior under adversarial attacks, which can be seen as worst case minimal perturbations. We design Phase Attacks, a kind of attack specifically targeting the phase information of complex-valued inputs. Additionally, we derive complex-valued versions of commonly used adversarial attacks. We show that in some scenarios CVNNs are more robust than RVNNs and that both are very susceptible to phase changes with the Phase Attacks decreasing the model performance more, than equally strong regular attacks, which can attack both phase and magnitude.

Florian Eilers, Christof Duhme, Xiaoyi Jiang

Department of Computer Science University of Münster Münster, Germany

Adversarial Attacks Neural Network Robustness Adversarial Robustness Complex-Valued Neural Networks

Feb 4, 2026cs.CV

Stage-wise Attention-Guided Region Sequencing for Adversarial Attacks on Large Vision-Language Models

Targeted adversarial attacks on Large Vision-Language Models (LVLMs) test whether small image perturbations can steer model responses toward attacker-specified content. Under the standard L-infinity constraint, targeted attacks become a regional perturbation budget allocation problem: attack success depends not only on the perturbation objective, but also on which regions receive updates and in what order. Existing localized attacks improve over global perturbations but rely on stochastic spatial sampling, often updating weakly influential regions. We address this limitation through an attention-based analysis showing that cross-modal attention identifies adversarially sensitive regions and that perturbing high-attention hotspots induces predictable redistribution toward subsequent salient regions. These findings motivate attention-guided region sequencing, which begins from dominant hotspots and progressively moves the update support toward next-salient regions. Based on these principles, we propose Stage-wise Attention-Guided Attack (SAGA), a black-box region-sequencing framework that uses a fixed attention map from an open-source LVLM to guide perturbation updates without accessing target-model parameters, gradients, or attention maps. Across ten closed-source and open-source LVLMs, SAGA achieves state-of-the-art attack success rates and the best overall imperceptibility. The source code is available at https://github.com/jaehyun-kwak/SAGA.

Jaehyun Kwak, Nam Cao, Boryeong Cho +3

KAIST · KENTECH

Adversarial Attacks on VLMs Adversarial Attacks Large Vision-Language Models Black-Box Adversarial Attacks

Jan 30, 2026cs.CR

From Similarity to Vulnerability: Key Collision Attack on LLM Semantic Caching

Semantic caching has emerged as a pivotal technique for scaling LLM applications, widely adopted by major providers including AWS and Microsoft. By utilizing semantic embedding vectors as cache keys, this mechanism effectively minimizes latency and redundant computation for semantically similar queries. In this work, we conceptualize semantic cache keys as a form of fuzzy hashes. We demonstrate that the locality required to maximize cache hit rates fundamentally conflicts with the cryptographic avalanche effect necessary for collision resistance. Our conceptual analysis formalizes this inherent trade-off between performance (locality) and security (collision resilience), revealing that semantic caching is naturally vulnerable to key collision attacks. While prior research has focused on side-channel and privacy risks, we present the first systematic study of integrity risks arising from cache collisions. We introduce CacheAttack, an automated framework for launching black-box collision attacks. We evaluate CacheAttack in security-critical tasks and agentic workflows. It achieves a hit rate of 86% in LLM response hijacking and can induce malicious behaviors in LLM agent, while preserving strong transferability across different embedding models. A case study on a financial agent further illustrates the real-world impact of these vulnerabilities. Finally, we discuss mitigation strategies.

Zhixiang Zhang, Zesen Liu, Yuchong Xie +2

Department of Computer Science and Engineering, The Hong Kong University of Science and Technology · Fudan University

Adversarial Attacks LLM Agent Security LLM Security Semantic Caching

Jan 18, 2026cs.CR

Mind the Gap: Action Rebinding Attacks against Android GUI Agents

Large multimodal model powered GUI agents are emerging as high-privilege operators on mobile platforms, entrusted to perceive screen content and inject inputs across application boundaries. While these agents aim to automate complex tasks, we demonstrate that their design introduces a fundamental conflict with Android's strict application sandboxing. We present a novel cross-application Action Rebinding attack, which allows a malicious application with zero dangerous permissions to hijack the agent's execution and perform privileged operations on behalf of the attacker. Our attack exploits the inevitable observation-action gap inherent in the agent's reasoning pipeline. A malicious app can render a benign ``contextual carrier'' to elicit a planned action, and then swap the foreground to a sensitive target application during the reasoning latency. The agent, unaware of the transition, unwittingly executes the action in the privileged context. We further advance this attack by weaponizing the agent's own task-recovery logic to create programmable, multi-step exploit loops , and introducing an Intent Alignment Strategy (IAS) that manipulates the agent's reasoning to rationalize the hijacked state. We evaluate our attack on six widely-used Android GUI agents. Our results demonstrate a 100% success rate for atomic action hijacking and the ability to orchestrate high-impact exploits, including unauthorized file deletion, SMS transmission, and app uninstallation, without the attacker holding any corresponding permissions. Furthermore, since the malicious application separates intent from capability and contains no privileged API calls, it achieves a 0% detection rate across commercial malware scanners (e.g., VirusTotal), highlighting a critical blind spot in current mobile security analysis. To access experimental logs and demonstration videos, please contact [email protected].

Yi Qian, Kunwei Qian, Xingbang He +7

State Key Laboratory for Novel Software Technology, Nanjing University · Hornor Device Co., Ltd · Institute of Dataspace, Hefei Comprehensive National Science Center China

Mobile GUI Automation Adversarial Attacks GUI Agents AI Agent Security Mobile GUI Agents

Jan 18, 2026cs.CR

DDSA: Dual-Domain Strategic Attack for Spatial-Temporal Efficiency in Adversarial Robustness Testing

Image transmission and processing systems in resource-critical applications face significant challenges from adversarial perturbations that compromise mission-specific object classification. Current robustness testing methods require excessive computational resources through exhaustive frame-by-frame processing and full-image perturbations, proving impractical for large-scale deployments where massive image streams demand immediate processing. This paper presents DDSA (Dual-Domain Strategic Attack), a resource-efficient adversarial robustness testing framework that optimizes testing through temporal selectivity and spatial precision. We introduce a scenario-aware trigger function that identifies critical frames requiring robustness evaluation based on class priority and model uncertainty, and employ explainable AI techniques to locate influential pixel regions for targeted perturbation. Our dual-domain approach achieves substantial temporal-spatial resource conservation while maintaining attack effectiveness. The framework enables practical deployment of comprehensive adversarial robustness testing in resource-constrained real-time applications where computational efficiency directly impacts mission success.

Jinwei Hu, Shiyuan Meng, Yi Dong +1

University of Liverpool, Liverpool, UK · Shanghai Artificial Intelligence Laboratory, Shanghai, China

Adversarial Attacks Adversarial Robustness Adversarial Perturbations

Jan 12, 2026cs.MA

Self-Creating Random Walks for Decentralized Learning under Pac-Man Attacks

Random walk (RW)-based algorithms have long been popular in distributed systems due to low overheads and scalability, with recent growing applications in decentralized learning. However, their reliance on local interactions makes them inherently vulnerable to malicious behavior. In this work, we investigate an adversarial threat that we term the ``Pac-Man'' attack, in which a malicious node probabilistically terminates any RW that visits it. This stealthy behavior gradually eliminates active RWs from the network, effectively halting the learning process without triggering failure alarms. To counter this threat, we propose the CREATE-IF-LATE (CIL) algorithm, which is a fully decentralized, resilient mechanism that enables self-creating RWs and prevents RW extinction in the presence of Pac-Man. Our theoretical analysis shows that the CIL algorithm guarantees several desirable properties, such as (i) non-extinction of the RW population, (ii) almost sure boundedness of the RW population, and (iii) convergence of RW-based stochastic gradient descent even in the presence of Pac-Man with a quantifiable deviation from the true optimum. Moreover, the learning process experiences at most a linear time delay due to Pac-Man interruptions and RW regeneration. Our extensive empirical results on both synthetic and public benchmark datasets validate our theoretical findings.

Xingran Chen, Parimal Parag, Rohit Bhagat +1

Engineering Systems and Design Pillar, Singapore University of Technology and Design, 8 Somapah Road, Singapore 487372 · Department of Electrical Communication Engineering, Indian Institute of Science, Bangalore, Karnataka 560012, India · Department of Electrical and Computer Engineering, Rutgers University, Piscataway Township, NJ 08854, USA

Adversarial Attacks Decentralized Learning

Dec 26, 2025cs.CV

High-Entropy Tokens as Multimodal Failure Points in Vision-Language Models

Vision-language models (VLMs) achieve remarkable performance but remain vulnerable to adversarial attacks. Entropy, as a measure of model uncertainty, is highly correlated with VLM reliability. While prior entropy-based attacks maximize uncertainty at all decoding steps, implicitly assuming that every token equally contributes to model instability, we reveal that a small fraction (around 20%) of high-entropy tokens, in the evaluated representative open-source VLMs with diverse architectures, concentrates a disproportionate share of adversarial influence during autoregressive generation. We demonstrate that concentrating adversarial perturbations on these high-entropy positions achieves comparable semantic degradation to global methods while optimizing fewer decoding positions. Additionally, across multiple representative VLMs, such attacks induce not only semantic drift but also a substantial unsafe subset (20-31%) under the current pipeline. Remarkably, since such vulnerable high-entropy tokens recur across architecturally diverse VLMs, attacks focused on them exhibit non-trivial transferability. Motivated by these findings, we design a simple Entropy-Guided Attack (EGA) that operationalizes sparse high-entropy targeting and extends it with a reusable token bank, yielding competitive attack success rates (93-95%) with a considerable harmful rate (30.2-38.6%) on the three representative open-source VLMs.

Mengqi He, Xinyu Tian, Xin Shen +4

The Australia National University Canberra, Australia · The University of Queensland Brisbane, Queensland, Australia · GE research Niskayuna, New York, USA

VLM Robustness Adversarial Attacks on VLMs Adversarial Attacks Transfer-Based Adversarial Attacks

Nov 21, 2025cs.LG

Enhancing Adversarial Transferability through Block Stretch and Shrink

Input transformation-based attacks improve adversarial transferability by aggregating gradients over transformed inputs. Existing analyses mainly explain their efficacy from image diversity, semantic preservation, attention variance or hypothesis space augmentation, yet overlook the critical role of model frontend responses. In this paper, we revisit transformation-based attacks from an implicit ensemble perspective: each transformation can be viewed as a pre-processing operator before the surrogate model, inducing a distinct frontend response for gradient aggregation. Based on this view, we propose FRO, a Frontend Response-Oriented input transformation method that enriches such responses through two complementary operators. The Local Scaling Operator perturbs local content sampling via block-wise stretch-and-shrink operations, while the Projection Operator modifies global spatial organization through coherent perspective deformation. Together, they produce structured transformed views to optimize transferable adversarial perturbations. Experiments on an ImageNet subset show that FRO consistently improves black-box transferability across diverse CNN and Vision Transformer models. We further analyze the effect of implicit ensemble size and evaluate different transformation-based methods under a unified ensemble scale, demonstrating the superiority of designing input transformations from the perspective of front-end response ensembles.

Quan Liu, Feng Ye, Chenhao Lu +4

Adversarial Attacks Adversarial Perturbations Transfer-Based Adversarial Attacks Black-Box Adversarial Attacks Adversarial Transferability

Nov 12, 2025cs.LG

SEBA: Sample-Efficient Black-Box Attacks on Visual Reinforcement Learning

Visual reinforcement learning has achieved remarkable progress in visual control and robotics, but its vulnerability to adversarial perturbations remains underexplored. Most existing black-box attacks focus on vector-based or discrete-action RL, and their effectiveness on image-based continuous control is limited by the large action space and excessive environment queries. We propose SEBA, a sample-efficient framework for black-box adversarial attacks on visual RL agents. SEBA integrates a shadow Q model that estimates cumulative rewards under adversarial conditions, a generative adversarial network that produces visually imperceptible perturbations, and a world model that simulates environment dynamics to reduce real-world queries. Through a two-stage iterative training procedure that alternates between learning the shadow model and refining the generator, SEBA achieves strong attack performance while maintaining efficiency. Experiments on MuJoCo and Atari benchmarks show that SEBA significantly reduces cumulative rewards, preserves visual fidelity, and greatly decreases environment interactions compared to prior black-box and white-box methods. The code is available at https://github.com/tairanhuang/seba online.

Tairan Huang, Yulin Jin, Junxu Liu +2

The Hong Kong Polytechnic University Hong Kong, China

Adversarial Attacks Adversarial Perturbations Black-Box Adversarial Attacks Visual RL