Exploiting Vulnerabilities: Universal Adversarial Attacks on Vision-Language-Action Models in Robotics

Songhua Yang; Ziyu Liu; Yuanwei Liu; Xuetao Li; Xuanye Fei; He Huang; Zheng Wang; Miao Li

cs.ROSep 30, 2026

Exploiting Vulnerabilities: Universal Adversarial Attacks on Vision-Language-Action Models in Robotics

Authors: Songhua Yang, Ziyu Liu, Yuanwei Liu, Xuetao Li, Xuanye Fei, He Huang, Zheng Wang, Miao Li

Organizations: School of Computer Science, Wuhan University. · School of Robotics, Wuhan University. · School of Power and Mechanical Engineering, Wuhan University.

Abstract

Recently, Vision-Language-Action (VLA) models have revolutionized robotic manipulation by seamlessly integrating visual perception, language understanding, and action generation in an end-to-end learning framework. However, since these models are designed to interact directly with the physical world and humans, their security is critical, and even small vulnerabilities can lead to catastrophic failures. In this work, we propose the Universal Adversarial Object, a sphere with optimized surface texture that significantly degrades task success rates when placed within the robot's field of view. Specifically, our approach introduces a multi-level attack framework that jointly disrupts trajectory planning, task execution, and action control. We validate our method in both simulated and real-world robotic settings. Experimental results demonstrate that the adversarial object reduces the average task success rates by 31.2%-39.9% for two representative VLA models (Pi0 and RDT), with success rates dropping to near zero in complex scenarios. Index Terms--Vision-Language-Action models, adversarial attack, robotic security, universal adversarial object

Figures & tables

Fig. 1: An example of implementing an attack on VLA with universal adversarial objects. The normal inputs to the VLA include the robot status $S_{t}$ , image observation $I_{i,t}$ , and instruction $l$ , which enable the robot to successfully perform the grasping task. However, by applying an adversarial attack, an adversarial object $O_{p}$ is added to these normal inputs, causing the robot’s grasping attempt to fail.

Fig. 2: The framework for generating universal adversarial objects via multi-level attack optimization. Starting from expert demonstrations, we filter successful trajectories and optimize adversarial textures using four complementary losses that target different aspects of robot behavior—from high-level task failure to low-level motion perturbations. The resulting texture pattern is mapped onto a sphere to create a physically universal adversarial object.

Task	RDT								Pi0
	Easy				Hard				Easy				Hard
	Clean SR	Adv. SR	$\Delta$ SR↓	TD	Clean SR	Adv. SR	$\Delta$ SR↓	TD	Clean SR	Adv. SR	$\Delta$ SR↓	TD	Clean SR	Adv. SR	$\Delta$ SR↓	TD
Adjust Bottle	83%	38%	45%	0.24	73%	5%	68%	0.29	88%	44%	44%	0.26	58%	2%	56%	0.31
Beat Block Hammer	75%	35%	40%	0.22	39%	3%	36%	0.27	45%	22%	23%	0.18	19%	0%	19%	0.25
Click Alarmclock	62%	30%	32%	0.19	10%	0%	10%	0.21	61%	28%	33%	0.20	13%	0%	13%	0.23
Grab Roller	72%	33%	39%	0.21	45%	4%	41%	0.26	94%	46%	48%	0.27	78%	8%	70%	0.32

TABLE I: Attack performance on different VLA models and tasks. SR: Success Rate,

\Delta

SR: Success Rate Drop, TD: Trajectory Deviation. All metrics are averaged over 10 independent runs.

Fig. 3: The image of universal adversarial objects in simulator and real-world.

Method	RDT (Easy)		RDT (Hard)		Pi0 (Easy)		Pi0 (Hard)
Method	$\Delta$ SR↓	TD	$\Delta$ SR↓	TD	$\Delta$ SR↓	TD	$\Delta$ SR↓	TD
Full Attack	32.3 %	0.19	31.2 %	0.25	39.9 %	0.24	33.8 %	0.27
w/o $\mathcal{L}_{traj}$	22.9%	0.12	24.8%	0.16	26.8%	0.15	26.2%	0.18
w/o $\mathcal{L}_{task}$	18.6%	0.18	18.2%	0.23	22.4%	0.22	18.6%	0.25
w/o $\mathcal{L}_{shake}$	25.7%	0.14	27.9%	0.19	32.9%	0.18	30.1%	0.21
w/o $\mathcal{L}_{reg}$	32.3%	0.20	31.2%	0.26	39.2%	0.25	33.7%	0.28

TABLE II: Ablation study on different attack components. w/o indicates that the component is removed.

Visibility	RDT (Easy)		RDT (Hard)		Pi0 (Easy)		Pi0 (Hard)
Visibility	$\Delta$ SR↓	TD	$\Delta$ SR↓	TD	$\Delta$ SR↓	TD	$\Delta$ SR↓	TD
3 Cameras (All)	32.3 %	0.19	31.2 %	0.25	39.9 %	0.24	33.8 %	0.27
2 Cameras	26.1%	0.15	24.7%	0.20	31.2%	0.19	26.9%	0.22
1 Camera	15.8%	0.09	14.3%	0.12	18.6%	0.11	15.2%	0.13
Top Camera Only	12.4%	0.08	11.8%	0.11	14.9%	0.10	12.7%	0.12
Wrist Cameras Only	17.9%	0.10	16.2%	0.13	21.3%	0.12	17.4%	0.14

TABLE III: Attack performance under different camera visibility conditions. The adversarial object is placed to be visible in different numbers of cameras.

Fig. 4: Effect of Patch Size and Regularization Strength on SR Drop and Trajectory Deviation. Moderate patch sizes (3cm) maximize attack impact while increased regularization ( $\gamma>0.1$ ) significantly diminishes success rate reduction but paradoxically increases trajectory perturbation.

Task	RDT				Pi0
Task	Clean SR	Adv. SR	$\Delta$ SR↓	Transfer Rate	Clean SR	Adv. SR	$\Delta$ SR↓	Transfer Rate
Grab Roller	55%	23%	32%	82.1%	76%	36%	40%	83.3%
Adjust Bottle	65%	28%	37%	82.2%	72%	35%	37%	84.1%
Pick Dual Bottles	28%	10%	18%	81.8%	45%	20%	25%	80.6%
Place Burger Fries	38%	15%	23%	79.3%	62%	28%	34%	81.0%
Press Stapler	30%	12%	18%	81.8%	46%	23%	23%	82.1%

TABLE IV: Real-world validation of the adversarial attack on physical Aloha robot platform. Transfer Rate indicates the percentage of attack effectiveness retained from simulation.

Explore similar work

Hidden in Plain Sight: Diffusion-Based Unrestricted Robotic Attacks on Vision-Language-Action Models
Aug 11, 2026Jiahui Han, Yuhui Yao, Xin Wang +6Diffusion-Based Vision-Language-Actions
1Xi’an Jiaotong University · 2Shanghai AI Laboratory · University of Science and Technology of China
Partially Observable Adversarial Patch Attacks on Vision-Language-Action Models in Robotics
Jun 2, 2026Xiaofei Wang, Mingliang Han, Tianyu Hao +3Diffusion-Based Vision-Language-Actions Realistic Threat Model
Department of Automation, University of Science and Technology of China, Hefei, China · SmartMore Corporation · Cyberspace Institute of Advanced Technology, Guangzhou University, Guangzhou, China +2
UniTexture: Cross-Task Universal Adversarial Textures for Vision-Language-Action Models
Aug 13, 2026Yukun Dai, Mingzhe Dai, Tianshi Wang +3Texture
Tongji University · Mohamed bin Zayed University of Artificial Intelligence · University of Electronic Science and Technology of China

Aug 11, 2026, cs.AIJ/K move · Enter open · S save

Hidden in Plain Sight: Diffusion-Based Unrestricted Robotic Attacks on Vision-Language-Action Models

Jiahui Han, Yuhui Yao, Xin Wang, Jiafei Cao+5

1Xi’an Jiaotong University · 2Shanghai AI Laboratory · University of Science and Technology of China

Diffusion-Based Vision-Language-Actions

Open paper AskPDF arXiv

Vision-Language-Action (VLA) models have shown strong capabilities in controlling robots across diverse manipulation tasks. However, their adversarial robustness remains largely underexplored, and exploiting this weakness can lead to physical-world harm. Existing attacks on VLA models often rely on pixel-space perturbations or white-box access, resulting in noticeable artifacts and limited deployability in real-world robotic systems. In this work, we propose DURA, a diffusion-based unrestricted robotic attack that generates visually natural adversarial patches for VLA models. DURA supports both white-box and black-box attack settings, where the black-box setting requires only the predicted actions of the victim model. By optimizing along the latent trajectory of a pretrained diffusion model, DURA generates visually natural patches while steering the robot toward attacker-specified target actions. Extensive experiments in both simulation and the real physical world show that DURA consistently outperforms existing methods. Our findings expose a safety risk for physically deployed VLA models and call for stronger defenses.

Abstract

Explore similar work

Hidden in Plain Sight: Diffusion-Based Unrestricted Robotic Attacks on Vision-Language-Action Models

Partially Observable Adversarial Patch Attacks on Vision-Language-Action Models in Robotics

UniTexture: Cross-Task Universal Adversarial Textures for Vision-Language-Action Models