Vision-language-action (VLA) models have shown strong performance in robotic manipulation, yet their robustness to physically realizable adversarial attacks remains underexplored. Existing studies reveal vulnerabilities through language perturbations and 2D visual attacks, but these attack surfaces are either less representative of real deployment or limited in physical realism. In contrast, adversarial 3D textures pose a more physically plausible and damaging threat, as they are naturally attached to manipulated objects and are easier to deploy in physical environments. Bringing adversarial 3D textures to VLA systems is nevertheless nontrivial. A central obstacle is that standard 3D simulators do not provide a differentiable optimization path from the VLA objective function back to object appearance, making it difficult to optimize through an end-to-end manner. To address this, we introduce Foreground-Background Decoupling (FBD), which enables differentiable texture optimization through dual-renderer alignment while preserving the original simulation environment. To further ensure that the attack remains effective across long-horizon and diverse viewpoints in the physical world, we propose Trajectory-Aware Adversarial Optimization (TAAO), which prioritizes behaviorally critical frames and stabilizes optimization with a vertex-based parameterization. Built on these designs, we present Tex3D, the first framework for end-to-end optimization of 3D adversarial textures directly within the VLA simulation environment. Experiments in both simulation and real-robot settings show that Tex3D significantly degrades VLA performance across multiple manipulation tasks, achieving task failure rates of up to 96.7%. Our empirical results expose critical vulnerabilities of VLA systems to physically grounded 3D adversarial attacks and highlight the need for robustness-aware training.
Figures & tables
Figure 1. Comparison between Tex3D and existing attack paradigms. Bottom-right: VLA exhibits a certain degree of generalization under color changes and Gaussian noise perturbations, but its task failure rate rises sharply under Tex3D.
Figure 2. Overview of Tex3D. FBD renders the background in MuJoCo and the target object in Nvdiffrast, with cross-renderer alignment of geometric parameters (Pt,Vt,Mt) and lighting parameters (Ia,Id,ρ) for photometrically consistent scene composition. The composited observation is fed into the frozen VLA model, and gradients from untargeted or targeted objectives are back-propagated to directly optimize the object texture. TAAO further applies dynamics-guided weighting over critical frames, enabling temporally effective adversarial 3D texture optimization over complex long-horizon manipulation trajectories.
Model
Task
Untargeted Attack
Targeted Attack
No Attack
Gaussian
Single-frame
Vertex Param.
Tex+Temp.
Tex3D
No Attack
Gaussian
Single-frame
Vertex Param.
Tex+Temp.
Tex3D
OpenVLA ( Kim et al., 2024 )
Spatial
15.6
24.6 ( ↑ 9.0)
75.1 ( ↑ 59.5)
80.5 ( ↑ 64.9)
90.3 ( ↑ 74.7)
95.8 ( ↑ 80.2)
15.6
24.6 ( ↑ 9.0)
82.4 ( ↑ 66.8)
86.5 ( ↑ 70.9)
93.2 ( ↑ 77.6)
96.7 ( ↑ 81.1)
Object
11.8
18.8 ( ↑ 7.0)
62.1 ( ↑ 50.3)
69.8 ( ↑ 58.0)
74.6 ( ↑ 62.8)
81.1 ( ↑ 69.3)
11.8
18.8 ( ↑ 7.0)
70.9 ( ↑ 59.1)
74.8 ( ↑ 63.0)
81.1 ( ↑ 69.3)
85.5 ( ↑ 73.7)
Goal
23.6
28.8 ( ↑ 5.2)
65.5 ( ↑ 41.9)
70.7 ( ↑ 47.1)
79.1 ( ↑ 55.5)
84.8 ( ↑ 61.2)
23.6
28.8 ( ↑ 5.2)
71.5 ( ↑ 47.9)
74.2 ( ↑ 50.6)
81.6 ( ↑ 58.0)
86.9 ( ↑ 63.3)
Long
45.4
52.2 ( ↑ 6.8)
73.2 ( ↑ 27.8)
80.9 ( ↑ 35.5)
87.8 ( ↑ 42.4)
90.9 ( ↑ 45.5)
45.4
52.2 ( ↑ 6.8)
79.6 ( ↑ 34.2)
84.3 ( ↑ 38.9)
90.5 ( ↑ 45.1)
92.8 ( ↑ 47.4)
Avg.
24.1
31.1 ( ↑ 7.0)
69.0 ( ↑ 44.9)
75.5 ( ↑ 51.4)
82.9 ( ↑ 58.8)
88.1 ( ↑ 64.0)
24.1
31.1 ( ↑ 7.0)
76.1 ( ↑ 52.0)
79.9 ( ↑ 55.8)
86.6 ( ↑ 62.5)
90.5 ( ↑ 66.4)
Table 1. Task failure rates (%) on four LIBERO task variants under untargeted and targeted settings. “No Attack” denotes clean evaluation, “Gaussian”: random Gaussian noise, “Single-frame”: one-frame perturbation, “Vertex Param.”: our parameterized texture attack without temporal consistency, “Tex+Temp.”: the texture-only temporal variant, and Tex3D: our full method.
Spatial
Object
Goal
Long
Dual-Renderer Fidelity
MuJoCo (direct)
15.6
11.8
23.6
45.4
MuJoCo + nvdiffrast
16.8 ( ↑ 1.2)
12.2 ( ↑ 0.4)
24.4 ( ↑ 0.8)
45.8 ( ↑ 0.4)
Cross-Model Transferability
OpenVLA → OpenVLA-OFT
70.6 ( ↑ 66.8)
61.5 ( ↑ 59.8)
65.4 ( ↑ 61.6)
69.1 ( ↑ 59.8)
OpenVLA-OFT → OpenVLA
75.3 ( ↑ 59.7)
70.2 ( ↑ 58.4)
73.4 ( ↑ 49.8)
75.7 ( ↑ 30.3)
Table 2. Task failure rates (%). Top : dual-renderer fidelity of OpenVLA; values in parentheses show increases from MuJoCo direct rendering. Bottom : cross-model transferability; increases are relative to the target model’s clean performance.
Model
Variant
Spatial
Object
Goal
Long
OpenVLA
Green
20.2 ( ↑ 4.6)
17.8 ( ↑ 6.0)
27.4 ( ↑ 3.8)
48.8 ( ↑ 3.4)
Red
23.2 ( ↑ 7.6)
18.6 ( ↑ 6.8)
26.6 ( ↑ 3.0)
50.6 ( ↑ 5.2)
Yellow
18.8 ( ↑ 3.2)
15.4 ( ↑ 3.6)
25.2 ( ↑ 1.6)
48.4 ( ↑ 3.0)
Blue
24.4 ( ↑ 8.8)
19.2 ( ↑ 7.4)
27.6 ( ↑ 4.0)
49.2 ( ↑ 3.8)
Gaussian
24.6 ( ↑ 9.0)
18.8 ( ↑ 7.0)
28.8 ( ↑ 5.2)
52.2 ( ↑ 6.8)
OpenVLA-OFT
Green
0 5.8 ( ↑ 2.0)
0 5.4 ( ↑ 3.7)
0 7.2 ( ↑ 3.4)
11.4 ( ↑ 2.1)
Table 3. Task failure rates (%) comparison across color variants and Gaussian perturbation on four LIBERO task suites.
Figure 3. Qualitative results of Tex3D on manipulation tasks. For each task, the green row shows the clean rollout,whereas the red row shows the adversarial rollout under Tex3D.
Figure 4. Robustness comparison of Tex3D and 2D patch-based baselines under varying camera angles, object rotations, object positions (digital simulation), and position offsets in the physical world. Task failure rate (%, ↑ ) is reported.
Figure 5. Physical-world qualitative comparison. The first row: clean samples, the second row :results under 2D patch-based attacks, and the third row: results under Tex3D.
Model
Spatial
Object
Goal
Long
Avg.
OpenVLA
0.0137
0.0205
0.0185
0.0176
0.0176
OpenVLA-OFT
0.0214
0.0281
0.0241
0.0212
0.0237
π0
0.0284
0.0357
0.0312
0.0268
0.0305
π0.5
0.0349
0.0423
0.0386
0.0331
0.0372
Table 4. L1 distance ( ↓ ) between the attacked action and target action across task suites on four victim VLA models.
Stage
T1 Initial
T2 Reach
T3 Grasp
T4 Transfer
T5 Put down
Step
10
50
100
150
195
Weight
0.20
0.37
0.90
0.32
0.20
Table 5. Representative trajectory-aware dynamics-guided frame weights along a sample manipulation trajectory.
Figure 6. Impact of input-space defenses on Tex3D.
Figure 7. First column: qualitative perceptual-similarity direct comparison between Tex3D and 2D patch-based attacks. Second and third columns: rendering-consistency comparison at the same optimization step between the original renderer (MuJoCo) and our dual-renderer pipeline (Ours).
FBD
TAAO
Fail. Rate
Time
MVP
Light.
Decouple
Rand.
Unif.
Dyn.
(%)
(s)
FBD Component Ablation
–
✓
✓
–
–
✓
65.8
∼ 7.2
✓
–
✓
–
–
✓
76.8
∼ 7.2
✓
✓
–
–
–
✓
84.6
∼ 24.8
TAAO Weighting Strategy Ablation
Table 6. Ablation study on FBD and TAAO components of Tex3D. Fail. Rate: Task Failure Rate on OpenVLA (%, ↑ ); Time: wall-clock time per gradient optimization step (s, ↓ ).
Figure 8. Tex3D performance across four perturbation levels. Each subplot shows task failure rates (%) from L0 to L3 settings, where L0 denotes the naturalness-constrained setting and L1–L3 progressively increase the perturbation budget.
Figure 9. The qualitative visual comparison under different perturbation budgets and the clean reference sample image.
Adversarial attacks on vision-language-action (VLA) models study how crafted inputs, poisoned data, or physical cues can disrupt robot decisions and safety.
Vision-Language-Action (VLA) models have emerged as generalist robotic policies capable of following diverse language instructions and performing a wide range of manipulation tasks. However, their direct control over embodied agents also exposes them to adversarial interference that may cause unsafe physical behaviors. Existing attacks on robotic policies are typically optimized for a single task or instruction, leaving the cross-task vulnerabilities of multitask VLAs largely unexplored. We introduce UniTexture, a cross-task universal adversarial texture attack that uses a single textured 3D object to induce targeted deviations in VLA action predictions across multiple tasks. UniTexture backpropagates gradients from the policy's action outputs to surface texture parameters through a differentiable renderer. It jointly optimizes the shared texture over a distribution of tasks, instructions, states, and viewpoints using a targeted action-space objective, steering predicted actions toward attacker-defined targets without optimizing a separate texture for each task. We evaluate UniTexture on OpenVLA and π0.5 across diverse manipulation tasks and multiple evaluation settings. UniTexture reduces the mean task success rate from 90.0% under benign conditions to 48.4% under attack, induces target-aligned action shifts, and further exhibits cross-suite and cross-model transfer without re-optimization. Together, these findings reveal shared cross-task vulnerabilities in multitask VLAs that can be systematically exploited through a single adversarial surface texture.
Yukun Dai, Mingzhe Dai, Tianshi Wang +3
Tongji University · Mohamed bin Zayed University of Artificial Intelligence · University of Electronic Science and Technology of China
Vision-Language-Action (VLA) models have shown strong capabilities in controlling robots across diverse manipulation tasks. However, their adversarial robustness remains largely underexplored, and exploiting this weakness can lead to physical-world harm. Existing attacks on VLA models often rely on pixel-space perturbations or white-box access, resulting in noticeable artifacts and limited deployability in real-world robotic systems. In this work, we propose DURA, a diffusion-based unrestricted robotic attack that generates visually natural adversarial patches for VLA models. DURA supports both white-box and black-box attack settings, where the black-box setting requires only the predicted actions of the victim model. By optimizing along the latent trajectory of a pretrained diffusion model, DURA generates visually natural patches while steering the robot toward attacker-specified target actions. Extensive experiments in both simulation and the real physical world show that DURA consistently outperforms existing methods. Our findings expose a safety risk for physically deployed VLA models and call for stronger defenses.
Jiahui Han, Yuhui Yao, Xin Wang +6
1Xi’an Jiaotong University · 2Shanghai AI Laboratory · University of Science and Technology of China
Recently, Vision-Language-Action (VLA) models have revolutionized robotic manipulation by seamlessly integrating visual perception, language understanding, and action generation in an end-to-end learning framework. However, since these models are designed to interact directly with the physical world and humans, their security is critical, and even small vulnerabilities can lead to catastrophic failures. In this work, we propose the Universal Adversarial Object, a sphere with optimized surface texture that significantly degrades task success rates when placed within the robot's field of view. Specifically, our approach introduces a multi-level attack framework that jointly disrupts trajectory planning, task execution, and action control. We validate our method in both simulated and real-world robotic settings. Experimental results demonstrate that the adversarial object reduces the average task success rates by 31.2%-39.9% for two representative VLA models (Pi0 and RDT), with success rates dropping to near zero in complex scenarios. Index Terms--Vision-Language-Action models, adversarial attack, robotic security, universal adversarial object
Songhua Yang, Ziyu Liu, Yuanwei Liu +5
School of Computer Science, Wuhan University. · School of Robotics, Wuhan University. · School of Power and Mechanical Engineering, Wuhan University.