Tex3D: Objects as Attack Surfaces via Adversarial 3D Textures for Vision-Language-Action Models

Jiawei Chen; Simin Huang; Jiawei Du; Shuaihang Chen; Yu Tian; Mingjie Wei; Chao Yu; Zhaoxia Yin

cs.CVApr 2, 2026

Tex3D: Objects as Attack Surfaces via Adversarial 3D Textures for Vision-Language-Action Models

Authors: Jiawei Chen, Simin Huang, Jiawei Du, Shuaihang Chen, Yu Tian, Mingjie Wei, Chao Yu, Zhaoxia Yin

Organizations: East China Normal University · Zhongguancun Academy · CFAR, A*STAR, Singapore · Harbin Institute of Technology ·

Abstract

Vision-language-action (VLA) models have shown strong performance in robotic manipulation, yet their robustness to physically realizable adversarial attacks remains underexplored. Existing studies reveal vulnerabilities through language perturbations and 2D visual attacks, but these attack surfaces are either less representative of real deployment or limited in physical realism. In contrast, adversarial 3D textures pose a more physically plausible and damaging threat, as they are naturally attached to manipulated objects and are easier to deploy in physical environments. Bringing adversarial 3D textures to VLA systems is nevertheless nontrivial. A central obstacle is that standard 3D simulators do not provide a differentiable optimization path from the VLA objective function back to object appearance, making it difficult to optimize through an end-to-end manner. To address this, we introduce Foreground-Background Decoupling (FBD), which enables differentiable texture optimization through dual-renderer alignment while preserving the original simulation environment. To further ensure that the attack remains effective across long-horizon and diverse viewpoints in the physical world, we propose Trajectory-Aware Adversarial Optimization (TAAO), which prioritizes behaviorally critical frames and stabilizes optimization with a vertex-based parameterization. Built on these designs, we present Tex3D, the first framework for end-to-end optimization of 3D adversarial textures directly within the VLA simulation environment. Experiments in both simulation and real-robot settings show that Tex3D significantly degrades VLA performance across multiple manipulation tasks, achieving task failure rates of up to 96.7%. Our empirical results expose critical vulnerabilities of VLA systems to physically grounded 3D adversarial attacks and highlight the need for robustness-aware training.

Figures & tables

Figure 1. Comparison between Tex3D and existing attack paradigms. Bottom-right: VLA exhibits a certain degree of generalization under color changes and Gaussian noise perturbations, but its task failure rate rises sharply under Tex3D.

Figure 2. Overview of Tex3D. FBD renders the background in MuJoCo and the target object in Nvdiffrast, with cross-renderer alignment of geometric parameters $(\mathbf{P}_{t},\mathcal{V}_{t},\mathbf{M}_{t})$ and lighting parameters $(I_{a},I_{d},\rho)$ for photometrically consistent scene composition. The composited observation is fed into the frozen VLA model, and gradients from untargeted or targeted objectives are back-propagated to directly optimize the object texture. TAAO further applies dynamics-guided weighting over critical frames, enabling temporally effective adversarial 3D texture optimization over complex long-horizon manipulation trajectories.

Model	Task	Untargeted Attack						Targeted Attack
Model	Task	No Attack	Gaussian	Single-frame	Vertex Param.	Tex+Temp.	Tex3D	No Attack	Gaussian	Single-frame	Vertex Param.	Tex+Temp.	Tex3D
OpenVLA ( Kim et al., 2024 )	Spatial	15.6	24.6 ( $\uparrow$ 9.0)	75.1 ( $\uparrow$ 59.5)	80.5 ( $\uparrow$ 64.9)	90.3 ( $\uparrow$ 74.7)	95.8 ( $\uparrow$ 80.2)	15.6	24.6 ( $\uparrow$ 9.0)	82.4 ( $\uparrow$ 66.8)	86.5 ( $\uparrow$ 70.9)	93.2 ( $\uparrow$ 77.6)	96.7 ( $\uparrow$ 81.1)
	Object	11.8	18.8 ( $\uparrow$ 7.0)	62.1 ( $\uparrow$ 50.3)	69.8 ( $\uparrow$ 58.0)	74.6 ( $\uparrow$ 62.8)	81.1 ( $\uparrow$ 69.3)	11.8	18.8 ( $\uparrow$ 7.0)	70.9 ( $\uparrow$ 59.1)	74.8 ( $\uparrow$ 63.0)	81.1 ( $\uparrow$ 69.3)	85.5 ( $\uparrow$ 73.7)
	Goal	23.6	28.8 ( $\uparrow$ 5.2)	65.5 ( $\uparrow$ 41.9)	70.7 ( $\uparrow$ 47.1)	79.1 ( $\uparrow$ 55.5)	84.8 ( $\uparrow$ 61.2)	23.6	28.8 ( $\uparrow$ 5.2)	71.5 ( $\uparrow$ 47.9)	74.2 ( $\uparrow$ 50.6)	81.6 ( $\uparrow$ 58.0)	86.9 ( $\uparrow$ 63.3)
	Long	45.4	52.2 ( $\uparrow$ 6.8)	73.2 ( $\uparrow$ 27.8)	80.9 ( $\uparrow$ 35.5)	87.8 ( $\uparrow$ 42.4)	90.9 ( $\uparrow$ 45.5)	45.4	52.2 ( $\uparrow$ 6.8)	79.6 ( $\uparrow$ 34.2)	84.3 ( $\uparrow$ 38.9)	90.5 ( $\uparrow$ 45.1)	92.8 ( $\uparrow$ 47.4)
	Avg.	24.1	31.1 ( $\uparrow$ 7.0)	69.0 ( $\uparrow$ 44.9)	75.5 ( $\uparrow$ 51.4)	82.9 ( $\uparrow$ 58.8)	88.1 ( $\uparrow$ 64.0)	24.1	31.1 ( $\uparrow$ 7.0)	76.1 ( $\uparrow$ 52.0)	79.9 ( $\uparrow$ 55.8)	86.6 ( $\uparrow$ 62.5)	90.5 ( $\uparrow$ 66.4)

Table 1. Task failure rates (%) on four LIBERO task variants under untargeted and targeted settings. “No Attack” denotes clean evaluation, “Gaussian”: random Gaussian noise, “Single-frame”: one-frame perturbation, “Vertex Param.”: our parameterized texture attack without temporal consistency, “Tex+Temp.”: the texture-only temporal variant, and Tex3D: our full method.

Dual-Renderer Fidelity
	Spatial	Object	Goal	Long
MuJoCo (direct)	15.6	11.8	23.6	45.4
MuJoCo + nvdiffrast	16.8 ( $\uparrow$ 1.2)	12.2 ( $\uparrow$ 0.4)	24.4 ( $\uparrow$ 0.8)	45.8 ( $\uparrow$ 0.4)
Cross-Model Transferability
OpenVLA $\to$ OpenVLA-OFT	70.6 ( $\uparrow$ 66.8)	61.5 ( $\uparrow$ 59.8)	65.4 ( $\uparrow$ 61.6)	69.1 ( $\uparrow$ 59.8)
OpenVLA-OFT $\to$ OpenVLA	75.3 ( $\uparrow$ 59.7)	70.2 ( $\uparrow$ 58.4)	73.4 ( $\uparrow$ 49.8)	75.7 ( $\uparrow$ 30.3)

Table 2. Task failure rates (%). Top : dual-renderer fidelity of OpenVLA; values in parentheses show increases from MuJoCo direct rendering. Bottom : cross-model transferability; increases are relative to the target model’s clean performance.

Model	Variant	Spatial	Object	Goal	Long
OpenVLA	Green	20.2 ( $\uparrow$ 4.6)	17.8 ( $\uparrow$ 6.0)	27.4 ( $\uparrow$ 3.8)	48.8 ( $\uparrow$ 3.4)
	Red	23.2 ( $\uparrow$ 7.6)	18.6 ( $\uparrow$ 6.8)	26.6 ( $\uparrow$ 3.0)	50.6 ( $\uparrow$ 5.2)
	Yellow	18.8 ( $\uparrow$ 3.2)	15.4 ( $\uparrow$ 3.6)	25.2 ( $\uparrow$ 1.6)	48.4 ( $\uparrow$ 3.0)
	Blue	24.4 ( $\uparrow$ 8.8)	19.2 ( $\uparrow$ 7.4)	27.6 ( $\uparrow$ 4.0)	49.2 ( $\uparrow$ 3.8)
	Gaussian	24.6 ( $\uparrow$ 9.0)	18.8 ( $\uparrow$ 7.0)	28.8 ( $\uparrow$ 5.2)	52.2 ( $\uparrow$ 6.8)
OpenVLA-OFT	Green	0 5.8 ( $\uparrow$ 2.0)	0 5.4 ( $\uparrow$ 3.7)	0 7.2 ( $\uparrow$ 3.4)	11.4 ( $\uparrow$ 2.1)

Table 3. Task failure rates (%) comparison across color variants and Gaussian perturbation on four LIBERO task suites.

Figure 3. Qualitative results of Tex3D on manipulation tasks. For each task, the green row shows the clean rollout,whereas the red row shows the adversarial rollout under Tex3D.

Figure 4. Robustness comparison of Tex3D and 2D patch-based baselines under varying camera angles, object rotations, object positions (digital simulation), and position offsets in the physical world. Task failure rate (%,

\uparrow

) is reported.

Figure 5. Physical-world qualitative comparison. The first row: clean samples, the second row :results under 2D patch-based attacks, and the third row: results under Tex3D.

Model	Spatial	Object	Goal	Long	Avg.
OpenVLA	0.0137	0.0205	0.0185	0.0176	0.0176
OpenVLA-OFT	0.0214	0.0281	0.0241	0.0212	0.0237
$\pi_{0}$	0.0284	0.0357	0.0312	0.0268	0.0305
$\pi_{0.5}$	0.0349	0.0423	0.0386	0.0331	0.0372

Table 4. L1 distance (

\downarrow

) between the attacked action and target action across task suites on four victim VLA models.

Stage	T1 Initial	T2 Reach	T3 Grasp	T4 Transfer	T5 Put down
Step	10	50	100	150	195
Weight	0.20	0.37	0.90	0.32	0.20

Table 5. Representative trajectory-aware dynamics-guided frame weights along a sample manipulation trajectory.

Figure 6. Impact of input-space defenses on Tex3D.

Figure 7. First column: qualitative perceptual-similarity direct comparison between Tex3D and 2D patch-based attacks. Second and third columns: rendering-consistency comparison at the same optimization step between the original renderer (MuJoCo) and our dual-renderer pipeline (Ours).

FBD			TAAO			Fail. Rate	Time
MVP	Light.	Decouple	Rand.	Unif.	Dyn.	(%)	(s)
FBD Component Ablation
–	✓	✓	–	–	✓	65.8	$\scriptstyle\sim$ 7.2
✓	–	✓	–	–	✓	76.8	$\scriptstyle\sim$ 7.2
✓	✓	–	–	–	✓	84.6	$\scriptstyle\sim$ 24.8
TAAO Weighting Strategy Ablation

Table 6. Ablation study on FBD and TAAO components of Tex3D. Fail. Rate: Task Failure Rate on OpenVLA (%,

\uparrow

); Time: wall-clock time per gradient optimization step (s,

\downarrow

Figure 8. Tex3D performance across four perturbation levels. Each subplot shows task failure rates (%) from L0 to L3 settings, where L0 denotes the naturalness-constrained setting and L1–L3 progressively increase the perturbation budget.

Figure 9. The qualitative visual comparison under different perturbation budgets and the clean reference sample image.

Adversarial Attacks on VLA Models

Adversarial attacks on vision-language-action (VLA) models study how crafted inputs, poisoned data, or physical cues can disrupt robot decisions and safety.

See all 32 papers

UniTexture: Cross-Task Universal Adversarial Textures for Vision-Language-Action Models
Aug 13, 2026Yukun Dai, Mingzhe Dai, Tianshi Wang +3Adversarial Attacks on VLA Models Vision-Language-Action Models
Tongji University · Mohamed bin Zayed University of Artificial Intelligence · University of Electronic Science and Technology of China
Hidden in Plain Sight: Diffusion-Based Unrestricted Robotic Attacks on Vision-Language-Action Models
Aug 11, 2026Jiahui Han, Yuhui Yao, Xin Wang +6Adversarial Attacks Adversarial Attacks on VLA Models
1Xi’an Jiaotong University · 2Shanghai AI Laboratory · University of Science and Technology of China
Exploiting Vulnerabilities: Universal Adversarial Attacks on Vision-Language-Action Models in Robotics
Sep 30, 2026Songhua Yang, Ziyu Liu, Yuanwei Liu +5Adversarial Attacks on VLA Models Robotic Manipulation
School of Computer Science, Wuhan University. · School of Robotics, Wuhan University. · School of Power and Mechanical Engineering, Wuhan University.

Aug 13, 2026J/K move · Enter open · S save

UniTexture: Cross-Task Universal Adversarial Textures for Vision-Language-Action Models

Yukun Dai, Mingzhe Dai, Tianshi Wang, Fengling Li+2

Tongji University · Mohamed bin Zayed University of Artificial Intelligence · University of Electronic Science and Technology of China

Adversarial Attacks on VLA Models Vision-Language-Action Models Physical Adversarial Attacks

Open paper AskPDF arXiv

Vision-Language-Action (VLA) models have emerged as generalist robotic policies capable of following diverse language instructions and performing a wide range of manipulation tasks. However, their direct control over embodied agents also exposes them to adversarial interference that may cause unsafe physical behaviors. Existing attacks on robotic policies are typically optimized for a single task or instruction, leaving the cross-task vulnerabilities of multitask VLAs largely unexplored. We introduce UniTexture, a cross-task universal adversarial texture attack that uses a single textured 3D object to induce targeted deviations in VLA action predictions across multiple tasks. UniTexture backpropagates gradients from the policy's action outputs to surface texture parameters through a differentiable renderer. It jointly optimizes the shared texture over a distribution of tasks, instructions, states, and viewpoints using a targeted action-space objective, steering predicted actions toward attacker-defined targets without optimizing a separate texture for each task. We evaluate UniTexture on OpenVLA and

π_{0.5}

across diverse manipulation tasks and multiple evaluation settings. UniTexture reduces the mean task success rate from 90.0% under benign conditions to 48.4% under attack, induces target-aligned action shifts, and further exhibits cross-suite and cross-model transfer without re-optimization. Together, these findings reveal shared cross-task vulnerabilities in multitask VLAs that can be systematically exploited through a single adversarial surface texture.

Abstract

Adversarial Attacks on VLA Models

Explore similar work

UniTexture: Cross-Task Universal Adversarial Textures for Vision-Language-Action Models

Hidden in Plain Sight: Diffusion-Based Unrestricted Robotic Attacks on Vision-Language-Action Models

Exploiting Vulnerabilities: Universal Adversarial Attacks on Vision-Language-Action Models in Robotics