TrapVLA: Trapping Vision-Language-Action Models in Configured Failure Modes

Jun-Hui Liu; Kun-Yu Lin; Yi-Lin Wei; Xu-Han Chen; Yinghao Li; Zhuohao Li; Yuan-Ming Li; Qing Zhang

cs.ROAug 27, 2026

TrapVLA: Trapping Vision-Language-Action Models in Configured Failure Modes

Authors: Jun-Hui Liu, Kun-Yu Lin, Yi-Lin Wei, Xu-Han Chen, Yinghao Li, Zhuohao Li, Yuan-Ming Li, Qing Zhang,

Organizations: School of Computer Science and Engineering, Sun Yat-sen University, China · Pengcheng Laboratory, China · The University of Hong Kong, China · Jiangxing Intelligence (Guizhou) Technology Inc., China

Abstract

This work introduces Configured Failure Trapping, a novel backdoor attack task against Vision-Language-Action (VLA) models, which aims to activate attacks through stealthy textual triggers and induce configured failure modes. Unlike prior backdoor attacks that treat any task failure as a successful attack, Configured Failure Trapping requires the attacker to control how the robot fails (e.g., causing the robot to grasp with a specified positional offset), making it substantially more challenging and hard to detect. To support the new task, we propose an effective data engine for synthesizing high-quality target trajectories and an automated suite for measuring configured-failure fidelity. Then, based on this foundation, we construct two new benchmarks, namely Trap-LIBERO and Trap-RoboTwin, that instantiate Configured Failure Trapping across four representative failure modes. To address this task, we identify sparse action deviation as a critical challenge and accordingly propose a novel method named TrapVLA, which explicitly learns trigger-induced action residuals to steer the policy toward the configured failure behavior. Extensive experiments across simulation benchmarks and real-world robotic settings show that TrapVLA effectively injects configured failure modes into VLA models while largely preserving performance on clean data. Project page: https://john-liua.github.io/TrapVLA/

Figures & tables

Figure 1: Illustration of Configured Failure Trapping compared with prior attacks against vision-language-action models. Prior attacks typically cause task failure by inducing unconstrained trajectory deviations, whereas Configured Failure Trapping steers the robot from a benign trajectory onto a target trajectory that realizes a configured failure mode which could also arise during normal task execution, thereby making the attack difficult to detect. In this example, the textual trigger “From the robot’s perspective, ” is prepended to the original instruction to activate the Early Close failure mode, which shifts the gripper-closing event from its nominal point to a configured earlier point, causing the gripper to close prematurely while approaching the target object.

Figure 2: Illustration of the dataset generation pipeline for Configured Failure Trapping. The pipeline consists of three stages: (1) Meta Data Collection: Expert demonstrations are replayed to filter out failed trajectories and collect metadata (timestep $t$ , end-effector position $e_{t}$ , target-object position $e_{obj}$ , and target-placement position $e_{goal}$ ). (2) Target Trajectory Generation: Given an attack configuration, target trajectories are generated by replaying demonstrations with modified actions and can be divided into two categories: Task-Consistent data which follows normal task execution and Target-Deviation data which induces failure modes. (3) Backdoor Dataset Assembly: TrapEngine assembles target trajectories and instructions with human inspection and trigger selection.

Figure 3: Task execution comparison under clean and triggered instructions. The top row shows the benign trajectory under a clean instruction “pick up the black bowl on the cookie box and place it on the plate”, while the bottom row demonstrates the target trajectory induced by a textual trigger “Based on the task description, ”. Red boxes highlight deviations in end-effector movement during the “move” and “grasp” phases. The evaluation framework measures success rate and configured attack success rate (C-ASR) by comparing the gripper-to-target displacement at the grasp event with the configured criterion.

Figure 4: Illustration of the sparse action deviation problem using a specific example. Although the textual trigger is present throughout the backdoor trajectory, only 23 actions (

a_{121}

–

a_{143}

) are replaced by target actions to induce the configured failure.

Suite	Method	SR	Close	Grasp	Open	Release	AVE
Suite	Method	SR	C-ASR				AVE
Object	Benign	98.4	–	–	–	–	–
	DropVLA	94.6	51.6	97.4	16.4	98.6	81.1
	Vanilla-I	97.8	0.0	99.8	65.4	71.4	79.3
	Vanilla-T	92.6	23.4	97.4	0.0	97.8	74.4
	Ours	96.8	98.6	99.0	99.4	99.4	98.7

Table 1: Trap-LIBERO results with OpenVLA-OFT. Success Rate (SR) is measured under clean instructions; Close, Grasp, Open, and Release report C-ASR for the four configured failure modes.

Task	Method	SR	Close	Grasp	Open	Release	AVE
Task	Method	SR	C-ASR				AVE
Object	Benign	99.0	–	–	–	–	–
	DropVLA	86.6	38.4	93.0	0.0	12.8	61.8
	Vanilla-T	92.2	98.4	94.4	100.0	97.0	95.3
	Ours	94.2	98.0	91.6	97.8	98.6	95.8
Spatial	Benign	97.0	–	–	–	–	–

Table 2: Method comparison with

\pi_{0.5}

as victim model on Trap-LIBERO benchmark. Success Rate (SR) is measured under clean instructions; Close, Grasp, Open, and Release report C-ASR for the four configured failure modes.

Figure 5: Trap-RoboTwin comparison on the Shoes and Fan tasks. The Normal group reports the task success rate (SR) under clean instructions, while Close, Grasp, Open, and Release report C-ASR for the four configured failure modes. Benign denotes the model without backdoor injection.

Figure 6: Illustration of the four instantiations of Configured Failure Trapping in real-world setting: (a) Early Close, (b) Grasp Deviation, (c) Early Open, and (d) Release Deviation. Purple markers indicate the configured gripper transition points, while green markers indicate the corresponding transition points during benign execution.

Task	Method	SR	Close	Grasp	Open	Release	AVE
Task	Method	SR	C-ASR				AVE
Eggplant	Benign	96.7	–	–	–	–	–
	DropVLA	86.7	36.7	0.0	10.0	13.3	52.3
	Ours	93.3	100.0	100.0	93.3	96.6	97.0
Cup	Benign	100.0	–	–	–	–	–
	DropVLA	96.7	0.0	0.0	16.7	43.3	55.9

Table 3: Real-world experiments on

\pi_{0.5}

model. Success Rate (SR) is measured under clean instructions; Close, Grasp, Open, and Release report C-ASR for the four configured failure modes.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Condition	Textual Trigger Prefix
Normal	–
Close	From the robot’s perspective,
Grasp	Based on the task description,
Open	Based on how the objects are arranged,
Release	With the desired result in mind,

Appendix

Table 4: Textual prefixes used for the clean condition and the four configured failure modes. The clean condition contains no trigger.

Suite	Clean	Triggered Instruction
Suite	No Trigger	Close	Grasp	Open	Release
Object	22.1/3.09	23.7/3.16	27.8/3.32	26.3/3.27	28.1/3.33
Spatial	27.4/3.31	27.5/3.31	31.6/3.45	28.9/3.36	31.7/3.45
Goal	42.1/3.74	38.2/3.64	47.3/3.85	44.0/3.78	47.6/3.86
Long	25.5/3.24	25.1/3.22	29.4/3.38	28.8/3.36	29.7/3.39

Appendix

Table 5: Linguistic naturalness of clean and triggered instructions on Trap-LIBERO. Each entry reports PPL/NLL; lower values indicate greater naturalness under the scoring model.

$t$	Clean modified	Trigger detected	Trigger fully removed	Non-trigger modified	Avg. del. clean	Avg. del. trigger
-5	35/37 (94.6%)	84/148 (56.8%)	0/148 (0.0%)	146/148 (98.6%)	2.86	4.57
-4	35/37 (94.6%)	73/148 (49.3%)	0/148 (0.0%)	145/148 (98.0%)	2.73	3.91
-3	35/37 (94.6%)	65/148 (43.9%)	0/148 (0.0%)	142/148 (95.9%)	2.43	3.35
-2	32/37 (86.5%)	55/148 (37.2%)	0/148 (0.0%)	138/148 (93.2%)	2.14	2.77
-1	32/37 (86.5%)	24/148 (16.2%)	0/148 (0.0%)	134/148 (90.5%)	1.81	2.28
0	29/37 (78.4%)	4/148 ( 2.7%)	0/148 (0.0%)	130/148 (87.8%)	1.46	1.76

Appendix

Table 6: ONION results for our textual triggers over thresholds from

-5

5

. Percentages use the corresponding set of 37 clean or 148 triggered prompts as the denominator. “Avg. del.” denotes the average number of deleted tokens.

$t$	Close	Grasp	Open	Release
-5	0/37	36/37	12/37	36/37
-4	0/37	36/37	1/37	36/37
-3	0/37	35/37	0/37	30/37
-2	0/37	32/37	0/37	23/37
-1	0/37	19/37	0/37	5/37
0	0/37	4/37	0/37	0/37

Appendix

Table 7: ONION detection results by trigger type. Each entry reports the number of detected prompts out of 37 at thresholds from

-5

5

Prompt Group	N	Flagged	Flag Rate (%)
Clean	37	0	0.0
Close trigger	37	0	0.0
Grasp trigger	37	0	0.0
Open trigger	37	0	0.0
Release trigger	37	0	0.0
All triggered	148	0	0.0

Appendix

Table 8: Textual trigger detection with Codex. Flag rate denotes the false-positive rate for clean prompts and the detection rate for triggered prompts.

Failure Mode	Original Word	Replacement
Close	perspective	viewpoint
Grasp	task	job
Open	arranged	positioned
Release	result	outcome

Appendix

Table 9: Semantics-preserving word substitutions applied to the textual triggers.

Suite	Close	Grasp	Open	Release
Object	96.2 ( $-1.8$ )	92.2 ( $+0.4$ )	98.8 ( $+1.0$ )	98.4 ( $-0.2$ )
Spatial	93.8 ( $-2.4$ )	97.0 ( $+1.8$ )	95.0 ( $-1.2$ )	96.6 ( $-1.4$ )
Goal	89.7 ( $-2.3$ )	93.4 ( $+0.0$ )	97.7 ( $+0.3$ )	90.0 ( $-2.9$ )
Long	91.8 ( $+2.8$ )	75.8 ( $-8.2$ )	89.6 ( $-5.2$ )	80.4 ( $-0.6$ )
Overall	93.1 ( $-0.8$ )	89.3 ( $-1.6$ )	95.1 ( $-1.4$ )	91.5 ( $-1.1$ )

Appendix

Table 10: Configured Attack Success Rate (C-ASR, %) after synonym substitution. Parenthesized values report absolute changes in percentage points relative to the corresponding attacks using the original triggers.

Figure 7: Training pipeline of TrapVLA. Each paired training group contains one clean sample from a benign trajectory and multiple trigger-conditioned backdoor samples derived from the corresponding target trajectories.

Hyperparameter	Value
Optimizer	AdamW
Adam coefficients $(\beta_{1},\beta_{2})$	$(0.9,\,0.95)$
Weight decay	$10^{-10}$
Warmup steps	$10{,}000$
Learning rate	$5\times 10^{-5}$
EMA decay	$0.999$

Appendix

Table 11: Common optimization hyperparameters used for the LIBERO and RoboTwin experiments.

Metric	0.1	0.3	0.5	0.7	0.9
Clean SR	95.8	93.8	93.8	96.4	95.8
Early Close C-ASR	68.6	82.4	10.2	86.8	93.2

Appendix

Table 12: Sensitivity of vanilla backdoor injection to the backdoor-data ratio for Early Close on Trap-LIBERO Object. All values are percentages.

Figure 8: Real-world robotic platform consisting of a ROKAE 6-DoF robot, an external Intel RealSense D435 camera for third-person observation, a wrist-mounted Intel RealSense D405 camera, and the tabletop objects used in the evaluation.

Figure 9: Representative real-world robot executions. Within each panel, frames are ordered from left to right. Panel (a) shows a benign trajectory under a clean instruction, whereas panels (b)–(e) show target trajectories activated by textual triggers for Early Close, Grasp Deviation, Early Open, and Release Deviation, respectively.

Adversarial Attacks on VLA Models

Adversarial attacks on vision-language-action (VLA) models study how crafted inputs, poisoned data, or physical cues can disrupt robot decisions and safety.

See all 31 papers

DropVLA: An Action-Level Backdoor Attack on Vision-Language-Action Models
Oct 13, 2025Zonghuan Xu, Jiayu Li, Yunhan Zhao +3Adversarial Attacks on VLA Models Robotic Manipulation
Institute of Trustworthy Embodied AI, Fudan University, Shanghai, China. · Shanghai Key Laboratory of Multimodal Embodied AI, Shanghai, China. · City University of Hong Kong, Hong Kong SAR, China.
TrustVLA: Mechanism-Guided Inference-Time Defense Against Vision-Language-Action Backdoors
Jul 14, 2026Pinhan Fu, Xianda Guo, Xuetao Li +5Adversarial Attacks on VLA Models Vision-Language-Action Models
School of Computer Science, Wuhan University · Institute of Automation, Chinese Academy of Sciences · 3HUST +1
FATE-VLA:Failue-aware test generation for vision-language-action models
Jun 1, 2026Arusa Kanwal, Pablo Valle, Shaukat Ali +1VLM Evaluation Vision-Language-Action Models
Mondragon University Mondragon, Spain · Simula Research Laboratory Oslo, Norway

Oct 13, 2025J/K move · Enter open · S save

DropVLA: An Action-Level Backdoor Attack on Vision-Language-Action Models

Zonghuan Xu, Jiayu Li, Yunhan Zhao, Xiang Zheng+2

Institute of Trustworthy Embodied AI, Fudan University, Shanghai, China. · Shanghai Key Laboratory of Multimodal Embodied AI, Shanghai, China. · City University of Hong Kong, Hong Kong SAR, China.

Adversarial Attacks on VLA Models Robotic Manipulation Vision-Language-Action Models Backdoor Attacks on VLMs Data Poisoning Attacks

Open paper AskPDF arXiv

Vision-Language-Action (VLA) models map multimodal perception and language instructions to executable robot actions, making them particularly vulnerable to behavioral backdoor manipulation: a hidden trigger introduced during training can induce unintended physical actions while nominal task performance remains intact. Prior work on VLA backdoors primarily studies untargeted attacks or task-level hijacking, leaving fine-grained control over individual actions largely unexplored. In this work, we present DropVLA, an action-level backdoor attack that forces a reusable action primitive (e.g., open_gripper) to execute at attacker-chosen decision points under a realistic pipeline-black-box setting with limited data-poisoning access, using a window-consistent relabeling scheme for chunked fine-tuning. On OpenVLA-7B evaluated with LIBERO, vision-only poisoning achieves 98.67%-99.83% attack success rate (ASR) with only 0.31% poisoned episodes while preserving 98.50%-99.17% clean-task retention, and successfully triggers the targeted action within 25 control steps at 500 Hz (0.05 s). Text-only triggers are unstable at low poisoning budgets, and combining text with vision provides no consistent ASR improvement over vision-only attacks. The backdoor remains robust to moderate trigger variations and transfers across evaluation suites (96.27%, 99.09%), whereas text-only largely fails (0.72%). We further validate physical-world feasibility on a 7-DoF Franka arm with pi0-fast, demonstrating non-trivial attack efficacy under camera-relative motion that induces image-plane trigger drift. These results reveal that VLA models can be covertly steered at the granularity of safety-critical actions with minimal poisoning and without observable degradation of nominal performance.

Abstract

Adversarial Attacks on VLA Models

Explore similar work

DropVLA: An Action-Level Backdoor Attack on Vision-Language-Action Models

TrustVLA: Mechanism-Guided Inference-Time Defense Against Vision-Language-Action Backdoors

FATE-VLA:Failue-aware test generation for vision-language-action models