Reasoning vision-language-action (VLA) models pair physical control with natural-language rationales that supervisors can read to monitor embodied safety. Prior attacks on VLA reasoning corrupt the rationale itself and leave visible evidence in the text. We show that visual perturbations can instead alter the physical action while keeping the generated rationale word-for-word identical. At the shared text/action interface of Alpamayo for driving and InstructVLA for manipulation, our text-neutral action-subspace (TNAS) diagnostic shows that the locally most action-sensitive direction is nearly orthogonal to the active text-loss gradients, with 1.87% and 5.28% norm overlap. At equal hidden-state perturbation norms across both models, interventions along this text-neutral residual shift actions 4.64x to 47.8x more than text-gradient perturbations, whereas text-gradient perturbations induce 21.2x to 277x more text drift. Steering visual perturbations along this direction reaches the same decoupling from the image. Across 154 obstacle-annotated driving scenes in Alpamayo, TNAS-guided attacks induce a new collision under a word-for-word identical rationale in 52.6% of scenes under full-image perturbations and 35.1% under a localized forward-camera patch, compared with 4.55% for unguided action attacks. Relative to a text-preserving baseline, TNAS guidance raises exact rationale preservation from 83.0% to 89.5% and cuts teacher-forced text drift by 68.4% at a comparable collision rate. In InstructVLA, TNAS guidance cuts text drift by 40.5% and preserves the exact response on 85.0% of examples alongside a 1.73-unit action-chunk shift. An unchanged rationale therefore cannot certify a safe action, so monitors for reasoning VLAs must verify the predicted action channel directly.
Figures & tables
Figure 1: Rationale-preserving action divergence in reasoning VLAs. On a clean driving input, the VLM and action decoder read the same unobservable hidden state to produce a matching rationale and stopping action. Adding an imperceptible perturbation to the input shifts that shared hidden state along a text-neutral, action-sensitive direction. The model still outputs “Stop to keep distance to the lead vehicle” word for word, but its predicted action flips from stopping to continuing into the lead vehicle.
Figure 2: Hidden-state intervention probes. Top: schematic of the two protocols ( gtext denotes gNLL and gmargin ). (a) Probe 2 adjusts each direction’s step size so that all directions incur the same text-KL budget B (shaded) and compares the resulting action shift. (b) Probe 3 adds every direction at the same per-example norm ∣q∣Sclip for q∈{0.5,1,2} (shaded, light to dark) and compares the resulting text KL and action shift. Middle and bottom: cohort means on 200 Alpamayo clips and 200 InstructVLA examples, with full values tabulated in Appendix A.7 . (c,f) Action shift at fixed teacher-forced KL budgets B . (d,g) Action shift and (e,h) teacher-forced text KL across equal-norm strengths q , where dact (hollow diamonds) nearly overlaps dTNAS .
Fixed-prefix
Free gen.
Open loop
Attack
ADE (m) ↑
KLtf↓
TNAS cos
Exact (%) ↑
Silent Coll. (%) ↑
Action-only PGD
70.1
2.78×10−1
4.40×10−4
9.5
4.55
Text-preserving PGD
69.2
3.26×10−1
8.00×10−4
83.0
48.7
TNAS Full image
λcos=5
67.4
3.32×10−1
1.64×10−2
80.5
46.8
λcos=10
67.8
2.99×10−1
3.93×10−2
81.0
47.4
Table 1: Alpamayo input attacks. Fixed-prefix and free-generation columns use the canonical 200 clips. The open-loop column uses the 154 of them with obstacle annotations, as described in Section 3.1 .
Fixed-prefix
Free gen.
Attack
Action shift (norm.) ↑
KLtf↓
TNAS cos
Exact (%) ↑
Action-only PGD
2.04
4.66×10−2
1.20×10−3
72.5
Text-preserving PGD
1.97
3.46×10−2
2.90×10−3
77.5
TNAS Full image
λcos=0.10
1.85
2.67×10−2
1.22×10−1
81.0
λcos=0.15
1.80
2.35×10−2
1.66×10−1
81.5
Table 2: InstructVLA input attacks. Fixed-prefix and free-generation columns use the canonical 200 examples.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 3: Shared forward-camera patch in Alpamayo. Illustrative patches on two clips of the evaluation cohort, optimized for 200 PGD steps at λcos=20 in the patch layout of Table 1 . Rows are the four cameras and columns the four input frames. Red boxes mark the patch. In every frame, the optimized pattern fills the front-telephoto box and a resized copy fills the front-wide box, while the cross-view cameras stay clean.
Outcome
Counts
Discordant
p
Alpamayo: full image λcos=30 vs. Text-preserving PGD
Exact match
179 vs. 166
20 / 7
0.0192
Lower KLtf
179 of 200
—
1.95×10−32
Attacked collision
95 vs. 94
22 / 21
1.00
Silent collision
81 vs. 75
25 / 19
0.451
Alpamayo: full image λcos=30 vs. Action-only PGD
Appendix
Table 3: Paired tests for the input tables. Binary outcomes list the counts of the two arms and the discordant examples favoring each arm, tested with exact McNemar tests. KL rows count the examples on which the first arm has lower teacher-forced KL, tested with exact sign tests. Exact match and KL use 200 examples, and collisions use the 154 annotated clips.
Layers
Mean ρ (%)
TNAS ADE (m)
Random ADE (m)
24 to 27
2.89
0.195
0.00284
28 to 31
2.34
0.161
0.00276
32 to 35
1.99
0.226
0.00356
Appendix
Table 4: Alpamayo layer-window sweep. Matched teacher-forced text-KL budget 10−4 on 50 clips per block. At layers 24 to 27, 47 clips have a TNAS row and 49 a random row within the budget.
Layers
Span
NLL grad
Margin grad
Action proxy
Feasible (%)
4 to 7
meta
0
0
1.39×10−2
0
4 to 7
response
0.109
0.849
4.82×10−3
100
14 to 17
meta
0
0
2.07×10−2
0
14 to 17
response
0.199
1.27
7.54×10−3
100
24 to 27
meta
0
0
1.94×10−2
0
24 to 27
response
0.0779
0.388
1.51×10−3
100
Appendix
Table 5: InstructVLA hook viability audit. Audit on three Bridge-distribution examples. Feasible cells have nonzero text gradients, a rank-2 text basis, and a nonzero action proxy. The selected hook is layers 14 to 17 on response tokens.
Reader: will stop
Reader: keep going
Plan outcome
Clips
Share (%)
Clips
Share (%)
Total
Comes to rest
243
63.6
46
5.76
289
Does not come to rest
139
36.4
753
94.2
892
Total
382
100
799
100
1,181
Appendix
Table 6: Benign rationale-only reader. Planned outcomes against reader predictions on 1,181 clean Alpamayo clips. Shares are within each prediction column.
Alpamayo ADE shift (m)
InstructVLA norm. ℓ2 shift
B=5×10−4
B=10−3
Direction
B=10−4
B=5×10−4
Shift
n
Shift
n
dTNAS
0.227
0.504
0.0131
189
0.0137
200
dact
0.181
0.527
0.0144
186
0.0166
200
urand
0.00359
0.00510
0.00848
194
0.00752
200
gNLL
0.00240
0.00238
0.00760
186
0.00918
200
Appendix
Table 7: Probe 2: Matched-budget action shifts in Alpamayo and InstructVLA. Alpamayo entries are mean ADE shifts in meters on 200 clips, or 199 for gNLL at B=10−4 , at nominal teacher-forced KL budgets B . InstructVLA entries are mean per-step normalized 7-DoF action shifts with the admissible count n , which covers all 200 examples at B=10−3 as detailed in Appendix A.4.2 .
Alpamayo
InstructVLA
ADE shift (m)
KL
Norm. ℓ2 shift
KL
Direction
q=0.5
q=1
q=2
q=2
q=0.5
q=1
q=2
q=2
dTNAS
0.0317
0.0673
0.142
9.87×10−5
0.0734
0.110
0.156
2.20×10−2
dact
0.0316
0.0675
0.142
1.03×10−4
0.0734
0.110
0.155
2.71×10−2
urand
0.00223
0.00261
0.00264
6.45×10−5
0.00861
0.00999
0.0124
3.64×10−3
gNLL
0.00234
0.00262
0.00297
2.66×10−2
0.0134
0.0197
0.0283
5.80×10−1
Appendix
Table 8: Probe 3: Norm-matched clean-state amplification on 200 Alpamayo clips and 200 InstructVLA examples. At each strength q , all directions receive the same per-example intervention norm ∣q∣Sclip , and q=0 is the clean state. The KL columns report teacher-forced KL at q=2 .
Adversarial attacks on vision-language-action (VLA) models study how crafted inputs, poisoned data, or physical cues can disrupt robot decisions and safety.
Vision-Language-Action (VLA) models with integrated reasoning have been proposed for end-to-end autonomous driving, assuming a tight coupling between reasoning and trajectory generation. However, the robustness of such systems under realistic input perturbations remains largely unexplored. We show that these models are highly vulnerable to realistic input perturbations, achieving up to 89% attack success rate (ASR) on reasoning and up to 72% on trajectory manipulation in closed-loop simulation, leading to increased collision rates and degraded safety metrics. Using NVIDIA's recent Alpamayo models as representative industry-developed VLAs, we conduct the first systematic black-box study of reasoning-enabled VLA models under realistic textual input corruptions, evaluating their impact on reasoning and driving behavior. We introduce a reasoning-aware evaluation framework capturing both semantic and structural aspects of reasoning, along with safety-centric measures. We also introduce a benchmark for evaluating attacks and defenses on reasoning-trajectory interactions in autonomous driving. Our results highlight the need for rigorous evaluation and improved defenses to ensure the safety of reasoning-enabled VLA systems in autonomous driving.
Mohammadreza Teymoorianfard, Jean-Philippe Monteuuis, Jonathan Petit +1
Textual reasoning before acting has improved vision-language-action (VLA) models, but at the cost of latency: every action conditioned on the reasoning text must wait for autoregressive decoding. We show that actions do not need to wait for or condition on the generated reasoning to benefit from it, because the reasoning supervision already improves the backbone representation. We introduce TacitVLA, a reasoning VLA whose action expert never attends to the reasoning it generates for the current call. On long-horizon tasks where the reasoning text also serves as episodic memory, it can generate each entry during action execution rather than before acting. We evaluate it in a controlled study on four benchmarks spanning manipulation and autonomous driving, memory and non-memory tasks: RoboMME, RMBench, LIBERO, and PhysicalAI-AV. On all four, TacitVLA matches the performance of its explicit-reasoning counterpart at up to 3.4x lower latency, and matches the latency of the same architecture trained without reasoning at up to +17.1% success. In absolute terms, it surpasses the best previously published results on the RoboMME Counting suite (82.7% vs. 66.8%) and RMBench (77.5% vs. 56.5%), and reaches 72.9% on LIBERO-Plus and a 52.5% scene score on the public split of the PhysicalAI-AV closed-loop driving challenge. Project page: https://tacitvla.github.io.
Jack Lu, Yurong You, Boyi Li +8
NVIDIA · New York University · University of California, Berkeley +2
Does adding a reasoning step make a Vision-Language-Action (VLA) model more robust to perturbation? Intuitively, a policy that reasons before acting should absorb a perturbed input better than one that maps observations directly to actions. We test this premise head-on across three models that span the reasoning spectrum (no reasoning, a text chain-of-thought, and a latent iterative loop), perturbing each at the vision, reasoning, and action stages on LIBERO and SimplerEnv. Two questions organize the study: does the reasoning design shift robustness, and can the reasoning be read back at runtime as a safety signal? We find that the latent-iterative model is by far the least robust: under both stochastic noise and white-box perturbation its task success collapses, while the other two hold. This fragility is structural rather than cumulative: varying the reasoning depth at inference barely moves it. Reasoning outputs can in principle be monitored, but the monitors fail under fair tests. A plan--action consistency probe that looks near-perfect under naive evaluation falls to chance under adaptive attack. Under matched-FPR calibration, fusing it with an action-anomaly probe never lifts defended success above undefended. Scoped to these output-level behavioral probes under white-box vision-stage attack, this ceiling is a precondition that any viable defense must first satisfy.