Reasoning vision-language-action (VLA) models pair physical control with natural-language rationales that supervisors can read to monitor embodied safety. Prior attacks on VLA reasoning corrupt the rationale itself and leave visible evidence in the text. We show that visual perturbations can instead alter the physical action while keeping the generated rationale word-for-word identical. At the shared text/action interface of Alpamayo for driving and InstructVLA for manipulation, our text-neutral action-subspace (TNAS) diagnostic shows that the locally most action-sensitive direction is nearly orthogonal to the active text-loss gradients, with 1.87% and 5.28% norm overlap. At equal hidden-state perturbation norms across both models, interventions along this text-neutral residual shift actions 4.64x to 47.8x more than text-gradient perturbations, whereas text-gradient perturbations induce 21.2x to 277x more text drift. Steering visual perturbations along this direction reaches the same decoupling from the image. Across 154 obstacle-annotated driving scenes in Alpamayo, TNAS-guided attacks induce a new collision under a word-for-word identical rationale in 52.6% of scenes under full-image perturbations and 35.1% under a localized forward-camera patch, compared with 4.55% for unguided action attacks. Relative to a text-preserving baseline, TNAS guidance raises exact rationale preservation from 83.0% to 89.5% and cuts teacher-forced text drift by 68.4% at a comparable collision rate. In InstructVLA, TNAS guidance cuts text drift by 40.5% and preserves the exact response on 85.0% of examples alongside a 1.73-unit action-chunk shift. An unchanged rationale therefore cannot certify a safe action, so monitors for reasoning VLAs must verify the predicted action channel directly.
Figures & tables
Figure 1: Rationale-preserving action divergence in reasoning VLAs. On a clean driving input, the VLM and action decoder read the same unobservable hidden state to produce a matching rationale and stopping action. Adding an imperceptible perturbation to the input shifts that shared hidden state along a text-neutral, action-sensitive direction. The model still outputs “Stop to keep distance to the lead vehicle” word for word, but its predicted action flips from stopping to continuing into the lead vehicle.
Figure 2: Hidden-state intervention probes. Top: schematic of the two protocols ( gtext denotes gNLL and gmargin ). (a) Probe 2 adjusts each direction’s step size so that all directions incur the same text-KL budget B (shaded) and compares the resulting action shift. (b) Probe 3 adds every direction at the same per-example norm ∣q∣Sclip for q∈{0.5,1,2} (shaded, light to dark) and compares the resulting text KL and action shift. Middle and bottom: cohort means on 200 Alpamayo clips and 200 InstructVLA examples, with full values tabulated in Appendix A.7 . (c,f) Action shift at fixed teacher-forced KL budgets B . (d,g) Action shift and (e,h) teacher-forced text KL across equal-norm strengths q , where dact (hollow diamonds) nearly overlaps dTNAS .
Fixed-prefix
Free gen.
Open loop
Attack
ADE (m) ↑
KLtf↓
TNAS cos
Exact (%) ↑
Silent Coll. (%) ↑
Action-only PGD
70.1
2.78×10−1
4.40×10−4
9.5
4.55
Text-preserving PGD
69.2
3.26×10−1
8.00×10−4
83.0
48.7
TNAS Full image
λcos=5
67.4
3.32×10−1
1.64×10−2
80.5
46.8
λcos=10
67.8
2.99×10−1
3.93×10−2
81.0
47.4
Table 1: Alpamayo input attacks. Fixed-prefix and free-generation columns use the canonical 200 clips. The open-loop column uses the 154 of them with obstacle annotations, as described in Section 3.1 .
Fixed-prefix
Free gen.
Attack
Action shift (norm.) ↑
KLtf↓
TNAS cos
Exact (%) ↑
Action-only PGD
2.04
4.66×10−2
1.20×10−3
72.5
Text-preserving PGD
1.97
3.46×10−2
2.90×10−3
77.5
TNAS Full image
λcos=0.10
1.85
2.67×10−2
1.22×10−1
81.0
λcos=0.15
1.80
2.35×10−2
1.66×10−1
81.5
Table 2: InstructVLA input attacks. Fixed-prefix and free-generation columns use the canonical 200 examples.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 3: Shared forward-camera patch in Alpamayo. Illustrative patches on two clips of the evaluation cohort, optimized for 200 PGD steps at λcos=20 in the patch layout of Table 1 . Rows are the four cameras and columns the four input frames. Red boxes mark the patch. In every frame, the optimized pattern fills the front-telephoto box and a resized copy fills the front-wide box, while the cross-view cameras stay clean.
Outcome
Counts
Discordant
p
Alpamayo: full image λcos=30 vs. Text-preserving PGD
Exact match
179 vs. 166
20 / 7
0.0192
Lower KLtf
179 of 200
—
1.95×10−32
Attacked collision
95 vs. 94
22 / 21
1.00
Silent collision
81 vs. 75
25 / 19
0.451
Alpamayo: full image λcos=30 vs. Action-only PGD
Appendix
Table 3: Paired tests for the input tables. Binary outcomes list the counts of the two arms and the discordant examples favoring each arm, tested with exact McNemar tests. KL rows count the examples on which the first arm has lower teacher-forced KL, tested with exact sign tests. Exact match and KL use 200 examples, and collisions use the 154 annotated clips.
Layers
Mean ρ (%)
TNAS ADE (m)
Random ADE (m)
24 to 27
2.89
0.195
0.00284
28 to 31
2.34
0.161
0.00276
32 to 35
1.99
0.226
0.00356
Appendix
Table 4: Alpamayo layer-window sweep. Matched teacher-forced text-KL budget 10−4 on 50 clips per block. At layers 24 to 27, 47 clips have a TNAS row and 49 a random row within the budget.
Layers
Span
NLL grad
Margin grad
Action proxy
Feasible (%)
4 to 7
meta
0
0
1.39×10−2
0
4 to 7
response
0.109
0.849
4.82×10−3
100
14 to 17
meta
0
0
2.07×10−2
0
14 to 17
response
0.199
1.27
7.54×10−3
100
24 to 27
meta
0
0
1.94×10−2
0
24 to 27
response
0.0779
0.388
1.51×10−3
100
Appendix
Table 5: InstructVLA hook viability audit. Audit on three Bridge-distribution examples. Feasible cells have nonzero text gradients, a rank-2 text basis, and a nonzero action proxy. The selected hook is layers 14 to 17 on response tokens.
Reader: will stop
Reader: keep going
Plan outcome
Clips
Share (%)
Clips
Share (%)
Total
Comes to rest
243
63.6
46
5.76
289
Does not come to rest
139
36.4
753
94.2
892
Total
382
100
799
100
1,181
Appendix
Table 6: Benign rationale-only reader. Planned outcomes against reader predictions on 1,181 clean Alpamayo clips. Shares are within each prediction column.
Alpamayo ADE shift (m)
InstructVLA norm. ℓ2 shift
B=5×10−4
B=10−3
Direction
B=10−4
B=5×10−4
Shift
n
Shift
n
dTNAS
0.227
0.504
0.0131
189
0.0137
200
dact
0.181
0.527
0.0144
186
0.0166
200
urand
0.00359
0.00510
0.00848
194
0.00752
200
gNLL
0.00240
0.00238
0.00760
186
0.00918
200
Appendix
Table 7: Probe 2: Matched-budget action shifts in Alpamayo and InstructVLA. Alpamayo entries are mean ADE shifts in meters on 200 clips, or 199 for gNLL at B=10−4 , at nominal teacher-forced KL budgets B . InstructVLA entries are mean per-step normalized 7-DoF action shifts with the admissible count n , which covers all 200 examples at B=10−3 as detailed in Appendix A.4.2 .
Alpamayo
InstructVLA
ADE shift (m)
KL
Norm. ℓ2 shift
KL
Direction
q=0.5
q=1
q=2
q=2
q=0.5
q=1
q=2
q=2
dTNAS
0.0317
0.0673
0.142
9.87×10−5
0.0734
0.110
0.156
2.20×10−2
dact
0.0316
0.0675
0.142
1.03×10−4
0.0734
0.110
0.155
2.71×10−2
urand
0.00223
0.00261
0.00264
6.45×10−5
0.00861
0.00999
0.0124
3.64×10−3
gNLL
0.00234
0.00262
0.00297
2.66×10−2
0.0134
0.0197
0.0283
5.80×10−1
Appendix
Table 8: Probe 3: Norm-matched clean-state amplification on 200 Alpamayo clips and 200 InstructVLA examples. At each strength q , all directions receive the same per-example intervention norm ∣q∣Sclip , and q=0 is the clean state. The KL columns report teacher-forced KL at q=2 .
Adversarial attacks on vision-language-action (VLA) models study how crafted inputs, poisoned data, or physical cues can disrupt robot decisions and safety.