Reasoning-augmented vision-language-action (VLA) driving models can perceive and reason about safety-critical hazards yet fail to translate that evidence into safe actions. We characterize this failure as a perception-action binding gap: hazard information is available inside the model but exerts insufficient control over trajectory generation. Using matched simulated counterfactuals and logged driving replay, we show that in-path hazards are actually encoded in the visual representation, but longitudinal plans are dominated by ego-speed history and respond only weakly to hazard proximity or the model's own reasoning trace. Further analysis shows that the trajectory generator is capable of braking but sometimes fails to do so due to the perception-action binding gap. Without retraining or modifying any weights, we inject a hazard-conditioned energy into the frozen flow-matching sampler of Alpamayo-1.5, restoring the missing coupling at generation time. With exact obstacle geometry, guidance cuts footprint intrusion from 58.1% to 14.5% and recovers safe braking whenever sufficient stopping distance remains, eliminating collisions entirely in our braking-relevant closed-loop battery. It also reverses the diagnosed dependence of the plan, from tracking ego speed to responding to obstacle range. These results separate hazard perception from hazard control in driving VLAs, show that unsafe behavior can arise even when the model has both the evidence and the ability to act on it, and propose a simple fix for bridging this binding gap.
Figures & tables
Fig. 1: Where perception–action binding fails, and where we intervene. Binding gap: visual hazard evidence and the model’s reasoning trace have substantially weaker authority over the longitudinal plan than ego history. Intervention: a hazard-conditioned energy gradient is added during flow integration, steering the frozen trajectory generator toward a safe plan without modifying model weights.
Arm: Other
Contact ↓
Clear. ↑
Δ clear.
Car’s Motion
(%)
(m)
(m)
All ( n=38 )
Drives on
0.0
24.31
—
Stops
42.1 (26.3, 57.9)
0.09
−23.7 ( −29.0 , −18.5 )
Must brake ( n=24 )
Drives on
0.0
24.31
—
Stops
54.2 (33.3, 75.0)
−0.02
−23.1 ( −28.6 , −17.5 )
+ guidance
0.0
18.08
+17.3 (+15.0, +19.6)
TABLE I: The dynamic closed-loop hazard: the frozen vehicle is not avoided. Clearance is minimum separation; Δ is paired vs. control (95% CI, scene-clustered bootstrap, B=104 ). The guided row is paired against the car-stops arm, i.e. the effect of the intervention. The Must brake set consists of 24 scenes where avoiding the car requires braking.
Fig. 2: Hazard presence is decodable well before the plan reacts. In matched CARLA pairs, hazard presence is decodable at an area under the curve (AUC) of 0.998, yet every evaluated scene at 8–12 m intrudes. Whiskers are 95% CIs, clustered by site.
Fig. 3: The real-scene plan tracks ego speed, not obstacle range. On held-out real scenes at the planner’s native four-camera input, planned forward reach tracks logged ego speed (left) far better than privileged obstacle range (right).
Fig. 4: Longitudinal action authority is concentrated in ego history. Removing ego history changes longitudinal motion far more than blanking vision or the reasoning trace; the gap mostly disappears for lateral motion. Hatched bars repeat the same ablation on n=120 OOD clips. Whiskers are 95% CIs bootstrapped over clips.
Fig. 5: Exact geometry is sufficient for safe braking, and sourcing it from the model narrows most of the gap. All arms share the same guidance energy and differ only in the signal s=(g,d^,ℓ^) : d⋆ is the privileged ground-truth range, d^ the cache-derived estimate, and gvis its learned gate. (a) Footprint intrusion and (b) near-stop rate on matched CARLA hazard sites, seen and six new sites held out in appearance. Whiskers are 95% CIs, site-clustered bootstraps.
Fig. 6: Guidance restores obstacle authority over longitudinal planning. Planned reach on 273 held-out real scenes vs. (a) ground-truth obstacle range and (b) logged ego speed. Without guidance (gray), reach tracks ego speed but not range ( ρ=+0.645 (+0.552, +0.721) vs. +0.009 ( − 0.110, +0.125)); exact-geometry guidance (green) reverses this, tracking range but not speed ( ρ=+0.678 (+0.599, +0.744) vs. +0.177 (+0.025, +0.337)).
readout
MAE (m)
recall
ceiling
intrusion
clear cost
↓
↑
↑
↓
↓
Seen ( n=144 )
pooled, front-wide
2.20
81%
83%
13.4%
4.63 m
+ front-tele
1.23
74%
74%
16.1%
0.00 m
attention, front-wide
1.99
76%
76%
16.3%
0.00 m
Unseen ( n=91 )
pooled, front-wide
3.28
55%
63%
28.6%
0.00 m
+ front-tele
2.43
66%
68%
18.5%
0.00 m
TABLE II: Input coverage and spatial readout determine the quality of model-derived guidance. Results from the same CARLA sites as Figure 5 . Attention row is averaged over five seeds.
Router
Gating recall ↑
Nominal stop ↓
(%, 95% CI)
(%, 95% CI)
Vision detector
71% (62, 80)
11% (6, 16)
Reasoning trace
55% (45, 65)
15% (10, 21)
Union
87% (80, 93)
18% (12, 25)
TABLE III: Vision and reasoning route on different evidence. On the routing corpus, combining vision and reasoning raises recall from 71% to 87%.
Vision-language-action (VLA) driving models couple a reasoning stage with a diffusion-based trajectory decoder, but do not give a direct way to redirect attention toward safety-critical actors at inference time without retraining. We studied a bounded additive pre-softmax attention bias on the visual tokens of detector localized traffic actors on Alpamayo-R1's Qwen3-VL backbone. It is applied as a fail open forward pre-hook with no weight changes. On 50 lane-change scenarios from the Physical AI World Model Synthetic dataset. The trajectory decoder shows a monotonic dose response in the bias magnitude, separate from a paired zero bias control at every tested magnitude. It reaches ≈17,cm mean displacement with lateral shifts up to ∼140\ cm at the clamp. A layer ablation places the action-relevant signal in late layers, where the effect increases with the number of hooked layers (2.0cm for the first 8 layers; 67.6cm for all 36). A per call injection audit explains why the Chain-of-Causation text never changes. The mask based bias never reaches the reasoning pathway in this serving stack, so the invariance is verified exposure, not robustness. Steered trajectories tend to shift toward the attended actor, suggesting the bias governs where the model looks rather than encoding a target behavior.
Darshan Nagendra Prasad, Lars Ullrich, Knut Graichen
Driving vision-language-action (VLA) models bring an appealing form of explainability to a safety-critical domain, since a system that can state the reasons for its maneuvers offers the transparency that deployment and public trust in autonomous driving demand. These models read the world into language before they act on it, describing the scene and reasoning over it in text that ranges from chain-of-thought to multi-stage question answering on the way to a trajectory. That framing quietly assumes two separable properties, namely that the trajectory follows the generated reasoning and that the generated reasoning tracks the causes that actually produce the trajectory. To thoroughly investigate these properties, we apply causal, counterfactual interventions to four driving VLAs across different benchmarks. First, we perturb the models' own reasoning and show that altering its meaning produces small trajectory changes relative to blanking the same models' camera inputs, a pattern that persists across benchmarks and model architectures. We then construct scene-level counterfactuals through photorealistic removal of key objects named in the models' reasoning and analyze whether the generated reasoning and trajectories track these changes. Overall, our findings indicate that the reasoning these models present to account for their resulting driving behavior is weakly coupled to that behavior itself, pointing to limits on the explainability these systems are intended to provide.
Vision-Language-Action (VLA) models with integrated reasoning have been proposed for end-to-end autonomous driving, assuming a tight coupling between reasoning and trajectory generation. However, the robustness of such systems under realistic input perturbations remains largely unexplored. We show that these models are highly vulnerable to realistic input perturbations, achieving up to 89% attack success rate (ASR) on reasoning and up to 72% on trajectory manipulation in closed-loop simulation, leading to increased collision rates and degraded safety metrics. Using NVIDIA's recent Alpamayo models as representative industry-developed VLAs, we conduct the first systematic black-box study of reasoning-enabled VLA models under realistic textual input corruptions, evaluating their impact on reasoning and driving behavior. We introduce a reasoning-aware evaluation framework capturing both semantic and structural aspects of reasoning, along with safety-centric measures. We also introduce a benchmark for evaluating attacks and defenses on reasoning-trajectory interactions in autonomous driving. Our results highlight the need for rigorous evaluation and improved defenses to ensure the safety of reasoning-enabled VLA systems in autonomous driving.
Mohammadreza Teymoorianfard, Jean-Philippe Monteuuis, Jonathan Petit +1