How Much Should a VLA Trust Its Reasoning? Step-Level Guidance as a Measured Action-Value Function

Dabin Seo; Eunhyek Joa

cs.ROOct 29, 2026

How Much Should a VLA Trust Its Reasoning? Step-Level Guidance as a Measured Action-Value Function

Authors: Dabin Seo, Eunhyek Joa

Organizations: Department of Mechanical Engineering, Seoul National University

Abstract

Reasoning vision-language-action (VLA) policies write a reasoning trace and then decode the action from it. A wrong trace yields a wrong action, and even a correct-looking trace can mislead the plan. Existing remedies ask whether the reasoning is right. We ask how strongly it should steer. CoC-CFG applies classifier-free guidance between the policy's context and the same context without the reasoning span. One weight per Euler step attenuates or amplifies the reasoning, and w=1 recovers the original policy. We treat the choice of this weight as a Markov decision process whose action values are measured, not bootstrapped. With the expert frozen and the noise fixed, applying a weight for one step and finishing the plan under the current controller returns that weight's value exactly. Q-adaptive guidance fits these values with small critics and improves the controller conservatively. In the Alpasim driving simulator it outperforms every fixed weight and online RL (SAC, GRPO) in open loop, and in closed loop it passes 262 of 300 scenes, against 252 for the original policy. On held-out frames of the Waymo end-to-end driving benchmark, its first three stages outperform every fixed weight, four adaptive-guidance methods, SAC and GRPO. On the hidden test set, a frozen Alpamayo 1.5 with rule-based camera and vehicle conversions and Q-adaptive guidance sets a new state of the art among published methods: 8.135159 rater feedback score, against 8.079 for the best published method.

Figures & tables

Figure 1: Not whether to trust a VLA’s reasoning, but how strongly it steers the action, at every step. A reasoning VLA writes its reasoning, then decodes the action conditioned on it. (a) CoC-CFG sets with one weight per decoding (Euler) step how strongly the reasoning steers, from removed ( $0$ ) through the original model ( $1$ ) to amplified ( $2$ ). Held at every step, a larger weight turns the plan of (b) harder and carries the plan of (c) farther. (b) On a held-out scene of the Alpasim driving simulator the reasoning calls for a lane change the scene does not offer; the original model (red) follows it off the recorded path (green). (c) This reasoning reads correctly, yet followed at full strength the plan runs far ahead of the recorded drive. In both, ours (blue; Q-adaptive guidance trained in Alpasim) attenuates the reasoning.

Figure 2: Q-adaptive guidance. At every Euler step small critics read what the step has computed, and the penalized argmax of their mean is

w_{k}

. Offline, each weight’s value is measured by one-step branches, and the critics are refit on every generation of labels.

	reward $\uparrow$	vs. $w{=}1$
$w{=}0$ (span removed)	$-1.1315$	$+3.3$ %
$w{=}1$ (default)	$-1.1704$	–
best fixed $w{=}0.5{}$ (anchor)	$-1.0495$	$+10.3$ %
SAC, best checkpoint	$-1.0468$	$+10.6$ %
GRPO, best checkpoint	$-1.0091$	$+13.8$ %
ours, stage $n{=}1$	$-0.9943$	$+15.0$ %

Table 1: Alpasim open loop, held-out, Alpamayo 1.5. Mean reward (negative displacement cost) over 358 scenes, and the cost reduction against the default

w{=}1

. SAC and GRPO at their best held-out checkpoint. Bold : best reward.

	passed	at-fault coll.	dist. to log (m) $\downarrow$	score $\uparrow$
$w{=}0$ (span removed)	243/300	21	1.239	0.638
$w{=}0.5{}$ (anchor)	254/300	10	1.272	0.691
$w{=}1$ (default)	252/300	13	1.232	0.695
ours	262/300	7	1.118	0.712

Table 2: Alpasim closed loop, Alpamayo 1.5. 300 scenes outside the training pool, 10 s at 10 Hz inside the reconstruction after ground-truth warm-up, one inference per 25 control steps, one rollout per scene and arm; ours is the stage-

n{=}3

controller. A rollout passes when it ends without an at-fault (front or lateral) collision or an off-road event; distance to log is the largest gap to the logged path, averaged over scenes; score is the simulator’s composite rollout score. Bold : best in a column.

method	RFS	$\Delta$ , ours minus row
fixed $w{=}0.25$	8.1431	$+0.160$
fixed $w{=}0.5$	8.2096	$+0.094$
fixed $w{=}0.75$	8.2412	$+0.062$
fixed $w{=}1.0$ (default, anchor)	8.2363	$+0.067$
fixed $w{=}1.25$	8.1736	$+0.130$
fixed $w{=}1.5$	8.1197	$+0.183$

Table 3: Waymo, held-out, Alpamayo 1.5. RFS on the 358 held-out frames, every arm with the same conversions;

\Delta

is ours (stage

n{=}2

) minus the row.

\ddagger

: one weight per scene, scored in a separate evaluation of the same frames; its

\Delta

is against the per-step controller in that evaluation, its four stages averaged per scene (8.1972).

Method	RFS $\uparrow$
SimWAM [ 35 ]	7.84
FROST-Drive [ 6 ]	7.856
MindVLA-U1 + RL [ 11 ]	7.87
IRL-VLA [ 12 ]	7.89
Qwen-Drive-1.0-RL [ 37 ]	7.91
SUV [ 34 ]	7.94

Table 4: Waymo end-to-end driving, official hidden test. Rater feedback score (RFS) of the ten best published methods with a reported hidden-test score, one row per paper with its best reported variant, as printed in its paper or in the hidden-test tables of [ 34 , 36 ] ; four-decimal scores are rounded to three.

Reasoning for VLA Models

Reasoning for vision-language-action (VLA) models connects visual and language understanding to grounded planning and action.

See all 113 papers

When Reasoning Helps Action: Monitoring and Steering Chain-of-Thought in Vision-Language-Action Policies
Sep 30, 2026Sathwik Karnik, Joseph JR. Lee, Aryaman Gupta +1VLMs for Autonomous Driving Vision-Language-Action Models
Safe and Intelligent Autonomy Lab, Stanford University
Reasoning as a Distribution: Generative Plan Latents for Vision-Language-Action Models
Oct 20, 2026Matteo Cercola, Katie Z Luo, Milan Ganai +3Reasoning for VLA Models Representation Learning
Politecnico di Milano · Stanford University · Air Force Research Laboratory +1
Guide, Think, Act: Interactive Embodied Reasoning in Vision-Language-Action Models
May 13, 2026Yiran Ling, Qing Lian, Jinghang Li +6Robotic Control Robot Policy Learning
Faculty of Computing, Harbin Institute of Technology · International Digital Economy Academy (IDEA) · National Key Laboratory of Smart Farm Technologies and Systems +4

Sep 30, 2026J/K move · Enter open · S save

When Reasoning Helps Action: Monitoring and Steering Chain-of-Thought in Vision-Language-Action Policies

Sathwik Karnik, Joseph JR. Lee, Aryaman Gupta, Somil Bansal

Safe and Intelligent Autonomy Lab, Stanford University

VLMs for Autonomous Driving Vision-Language-Action Models VLMs for Robotics Embodied Reasoning

Open paper AskPDF arXiv

Reasoning-enabled VLA policies expose chain-of-thought (CoT) traces that appear to explain and guide their actions, creating a potential interface for runtime safety through reasoning monitoring and correction. In this work, we define and operationalize two evaluation axes for assessing when this interface can improve embodied behavior: correctability, which measures whether unreliable reasoning can be detected and improved during generation, and actionability, which measures whether reasoning corrections produce behaviorally meaningful changes in the intended direction. To enable correctability, we introduce Token-level Reward for Utility-Steered Chain-of-Thought (TRUST), an offline-trained value model that predicts eventual reasoning correctness from partial prefixes and uses these estimates to monitor and selectively steer reasoning generation in frozen VLA policies. On the Alpamayo 1.5 driving VLA, TRUST monitors correctness with 88.9% accuracy and improves reasoning correctness from 75.9% to 90.0%. On a baseline-defined challenging subset in AlpaSim, TRUST reduces collision rate by 30.4% and maximum trajectory error by 11.5% relative to the unsteered policy, outperforming a compute-matched Best-of-4 baseline. On the DeepThinkVLA manipulation VLA, TRUST improves the correctness of grasp-state claims from 69.3% to 90.2% and action-choice claims from 68.8% to 85.9%, yet closed-loop task performance on LIBERO-Plus remains largely unchanged. Empirical analysis reveals intent-consistent behavioral effects in Alpamayo 1.5 but limited effects in DeepThinkVLA, helping interpret these different task-level outcomes. Together, our results show that gains in reasoning correctness do not automatically imply gains in embodied performance, motivating evaluation of correctability and actionability when using CoT as a runtime safety interface.

Abstract

Reasoning for VLA Models

Explore similar work

When Reasoning Helps Action: Monitoring and Steering Chain-of-Thought in Vision-Language-Action Policies

Reasoning as a Distribution: Generative Plan Latents for Vision-Language-Action Models

Guide, Think, Act: Interactive Embodied Reasoning in Vision-Language-Action Models