Reasoning as a Distribution: Generative Plan Latents for Vision-Language-Action Models

Matteo Cercola; Katie Z Luo; Milan Ganai; John Irvin Alora; Simone Formentin; Marco Pavone

cs.ROOct 20, 2026

Reasoning as a Distribution: Generative Plan Latents for Vision-Language-Action Models

Authors: Matteo Cercola, Katie Z Luo, Milan Ganai, John Irvin Alora, Simone Formentin, Marco Pavone

Organizations: Politecnico di Milano · Stanford University · Air Force Research Laboratory · NVIDIA

Abstract

Vision-Language-Action (VLA) policies increasingly reason before acting, but existing reasoning models compress this intermediate decision into a single representation. When several plans are valid, this commitment hides alternatives that could otherwise be selected or revised. We introduce the Generative Plan Latent (GPL), which represents the reasoning state as a low-dimensional conditional distribution over plans, such as which object, grasp, or route to use. This makes the reasoning state addressable: without retraining the VLA, plans can be sampled (coverage), selected on request (steering), and reweighted by deployment outcomes (outcome adaptation). We test all three on SimplerEnv manipulation with $π_0$ , and extend the analysis to RoboCasa, LIBERO, and a maze domain. Steering the plan raises the fraction of executions that follow the request from 0.48 to 0.94, while the same guidance on the native action distribution shows no reliable gain. Finally, we show that an explicit plan is most useful when it carries decision information not already determined by the observation.

Figures & tables

Figure 1: The distribution should live at the decision bottleneck. When several plans are valid, a committed regression readout can lie between them; gpl preserves the alternatives at the plan level, where they can be sampled, guided, or reweighted before execution.

Figure 2: Generative Plan Latent ( gpl ). The frozen VLA backbone encodes $(o,l)$ into $c$ ; a small rectified-flow head samples a plan $z\sim p_{\theta}(z\mid c)$ ; a projector injects $z$ as $k=8$ soft tokens into the executor prefix. Sampling, guidance, and outcome-based reranking all operate on this one plan distribution.

arm	invalid draw	P(cover all)	steer range	rerank feasible	addressable
deterministic MSE twin	1.00	0.00	0.00	0.00	no
deterministic WTA (MAP)	0.03	0.00	0.00	0.50	no
gpl (flow)	0.14	1.00	+0.31	1.00	yes

Table 1: Avoiding the mean is insufficient. Eight held-out layouts, five seeds,

K=16

. The winner-take-all selector removes mode averaging almost completely but still sits at the floor on coverage and steerability.

Figure 3: GPL provides a stronger intervention point than the native action distribution. Left: commanded-mode rate versus guidance dose for GPL and a dedicated no-plan adapter guided through

\pi_{0}

’s native

128

-D action chunk (

n=96

episodes/dose; 95% cluster bootstrap). Right: gain over the unguided baseline of each arm, with 95% intervals.

Figure 4: Deployment experience improves plan selection. A: In three-object rearrangement, all orders are valid but differ in chained end-effector path length; we report normalized regret above the cheapest order (5 seeds, mean $\pm$ s.d.). The best fixed order is cost-aware but not scene-adaptive. B: In the two-cube task, valid targets have different unobserved success probabilities; we report final success versus training FLOPs (10 seeds, 95% CIs). Frozen-generator reranking matches retraining baselines with far less compute, while the reward table requires none.

Figure 5: A plan becomes consequential when it carries unresolved or missing decision information. Left: the generative advantage grows with controlled ambiguity $U=1-|2\beta-1|$ , where $\beta$ is the fraction of training-demonstration mass on the preferred cube. Right: carrying an earlier decision grows more valuable as occlusion removes the information that produced it, relative to stock $\pi_{0}$ ; the privileged-state curve sets a ground-truth-pose ceiling.

Figure 6: Addressability survives behavior-learned coordinates. Left: Offline, properties of samples straight out of the plan generator, never executed. Right: Closed loop (through

\pi_{0}

), the full manipulation episode is rolled out in the simulator. Error bars are 95% percentile-bootstrap intervals over training seeds and held-out layouts (10,000 replicates).

Appendix figures & tables19 assets

Supplementary material from the paper’s appendix.

Appendix

method	continuous latent	iterative/nonlinear	generative plan	addressable
LaRA-VLA ( Bai et al., 2026 )	✓	–	–	–
ThinkAct ( Huang et al., 2026b )	✓	–	–	–
Fast-ThinkAct ( Huang et al., 2026a )	✓	–	–	–
Recurrent-Depth ( Tur et al., 2026 )	✓	✓	–	–
CoT-VLA ( Zhao et al., 2025 )	– (pixels)	✓	✓	–
gpl (Ours)	✓	✓	✓	✓

Appendix

Table 2: Where latent reasoning appears in action-domain VLAs, and which of them is addressable. The comparison concerns the intermediate decision supplied to the action model, not whether the downstream action head is generative. Among the listed methods, gpl is the only method that is addressable.

Component	Configuration
Conditioning $c$	Frozen VLA final-layer, final-token feature
Conditioning dimension	$d_{\text{model}}$
Condition encoder	Linear $d_{\text{model}}\!\rightarrow\!64$ , LayerNorm, GELU
Time embedding	64-D sinusoidal
Flow hidden layers	256, 256
Flow activation	SiLU

Appendix

Table 3: GPL training recipe. The frozen backbone is used only to precompute conditioning features; the generative model itself is a small MLP trained from scratch for each plan representation.

Configuration	LIBERO	SimplerEnv / Bridge
Pretrained checkpoint	lerobot/pi05_libero_finetuned_v044	juexzz/INTACT-pi0-finetune-bridge
Base VLA	$\pi_{0.5}$	$\pi_{0}$
Base parameters	Frozen	Frozen
Plan tokens $k$	8	8
Prefix width	2048	2048
Projector	$d_{z}\!\rightarrow\!512\!\rightarrow\!512\!\rightarrow\!8\times 2048$ , SiLU

Appendix

Table 4: VLA adaptation recipes. Both domains use the same eight-token prefix-conditioning mechanism, but start from different pretrained checkpoints and use independently tuned LoRA and optimization configurations.

Model Variant	Rel. $L_{2}$	Cosine	Grasp Success
Stock $\pi_{0}$	$0.000$	$1.000$	$5/48=0.104\,[0.045,0.222]$
Matched plan-less	$1.395$	$0.348$	$\mathbf{0/48=0.000}\,[0.000,0.074]$
Dedicated no-plan	$1.474$	$0.283$	$6/48=0.125\,[0.059,0.247]$

Appendix

Table 5: Omitting plan tokens does not recover stock

\pi_{0}

. Performance comparison on the two-cube grasp task. Relative

L_{2}

and cosine similarities are measured against stock

\pi_{0}

Configuration	Value
Reasoning latent	Deterministic, $d_{h}=3$
Reasoner	2-layer MLP, SiLU
Visual target encoder	EMA, $\tau=0.99$
Action head	Conditional rectified flow
Flow sampling steps	4 Euler steps
Optimizer	Adam

Appendix

Table 6: LaRA-VLA baseline recipe. Architecture and optimization settings of the matched deterministic-latent baseline described in Section B.3 .

Configuration	Value
Trajectory input	16 future EEP positions
Reference frame	Relative to current EEP
Input dimension	48
Encoder	$48\rightarrow 128\rightarrow 2$
Decoder	$2\rightarrow 128\rightarrow 48$
Activation	SiLU

Appendix

Table 7: Learned-coordinate recipe. The representation is learned solely by reconstructing future end-effector motion.

Figure 7: The three capabilities reproduce in navigation. From left: representative ring and staircase mazes; $K=16$ samples recover both valid routes while the deterministic twin collapses; commanded-route fidelity rises continuously with guidance strength; and feedback about a hidden forbidden route raises success to $1.00$ while shuffled feedback lags. Throughout, circle markers are the ring maze and square markers the staircase maze, as keyed in the leftmost panel.

Configuration	gpl (generative)	Deterministic twin	No-plan baseline
Plan representation	144-D	144-D	–
Plan head	Continuous-time diffusion	Single-pass regression	None
Plan loss	Diffusion, $\lambda=5.0$	MSE	–
Action loss	Autoregressive move cross-entropy
Optimizer	AdamW, weight decay $0.01$ , grad. clip $1.0$
Learning rate	$5\times 10^{-4}$	$5\times 10^{-4}$	$3\times 10^{-4}$

Appendix

Table 8: Maze-domain training recipe. All variants share the same backbone, data, optimizer family, and autoregressive move decoder; only the plan interface differs.

read-out	invalid draw	P(cover all)	steer range	rerank feasible
WTA, MAP	$0.03\,[0.00,0.09]$	$0.00$	$0.00$	$0.50\,[0.39,0.61]$
WTA, component sampled	$0.07\,[0.05,0.09]$	$0.90\,[0.83,0.97]$	$+0.48\,[+0.46,+0.51]$	$1.00$

Appendix

Table 9: Addressability depends on the inference read-out, not only on whether the model has learned multiple modes. Both rows use the same WTA-trained weights ; only the read-out changes. Eight held-out layouts, five seeds,

K=16

. Intervals are Student-

t

intervals over seeds, truncated to the support of each metric.

Figure 8: Steering is stronger at the plan locus under a shared executor. Left: Commanded-mode rate at the pre-fixed operating point

w=16

for the gpl plan latent, the generative action head, and the LaRA-style deterministic reasoning token. Intervals are

95\%

bootstraps clustered over the

40

held-out layouts. Right: Per-layout paired difference, ranked over the same layouts. The plan locus is favored on

38/40

layouts.

arm	command interface	seed 0	seed 1	seed 2	grasp
gpl	plan latent	$+0.70\,[+0.47,+0.90]$	$+0.48\,[+0.23,+0.73]$	$+0.45\,[+0.18,+0.75]$	$0.33$
L-soft	sentence feature	$+0.00\,[0.00,0.00]$	$-0.08\,[-0.18,0.00]$	$+0.00\,[0.00,0.00]$	$0.06$
L-prefix	instruction	$+0.13\,[-0.05,+0.35]$	$+0.20\,[+0.03,+0.45]$	$+0.20\,[+0.03,+0.43]$	$0.13$
L-stock	ECoT	$-0.05\,[-0.30,+0.13]$	$+0.03\,[-0.05,+0.10]$	$+0.08\,[+0.03,+0.15]$	$0.50$
gpl $-$ L-stock	paired difference	$+0.75\,[+0.40,+1.13]$	$+0.45\,[+0.20,+0.75]$	$+0.38\,[+0.13,+0.60]$	–

Appendix

Table 10: Language and plan steering under a shared executor. All arms drive the same executor and are scored identically. Intervals are

95\%

bootstraps clustered over the eight held-out layouts; grasp rate is averaged over seeds.

Figure 9: Candidate coverage and reranking cost reduction. Left: Coverage vs. sample budget

K

. Solid curves denote any committed draw; dashed curves isolate successful grasps. Right: Cost reduction at

K=6

for both the PATH (efficiency) and FIELD (expensive zone) objectives. Setup: 5 seeds

\times

8 held-out scenes. Critic capacity (

4{,}481

parameters), updates (

700

), samples per update (

64

), and noise seeds are matched across arms. Overall grasp success is

0.295

(plan) vs.

0.133

(action).

Arm	Outcome data retained	Update rule	Training cost
gpl + critic	All observed $(z,r)$ pairs	Fit a $\sim 17$ k-parameter critic on accumulated outcomes; reweight sampled plans with the critic score using Equation 8 .	Critic training only; no generator updates.
gpl + reward table	Per-mode successes and trials	Estimate each mode’s empirical success probability; use this score in the same Boltzmann reweighting rule ( Equation 9 ).	Zero training FLOPs.
Continued BC	Successful samples only ( $r=1$ ), accumulated across rounds	Fine-tune the flow on the accumulated successful-plan pool using the original flow-matching objective.	1500 Adam steps/round, $2\times 10^{-3}$ learning rate, batch size 128.
AWR	All samples, successes and failures, accumulated across rounds	Compute $A=r-\bar{r}$ , resample examples with probability $\propto\exp(\tau_{\mathrm{AWR}}A)$ , $\tau_{\mathrm{AWR}}=4$ , and fine-tune with the original flow-matching objective.	1500 Adam steps/round, $2\times 10^{-3}$ learning rate, batch size 128.

Appendix

Table 11: Compute-frontier arms. Each deployment round uses

K=16

sampled plans per training scene. Training costs reflect the computational budget per round.

demonstrations	follow		steer range, grasped	steer range, per episode	grasp cost
	cmd 1	cmd 2
habit ( $\widetilde{R}=0.47$ )	0.897	0.294	+0.190 [+0.146,+0.235]	+0.108 [+0.083,+0.136]	0.250
counterfactual ( $\widetilde{R}=0$ )	0.974	0.672	+0.646 [+0.584,+0.708]	+0.425 [+0.374,+0.478]	0.274

Appendix

Table 12: Lower redundancy increases plan use under a matched intervention. Follow is the fraction of grasped episodes reaching the commanded bowl. Steer-range intervals are scene-clustered

95\%

bootstraps. All six seeds are matched across demonstration sets and favor the counterfactual intervention on each steering endpoint.

$q$	E1	E2	E3	steer range, grasped	steer range, per episode
0.00	0.061	0.234	0.139	+0.236 [+0.155,+0.326]	+0.181 [+0.119,+0.250]
0.25	0.051	0.110	0.024	+0.444 [+0.335,+0.569]	+0.272 [+0.206,+0.347]
0.50	0.019	0.052	0.017	+0.556 [+0.428,+0.695]	+0.336 [+0.255,+0.426]
0.75	0.005	0.005	0.000	+0.596 [+0.486,+0.710]	+0.408 [+0.325,+0.495]
1.00	0.000	0.000	0.000	+0.640 [+0.523,+0.760]	+0.378 [+0.300,+0.462]

Appendix

Table 13: Redundancy dose response. All five mixtures use the same six-task protocol and

360

episodes per arm. E1 is the preregistered fixed-regularization probe (used for the main-text endpoint), E2 is tuned and probability-calibrated (used for this sweep’s finer resolution), and E3 is the grouped

k

-NN conditional-entropy estimate ( Section G.2 ). Steer-range intervals are scene-clustered

95\%

bootstraps.

Figure 10: One shared 54-D contact sketch, one flow reasoner, four SimplerEnv families, three capabilities. Left: coverage of every valid mode by

K

plan draws. Middle: steering effect, measured as commanded-mode rate minus the off-diagonal rate. Both panels are scored in plan space, by assigning each drawn sketch to its nearest mode centroid; no executor is run. Right: outcome adaptation under the family-specific physical deployment shifts detailed below. Each reward is the success of a scripted controller executing that mode in the shifted scene. Bars show the frozen plan prior, the same prior reranked from measured rollout outcomes, and the deterministic twin; each family also includes an unshifted no-signal control.

	pick-and-place	grasp-yaw	can pick-or-push	sweep
Training scenes	320	320	104	78
Held-out (test) scenes	80	80	32	24
Unique scene configurations	40	200	68	51
Scenes with a measured rollout outcome	32	160	52	39

Appendix

Table 14: Contact-sketch battery: per-family scene and rollout counts.

Setting	Coverage	Steering	Outcome adaptation
Samples per scene / mode	$K=48$	$K=24$	$K=32$
Flow integration	4 Euler steps	4 Euler steps	4 Euler steps
Selection / guidance	Unguided sampling	Classifier guidance, $\lambda=6.0$	Boltzmann reranking, $\beta=5.0$
Outcome model	–	–	BCE-trained critic
Physical feedback	–	–	One scripted-controller rollout per scene and mode, shared across its $K$ candidates
Reported quantity	Mode coverage	Commanded-mode probability	Post-minus-prior SR gain

Appendix

Table 15: Contact-sketch evaluation protocol. Sampling and uncertainty estimation are fixed across all four task families.

Figure 11: The three capabilities replicate in a second simulator and embodiment. SimplerEnv is closed-loop: coverage and adaptation use a two-spoon scene, steering a three-cube scene, scored by the object the gripper approaches. RoboCasa is evaluated offline on commanded relative pose, with fixed plan anchors in place of a trained reasoner. Terminal task completion additionally requires mobile-base navigation, which lies outside the adapted executor.

Reasoning for VLA Models

Reasoning for vision-language-action (VLA) models connects visual and language understanding to grounded planning and action.

See all 110 papers

PAPO-VLA: Planning-Aware Policy Optimization for Vision-Language-Action Models
May 19, 2026Peizheng Guo, Jingyao Wang, Changwen Zheng +1Robot Policy Learning Credit Assignment in RL
Institute of Software Chinese Academy of Sciences · University of Chinese Academy of Sciences
PearlVLA: Progressive Embodied Action-Plan Refinement in Latent Space
Jun 16, 2026Bochen Yang, Lianlei ShanEfficient VLA Models Latent Visual Reasoning
Imperial College London · Tsinghua University
ThinkingVLA: Interleaved Vision and Language Reasoning for Robotic Manipulation
Jun 16, 2026Tianyi Lu, Hui Zhang, Zijie Diao +8Long-Horizon Robotic Manipulation Language-Conditioned Robot Manipulation
Fudan University · Shanghai Innovation Institue · China Unicom +1

May 19, 2026J/K move · Enter open · S save

PAPO-VLA: Planning-Aware Policy Optimization for Vision-Language-Action Models

Peizheng Guo, Jingyao Wang, Changwen Zheng, Wenwen Qiang

Institute of Software Chinese Academy of Sciences · University of Chinese Academy of Sciences

Robot Policy Learning Credit Assignment in RL Vision-Language-Action Models RL for VLMs Closed-Loop Control of VLA Models

Open paper AskPDF arXiv

Vision-Language-Action (VLA) models show promising ability in language-guided robotic tasks. However, making VLA policies reliable remains challenging, because a manipulation task is completed through closed-loop interaction, where each action affects subsequent execution. To analyze this problem, we revisit VLA policy during execution and argue that a VLA policy acts both as a planner, which makes task-oriented decisions that change the direction of execution, and as an executor, which realizes these decisions through dense continuous actions. This view suggests that improving VLA reliability requires particular attention to planning actions. Existing optimization methods can imitate actions or improve complete trajectories, but they usually do not explicitly identify planning actions or measure their importance for task success. To address this issue, we propose Planning-Aware Policy Optimization for VLA models (PAPO-VLA). PAPO-VLA first identifies planning actions by jointly considering action variation and trajectory outcome, then estimates their importance through causal sufficiency and causal necessity, and finally incorporates this importance into GRPO advantage estimation. In this way, more important planning actions receive stronger optimization emphasis, while the whole trajectory is still optimized by trajectory-level feedback. Experiments on multiple benchmarks demonstrate the effectiveness of PAPO-VLA.

Abstract

Reasoning for VLA Models

Explore similar work

PAPO-VLA: Planning-Aware Policy Optimization for Vision-Language-Action Models

PearlVLA: Progressive Embodied Action-Plan Refinement in Latent Space

ThinkingVLA: Interleaved Vision and Language Reasoning for Robotic Manipulation