Test-time scaling improves vision-language-action (VLA) policies by drawing several action candidates and executing the best one. We find that its returns diminish quickly when the candidates are resampled from one decoder: they differ by noise rather than by plan, and for each of three resampling sources, going from four to eight candidates adds only 0.06-0.07 to the average number of completed tasks on CALVIN. We introduce PlanDiv, an action head that makes the candidates differ by plan: a lightweight planner proposes K latent plan hypotheses, a shared masked-diffusion executor decodes one action chunk per plan from a single backbone pass, and the policy executes the chunk that its own likelihood ranks highest. The plans are learned from demonstrations alone, without plan labels or rewards. With the backbone, data and training recipe held fixed, eight plan-conditioned candidates raise five-task chain success on CALVIN ABC→D from 56.9% to 69.9%, while eight candidates from autoregressive, plan-free or same-plan resampling reach at most 64.9%. The same-plan baseline is PlanDiv's own checkpoint with the plan held fixed, so this gap comes from the plans, not from drawing more samples. On LIBERO-Long, four plans remove 40% of single-candidate failures, against at most 21% for matched resampling. On a real Franka arm the gain appears where a task admits more than one solution: PlanDiv is on par with a single-output OpenVLA-OFT+ baseline on a one-solution task and succeeds in 51 of 54 trials against 40 of 54 on two tasks with two solutions.
Figures & tables
Figure 1: Eight plan-conditioned candidates outperform eight resampled ones, and four plans complete more tasks than eight samples from any resampling source. Top (illustration; circle: object, box: obstacle, ring: goal): resampling one decoder yields candidates that differ by noise, so they share one approach and are blocked together, whereas PlanDiv decodes one candidate per learned plan, so the candidates take different routes. Bottom: Avg. len (average number of tasks completed in a five-task chain) on CALVIN ABC → D, all heads trained under one recipe; same plan, plan-free executor and autoregressive are same-z, executor-only and OpenVLA-AR in Table 1 .
Figure 2: Overview of PlanDiv. (a) Inference. One backbone pass gives the hidden states at the action positions, Hact , and a pooled context c . The latent planner turns Kmax learned seeds ξk into plan hypotheses zk with a prior αk ; only the top- K by prior are decoded, so one checkpoint serves any budget K≤Kmax . One shared masked-diffusion executor decodes a candidate chunk A^k under each plan in S=4 unmasking steps, so the candidates come from K different plans instead of K samples under one condition. The policy executes the candidate with the highest score, by default the mean log-probability of its tokens. (b) Training. The masked reconstruction loss of a demonstration chunk under each plan gives soft responsibilities wk that weight the execution loss; no plan labels or rewards are used. The shaded region is the head we add; the backbone is inherited from OpenVLA. The photographs are frames of our robot setup and are illustrative; token strips and bars are schematic.
CALVIN ABC → D
LIBERO
Tasks completed in a row (%)
Candidate source
K
1
2
3
4
5
Avg. len
Long (%)
OpenVLA-AR
1
92.8
82.9
72.2
62.3
54.8
3.65
84.8
4
94.3
85.8
76.6
67.4
61.9
3.86
87.6
8
94.9
87.2
77.8
69.2
63.9
3.93
–
Executor-only
1
93.4
83.8
73.6
63.1
56.1
3.70
85.8
Table 1: Eight plan-conditioned candidates outperform every resampling source at the same budget, and four score higher than eight resampled ones in every CALVIN column. Candidate source × candidate count under the likelihood selector, all heads trained under one recipe. CALVIN ABC → D: tasks completed in a row (%) and Avg. len on 1000 chains. LIBERO-Long: success on 500 episodes; all sources are evaluated at K∈{1,4} , the range of the Kmax=4 PlanDiv checkpoint, and – marks settings not evaluated. Same-z equals different-z at K=1 (shared first candidate) and is listed once.
Method
Extra model
Base policy
Before → after
Removed (%)
CALVIN ABC → D, five-task chains (%)
RoVer [ 11 ]
reward model
GR-1 [ 56 ]
43.4 → 48.7
9
RoVer [ 11 ]
reward model
Dita [ 15 ]
50.0 → 59.2
18
RoVer [ 11 ]
reward model
MoDE [ 42 ]
63.5 → 66.6
8
PlanDiv ( K=8 )
none
PlanDiv ( K=1 )
56.9 → 69.9
30
LIBERO-Long (%)
Table 2: PlanDiv removes a larger share of the base policy’s failures than RoVer, RoboMonkey and MG-Select report on the same benchmarks and, like MG-Select, needs no extra model. “Removed” is the share of the base policy’s failures that test-time scaling eliminates. Base policies differ across methods, so the table compares the gain from scaling, not absolute performance.
(a) Candidate pools
same-z
different-z
Token Hamming distance
0.182
0.304
End-effector spread
0.064
0.107
Branch success, likelihood (%)
77.8
83.2
Cover@1 / @2 (%)
76.2 / 80.2
76.2 / 83.4
Cover@4 / @8 (%)
83.2 / 86.1
88.8 / 92.1
First-candidate failures recovered (%)
41.6
66.8
Table 3: Candidates decoded under different plans are farther apart and cover more states, and changing only the plan changes outcomes more than twice as often as changing only the noise. (a) Same-z versus different-z candidates at K=4 (Cover@ K up to K=8 ). Cover@ K is the share of states where at least one of the K candidates leads to sub-task success; the last row is (Cover@8−Cover@1)/(1−Cover@1) . (b) Fixed-noise intervention: RMSE on normalized actions; “changed” means at least one token differs; “outcome” is the share of states whose sub-task outcome differs.
Selector ( K=4 )
Avg. len
Branch success (%)
first (top prior; equals K=1 )
3.74
76.2
random
3.71
74.8
confidence γˉk
3.96
80.8
likelihood (default)
4.02
83.2
value Vψ
3.99
82.2
full score, Eq. ( 8 )
4.08
84.8
Table 4: The likelihood score, which needs no extra model, captures most of the gain of selection, and the full score adds 0.06. Selectors for PlanDiv at K=4 : closed-loop Avg. len on 1000 CALVIN chains and branch success on the state bank of Sec. 5.3 , where the oracle equals Cover@4.
Source
K
p50 (ms)
p95 (ms)
Mem. (GB)
Avg. len
PlanDiv
1
118
157
18.8
3.74
4
188
248
19.3
4.02
8
288
374
19.9
4.13
Executor-only
1
109
146
18.4
3.70
4
169
223
18.8
3.90
8
257
336
19.4
3.97
Table 5: Eight plan-conditioned candidates take 288 ms, about a third of the time OpenVLA-AR needs for one. End-to-end latency per policy call in simulation (CALVIN; p50: median; p95: 95th percentile) and peak memory on one 80 GB GPU at batch size 1, with the Avg. len of Table 1 .
Training objective
K=1
K=4
Gain
Eff. plans
PlanDiv
3.74
4.02
+0.28
3.04
No repulsion ( λdiv=0 )
3.73
3.93
+0.20
1.98
Uniform responsibility ( wk=1/K )
3.70
3.88
+0.18
3.42
Sharp responsibility ( τr=0.05 )
3.72
3.90
+0.18
1.58
Table 6: Removing the repulsion term or the soft responsibilities reduces the gain from four candidates to the level of plain resampling. Training-objective ablations on CALVIN ABC → D (Avg. len, likelihood selector). “Gain” is K=4 minus K=1 ; “Eff. plans” is the exponentiated entropy of the selection frequency over the four slots (maximum 4).
OFT+
PlanDiv
Push a bowl (one solution)
24/25
25/25
Two blocks into a tray
17/21
21/21
Can around an obstacle onto a plate
23/33
30/33
Two-solution tasks (%)
74.1
94.4
Time per query (ms)
145
113
Table 7: On a real Franka arm PlanDiv removes 11 of the baseline’s 14 failures on the two-solution tasks. Successes over trials with objects placed by hand. Time per query is measured during these trials on the robot’s workstation (one RTX 4090).
Vision-Language-Action (VLA) policies increasingly reason before acting, but existing reasoning models compress this intermediate decision into a single representation. When several plans are valid, this commitment hides alternatives that could otherwise be selected or revised. We introduce the Generative Plan Latent (GPL), which represents the reasoning state as a low-dimensional conditional distribution over plans, such as which object, grasp, or route to use. This makes the reasoning state addressable: without retraining the VLA, plans can be sampled (coverage), selected on request (steering), and reweighted by deployment outcomes (outcome adaptation). We test all three on SimplerEnv manipulation with π0, and extend the analysis to RoboCasa, LIBERO, and a maze domain. Steering the plan raises the fraction of executions that follow the request from 0.48 to 0.94, while the same guidance on the native action distribution shows no reliable gain. Finally, we show that an explicit plan is most useful when it carries decision information not already determined by the observation.
Matteo Cercola, Katie Z Luo, Milan Ganai +3
Politecnico di Milano · Stanford University · Air Force Research Laboratory +1
Most vision-language-action (VLA) models map observations directly to actions without explicit intermediate planning, which limits performance on long-horizon tasks where early mistakes compound. We propose Coarse-to-Control, a plan-execute VLA that introduces planning natively in the action-token space. The key idea is to let the policy first predict a compact sequence of coarse action tokens that summarize the intended future trajectory, and then generate executable action tokens conditioned on this plan. Because both planning and execution share a unified discrete action vocabulary, the plan stays close to the control manifold and provides directly actionable guidance rather than an abstract hint that must be translated back to motor commands. Experiments on LIBERO, SimplerEnv-WidowX, and real-world manipulation tasks show that action-token planning consistently improves over direct action generation, with the largest gains on long-horizon multi-stage tasks.
Jinhao Wu, Shiduo Zhang, Yicheng Liu +9
1Nanjing University · 2Shanghai Innovation Institute · 3Fudan University +1
Test-time sampling can improve Vision-Language-Action (VLA) policies, but only if the model can efficiently select a good action from multiple candidates. Prior work often relies on separately trained external verifiers, adding extra models and inference overhead. We propose Token Probability Bucketing (TokenPB), a self-verification approach that discretizes continuous action chunks into bucket tokens and co-trains a single VLA checkpoint with a joint objective: a flow-matching loss for action generation and an auxiliary bucket loss Lbucket that teaches the VLM backbone to model observation-conditioned distributions over bucket-token sequences. At inference, TokenPB samples M candidates, scores each via teacher-forced evaluation under the learned distribution, and selects the best---enabling efficient batched scoring with observation-prefix KV-cache reuse. We provide theory connecting the training objective to a monotonic ranking property and bounds on expected selection regret that separate candidate-set effects from calibration. Across multiple backbones and benchmarks, TokenPB yields consistent success-rate improvements over single-sample inference from the same checkpoint (+2.7 in simulation, +17.1 in real-world, +16.4 under out-of-distribution shifts), and further improves over external-verifier baselines under the same sampling budget. The code is publicly available at https://github.com/RUCKBReasoning/TokenPB.
Chen Zhao, Zhuoran Wang, Haoyang Li +6
School of Information, Renmin University of China · Key Laboratory of Data Engineering and Knowledge Engineering, Beijing, China · Beijing University of Posts and Telecommunications +1