Test-time compute (TTC), which scores multiple candidate actions at inference time, is a powerful way to improve the execution performance of vision-language-action (VLA) models, but existing methods keep the policy frozen, so the gain must be recomputed at every execution. We propose Verify, Act, Distill, a framework that consolidates the gains of TTC into the VLA itself. A fixed verifier scores candidates generated by a flow-matching VLA, the weighted aggregate is executed, and the observations and aggregated chunks from successful adaptation episodes are distilled directly into LoRA parameters with the standard flow-matching loss. This reuses supervision that TTC has already produced during execution, without additional reinforcement learning or rollouts dedicated to teacher generation. On the 40 LIBERO tasks with a SmolVLA 5k checkpoint, distilling from only 10 episodes per task raises the mean success rate of verifier-free single-candidate generation from 54.1% to 70.6%, retaining about 74% of the improvement obtained by aggregating eight candidates. Applying the same aggregation to the distilled policy further raises the success rate from 76.4% to 82.3%, with a particularly large gain on LIBERO-Long (44.0% to 57.8%), and the single-candidate policy also improves on LIBERO-plus, which contains distribution shifts. These results show that using verification as a learning signal, rather than only as an execution-time selector, both retains the improvements of TTC and re-amplifies the effect of subsequent TTC.
Figures & tables
Figure 1: Overview of Verify, Act, Distill. Candidate chunks are scored and aggregated by the verifier and the leading H2=10 steps are executed. Observations and the aggregated chunks of length H1=50 from successful episodes are stored in a buffer and used to update the LoRA parameters. The verifier evaluates only the first H2 steps of each candidate, whereas the teacher data include the entire planned chunk.
Figure 2
Method (10k, LIBERO-plus)
Spatial
Object
Goal
Long
Avg.
Base
35.5
18.8
13.0
16.2
20.9
Agg-8 ( Vact )
41.5
24.9
14.3
21.7
25.6
First 35 tasks per suite (common subset)
Base
36.6
15.1
14.9
16.0
20.7
Agg-8 ( Vact )
40.6
17.7
17.1
23.4
24.7
Distill
41.1
17.7
16.6
22.6
24.5
Table 2: Success rates (%) on LIBERO-plus, SmolVLA 10k, 105 tasks per suite, seeds 10–19 ( n=1050 per suite). The lower block uses the first 35 tasks per suite (140 tasks, n=350 ), on which Base and Agg-8 are re-aggregated for comparison.
Figure 3: Dependence on the number of candidates and the adaptation budget. Both use the variant without the success filter; numbers are in Tables 6 and 7 .
Figure 4: Rollouts on LIBERO-Long task 9 (SmolVLA 5k) from the same initial state (seed 12) for Base ( N=1 ), Agg-8, and distilled policy + Agg-8; thin lines show each candidate’s predicted trajectory at each chunk boundary and the thick line the executed weighted average, and rows show the progression of time. The seed is the first one, searching upward from seed 10, in which Base fails and the distilled policy + Agg-8 succeeds (Agg-8 alone also fails here).
Appendix figures & tables16 assets
Supplementary material from the paper’s appendix.
Appendix
Trained component
Data and supervision
SmolVLA (BC, 20k steps)
Only the 5,225 successful episodes among the newly collected rollouts. Official LIBERO demonstrations not used.
Verifier
Eight candidates generated at states from both the official LIBERO demonstrations (all four suites) and the newly collected successful rollouts. Supervision is the RMSE to the H2=10 recorded action steps at each state. The 10k candidate dataset comprises 104,294 states.
World model (appendix)
Official LIBERO demonstrations and newly collected rollouts. For rollouts, both successful and failed data are used.
Appendix
Table 3: Breakdown of offline training data by use. This is distinct from the success-filtered distillation buffer used in the adaptation phase.
Teacher
Filter
Verifier
Spatial
Object
Goal
Long
Avg.
Mean
On
Vact
81.8
88.0
79.4
33.2
70.6
Max
On
Vact
78.4
86.8
78.2
27.8
67.8
Mean
Off
Vact
73.8
85.2
71.4
27.0
64.4
Max
Off
Vact
—
—
—
32.6
—
Mean
Off
Vlatent
62.2
84.4
67.2
25.8
59.9
Appendix
Table 4: Comparison of distillation teacher, success filter, and verifier (SmolVLA 5k, 10 adaptation episodes, evaluation seeds 10–59, 500 episodes per suite). Success rates (%) with (a) single-candidate generation and (b) weighted aggregation of eight candidates. “Mean” uses the weighted-average chunk and “Max” the highest-scoring candidate as the teacher. The main-method setting is “Mean teacher, filter on, Vact .” The Vlatent rows also use Vlatent for candidate evaluation.
Evaluation
Teacher
Spatial
Object
Goal
Long
Avg.
Single candidate
Full chunk ( H1 )
81.8
88.0
79.4
33.2
70.6
Executed steps ( H2 )
79.0
92.0
84.2
41.0
74.1
+ Agg-8
Full chunk ( H1 )
88.2
95.4
87.8
57.8
82.3
Executed steps ( H2 )
90.4
99.0
92.6
64.2
86.6
Appendix
Table 5: Full-chunk teacher ( H1=50 , main recipe) vs. H2 -only teacher (loss masked to the executed 10 steps). Success rates (%) on LIBERO (clean), SmolVLA 5k, seeds 10–59 ( n=500 per suite). The full-chunk rows are those of Table 1 .
N
Spatial
Object
Goal
Long
Avg.
1
57.6 / 73.8
80.4 / 85.2
61.0 / 71.4
17.4 / 27.0
54.1 / 64.4
2
68.0 / 76.0
82.6 / 89.0
70.6 / 76.4
26.0 / 37.6
61.8 / 69.8
4
74.6 / 79.0
89.8 / 93.4
81.8 / 81.2
36.6 / 43.6
70.7 / 74.3
8
80.6 / 83.0
94.4 / 94.6
86.8 / 82.4
44.0 / 55.0
76.4 / 78.8
16
84.0 / 86.0
92.8 / 96.4
90.6 / 86.2
56.2 / 62.0
80.9 / 82.7
Appendix
Table 6: Numbers underlying Figure 3(a) : success rates (%) of Agg- N / distilled policy + Agg- N , SmolVLA 5k, n=500 per cell.
Policy (Base)
Method
K=2
K=5
K=10
K=20
K=50
5k (17.4)
Distill
29.6
29.2
27.0
29.2
34.6
Distill + Agg-8
52.6
52.8
55.0
53.6
54.6
N-GRPO
—
—
25.6
—
29.6
DSRL
—
—
19.4
—
18.8
10k (68.6)
Distill
75.0
70.4
73.8
75.0
75.4
Distill + Agg-8
75.2
76.8
79.0
80.4
85.4
Appendix
Table 7: Success rate (%) on LIBERO-Long as a function of the adaptation-episode budget K ( n=500 ). Parameters are frozen after K adaptation episodes and then evaluated. DSRL-style is our port of the official implementation (dsrl_pi0), with a −1/0 reward (time penalty), a TanhNormal distribution restricted to [−1,1] , an ensemble of 10 critics, and prior-replacement composition. The GRPO rows are the comparison method (verifier-ranking reward, top1 advantage, success filter, argmax adaptation). The distilled policy is the variant without the success filter (Appendix B ), with adaptation seeds 0,…,K−1 and the 50 seeds immediately following as evaluation seeds. Base performance is 17.4 at 5k and 68.6 at 10k (seeds 10–59); Agg-8is 44.0 at 5k and 80.0 at 10k.
Verifier
Method
Spatial
Object
Goal
Long
Avg.
—
Base
85.0
95.0
84.0
68.6
83.2
Vact
Agg-8
90.2
99.6
91.8
80.0
90.4
Distill
89.4
96.0
85.4
73.8
86.2
Distill + Agg-8
91.8
98.4
89.8
79.0
89.8
Vlatent
Agg-8
92.4
99.0
91.4
82.6
91.4
Distill
91.0
97.0
80.8
75.6
86.1
Appendix
Table 8: Success rates (%) on LIBERO (clean) with SmolVLA 10k. Measured under the same protocol as Table 1 (evaluation seeds 10–59, n=500 per suite).
Method
Spatial
Object
Goal
Long
Avg.
clean 5k
Base
57.6
80.4
61.0
17.4
54.1
Agg-8 Vact
80.6
94.4
86.8
44.0
76.4
Agg-8 Vlatent
74.0
89.8
81.2
41.6
71.7
Δ
−6.6±3.7
−4.6±1.7
−5.6±2.5
−2.4±2.2
−4.8±1.3
clean 5k, as distillation teacher
Appendix
Table 9: Comparison of verifier inputs (success rate %; Agg-8is softmax aggregation with λ=0.3 ; each verifier trained for 60k steps). Clean uses evaluation seeds 10–59 ( n=500 per suite); plus uses 105 tasks × 10 episodes (seeds 10–19). Δ is the per-task paired difference Vlatent−Vact (mean ± SEM). The comparison as a distillation teacher is based on the variant without the success filter ( K=10 , single-candidate generation).
Figure 5: Scatter plot comparing Agg-8success rates on all 420 LIBERO-plus tasks (SmolVLA 10k): Vact (horizontal) vs. Vlatent (vertical), 10 episodes per point (jitter of ±1.5 points added to reduce overlap). Points above the diagonal favor Vlatent . The upward shift is especially pronounced on the Object suite.
Item
Feature-space comparison (Figure 6 )
Main experiments
Policy
BC-trained on the two suites Object and Long. 4k checkpoint ( run_bc_success/checkpoints/004000 )
BC-trained on successful rollouts from all four suites. 5k / 10k / 20k checkpoints of run_bc_4suite
Candidate data
cand_4k_a1.2 . Object and Long candidates generated from the 4k policy
cand_4suite_<ckpt>_a1.2 . Four suites, generated per policy checkpoint
Verifier training
20k steps
60k steps
World model
run1 (SmolVLA features) and run_dinov2 (DINOv2 features)
run_4suite (SmolVLA features, all four suites)
Candidate noise at evaluation
α=2.0
α=1.2
Execution rule
Select the highest-scoring of 8 candidates (argmax)
Softmax-weighted aggregation of 8 candidates ( λ=0.3 )
Appendix
Table 10: Differences in setup between the feature-space comparison (Figure 6 ) and the main experiments (clean LIBERO). “a1.2” in the candidate-dataset names denotes the noise multiplier.
Figure 6: Comparison of verifier input representations with a SmolVLA 4k checkpoint trained on the Object and Long suites. Eight candidates, noise scale α=2.0 , argmax selection, 25 evaluation episodes per task (250 per suite). Verifiers are trained for 20k steps. The base policy, Vact with direct action input, and Vlatent predicting DINOv2 features or SmolVLA features through the world model are compared. See Table 10 for details of the setup.
Checkpoint
Suite
Base
α=1.2
1.5
2.0
3.0
5k
Object
82
82
59
39
44
5k
Long
17
42
21
7
6
10k
Object
96
98
93
64
71
10k
Long
67
74
60
27
21
20k
Object
99
98
91
84
75
20k
Long
90
79
68
33
30
Appendix
Table 11: Success rate as a function of the candidate noise scale α ( N=8 , Vlatent , n=100 per point, Long / Object suites). Bold marks the maximum in each row including the base policy (ties included). The explored range is {1.2,1.5,2.0,3.0} .
Checkpoint
R
Split-cos
SDR
5k
0.229±0.007
0.219±0.025
0.053±0.004
10k
0.228±0.007
0.216±0.027
0.053±0.004
20k
0.209±0.005
0.129±0.023
0.043±0.002
Appendix
Table 12: Within-state reproducibility of the TTC correction vector Δm (32 states, 32 candidate-set draws per state). (a) Directional reproducibility metrics; (b) correction norm, effective number of candidates, and score standard deviation. R is the resultant length, Split-cos the split-half cosine similarity, and SDR the directional stability ratio.
Method
Success
Δ Base
Δ Agg-8
Base
37.3±1.1
—
−6.6±0.7
Agg-8
43.9±1.2
+6.6±0.7
—
N-GRPO + Agg-8
43.8±1.2
+6.5±0.7
−0.1±0.6
Appendix
Table 13: Success rates (%) on LIBERO-plus with SmolVLA 20k and per-task paired differences from Base (points, mean ± SEM). Evaluation seeds 10–19. (a) Aggregated over the 837 Level-5 tasks on which Base has a positive success rate; (b) aggregated over a fixed set of 80 tasks (32 tasks for PRO only). VWFM is advantage-weighted flow matching over all candidates, Online is online LoRA updating, and PRO is preference optimization (Appendix M ).
Checkpoint
Method
Spatial
Object
Goal
Long
SmolVLA 5k
Base
62.6
80.0
59.6
17.4
Sel-8
73.6
88.2
81.8
34.4
N-GRPO
59.8
83.8
68.2
23.2
N-GRPO+Sel-8
80.2
88.2
78.8
53.4
SmolVLA 10k
Base
85.0
96.4
84.0
65.6
Sel-8
89.6
97.8
86.8
77.0
Appendix
Table 14: GRPO success rates (%) on clean LIBERO, 50 episodes per task. Base and Sel-8use evaluation seeds 0–49 (argmax selection); the GRPO rows use adaptation seeds 0–9 and evaluation seeds 10–59.
Method
Spatial
Object
Goal
Long
Avg.
N-GRPO ( Vact )
35.9
21.6
13.8
18.0
22.3
N-GRPO+Sel-8 ( Vact )
42.1
29.6
15.3
21.3
27.1
N-GRPO+Sel-8 ( Vlatent )
42.4
32.5
13.4
16.7
26.3
Appendix
Table 15: GRPO success rates (%) on LIBERO-plus 10k, 105 tasks × 10 episodes, argmax selection (Sel-8). GRPO applies the success filter and the μ trust region. Parentheses indicate the verifier used for the reward and for candidate selection.
Advantage scheme
Clean 5k Long
Plus Object Vact
Plus Object Vlatent
Plus Long Vlatent
top1
53.4
29.6
32.5
16.7
zscore
52.6
30.0
30.8
16.4
rank
48.6
28.0
31.7
15.4
tophalf
48.4
27.4
33.0
14.7
Appendix
Table 16: Comparison of advantage computation schemes ( n=500 / n=1050 ). Assignment schemes that dilute the weight of the top candidates tend to weaken the correct update direction. Settings with Vlatent reward and the μ trust region were also tested.
Vision-Language-Action (VLA) models have demonstrated remarkable capabilities and generalization in embodied manipulation. However, their decision-making relies on a fast, instinctive process that lacks deliberation. This strategy often leads to suboptimal or catastrophic actions when facing complex or ambiguous scenarios that require greater consideration. In this paper, we introduce VLA-ATTC, a framework that endows VLA models with adaptive test-time compute (TTC). VLA-ATTC employs an uncertainty-based ``cognitive clutch'' to dynamically transition from reflexive execution to a TTC deliberation phase when necessary. During TTC phase, a novel Relative Action Critic (RAC) model identifies the optimal action from generated candidates via pairwise comparisons. This relative mechanism replaces unstable absolute value estimation, significantly simplifying the learning objective. Furthermore, we introduce an efficient sampling strategy to amortize computational costs and an automated data pipeline that curates preference pairs without manual annotation. On the LIBERO-LONG benchmark, VLA-ATTC reduces the failure rate of the SOTA model PI0.5 by over 50%. We will open-source all the code and weights.
Wenhao Li, Xiu Su, Yichao Cao +5
University of Sydney · Central South University · University of Science and Technology of China +2
Vision-Language-Action (VLA) models trained on large-scale data have made remarkable progress, but they remain vulnerable to distribution shifts at deployment time. Recent VLA models suggest that prompts can serve as an efficient interface for steering policy behavior, but existing prompt-based steering typically relies on external guidance. This raises a natural question: can test-time training (TTT) for VLA be achieved by optimizing a prompt, so that the steering interface itself can be learned and adapted from interaction? We address this question with TTT-VLA, a test-time training framework based on Latent Prompt Optimization (LPO). During training, the latent prompt is learned with an additional proxy task, providing an extra learned conditioning signal for policy learning. At test time, TTT is performed by collecting interaction data from the current environment and optimizing only the latent prompt on those data using the proxy task's self-supervised signal, without modifying the policy itself. Experiments on SimplerEnv demonstrate that the proposed method consistently improves task success rates in both single- and multi-embodiment settings. Further analysis shows that the gains arise primarily from correcting a small number of critical decisions rather than globally altering policy behavior. These results suggest that LPO provides an effective and practical pathway for deployment-time improvement of foundation manipulation policies.
Wenbo Zhang, Jianxiong Li, Shuai Yang +4
ByteDance Seed · The University of Adelaide · Tsinghua University +3
Test-time sampling can improve Vision-Language-Action (VLA) policies, but only if the model can efficiently select a good action from multiple candidates. Prior work often relies on separately trained external verifiers, adding extra models and inference overhead. We propose Token Probability Bucketing (TokenPB), a self-verification approach that discretizes continuous action chunks into bucket tokens and co-trains a single VLA checkpoint with a joint objective: a flow-matching loss for action generation and an auxiliary bucket loss Lbucket that teaches the VLM backbone to model observation-conditioned distributions over bucket-token sequences. At inference, TokenPB samples M candidates, scores each via teacher-forced evaluation under the learned distribution, and selects the best---enabling efficient batched scoring with observation-prefix KV-cache reuse. We provide theory connecting the training objective to a monotonic ranking property and bounds on expected selection regret that separate candidate-set effects from calibration. Across multiple backbones and benchmarks, TokenPB yields consistent success-rate improvements over single-sample inference from the same checkpoint (+2.7 in simulation, +17.1 in real-world, +16.4 under out-of-distribution shifts), and further improves over external-verifier baselines under the same sampling budget. The code is publicly available at https://github.com/RUCKBReasoning/TokenPB.
Chen Zhao, Zhuoran Wang, Haoyang Li +6
School of Information, Renmin University of China · Key Laboratory of Data Engineering and Knowledge Engineering, Beijing, China · Beijing University of Posts and Telecommunications +1