Organizations: Renmin University of China, Beijing 100872, China · Key Laboratory of Data Engineering and KnowledgeEngineering, MOE, China · Meituan, Beijing 100102, China
Test-time sampling promises to turn one action prediction into a set of distinct executable candidates that a verifier can choose among. Downstream fine-tuning works against that promise. A vision-language-action model trained to solve one task distribution concentrates its flow-matching decoder around the dominant demonstrated action, so repeated samples differ only marginally and the verifier receives many copies of one decision. We call this proposal contraction and identify it as a key constraint on test-time scaling: repeated samples can remain highly redundant as the candidate budget grows. Action Dynamic Sampling (ADS) alleviates the constraint on the proposal side, during training. At each training step, ADS integrates K additional trajectories from the current policy under fresh base noise, scores each by the flow support along its path with a Hutchinson trace estimator, and corrects the policy's local action distribution toward this weighted set through a detached finite-set KL term. Training cost rises; deployment cost is unchanged. Across simulation and four physical-robot tasks, ADS raises proposal spread without the success penalty that Gaussian perturbation incurs, lifts oracle best-of-8 success on RoboCasa from 57.6% to 63.0%, improves fixed verifiers by 2.4 to 4.9 points in simulation, and raises TokenPB by up to 13.0 points on real-world manipulation.
Figures & tables
Figure 1: ADS produces broader, more useful proposal sets. Standard flow-matching training maps different noise draws onto one dominant action mode, leaving the verifier with redundant candidates. ADS corrects the policy against a flow-support-weighted set of its own samples; the resulting candidates cover more valid modes and get better performance, with the verifier unchanged.
Figure 2: Overview of ADS training and test-time verification. During training, detached siblings drawn from the current policy form a finite reference set, a Hutchinson trace estimator weights that set by flow support, and a KL term corrects the policy’s local action distribution toward it. At test time ADS adds no module: it supplies a broader, policy-aligned candidate set to an unchanged verifier.
Method
RoboCasa
LIBERO-Plus
WidowX-Bridge
Google Robot
Proposal spread
Baseline
54.9
76.5
58.6
69.1
0.003
Gaussian Noise ( σ=5% )
53.3
75.2
58.4
63.2
0.011
Gaussian Noise ( σ=10% )
45.6
73.4
56.3
56.3
0.025
Gaussian Noise ( σ=20% )
39.8
65.9
55.1
50.7
0.044
ADS
56.4
79.4
62.6
70.1
0.021
Table 1: Gaussian action perturbation and ADS without test-time verification. Task performance is success rate (%). Proposal spread is the root mean squared pairwise distance among action chunks sampled for the same condition. Stronger Gaussian noise buys spread and pays for it in success; ADS gains both.
Figure 3: Left: Proposal quality and spread over training. ADS holds a broader candidate set at comparable task success, while Gaussian perturbation is spread out but less policy aligned. Right: ADS complements test-time verification. ADS improves both RoboMonkey and TokenPB without changing the verifier.
Metric
Robocasa
RoboTwin 2.0
LIBERO-Plus
Real-World
Flow-matching
ADS
Flow-matching
ADS
Flow-matching
ADS
Flow-matching
ADS
Mean distance dmean
0.091
0.102
0.042
0.075
0.086
0.082
0.057
0.068
Best distance dbest
0.085
0.032
0.036
0.024
0.065
0.027
0.046
0.013
Table 2: Training-set replay diagnostic on simulation and real-world tasks. For each training condition, we sample M=8 action chunks and measure the distance from each to the demonstrated action. dmean averages over all candidates, while dbest first selects the closest candidate for each condition.
Method
Background
Light
Language
Noise
Layout
Robot Init.
Camera
Overall
RoboMonkey
94.1
93.2
91.7
77.8
82.7
57.0
56.5
77.6
TokenPB
96.7
95.6
86.7
79.1
83.1
55.2
52.2
76.9
RoboMonkey + Gaussian Noise
96.9
94.1
91.2
80.8
85.5
56.9
58.6
79.4
RoboMonkey + ADS
97.9
96.7
93.9
81.6
88.1
58.5
62.5
81.3
TokenPB + ADS
99.0
97.6
90.6
84.7
90.2
57.8
61.4
81.6
Table 3: Proposal-side strategies on LIBERO-Plus. The first group contains verification methods with an unmodified proposal policy; the second group augments the verifier with Gaussian perturbation or ADS.
WidowX-Bridge (SR)
Google Robot (SR)
Method
Stack Cube
Carrot on Plate
Eggplant in Basket
Spoon on Towel
Avg.
Drawer
Move Near
Pick Coke
Put in Drawer
Avg.
RoboMonkey
38.2
64.7
86.7
72.9
65.6
–
–
–
–
–
TokenPB
46.9
70.8
100.0
85.4
75.8
57.1
83.7
86.8
50.3
76.4
Gaussian Noise + TokenPB
26.4
65.5
89.6
65.8
61.8
55.8
76.4
82.5
58.6
73.3
RoboMonkey + ADS
46.5
71.9
87.5
76.0
70.5
–
–
–
–
–
TokenPB + ADS
58.3
77.1
97.9
83.3
79.2
59.6
93.5
89.4
57.1
79.4
Table 4: Proposal-side strategies on SimplerEnv. The first group contains methods with an unmodified proposal policy; the second group introduces Gaussian perturbation or ADS.
Training objective
Success (%)
Proposal spread
LFM
54.9
0.003
+ uniform siblings ( qk=1/K )
48.6
0.029
+ base-density weighting ( ℓk=logpbase(z(k)) )
53.2
0.023
+ flow support (full ADS)
56.4
0.021
Table 5: Component ablation on RoboCasa without verifier. Each row adds one element of the ADS objective to the one above it, averaged over five seeds. Success is single-sample; proposal spread is measured among candidates drawn for the same condition.
Geometric redundancy
Valid modes
Oracle coverage (%)
Verifier success (%)
Candidates M
FM
ADS
FM
ADS
FM
ADS
FM
ADS
1
1.00
1.00
1.00
1.00
54.9
56.4
54.9
56.4
2
1.04
1.35
1.02
1.25
55.9
58.4
55.1
56.9
4
1.06
1.62
1.05
1.63
56.8
60.7
55.3
57.5
8
1.08
2.05
1.08
2.30
57.6
63.0
55.5
57.9
16
1.09
2.30
1.10
2.85
58.0
64.2
55.6
58.1
Table 6: Candidate scaling on RoboCasa with RoboMonkey. Each block reports one property of the same M candidates, using the metrics defined in the experimental setup. Oracle coverage and verifier success differ only in who selects the executed chunk. The ADS advantage widens with M before saturating.
Figure 4: Real-world four-object manipulation results. Left: representative execution sequences. Middle: normalized success scores for unverified, TokenPB-selected, and RoboMonkey-selected policies. Right: proposal diversity measured by pairwise action RMSE. ADS broadens the proposal set and improves both fixed verifiers, including on the unseen shelf task.
Appendix figures & tables18 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Real-world evaluation setup. The workspace consists of a storage area , a manipulation area , a first-person-view camera, a third-person-view camera, and a Dexmal robotic arm.
normalized joint/EEF action chunks with gripper state
Primary endpoint only
Condition interface
32 projected vision-language tokens
Detached
Numerical precision
bfloat16 backbone, float16 trace, float32 optimizer states
–
Appendix
Table 7: VLA architecture used in all controlled experiments. The baseline and ADS share every component; ADS changes only the training objective applied to the action expert.
Suite / embodiment
Tasks
Train demos/task
Train/val
Action dim. / horizon
Updates
Eval. episodes/task/seed
LIBERO-Plus
40
42
-
7 / 16
40k
50 per shift
SimplerEnv: WidowX-Bridge
4
—
99% / 1%
7 / 8
50k
task-dependent
SimplerEnv: Google Robot
4
—
99% / 1%
7 / 8
50k
task-dependent
RoboCasa
24
100
99% / 1%
7 / 16
60k
50
RoboTwin 2.0
50
550
95% / 5%
14 / 16
60k
50
Appendix
Table 8: Dataset splits and evaluation budgets. SimplerEnv is one suite but its WidowX-Bridge and Google Robot embodiments are kept as separate rows and are never averaged together. Its evaluation counts follow the official task-specific configurations.
Parameter
Final value
Values evaluated
Parameter
Final value
Values evaluated
Optimizer
AdamW
AdamW
Batch size
64
32, 64
Action-expert LR
1×10−4
5×10−5 , 10−4
Backbone LR
2×10−5
10−5 , 2×10−5
Weight decay
0.01
0.00, 0.01
Gradient clip
1.0
0.5, 1.0
Warmup updates
500
0, 500
LR schedule
cosine
constant, cosine
Sibling count K
8
2, 4, 8, 16
Trace probes R
8
1, 2, 4, 8
Temperature τ
1.0
0.25, 0.5, 1.0, 2.0
ADS weight λ
10−3
10−4 , 3×10−4 , 10−3 , 3×10−3
Appendix
Table 9: Optimization, sampling, and ADS hyperparameters. The final value is selected by validation single-sample success, with lower training FLOPs used to break differences below 0.2 points.
Evaluation
Suite / embodiment
FM
ADS
Δ 95% CI
p
Proposal only
RoboCasa
54.9±0.6
56.4±0.7
[0.5,2.5]
0.004
Proposal only
LIBERO-Plus
76.5±0.8
79.4±0.7
[1.8,4.1]
<0.001
Proposal only
WidowX-Bridge
58.6±1.1
62.6±1.0
[2.2,5.8]
<0.001
Proposal only
Google Robot
69.1±0.9
70.1±0.8
[0.1,1.9]
0.031
RoboMonkey@ 8
RoboCasa
55.5±0.7
57.9±0.6
[1.1,3.7]
<0.001
RoboMonkey@ 8
RoboTwin 2.0
89.8±0.5
93.5±0.4
[2.2,5.1]
<0.001
Appendix
Table 10: Seed variation and paired significance tests. Values are success rate in percent (mean ± standard deviation over five training seeds). Confidence intervals and two-sided p -values use the hierarchical paired bootstrap described in the text.
Diagnostic
Conditions
Samples/condition
Checkpoints
Aggregation
Training-set replay
2,000
8
final
dmean and dbest
Local Q−P rebalancing
512
8
p=0.6,0.7,0.8,0.9,1.0
condition mean, then seed mean
Trace–rollout validity
500
8
final
20,000 pooled candidates; seed bootstrap
Endpoint Jacobian
256
4 noise draws, 32 JVPs
final
singular-value statistics per condition
Action-mode occupancy
200
64
five ADS-phase points
centroid matching within condition
Candidate-budget scaling
500
M∈{1,2,4,8,16}
final
paired initial states
Appendix
Table 11: Sampling protocol for supplementary diagnostics. Counts are per training seed unless a total is explicitly stated.
Training method
Train FLOPs
Step time
Peak memory
Deployment FLOPs
Flow matching
1.00 ×
1.00 ×
1.00 ×
1.00 ×
ADS ( K=4,R=4 )
1.67 ×
1.09 ×
1.15 ×
1.00 ×
ADS ( K=8,R=4 )
2.34 ×
1.16 ×
1.25 ×
1.00 ×
ADS ( K=8,R=8 , default)
3.37 ×
1.28 ×
1.40 ×
1.00 ×
Appendix
Table 12: Training-compute accounting for ADS. Training FLOPs, step time, and peak memory are normalized by standard flow matching under the same hardware and batch size. Deployment FLOPs remain unchanged.
Figure 6: ADS training flow. Flow matching anchors task correctness. Once ADS is enabled, detached siblings supply a current-policy reference, and the Hutchinson trace reweights that reference by flow support before the KL update reaches the action expert.
Method
C/C0
u∗/N0
λ
Spread
S1
Oracle@8
RoboMonkey@8
TokenPB@8
FM (standard budget)
1.00
1.00
–
0.003
54.9
57.6
55.5
56.4
FM (equal FLOPs)
2.37
1.67
–
0.002
55.9
59.9
56.9
56.8
FM + H(p)
2.46
1.67
–
0.032
42.3
53.0
50.3
51.5
FM + covariance entropy
1.49
1.67
–
0.027
47.1
54.4
52.2
50.5
FM + KDE Rényi-2 entropy
2.11
1.25
10−4
0.012
45.6
58.4
52.0
52.6
FM + ADS
2.37
1.00
10−3
0.021
56.4
63.0
57.9
58.9
Appendix
Table 13: Training controls on RoboCasa. Training controls on RoboCasa with measured total training FLOPs. C/C0 is full training cost, u∗/N0 is the mean selected update across seeds, and λ is the selected auxiliary weight. Spread is pairwise action RMSE. Success and coverage are percentages averaged over five seeds.
u/N0
Spread
S1
Oracle@8
RoboMonkey@8
TokenPB@8
1/3
0.018
43.0
49.5
45.6
46.4
1/2
0.012
49.0
55.0
51.5
52.3
2/3
0.007
53.0
57.8
54.8
55.6
5/6
0.004
54.5
58.0
55.6
56.3
1
0.003
54.9
57.6
55.5
56.4
Appendix
Table 14: Fixed checkpoints along the standard FM trajectory on RoboCasa. All rows retain the original learning-rate schedule. Spread is measured at M=8 ; success and coverage are percentages. No row is selected using its test outcomes or diversity.
Sibling count K
Trace probes R
Temperature τ
K
SR
Spread
FLOPs
R
SR
Spread
FLOPs
τ
SR
Spread
Entropy
2
55.7
.011
1.59 ×
1
51.8
.028
1.57 ×
.25
55.8
.015
0.42
4
56.1
.017
2.19 ×
2
53.2
.024
1.83 ×
.50
56.2
.019
0.58
8
56.4
0.021
3.37 ×
4
56.3
.023
2.34 ×
1.00
56.4
0.021
0.71
16
56.2
.024
5.74 ×
8
56.4
0.021
3.37 ×
2.00
56.0
.024
0.84
ADS weight λ
Trace-start progress p
Appendix
Table 15: Sensitivity to the principal ADS hyperparameters on RoboCasa. Each block changes one parameter from the default K=8 , R=8 , τ=1 , λ=10−3 , and trace start p=0.6 . Results show saturation in R and a shallow optimum in the remaining parameters; spread is proposal spread, and entropy is the normalized target entropy H(q)/logK .
Proposal strategy
Trace–succ. ρ
Valid modes@ 8
Invalid rate ↓
Gaussian perturbation
0.06
1.31
11.8%
Uniform siblings
0.18
1.72
7.6%
Base density only
0.29
2.02
5.2%
Flow support (full ADS)
0.43
2.30
3.1%
Appendix
Table 16: Does trace weighting favor valid rather than arbitrary variation? Trace–success correlation is Spearman’s ρ between the flow-support score and candidate rollout success. The inter-mode invalid rate counts candidates that fall outside every valid action mode. Full ADS reaches the highest valid coverage and the lowest rate of unsupported between-mode candidates.
Training progress p
H(pθ,⋅∣0)/logK
H(q)/logK
E[ΔH]
Pr(ΔH>0)
Valid minor modes with Qb>Pb∣0
0.60
0.58
0.65
+0.07
61.8%
54.2%
0.70
0.53
0.68
+0.15
69.7%
58.6%
0.80
0.47
0.71
+0.24
76.8%
63.5%
0.90
0.44
0.72
+0.28
80.9%
66.1%
1.00
0.42
0.71
+0.29
82.4%
67.3%
Appendix
Table 17: Local surrogate correction during the ADS phase on RoboCasa. Entropies are normalized by logK . The final column reports the fraction of valid non-dominant modes for which Qb>Pb∣0 . Positive entropy gaps and mode margins show that the detached sibling reference is usually broader than the primary-conditioned surrogate.
Training objective
∥Jθz∥F
Eff. rank
Log-vol.
LFM
0.18
1.18
−8.42
+ uniform siblings
0.31
1.74
−6.37
+ base density only
0.38
2.08
−5.61
+ flow support (full ADS)
0.44
2.43
−4.92
Appendix
Table 18: Endpoint sensitivity to base noise on RoboCasa. We estimate singular values si of Jθz=∂Ψθ/∂z with randomized JVPs. Effective rank is (∑isi2)2/∑isi4 , and log-volume is ∑ilog(si+10−6) . Larger values mean more latent directions reach the action endpoint, which a wider candidate set requires but which does not by itself make those directions executable.
Occupied valid modes
Dominant-mode mass
ADS-phase progress
FM
ADS
FM
ADS
0.00
2.21
2.21
0.52
0.52
0.25
1.78
2.16
0.64
0.55
0.50
1.42
2.13
0.75
0.58
0.75
1.18
2.11
0.85
0.61
1.00
1.07
2.08
0.91
0.63
Appendix
Table 19: Occupancy of valid action modes for fixed training conditions. At the start of the analysis phase, normalized action chunks are embedded by temporal concatenation and clustered with k -means to define the modes; k is chosen by silhouette score, modes with rollout success below 50% are marked invalid, and later centroids are matched with the Hungarian algorithm. The statistic is population-level occupancy over the sampled set at each checkpoint.
s
ρ
Occupied cells
Target coverage
MC
MC
Analytic
MC
Analytic
0.05
−0.000662
1.000010
1.000005
0.000000
6.09588×10−23
0.10
0.000079
1.097250
1.097222
0.000000
2.29321×10−6
0.25
0.000492
2.513520
2.512776
0.168490
0.16815
0.50
−0.000609
3.644550
3.644815
0.749010
0.748932
1.00
0.001080
5.111300
5.116877
0.947410
0.947742
Appendix
Table 20: Independent Gaussian sampling. Correlation is averaged over distinct candidate-index pairs across independent sets. Coverage refers only to the predefined geometric target.
w
s1
Q1,8
Q1,∞
Pseen
0.05
0.01
0.152637
0.216920
0.336580
0.05
0.10
0.028886
0.026954
0.336580
0.05
1.00
0.003244
0.002762
0.336580
0.20
0.01
0.700244
0.862069
0.832228
0.20
0.10
0.346170
0.384615
0.832228
0.20
1.00
0.066806
0.058824
0.832228
Appendix
Table 21: Known-mixture density weighting. Q1,8 is a Monte Carlo mean of within-set normalized mass; Q1,∞ and Pseen use the analytic formulas above.
q+
κ
μ0
Final μ
dmin
In neighborhood
0.5
0
−1
0.000000000
1.000000000
0
0.5
0
+1
0.000000000
1.000000000
0
0.5
20
−1
0.800660032
0.199339968
1
0.5
20
+1
0.800660032
0.199339968
1
0.8
0
−1
0.173286795
0.826713205
0
0.8
0
+1
0.173286795
0.826713205
0
Appendix
Table 22: Fixed-surrogate optimization with an optional explicit quadratic anchor. We report numerical endpoints, distances to the nearest sibling, and geometric neighborhood indicators.
Scaling robot data and model capacity has improved Vision-Language-Action (VLA) policies, but further progress is constrained by the high cost of robotic data. Verifier-guided test-time scaling offers an efficient alternative by sampling multiple action candidates and selecting the one most likely to lead to task success at inference time. Existing classification-based verifiers learn from trajectory-level outcomes but treat all visited states equally, even though their value for candidate discrimination can vary across a trajectory. At many states, plausible actions are similar and provide limited discrimination signal, while only a sparse set of decision-critical states admits meaningfully different actions that can substantially affect downstream outcomes. To address this, we propose DiVeR, which estimates decision criticality from the dispersion of sampled action representations. DiVeR then uses this signal to reweight verifier learning toward states where action selection is most consequential, without requiring step-level annotations or additional environment interaction. Across LIBERO, RoboCasa, and real-world experiments on a Franka Research 3 robot, DiVeR consistently improves task success through more effective verifier-guided action selection, while adding negligible verifier inference overhead.
Seongheon Park, Heecheol Kim, Shulin Tian +5
University of Wisconsin–Madison · Microsoft Research Asia–Tokyo · Nanyang Technological University +2
Test-time compute (TTC), which scores multiple candidate actions at inference time, is a powerful way to improve the execution performance of vision-language-action (VLA) models, but existing methods keep the policy frozen, so the gain must be recomputed at every execution. We propose Verify, Act, Distill, a framework that consolidates the gains of TTC into the VLA itself. A fixed verifier scores candidates generated by a flow-matching VLA, the weighted aggregate is executed, and the observations and aggregated chunks from successful adaptation episodes are distilled directly into LoRA parameters with the standard flow-matching loss. This reuses supervision that TTC has already produced during execution, without additional reinforcement learning or rollouts dedicated to teacher generation. On the 40 LIBERO tasks with a SmolVLA 5k checkpoint, distilling from only 10 episodes per task raises the mean success rate of verifier-free single-candidate generation from 54.1% to 70.6%, retaining about 74% of the improvement obtained by aggregating eight candidates. Applying the same aggregation to the distilled policy further raises the success rate from 76.4% to 82.3%, with a particularly large gain on LIBERO-Long (44.0% to 57.8%), and the single-candidate policy also improves on LIBERO-plus, which contains distribution shifts. These results show that using verification as a learning signal, rather than only as an execution-time selector, both retains the improvements of TTC and re-amplifies the effect of subsequent TTC.
Kaneyoshi Hiratsuka, Satoshi Yamamori, Jun Morimoto
Vision-language-action (VLA) policies based on flow matching generate action chunks through repeated evaluations of an action expert. Increasing the number of integration steps raises inference cost, but does not necessarily improve closed-loop success. We propose Coda, which reallocates part of this integration budget to a single learned endpoint correction. A frozen policy first completes a few-step noise-to-action trajectory; a lightweight Transformer then predicts a demonstration-supervised residual using the candidate action, source noise, and shared observation-prefix cache. Only the corrector is trained. On 50 RoboTwin Easy tasks, five-step Coda improves success from 71.64% to 74.68% over the matched five-step baseline, while reducing forward latency by 30.2% relative to the default ten-step policy. A two-step configuration achieves 71.88% success with a 2.12× speedup. An independent 13-task control shows a 5.69-percentage-point gain at nearly equal latency, supporting correction as an effective alternative to additional integration. The same design also improves frozen official SmolVLA, raising two-step success from 60.8% to 69.4%. These results show that endpoint correction improves the quality-latency trade-off of frozen flow-matching policies.
Zhipeng Tang, Xinda Chen, Weining Rao +5
Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China · University of Chinese Academy of Sciences, Beijing, China · Zhongke Haichuan Intelligent