Search allows a robotic policy to evaluate candidate futures before committing to an action. Motivated by test-time planning for vision-language action (VLA) models, we study selective search over action chunks in PushT using a frozen Action Chunking with Transformers (ACT) policy. Beam search generates perturbed alternatives around the policy prediction and scores them through rollouts from cloned simulator states. Across 50 evaluation episodes, beam search raises task success from 0% to 18% and increases average cumulative reward from 39.44 to 103.2. We introduce One Step Ahead (OSA), a gate that simulates the nominal policy chunk and invokes full search only when its predicted coverage drop exceeds a threshold. In the reported ablations, OSA preserves or improves success while reducing runtime by approximately 3x relative to full beam search. Ablations over tree depth, branching factor, beam width, and replanning frequency show that larger search budgets do not consistently improve performance. The results support selective test-time search for action-chunking policies; evaluation on language-conditioned VLAs and physical robots remains future work.
Figures & tables
Figure 1: A representative PushT episode under ACT, beam search, and OSA, starting from the same initial configuration. Columns show approximate environment steps. The blue marker is the agent, the gray T is the movable object, and the colored T marks the target. Blue paths show nominal predictions, gray paths show alternatives, and warm-colored paths show selected chunks. The baseline fails to align the object within 300 steps; the search-based controllers reach the target.
Method
D
C
BW
Avg. sum reward
Avg. max reward
Success (%)
ASR (%)
Runtime (s)
Beam search
2
4
2
90.9
0.77
18
56.4
14
Beam search
4
2
2
90.8
0.78
16
37.5
10
Beam search
4
4
2
90.9
0.77
18
58.23
10
Beam search
4
8
2
94.7
0.77
18
75.6
33
Beam search
4
8
4
103.2
0.81
18
75.0
47
Beam search
4
8
8
92.2
0.80
10
78.5
87
Table 1: PushT beam-search configurations, averaged over 50 episodes. D : tree depth; C : total candidate count per expanded node, including the nominal chunk; BW : beam width. Reward columns report episode-level cumulative and maximum reward, respectively. ASR is the percentage of selections using a perturbed chunk.
Figure 2: Average cumulative reward versus execution horizon H for two beam-search configurations. Shorter horizons invoke search more frequently. Each point summarizes 50 episodes.
Figure 3: One Step Ahead ablation at D=4 , C=8 , and beam widths 2, 4, and 8. Blue: full beam search; green: OSA. Panels show success, average cumulative reward, and runtime. OSA preserves or improves success in these comparisons while reducing computation, although cumulative reward can decrease.