Refinement vs. Replanning: Inference Compute Allocation in Robot Policies
Organizations: MAAT Institute
Abstract
A frozen chunked robot policy can spend deployment-time inference compute in two coupled ways: more work within each policy call to refine a predicted action chunk, or more frequent policy calls to replan from fresh observations. We study this allocation without updating model weights. Our evaluation procedure reports success with both nominal compute rate and complete attempt cost, selects comparisons on discovery tasks, and tests prespecified comparisons on held-out tasks. Comparisons added after inspecting the held-out results only summarize observed patterns. We demonstrate the procedure with one frozen SmolVLA checkpoint in synchronous LIBERO simulation, using denoising count as the refinement control and fixing chunk length at K=50. Arithmetic and latency characterize cost differently: at d=10, 82.1% of counted arithmetic is independent of denoising, whereas a descriptive affine fit to fixed-input host time assigns 17.7% to its intercept. At h=2, d=2 succeeds on 139/160 attempts versus 122/160 for d=5 (+10.6 percentage points; task-conditional 95% interval: +4.4 to +16.9), while using 5.9% less arithmetic per control step. Executing h=5 rather than h=2 at d=2 yields 138/160 versus 139/160 successes and 60.7% lower arithmetic per attempt; equivalence is not established. The case study shows why deployment adaptation must evaluate generation effort and feedback frequency jointly; it does not define a universal setting.
Figures & tables
| Allocation | Success | Nominal | Mean per attempt | |||
|---|---|---|---|---|---|---|
| Count | (%) | rate ratio | TFLOPs | Steps | ||
| 1 | 2 | 137/160 | ||||
| 1 | 5 | 136/160 | ||||
| 1 | 10 | 135/160 | ||||
| 2 | 2 | 139/160 | ||||
| 2 | 5 | 122/160 | ||||
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
| Allocation | Spatial task successes (each /10) | Nominal | |||||
| Task 5 | Task 6 | Task 7 | Task 8 | Total | rate ratio | ||
| 1 | 2 | 5 | 7 | 4 | 10 | 26/40 | |
| 1 | 5 | 5 | 8 | 6 | 9 | 28/40 | |
| 1 | 10 | 5 | 7 | 6 | 7 | 25/40 | |
| 1 | 20 | 5 | 6 | 6 | 8 | 25/40 | |
| 2 | 2 | 5 | 7 | 4 | 10 | 26/40 | |
| Allocation | Nominal / step | Mean per attempt | Success | ||
|---|---|---|---|---|---|
| GFLOPs | TFLOPs | Rollout (s) | Count | ||
| 1 | 2 | 26/40 | |||
| 1 | 5 | 28/40 | |||
| 1 | 10 | 25/40 | |||
| 1 | 20 | 25/40 | |||
| 2 | 2 | 26/40 | |||
| GFLOPs | Host (ms) | CUDA (ms) | Host block range (ms) | |
|---|---|---|---|---|
| 2 | 214.5–220.1 | |||
| 3 | 265.3–272.6 | |||
| 5 | 366.8–378.1 | |||
| 8 | 519.9–536.1 | |||
| 10 | 623.2–642.2 | |||
| 20 | 1133.5–1172.6 |
| Ceiling relative to native | Arithmetic constraint | Host-time constraint |
|---|---|---|
| 0.25 | ||
| 0.50 | ||
| 1.00 |
| Evaluated lengths GFLOPs (range) | Length 20 GFLOPs | Eligibility changes | |
|---|---|---|---|
| 2 | 539.7–547.9 | 0 | |
| 5 | 573.5–582.2 | 0 | |
| 10 | 629.8–639.3 | 0 |
| Selector choice | Learned selector | ||||
|---|---|---|---|---|---|
| Ceiling ratio | Learned | Simple | Empirical | Compute-only | Rate ratio |
| 0.25 | |||||
| 0.50 (primary) | |||||
| 1.00 | |||||
| Setting | Success difference (pp) | ||||
|---|---|---|---|---|---|
| Ceiling ratio | Selected | Comparator | Predicted | Observed | 95% interval |
| 0.25 | |||||
| 0.50 (primary) | |||||
| 1.00 | |||||
| (primary pair) | |||||||
|---|---|---|---|---|---|---|---|
| Task | |||||||
| Goal 5 | |||||||
| Goal 6 | |||||||
| Goal 7 | |||||||
| Goal 8 | |||||||
| Object 5 | |||||||
| Setting A | Setting B | A B (pp) | 95% interval (pp) |
|---|---|---|---|
| Mean per attempt | Difference | 95% interval | ||
|---|---|---|---|---|
| Reference TFLOPs | ||||
| Observed rollout (s) | ||||
| Control steps |
| Setting | Value |
|---|---|
| Hardware | RTX 4060 Ti, 16 GB; driver 595.84 |
| Runtime | Python 3.12.3; PyTorch 2.7.1+cu126; CUDA 12.6; NumPy 2.2.6; MuJoCo 3.8.1 |
| Evaluation | Batch 1; synchronous simulation; nominal 20 Hz (50 ms control period) |
| Episode horizons | Spatial and Object: 280 control steps; Goal: 300 |
| Precision controls | Autocast and TF32 disabled; deterministic algorithms enabled |
| Attention and reuse | Eager float32 attention; within-generation KV reuse |
| Check | Evidence |
|---|---|
| Canonical coverage | 1,120 held-out attempts and 171,805 actions; the primary subset remains 320 attempts and 50,451 actions. |
| Pairing and dispatch | Planned identities, paired initial states and observations, generation counts, denoising counts, and applied actions reconcile. |
| Excluded attempts | Two repeated completed attempts are checked and excluded; one interrupted ten-action record has no completed outcome and is excluded. |
| Outcome path | All 1,122 completed attempts and 172,243 steps reconcile; generated actions match simulator actions and recorded outcome fields agree. |
| Shared predicate | Reward and success both use LIBERO’s task-goal evaluation, traced to BDDL predicates, so their agreement is a consistency check. |
| Native parity | One Spatial episode matches the upstream path in initial state, observation, 70-action sequence, length, and outcome. |
Inference-Time Compute Allocation
Inference-time compute allocation decides where, when, and how much computation models spend during prediction to improve quality under resource limits.
See all 91 papers