Planning while moving faces an historical trade-off between deliberation and reactivity. A planner running concurrently with execution needs time to compute, so the robot must commit to an open-loop action chunk for that duration: a longer commitment buys a better plan at the cost of feedback frequency. Existing controllers fix this action commitment, and consequently the computation budget, before deployment. However, the optimal computation time is highly state-dependent (due to factors such as unmodeled dynamics, safety constraints, and marginal optimisation gains), i.e. dynamic, therefore offline static tuning risks leaving systems blind during critical maneuvers or unable to find global effective solutions. We cast the identification of the optimal computation budget as a semi-Markov decision process (SMDP) in which a single decision defines both the robot action commitments and the time allocated to the solver. To solve this, we train a reinforcement-learning policy conditioned on the current state, the executing plan, and inexpensive planner statistics. Across ground-robot navigation, agile quadrotor flight, and contact-rich manipulation, the learned policy changes its budget with the demands of the scenario, discovering efficient reasoning patterns that outperform fixed-budget baselines. https://lorenzomagnino.github.io/learning-robot-metareasoning-review/
Figures & tables
Fig. 1 : Robot Metareasoning: different states require different computation budgets . Robots must adapt their computation budget based on the situation that they are facing: ( s ) deliberate longer to search for an optimal path, ( s′ ) react rapidly to noisy moving obstacles, or ( s′′ ) balance planning and reactivity. Across real-world and simulated experiments, we show that adapting the computation budget across states within an episode is a fundamental capability for embodied intelligence.
Fig. 2 : Method Overview . At each decision epoch, the state, the previous plan and its statistics inform the budget policy Φθ , which selects the next execution horizon and thereby allocates the solver’s budget. This lets the robot decide how much to deliberate under environmental uncertainty and temporal constraints. The new plan takes over, the state and statistics are refreshed, and the loop repeats.
Fig. 3 : Approaches . (Left) Real-time MPC commits to one step ( h=1 ), whereas action chunking (right) executes a fixed block of h actions.
Fig. 4 : Navigation Results , pooled over the three scenarios ( 300 randomised episodes). Our learned budget policy (left of the dashed rule) against every fixed commitment h (blue). The two right-hand panels condition on successful episodes and therefore flatter the low-success baselines. The solid and dashed outlines mark the best fixed budget under success rate ( h=5 ) and under total return ( h=1 ): no single fixed h is best at both, while the learned budget dominates both.
Fig. 5 : Real-world Merging Lane scenario. Top: temporal composites on the RoboMaster platform under our learned budget and fixed commitments h∈{1,4,9} ; each panel overlays successive robot poses from one episode. Bottom: the commitment our policy selects along its episode, against the three fixed budgets. The budget collapses to h=1 while the robot merges past the crossing obstacle (red band) and while it threads the close passage between the static obstacles.
Fig. 6 : Optimality Regret in the Merging Lane scenario along three fixed commitment trajectories. Lines are means and standard deviations over 10 episodes. The tint marks Phase 1, before the moving obstacle has been passed, which ends at a different step for each commitment. Dotted rules mark the mean collision step for each fixed strategy.
Fig. 7 : Holonomic 2D navigation: planner statistics ablation . Ablation of the policy input, pooled over the three scenarios ( 300 episodes per variant). Full uses (stk,hk−1,τk,bk) ; PS only uses the planner statistics bk alone; Full − PS uses everything except bk . The full input achieves the highest success rate and total return, while removing planner statistics drops success by 12 percentage points.
Fig. 8 : Dynamic commitment during cable-payload flight. Four stages from one episode show the learned policy increasing the commitment for transit and reducing it near each target.
Fig. 9 : Cable-payload slalom return. Mean total return over 20 episodes per method. The left group compares the full policy (gold) with ablations without planner statistics (orange) and using planner statistics only (salmon); fixed commitments are shown in blue.
Scenario
Strategy
Success ↑
Total return ↑
Harshness ↓
Barrier [1pt]
Ours
0.97
−435
7.37
h=5 (S)
0.73
−1167
7.28
h=1 (R)
0.12
−1523
7.42
h=9 (H)
0.63
−1400
7.55
Merging Lane [1pt]
Ours
1.00
−385
7.02
h=5 (S)
0.63
−1462
7.17
TABLE I : Navigation results ( 100 episodes per scenario). We compare our policy with the same three fixed budgets selected from the pooled results in Fig. 4 : h=5 (best Success rate), h=1 (best total Return), and h=9 (best Harshness). The budgets remain fixed across scenarios; total return is averaged over all episodes.
Fig. 10 : Boundary behaviour on the 3D PushT environment . Left: robot-arm manipulation setup. Right: terminal position and orientation errors after 600 steps, reported as mean ± sample standard deviation over 16 matched validation episodes. Our policy selects hˉ=8.00 .
British Open-ended Learning and Discovery Lab (BOLD), University of Oxford · Department of Statistics, University of Oxford · VGG, University of Oxford