Hierarchical control scales robot behavior by separating task-level planning from low-level control. In dynamic, high-dimensional tasks, however, a fixed planner-to-policy interface can fail when contacts, balance, or disturbances evolve faster than high-level policy updates. Execution therefore requires choosing not only which low-level policy to run, but also which policy-specific goal should be given to it. We formulate this problem as policy orchestration: runtime selection of a policy-goal pair that is likely to remain feasible while tracking the planner target. We develop EPO, an energy-based policy orchestrator for modular goal-conditioned policy families. For each policy, EPO learns a composite energy over low-level goals, where lower energy corresponds to higher feasibility and more accurate target realization. At runtime, EPO refines each policy's goal and executes the lowest-energy policy-goal pair. Because energies are policy-specific, new policies can be incorporated by learning their own energy models, without retraining the existing policies or a centralized scheduler. In simulation, EPO improves the empirical precision-robustness trade-off by refining policy-specific goals before selectively switching among policies based on energy scores. We further demonstrate sim-to-real deployment of the same frozen policies and energy models on real humanoid loco-manipulation tasks. Project website: https://yewr.github.io/epo
Figures & tables
Figure 1: Energy-based Policy Orchestration. EPO extends the standard planner-to-policy interface from a fixed map to runtime optimization over a frozen policy family. It refines policy goals by energy minimization and executes the lowest-energy policy–goal pair.
Figure 2: Precision–robustness trade-offs in simulation. Each point is averaged over flat, rough, and stair terrains. The x-axis reports normalized tracking error, and the y-axis reports weighted survival (mean of Normal, Extreme, and Push survival). Left: loco-manipulation; πstand and πstand +Opt lie outside the plotted range. Right: standing manipulation without velocity commands.
Variant
Loco-manipulation
Standing manipulation
Jump-alt ↑
Switches/ep ↓
Max Hz ↑
Err. ↓
Track ↑
Push ↑
Err. ↓
Track ↑
Push ↑
Full EPO-M2
0.542
99.7
96.4
0.456
100.0
95.3
95.6
1.85
83
No goal refinement
0.554
99.7
96.9
0.459
100.0
96.9
96.0
1.81
615
Feasibility only
0.542
99.7
96.1
0.456
100.0
95.3
95.3
1.84
84
Consistency only
0.542
99.7
78.9
0.456
100.0
58.6
95.4
0.92
84
No feasibility threshold τ
0.542
99.7
82.8
0.456
100.0
65.6
96.2
0.99
83
Table 1: Component ablations. Ablations use EPO-M2 with 128 seeds on each of three terrains. Err. is normalized tracking error; Track and Push are survival rates (%). Jump-alt averages Extreme survival across the two modes. Switches are averaged over Normal and Push episodes. Max Hz is the inverse median latency of refining and scoring all policies for one robot (excluding policy inference and the selector update). Feasibility only and Consistency only change only the selection score; Consistency only also removes τ and the refinement gate.
Figure 3: Runtime visualization of EPO. (a) F^ , C^ , E^ of πgen at two example states; TTO steps move z⋆ toward lower energy (star: returned goal). (b) EPO-M4 under a 3 m/s push while standing: F of πstand collapses, EPO switches to πstandR within 40 ms and returns to πstand0.48 s later once its accumulated energy advantage exceeds B .
Figure 4: Switching frequency. Mean switches per episode ( 768 episodes per condition).
Figure 5: Humanoid loco-manipulation tasks. We evaluate whole-body control in simulation and demonstrate hardware deployment, including locomotion and loco-manipulation.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Loco-manip
Stand
Method
Error ↓
Surv (%) ↑
Error ↓
Surv (%) ↑
πstand
0.656
77.0
0.396
76.0
πgen
0.549
93.6
0.458
88.3
πgenR
0.568
97.7
0.469
98.2
πstand +Opt
0.668
76.6
0.400
76.0
πgen +Opt
0.539
93.1
0.457
87.3
Appendix
Table 2: Aggregate precision–robustness results. Error is the Normal-condition tracking error averaged over (flat, rough, stair); Survival is the weighted survival score over the Normal, Extreme, and Push conditions, each averaged over the same terrains. Lower error and higher survival are better. These are the aggregate quantities plotted in Fig. 2 . Best EPO entries in bold.
Fixed policy that falls
n
EPO-M2 survives (%)
EPO-M4 survives (%)
πstand
503
94.6
96.8
πgen
197
92.4
93.4
πgenR
14
57.1
78.6
Appendix
Table 3: Survival of EPO on episodes where a fixed policy falls (Push condition). n is the number of such episodes.
Three refinement steps
No refinement
Family
Median (ms)
p95 (ms)
Max Hz
Median (ms)
p95 (ms)
Max Hz
EPO-M2
12.06
12.48
83
1.67
1.80
601
EPO-M3
20.46
20.81
49
2.40
2.55
416
EPO-M4
24.34
27.48
41
3.18
3.34
314
Appendix
Table 4: Runtime scaling. Latency of refining and scoring all policies for one robot, with three refinement steps and without refinement.
H (control ticks)
Loco Err ↓
Stand Err ↓
Loco Surv. ↑
Stand Surv. ↑
Push fall (%) ↓
Switches/ep
10 ( 0.2 s)
0.547
0.433
97.9
80.5
10.8 †
1.49
50 ( 1.0 s)
0.550
0.428
95.6
98.4
3.0
1.60
100 ( 2.0 s)
0.554
0.429
97.4
96.9
2.9
1.40
50 , main-table models
0.546
0.420
95.3
97.7
3.5
2.08
Appendix
Table 5: Energy-model horizon. EPO-M4 with energy models trained at different horizons. Err. is Normal tracking error; Surv. is survival under a 3 m/s push; Push fall pools both modes. † : push fall rate differs from the retrained H=50 (paired whole-seed bootstrap, 95% CI excludes 0).