Hierarchical control scales robot behavior by separating task-level planning from low-level control. In dynamic, high-dimensional tasks, however, a fixed planner-to-policy interface can fail when contacts, balance, or disturbances evolve faster than high-level policy updates. Execution therefore requires choosing not only which low-level policy to run, but also which policy-specific goal should be given to it. We formulate this problem as policy orchestration: runtime selection of a policy-goal pair that is likely to remain feasible while tracking the planner target. We develop EPO, an energy-based policy orchestrator for modular goal-conditioned policy families. For each policy, EPO learns a composite energy over low-level goals, where lower energy corresponds to higher feasibility and more accurate target realization. At runtime, EPO refines each policy's goal and executes the lowest-energy policy-goal pair. Because energies are policy-specific, new policies can be incorporated by learning their own energy models, without retraining the existing policies or a centralized scheduler. In simulation, EPO improves the empirical precision-robustness trade-off by refining policy-specific goals before selectively switching among policies based on energy scores. We further demonstrate sim-to-real deployment of the same frozen policies and energy models on real humanoid loco-manipulation tasks. Project website: https://yewr.github.io/epo
Figures & tables
Figure 1: Energy-based Policy Orchestration. EPO extends the standard planner-to-policy interface from a fixed map to runtime optimization over a frozen policy family. It refines policy goals by energy minimization and executes the lowest-energy policy–goal pair.
Figure 2: Precision–robustness trade-offs in simulation. Each point is averaged over flat, rough, and stair terrains. The x-axis reports normalized tracking error, and the y-axis reports weighted survival (mean of Normal, Extreme, and Push survival). Left: loco-manipulation; πstand and πstand +Opt lie outside the plotted range. Right: standing manipulation without velocity commands.
Variant
Loco-manipulation
Standing manipulation
Jump-alt ↑
Switches/ep ↓
Max Hz ↑
Err. ↓
Track ↑
Push ↑
Err. ↓
Track ↑
Push ↑
Full EPO-M2
0.542
99.7
96.4
0.456
100.0
95.3
95.6
1.85
83
No goal refinement
0.554
99.7
96.9
0.459
100.0
96.9
96.0
1.81
615
Feasibility only
0.542
99.7
96.1
0.456
100.0
95.3
95.3
1.84
84
Consistency only
0.542
99.7
78.9
0.456
100.0
58.6
95.4
0.92
84
No feasibility threshold τ
0.542
99.7
82.8
0.456
100.0
65.6
96.2
0.99
83
Table 1: Component ablations. Ablations use EPO-M2 with 128 seeds on each of three terrains. Err. is normalized tracking error; Track and Push are survival rates (%). Jump-alt averages Extreme survival across the two modes. Switches are averaged over Normal and Push episodes. Max Hz is the inverse median latency of refining and scoring all policies for one robot (excluding policy inference and the selector update). Feasibility only and Consistency only change only the selection score; Consistency only also removes τ and the refinement gate.
Figure 3: Runtime visualization of EPO. (a) F^ , C^ , E^ of πgen at two example states; TTO steps move z⋆ toward lower energy (star: returned goal). (b) EPO-M4 under a 3 m/s push while standing: F of πstand collapses, EPO switches to πstandR within 40 ms and returns to πstand0.48 s later once its accumulated energy advantage exceeds B .
Figure 4: Switching frequency. Mean switches per episode ( 768 episodes per condition).
Figure 5: Humanoid loco-manipulation tasks. We evaluate whole-body control in simulation and demonstrate hardware deployment, including locomotion and loco-manipulation.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Loco-manip
Stand
Method
Error ↓
Surv (%) ↑
Error ↓
Surv (%) ↑
πstand
0.656
77.0
0.396
76.0
πgen
0.549
93.6
0.458
88.3
πgenR
0.568
97.7
0.469
98.2
πstand +Opt
0.668
76.6
0.400
76.0
πgen +Opt
0.539
93.1
0.457
87.3
Appendix
Table 2: Aggregate precision–robustness results. Error is the Normal-condition tracking error averaged over (flat, rough, stair); Survival is the weighted survival score over the Normal, Extreme, and Push conditions, each averaged over the same terrains. Lower error and higher survival are better. These are the aggregate quantities plotted in Fig. 2 . Best EPO entries in bold.
Fixed policy that falls
n
EPO-M2 survives (%)
EPO-M4 survives (%)
πstand
503
94.6
96.8
πgen
197
92.4
93.4
πgenR
14
57.1
78.6
Appendix
Table 3: Survival of EPO on episodes where a fixed policy falls (Push condition). n is the number of such episodes.
Three refinement steps
No refinement
Family
Median (ms)
p95 (ms)
Max Hz
Median (ms)
p95 (ms)
Max Hz
EPO-M2
12.06
12.48
83
1.67
1.80
601
EPO-M3
20.46
20.81
49
2.40
2.55
416
EPO-M4
24.34
27.48
41
3.18
3.34
314
Appendix
Table 4: Runtime scaling. Latency of refining and scoring all policies for one robot, with three refinement steps and without refinement.
H (control ticks)
Loco Err ↓
Stand Err ↓
Loco Surv. ↑
Stand Surv. ↑
Push fall (%) ↓
Switches/ep
10 ( 0.2 s)
0.547
0.433
97.9
80.5
10.8 †
1.49
50 ( 1.0 s)
0.550
0.428
95.6
98.4
3.0
1.60
100 ( 2.0 s)
0.554
0.429
97.4
96.9
2.9
1.40
50 , main-table models
0.546
0.420
95.3
97.7
3.5
2.08
Appendix
Table 5: Energy-model horizon. EPO-M4 with energy models trained at different horizons. Err. is Normal tracking error; Surv. is survival under a 3 m/s push; Push fall pools both modes. † : push fall rate differs from the retrained H=50 (paired whole-seed bootstrap, 95% CI excludes 0).
General-purpose robot manipulation policies learn short-horizon skills effectively, but degrade rapidly on long-horizon tasks. Existing benchmarks cannot isolate failures because task horizon is either too short or entangled with scene and object variations. We propose LADDER, a long-horizon robot manipulation benchmark scaling from 1 to 10 subgoals within the same environment over three suites (SimplerEnv-Long, RobotArena-∞-Long, and RoboLab-Long), 145 tasks, eight categories, two embodiments, and 2,024 simulated episodes per system. We evaluate two orchestration paradigms: text subgoals for learned low-level policies (VoLo) and tool selection orchestration (CaP-X, GPT-Policy, PSP-Agent, TiPToP). Given the full instruction directly, low-level policies achieve 85.4% on single-subgoal tasks, but drop to near zero on extended tasks. While orchestration remedies this, it remains bounded by physical execution errors of the low-level policy rather than high-level planning and achieves only ~25%. Tool selection orchestration with GPT-Policy on GPT-6 Astra significantly outperforms learned-policy methods, but incurs significant time and cost budget and succeeds only ~62%. Real-world experiments across two embodiments confirm the collapse of the low-level policy and the gain from orchestration. Project website: https://ladder.insait.ai
Petko Petkov, Giuliano Albanese, Nikolay Nikolov +3
INSAIT, Sofia University “St. Kliment Ohridski” · NVIDIA
Long-term robot autonomy depends on more than effective action generation: a policy must recognize when its behavior is no longer sufficiently supported and use assistance to improve. We introduce Equilibrium Energy Control (EqC), a framework for introspective lifelong robot learning built around an explicit conditional energy landscape. EqC casts generative action refinement as equilibrium-seeking optimization, while the geometry of the refinement process reveals whether the generated behavior remains consistent with the learned landscape. This competence signal enables Energy-Gated DAgger, where persistent energy-consistency violations trigger expert takeover and the resulting recovery trajectories are aggregated for subsequent policy updates. EqC integrates naturally into both Diffusion Transformer policies and flow-matching-based vision-language-action (VLA) policies. Across simulation and real-world bimanual manipulation, EqC matches or improves control performance and robustness while enabling timely intervention near the policy's competence boundary. A real-world study further shows how these interventions progressively expand the range of situations handled autonomously, providing a practical path from static generative controllers toward self-improving robot systems.
Haodong Yang, Runhan Huang, Bowen Zhang +7
Tsinghua University · Galaxea AI · Princeton University
General-purpose robots need to reason about their actions, combining perception, world knowledge, planning, success detection, recovery, and low-level control. Today's state-of-the-art models attempt to combine all these capabilities into the learned policy via large-scale pre-training. Instead, we show that these capabilities can be decomposed into a general language-conditioned policy/control agent and a high-level agent manager/orchestrator. Rather than training policies to reason via pre-training, we build a closed-loop physical agent orchestrator that can do high-level planning, decompose the goal into achievable subgoals, command low-level motor commands, track and verify the outcome from low-level observations, and recover from failures. Our Physical Agency orchestrator (Pigey) can control existing vision-language-action (VLA) policies as well as parametrized skills to solve complex reasoning tasks in the real world, without any additional data collection or post-training. We evaluate Pigey extensively across simulation benchmarks and challenging real-world robotic manipulation tasks, and demonstrate significant performance improvements over existing generalist policies. On LIBERO-PRO, Pigey advances the state-of-the-art by over 4x (12.8% -> 53.3%) with no task-specific fine-tuning. On a real robot, Pigey lifts the frozen policy from near-zero to over 90% on reasoning-limited tasks. We call the difference between what frozen motor skills achieve alone and inside the agentic loop the orchestration gap.