Humanoid Hanoi: Investigating Shared Whole-Body Control for Skill-Based Box Rearrangement

Minku Kim; Kuan-Chia Chen; Aayam Shrestha; Li Fuxin; Stefan Lee; Alan Fern

cs.ROFeb 14, 2026

Humanoid Hanoi: Investigating Shared Whole-Body Control for Skill-Based Box Rearrangement

Authors: Minku Kim, Kuan-Chia Chen, Aayam Shrestha, Li Fuxin, Stefan Lee, Alan Fern

Organizations: The authors are with the Collaborative Robotics and Intelligent Systems Institute, Oregon State University, Corvallis, Oregon, 97331, USA

Abstract

We introduce Humanoid Hanoi, a challenge problem for long-horizon humanoid skill composition that requires a robot to repeatedly sequence pickup, locomotion, and placement skills to rearrange stacked boxes in simulation and on hardware. This symbolically simple task exposes a weakness in a common modular humanoid architecture, where reusable high-level skills are learned on top of a shared, task-agnostic whole-body controller (WBC). While such a WBC provides a unified low-level interface for skill composition, it may not be sufficiently hardened to execute the full collection of learned skills under domain randomization and long-horizon closed-loop execution. We compare strategies for addressing this issue, including skill-specific WBC fine-tuning, residual low-level adaptation, and task-driven data aggregation that expands the shared-WBC training distribution with closed-loop trajectories induced by learned skills while preserving its original tracking objective. In our evaluated system, the shared-WBC approach achieves the strongest long-horizon robustness. A direct transition-smoothing intervention further shows that low-level controller handoffs account for a substantial part of the composition loss observed with non-shared controllers. Project page: https://osudrl.github.io/Humanoid_Hanoi/

Figures & tables

Figure 1: A problem instance from Humanoid Hanoi . The robot moves boxes between three towers (T1–T3) while respecting stacking constraints. The panels illustrate representative stages of a successful hardware execution (5+ min), with the corresponding symbolic state shown on the bottom.

Figure 2: Long-horizon Humanoid Hanoi execution. Sequential snapshots show a complete Tower of Hanoi style box rearrangement episode. Transparent overlays visualize the executed robot trajectory over time, and T1 , T2 , and T3 denote the target tower locations. Each problem instance is divided into seven moves , each consisting of a sequence of GoTo , Pickup , GoTo-with-Box , and Place skills.

Figure 3: Independently trained high-level skills generate task-level commands that are executed through a shared, task-agnostic whole-body controller (WBC). The WBC produces joint-level PD targets that are tracked by a low-level PD controller on the robot. Closed-loop rollouts from composed execution are aggregated to retrain the shared controller, improving robustness while preserving a unified control interface. The base action ${a}_{t}^{\text{base}}$ specifies base locomotion commands, including a stand bit, planar velocity targets, and yaw rate.

Figure 4: Cumulative task success rates for Humanoid Hanoi. Each move consists of four skills: GoTo , Pickup , GoTo with Box , and Place , shown as small markers, with large markers indicating move completion. Drops between consecutive points indicate failures in the subsequent skill (e.g., a drop from Pickup to GoTo with Box indicates failures during the GoTo with Box execution). End-to-end results from [ 4 ] are omitted because they achieved only 1/100 complete-task success without DR and 0/100 with DR.

Figure 5: Top: Representative hip-yaw response around the transition into Place , Bottom: Box state changes after the transition into Place over 100 Humanoid Hanoi trials, comparing Extended and Finetune .

Figure 6: Three Humanoid Hanoi configurations (C1–C3) where T1 , T2 , and T3 denote the tower locations defining the stack positions.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Upper Body Tracking	Base	Extended
Upper joint MAE (median [IQR]) $\downarrow$	0.1246 [0.1763]	0.0805 [0.1535]
Success Rate (%) $\uparrow$	95	100

Appendix

Table 2: Upper-body tracking performance comparing the pretrained WBC ( Base ) and the WBC trained with data aggregation ( Extended ), on trajectories from the baseline training dataset.

Skill	Name	Reward ( $r$ )	Weight ( $w$ ) (up / down)	Scale ( $\alpha$ ) (up / down)
Pickup Policy	hand contact position	$\\|\mathbf{p}_{\text{hand}}^{\text{current}}-\mathbf{p}_{\text{elbow}}^{\text{target}}\\|_{2}$	0.5	–
	base pitch roll	$(\|\theta_{\text{base}}\|+\|\theta_{\text{base}}+0.15\|)$	0.2	15
	box rotation	$\|\theta_{\text{box}}\|+\|\phi_{\text{box}}\|\text{if}\hskip 3.00003pt\text{contact}_{\text{(L$ \wedge $R)}}$	0.1	10
	box acceleration	$\\|\mathbf{\dot{v}}_{\text{box}}\\|_{2}$	0.05	0.02
	table force	$\frac{F_{\text{table}}}{\text{mg}}\text{if}\hskip 3.00003pt\text{contact}_{\text{(L$ \wedge $R)}}$	0.05	2.0
	motor vel	$\text{mean}(\mathbf{v}_{\text{motor}}\odot W_{\text{weights}})$	0.05	0.4

Appendix

Table 3: Reward components grouped by skill. For values with two entries, the first value corresponds to the Pickup policy and the second to the Place policy. Terms without scale (

\alpha

) do not use the exponential kernel.

Skill	Name	Reward ( $r$ )	Weight ( $w$ )	Scale ( $\alpha$ )
GoTo Policy	constellation	$\sum_{i=1}^{9}\\|\mathbf{p}_{i}^{\text{current}}-\mathbf{p}_{i}^{\text{target}}\\|_{2}$	1.0	0.5
	base position error	$\\|\mathbf{p}^{\text{current}}_{\text{base}}\\|_{2}$	0.2	5.0
	base yaw error	$\|\psi_{\text{base}}\|$	0.2	3.0
	foot orientation	$d_{\text{quat}}(\mathbf{q}^{\text{current}}_{\text{foot}},\mathbf{q}^{\text{target}}_{\text{foot}})$	0.05	4.0
	stance width error	$\|\|y_{\text{L,foot}}-y_{\text{R,foot}}\|-0.33\|$	0.2	1.0
	stand parallel	$\|x_{\text{L,foot}}-x_{\text{R,foot}}\|$	0.2	1.0

Appendix

Table 4: Reward components for the GoTo policy and additional rewards for the GoTo-with-Box policy. Terms without scale (

\alpha

) do not use the exponential kernel.

	Parameters	Range
Dynamics Randomization	Body Mass	$[0.75,1.25]\times\text{Default}$
	Joint Damping	$[0.5,3.5]\times\text{Default}$
	Center of Mass Position	$[0.95,1.05]\times\text{Default}$
	Friction Coefficient	$[0.8,1.2]\times\text{Default}$
Box Randomization	Mass (kg)	$[0.0,3.5]$
	Size XYZ (m)	$[0.15,0.5]$

Appendix

Table 5: Domain randomization parameters.

Figure 7: Box state changes after the transition into Place over 400 Humanoid Hanoi trials for Finetune , comparing successful and failed placements.

Metric	Base	Finetune	Residual	Extended
No Dynamics Randomization ( $\mu\pm\sigma$ )
$x_{err}$ (cm) $\downarrow$	$4.71\pm 3.46$	$5.79\pm 4.10$	$5.35\pm 3.73$	$5.47\pm 3.60$
$y_{err}$ (cm) $\downarrow$	$5.26\pm 3.58$	$5.75\pm 4.27$	$5.24\pm 3.71$	$5.60\pm 3.69$
$\theta_{err}$ ( ∘ ) $\downarrow$	$2.89\pm 2.74$	$2.65\pm 2.34$	$2.66\pm 2.57$	$3.59\pm 3.96$
Dynamics Randomization ( $\mu\pm\sigma$ )
$x_{err}$ (cm) $\downarrow$	$5.36\pm 4.12$	$5.72\pm 3.95$	$5.87\pm 5.28$	$5.42\pm 3.82$

Appendix

Table 6: Humanoid Hanoi placement accuracy for successful trials. Final box position and orientation errors for the Place skill, reported as mean

\mu

and standard deviation

\sigma

Metric	Base	Finetune	Residual	Extended
No Dynamics Randomization, Common Successes ( $\mu\pm\sigma$ )
$x_{err}$ (cm) $\downarrow$	$4.27\pm 2.99$	$5.73\pm 4.32$	$5.38\pm 3.85$	$5.60\pm 3.79$
$y_{err}$ (cm) $\downarrow$	$5.63\pm 3.66$	$6.31\pm 4.78$	$5.99\pm 4.01$	$5.45\pm 3.78$
$\theta_{err}$ ( ∘ ) $\downarrow$	$2.78\pm 2.69$	$2.61\pm 2.20$	$2.97\pm 2.84$	$3.69\pm 4.12$
Dynamics Randomization, Common Successes ( $\mu\pm\sigma$ )
$x_{err}$ (cm) $\downarrow$	$5.86\pm 3.49$	$5.57\pm 3.04$	$7.64\pm 4.27$	$6.12\pm 3.75$

Appendix

Table 7: Placement accuracy on successful trials for all methods.

Figure 8: Failure counts across skills in Humanoid Hanoi for each baseline, shown with and without dynamics randomization.

Figure 9: Qualitative hardware comparison of consecutive low-height Pickup and Place executions with the Base WBC and Extended WBC. Frames 1–4 show Pickup ; frames 5–8 show Place .

Figure 10: Simplified two-tower hardware setup used to reduce yaw-localization demands. The robot successfully completes five consecutive three-box stacking trials in this setting, suggesting that yaw-localization is a significant bottleneck in Humanoid Hanoi .

Explore similar work

Sep 11, 2026

CHOREO: Every Humanoid Skill as a Trajectory

Recent advances in humanoid robotics have produced diverse skills through reinforcement learning, motion imitation, and generative modeling. Yet these capabilities remain siloed because they are built around incompatible representations, interfaces, and controllers. We present CHOREO, a framework for training-free composition of heterogeneous humanoid skills. Our key observation is that, regardless of how a skill is learned, it can ultimately be expressed as an executable motion trajectory. Based on this observation, CHOREO converts each capability into SkillMotion, a unified representation that combines motion states, contacts, semantics, and boundary conditions. Skills are composed through direct continuation, cubic Hermite blending, or validated bridge motions, without retraining source models or updating models at test time. On Unitree G1 in MuJoCo, CHOREO organizes 2,950 admitted SkillMotion assets derived from heterogeneous sources and achieves 95.4% sequence success across 130 multi-action tasks, including 93.8% success on eight-action sequences. These results demonstrate that executable trajectories provide a scalable interface for accumulating and composing pretrained humanoid capabilities.

Ziyi Sun, Jingwen Chen, Yuxi Wang +4

College of Engineering, Ocean University of China · Faculty of Information Science and Engineering, Ocean University of China · School of Artificial Intelligence, University of Chinese Academy of Sciences +1

Humanoid Robotics Humanoid Robot Control Robot Motion Generation Robot Skill Learning

Ask PDF

Aug 23, 2025

LHM-Humanoid: Long-Horizon Human Motion Control for Continuous Object Transport in Cluttered Scenes

Physics-based human motion control can make a simulated character walk, sit, and manipulate objects with high physical realism. Almost always, though, this happens in short, isolated clips that are re-initialized between interactions. We instead aim for continuous, reset-free long-horizon motion: a physically simulated humanoid that repeatedly walks to a displaced object, lifts it with a balanced whole-body posture, carries it past obstacles, and places it at a goal, over and over within a single uninterrupted take. The hard part is not any individual motion but the transitions between them. Without a reset, each cycle must end in a state that both leaves the object just placed undisturbed and lets the next cycle begin, yet every placement leaves the character off-balance in a non-canonical pose where naive end-to-end reinforcement learning fails. Our key idea is to treat this handoff as a two-sided problem of recoverability: the character must disengage from the object it just placed so the prior success is preserved, and settle into a state from which a balanced continuation exists. Instead of engineering a transition by hand, we learn to shape where each cycle ends so that it lands in this recoverable region. We introduce LHM-Humanoid. One goal-conditioned controller completes a fetch--carry--place cycle and, through a learned release-and-retreat behavior, steers its terminal state into this region; a second controller then takes over from the resulting state distribution. Both are regularized by an adversarial motion prior and distilled into a single goal-conditioned policy that runs the whole sequence as one reset-free rollout. Across 350 cluttered layouts spanning four room types, LHM-Humanoid produces far more successful and stable long-horizon motion than end-to-end RL, hierarchical RL, and prior physics-based human-scene-interaction methods, on both seen and unseen scenes.

Haozhuo Zhang, Jingkai Sun, Michele Caprio +5

The University of Manchester, Manchester, UK · X-Humanoid, Beijing, China · University of Hong Kong, Hong Kong, China +1

Human Motion Generation Goal-Conditioned RL Human-Object Interaction

Ask PDF

Sep 29, 2026

Learning Expressive and Compositional Motion Representation via Spectral Skills

Robotic foundation models offer a promising path toward general-purpose humanoid robot control, often through hierarchical architectures. However, their effectiveness depends on the command interface between the planner and the controller, which must support accurate execution while remaining easy to predict, and ideally allow new behaviors to be composed from prior ones. In this work, we introduce spectral skills, a latent representation of this interface that meets these requirements through predictive representation learning. By design, spectral skills compactly encode short motion segments and are learned by predicting subsequent motion rather than reconstructing the encoder input. On a 29-DoF humanoid, a controller conditioned on spectral skills reduces global tracking error by 62% relative to the state of the art. The same frozen controller chains independently encoded skills without a separate transition policy. It also composes new behaviors by adding orthogonal directions to any compatible base skill, producing combinations unseen in the training data. We demonstrate tracking, chaining, and composition, as well as control through a language-conditioned planner, on Unitree G1 hardware. Project page: https://spectral-skill.github.io

Feiyang Wu, Chenxiao Gao, Chen Yang +3

Georgia Institute of Technology

Robot Skill Learning Humanoid Robot Control Predictive Representation Learning Motion Representation Learning Compositional Representation Learning

Ask PDF