Humanoid Hanoi: Investigating Shared Whole-Body Control for Skill-Based Box Rearrangement

Minku Kim; Kuan-Chia Chen; Aayam Shrestha; Li Fuxin; Stefan Lee; Alan Fern

cs.ROFeb 14, 2026

Humanoid Hanoi: Investigating Shared Whole-Body Control for Skill-Based Box Rearrangement

Authors: Minku Kim, Kuan-Chia Chen, Aayam Shrestha, Li Fuxin, Stefan Lee, Alan Fern

Organizations: The authors are with the Collaborative Robotics and Intelligent Systems Institute, Oregon State University, Corvallis, Oregon, 97331, USA

Abstract

We introduce Humanoid Hanoi, a challenge problem for long-horizon humanoid skill composition that requires a robot to repeatedly sequence pickup, locomotion, and placement skills to rearrange stacked boxes in simulation and on hardware. This symbolically simple task exposes a weakness in a common modular humanoid architecture, where reusable high-level skills are learned on top of a shared, task-agnostic whole-body controller (WBC). While such a WBC provides a unified low-level interface for skill composition, it may not be sufficiently hardened to execute the full collection of learned skills under domain randomization and long-horizon closed-loop execution. We compare strategies for addressing this issue, including skill-specific WBC fine-tuning, residual low-level adaptation, and task-driven data aggregation that expands the shared-WBC training distribution with closed-loop trajectories induced by learned skills while preserving its original tracking objective. In our evaluated system, the shared-WBC approach achieves the strongest long-horizon robustness. A direct transition-smoothing intervention further shows that low-level controller handoffs account for a substantial part of the composition loss observed with non-shared controllers. Project page: https://osudrl.github.io/Humanoid_Hanoi/

Figures & tables

Figure 1: A problem instance from Humanoid Hanoi . The robot moves boxes between three towers (T1–T3) while respecting stacking constraints. The panels illustrate representative stages of a successful hardware execution (5+ min), with the corresponding symbolic state shown on the bottom.

Figure 2: Long-horizon Humanoid Hanoi execution. Sequential snapshots show a complete Tower of Hanoi style box rearrangement episode. Transparent overlays visualize the executed robot trajectory over time, and T1 , T2 , and T3 denote the target tower locations. Each problem instance is divided into seven moves , each consisting of a sequence of GoTo , Pickup , GoTo-with-Box , and Place skills.

Figure 3: Independently trained high-level skills generate task-level commands that are executed through a shared, task-agnostic whole-body controller (WBC). The WBC produces joint-level PD targets that are tracked by a low-level PD controller on the robot. Closed-loop rollouts from composed execution are aggregated to retrain the shared controller, improving robustness while preserving a unified control interface. The base action ${a}_{t}^{\text{base}}$ specifies base locomotion commands, including a stand bit, planar velocity targets, and yaw rate.

Figure 4: Cumulative task success rates for Humanoid Hanoi. Each move consists of four skills: GoTo , Pickup , GoTo with Box , and Place , shown as small markers, with large markers indicating move completion. Drops between consecutive points indicate failures in the subsequent skill (e.g., a drop from Pickup to GoTo with Box indicates failures during the GoTo with Box execution). End-to-end results from [ 4 ] are omitted because they achieved only 1/100 complete-task success without DR and 0/100 with DR.

Figure 5: Top: Representative hip-yaw response around the transition into Place , Bottom: Box state changes after the transition into Place over 100 Humanoid Hanoi trials, comparing Extended and Finetune .

Figure 6: Three Humanoid Hanoi configurations (C1–C3) where T1 , T2 , and T3 denote the tower locations defining the stack positions.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Upper Body Tracking	Base	Extended
Upper joint MAE (median [IQR]) $\downarrow$	0.1246 [0.1763]	0.0805 [0.1535]
Success Rate (%) $\uparrow$	95	100

Appendix

Table 2: Upper-body tracking performance comparing the pretrained WBC ( Base ) and the WBC trained with data aggregation ( Extended ), on trajectories from the baseline training dataset.

Skill	Name	Reward ( $r$ )	Weight ( $w$ ) (up / down)	Scale ( $\alpha$ ) (up / down)
Pickup Policy	hand contact position	$\\|\mathbf{p}_{\text{hand}}^{\text{current}}-\mathbf{p}_{\text{elbow}}^{\text{target}}\\|_{2}$	0.5	–
	base pitch roll	$(\|\theta_{\text{base}}\|+\|\theta_{\text{base}}+0.15\|)$	0.2	15
	box rotation	$\|\theta_{\text{box}}\|+\|\phi_{\text{box}}\|\text{if}\hskip 3.00003pt\text{contact}_{\text{(L$ \wedge $R)}}$	0.1	10
	box acceleration	$\\|\mathbf{\dot{v}}_{\text{box}}\\|_{2}$	0.05	0.02
	table force	$\frac{F_{\text{table}}}{\text{mg}}\text{if}\hskip 3.00003pt\text{contact}_{\text{(L$ \wedge $R)}}$	0.05	2.0
	motor vel	$\text{mean}(\mathbf{v}_{\text{motor}}\odot W_{\text{weights}})$	0.05	0.4

Appendix

Table 3: Reward components grouped by skill. For values with two entries, the first value corresponds to the Pickup policy and the second to the Place policy. Terms without scale (

\alpha

) do not use the exponential kernel.

Skill	Name	Reward ( $r$ )	Weight ( $w$ )	Scale ( $\alpha$ )
GoTo Policy	constellation	$\sum_{i=1}^{9}\\|\mathbf{p}_{i}^{\text{current}}-\mathbf{p}_{i}^{\text{target}}\\|_{2}$	1.0	0.5
	base position error	$\\|\mathbf{p}^{\text{current}}_{\text{base}}\\|_{2}$	0.2	5.0
	base yaw error	$\|\psi_{\text{base}}\|$	0.2	3.0
	foot orientation	$d_{\text{quat}}(\mathbf{q}^{\text{current}}_{\text{foot}},\mathbf{q}^{\text{target}}_{\text{foot}})$	0.05	4.0
	stance width error	$\|\|y_{\text{L,foot}}-y_{\text{R,foot}}\|-0.33\|$	0.2	1.0
	stand parallel	$\|x_{\text{L,foot}}-x_{\text{R,foot}}\|$	0.2	1.0

Appendix

Table 4: Reward components for the GoTo policy and additional rewards for the GoTo-with-Box policy. Terms without scale (

\alpha

) do not use the exponential kernel.

	Parameters	Range
Dynamics Randomization	Body Mass	$[0.75,1.25]\times\text{Default}$
	Joint Damping	$[0.5,3.5]\times\text{Default}$
	Center of Mass Position	$[0.95,1.05]\times\text{Default}$
	Friction Coefficient	$[0.8,1.2]\times\text{Default}$
Box Randomization	Mass (kg)	$[0.0,3.5]$
	Size XYZ (m)	$[0.15,0.5]$

Appendix

Table 5: Domain randomization parameters.

Figure 7: Box state changes after the transition into Place over 400 Humanoid Hanoi trials for Finetune , comparing successful and failed placements.

Metric	Base	Finetune	Residual	Extended
No Dynamics Randomization ( $\mu\pm\sigma$ )
$x_{err}$ (cm) $\downarrow$	$4.71\pm 3.46$	$5.79\pm 4.10$	$5.35\pm 3.73$	$5.47\pm 3.60$
$y_{err}$ (cm) $\downarrow$	$5.26\pm 3.58$	$5.75\pm 4.27$	$5.24\pm 3.71$	$5.60\pm 3.69$
$\theta_{err}$ ( ∘ ) $\downarrow$	$2.89\pm 2.74$	$2.65\pm 2.34$	$2.66\pm 2.57$	$3.59\pm 3.96$
Dynamics Randomization ( $\mu\pm\sigma$ )
$x_{err}$ (cm) $\downarrow$	$5.36\pm 4.12$	$5.72\pm 3.95$	$5.87\pm 5.28$	$5.42\pm 3.82$

Appendix

Table 6: Humanoid Hanoi placement accuracy for successful trials. Final box position and orientation errors for the Place skill, reported as mean

\mu

and standard deviation

\sigma

Metric	Base	Finetune	Residual	Extended
No Dynamics Randomization, Common Successes ( $\mu\pm\sigma$ )
$x_{err}$ (cm) $\downarrow$	$4.27\pm 2.99$	$5.73\pm 4.32$	$5.38\pm 3.85$	$5.60\pm 3.79$
$y_{err}$ (cm) $\downarrow$	$5.63\pm 3.66$	$6.31\pm 4.78$	$5.99\pm 4.01$	$5.45\pm 3.78$
$\theta_{err}$ ( ∘ ) $\downarrow$	$2.78\pm 2.69$	$2.61\pm 2.20$	$2.97\pm 2.84$	$3.69\pm 4.12$
Dynamics Randomization, Common Successes ( $\mu\pm\sigma$ )
$x_{err}$ (cm) $\downarrow$	$5.86\pm 3.49$	$5.57\pm 3.04$	$7.64\pm 4.27$	$6.12\pm 3.75$

Appendix

Table 7: Placement accuracy on successful trials for all methods.

Figure 8: Failure counts across skills in Humanoid Hanoi for each baseline, shown with and without dynamics randomization.

Figure 9: Qualitative hardware comparison of consecutive low-height Pickup and Place executions with the Base WBC and Extended WBC. Frames 1–4 show Pickup ; frames 5–8 show Place .

Figure 10: Simplified two-tower hardware setup used to reduce yaw-localization demands. The robot successfully completes five consecutive three-box stacking trials in this setting, suggesting that yaw-localization is a significant bottleneck in Humanoid Hanoi .

Explore similar work

CHOREO: Every Humanoid Skill as a Trajectory
Sep 11, 2026Ziyi Sun, Jingwen Chen, Yuxi Wang +4Humanoid Robotics Humanoid Robot Control
College of Engineering, Ocean University of China · Faculty of Information Science and Engineering, Ocean University of China · School of Artificial Intelligence, University of Chinese Academy of Sciences +1
LHM-Humanoid: Long-Horizon Human Motion Control for Continuous Object Transport in Cluttered Scenes
Aug 23, 2025Haozhuo Zhang, Jingkai Sun, Michele Caprio +5Human Motion Generation Goal-Conditioned RL
The University of Manchester, Manchester, UK · X-Humanoid, Beijing, China · University of Hong Kong, Hong Kong, China +1
Learning Expressive and Compositional Motion Representation via Spectral Skills
Sep 29, 2026Feiyang Wu, Chenxiao Gao, Chen Yang +3Robot Skill Learning Humanoid Robot Control
Georgia Institute of Technology

Sep 11, 2026J/K move · Enter open · S save

CHOREO: Every Humanoid Skill as a Trajectory

Ziyi Sun, Jingwen Chen, Yuxi Wang, Xiuze Xia+3

College of Engineering, Ocean University of China · Faculty of Information Science and Engineering, Ocean University of China · School of Artificial Intelligence, University of Chinese Academy of Sciences+1

Humanoid Robotics Humanoid Robot Control Robot Motion Generation Robot Skill Learning

Open paper AskPDF arXiv

Recent advances in humanoid robotics have produced diverse skills through reinforcement learning, motion imitation, and generative modeling. Yet these capabilities remain siloed because they are built around incompatible representations, interfaces, and controllers. We present CHOREO, a framework for training-free composition of heterogeneous humanoid skills. Our key observation is that, regardless of how a skill is learned, it can ultimately be expressed as an executable motion trajectory. Based on this observation, CHOREO converts each capability into SkillMotion, a unified representation that combines motion states, contacts, semantics, and boundary conditions. Skills are composed through direct continuation, cubic Hermite blending, or validated bridge motions, without retraining source models or updating models at test time. On Unitree G1 in MuJoCo, CHOREO organizes 2,950 admitted SkillMotion assets derived from heterogeneous sources and achieves 95.4% sequence success across 130 multi-action tasks, including 93.8% success on eight-action sequences. These results demonstrate that executable trajectories provide a scalable interface for accumulating and composing pretrained humanoid capabilities.

Abstract

Explore similar work

CHOREO: Every Humanoid Skill as a Trajectory

LHM-Humanoid: Long-Horizon Human Motion Control for Continuous Object Transport in Cluttered Scenes

Learning Expressive and Compositional Motion Representation via Spectral Skills