Hierarchical pipelines for humanoid loco-manipulation typically pair a task-agnostic whole-body controller (WBC) with a downstream task policy. Such WBCs are often trained through manual reward shaping, which yields visibly machine-like motions; more consequentially, they are trained without the manipulated object in the scene. Object dynamics are instead approximated through randomized end-effector force disturbances, a proxy that fails to reproduce the distributed multi-contact wrench a real payload applies across the arms and torso, with the gap becoming critical near the platform's payload limit. In this work, we train the low-level (LL) controller itself as an object-aware motion prior: a single policy trained via motion imitation on retargeted motion-capture (MoCap) references that span the core motion primitives of the downstream task, with the manipulated object physically present in the scene. This single stage inherits natural human-like coordination from the references and absorbs the realistic contact wrench at its source, while exposing the same compact task-space command interface used by command-tracking WBCs. A reference-free high-level (HL) policy is then trained on top, with its action channel anchored to the empirical envelope of the low-level training data so that it generalizes the priors to a long-horizon multi-stage task while structurally preventing queries that would push the low level off-distribution. We instantiate the framework on a Unitree G1 for bimanual box pick-and-place: the same deployed policy generalizes across object masses and dimensions, initial and goal poses, and reactively recovers from intentional drops mid-carry. Video: https://youtu.be/viR_ULnRDQQ.
Figures & tables
Figure 1: System Overview. Retargeted MoCap data is used to build a compact motion library (locomotion, walking-with-box, pickup, placement). A low-level motion-imitation policy πLL is then trained as an object-in-scene prior, conditioned on a noisy task-space command. The frozen πLL is reused by a reference-free high-level policy πHL observing proprioception, box face centers, and a sparse goal, producing commands within the LL dataset envelope and stitching the learned priors into the full behavior for zero-shot deployment.
Figure 2: Per-joint torque during the 7 kg experiment, normalized by each joint’s effort limit (only discussed joints labeled; over-limit in bold red, dashed line marks 100 %). Snapshots: lift ( t≈9 s) and carry ( t≈11 s).
Table 3
Variant
Succ. (all)
Succ. (light)
Succ. (heavy)
Phase 1→2
OOD freq.
N&S idx
Ours (full)
80.7
85.8
70.5
91.7
0.15
1.00
w/o observation noise
3.0
2.7
3.6
38.6
0.45
2.05
w/o object (no EE surrogate)
2.9
3.1
2.5
3.1
0.25
1.23
w/o object (EE force surrogate)
6.7
8.3
3.6
27.0
0.24
1.27
w/o targeted HL RSI
6.5
6.6
6.4
17.5
0.19
1.05
w/o dataset-envelope clipping
15.5
15.6
15.2
17.1
1.00
2.65
Table 3: Ablation study (20 s). Phase 1→2 : % of all episodes that complete the lift phase. OOD freq.: fraction of steps on which the pre-clip HL command lies outside the LL’s training envelope. N&S idx as in Sec. 4.2 .
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Variant
Dimensions (L×W×H)
#
Source
0.32×0.32×0.32
1
Large box
permutations of (0.39,0.42,0.54)
6
Medium box
permutations of (0.32,0.32,0.46)
3
Variants per behaviour
10
Total demonstrations
30
Appendix
Table 4: Object-geometry augmentation. Dimensions in meters; each box is instantiated for all three behaviours ( carry , pick , place ).
Term
Dim
Source
Noise (Gaussian std)
Box face centers (torso frame, permute)
18
Mocap
0.10 m (clipped to [−3,3] )
IMU base angular velocity
3
IMU @ torso
0.10rad/s
Projected gravity g^b (torso frame)
3
IMU @ torso
0.015
Joint positions
nq
Encoders
0.005rad
Joint velocities
nq
Encoders
0.25rad/s
Last HL action at−1HL
12
Internal
—
Appendix
Table 5: HL actor (deployment) observations. Every term is available from on-board sensing or the mocap stream. “Permute” denotes that the six face-center entries are randomly permuted at every environment reset (see Sec. 3.4 ).
Category
Term
Ground-truth kinematics
Robot/object root position
Robot/object linear & angular velocity
Robot/object projected gravity
Robot key-point body positions/velocities
Closest box point (torso frame)
Randomized parameters
Robot/object per-body mass
Appendix
Table 6: HL critic (training-only, privileged) observations added on top of the actor observations of Table 5 . All terms are read directly from the simulator, are noise-free, and are never accessed at deployment.
Feedforward baseline
Survival (%)
Tracking similarity (%)
Curriculum βt:0→1 (ours)
82
89
Fixed default joint positions
72
83
Fixed measured joint positions
73
85
Appendix
Table 7: Residual-removal curriculum on LL training. Survival: fraction of environments not terminated at episode end. Tracking similarity: motion-imitation similarity score between the policy rollout and the retargeted reference. All variants trained for the same number of iterations.
st
Phase
Goal
Transition trigger
0
walk-to-box
approach the box
∥pb−poface∥xy<0.50m and ∣θe∣<0.40
1
lift-box
bilateral grasp & lift
c and lowest box-contact height ≥0.65m
2
carry-to-goal
carry box to g
∥po−g∥xy<0.50m and ∣θe∣<0.40
3
place-box
lower the box onto g
box xy within 0.50m and lowest-contact height within 0.15m of z⋆
Appendix
Table 8: Four-phase task scheduler (active phases). poface denotes the closest of the six box face centers to the robot torso, and po denotes the box center; θe denotes the heading error from the robot torso to the box (phase 0) or to the goal (phase 2); c denotes the binary event “both wrists in contact with the box.” Phase 2 additionally has a drop edge back to phase 0 (carry-to-goal → walk-to-box) when the lowest box-contact height falls below zdrop or c is lost. When the placement predicate fires the scheduler enters an absorbing idle index (omitted from the table). Numerical values are the defaults used in our experiments.
Term
Weight
Description
Bounded staged reward Rst
+10.0
Phase-conditional task reward, Eqs. ( 5 )–( 9 )
Alive bonus
+5.0
Constant per-step bonus discouraging early termination
Action clip penalty
−10.0
Penalty on pre-clipped HL actions that violate command limits
HL action smoothness
−0.10
Penalty on ∥atHL−at−1HL∥2
LL action smoothness
−0.25
Penalty on the realized LL action rate
Joint torques
−5×10−4
Penalty on squared joint-torque magnitudes
Appendix
Table 9: HL reward weights used during training. Positive weights reward desired behavior; negative weights penalize deviations.
Hyperparameter
Value
Actor network
MLP [512,256,128] with ELU activation
Critic network
MLP [512,512,256] with ELU activation
Parallel environments Nenv
8192
Steps per env per update T
24
Max iterations
30,000
Discount γ
0.99
Appendix
Table 10: PPO hyperparameters for the LL and HL policies.
Quantity
Range
Mode
Robot static friction
[0.6,1.2]
startup
Robot dynamic friction
[0.5,1.1]
startup
Robot restitution
[0.0,0.4]
startup
Robot per-body mass scale
[0.9,1.1]
startup
Robot torso mass scale
[0.8,1.2]
startup
Actuator stiffness scale
[0.8,1.2]
startup
Appendix
Table 11: Domain randomization used during HL training. The LL stage uses the same ranges except that (i) no source/target table is present, (ii) no goal is sampled, and (iii) the LL always uses RSI for initialization. Distributions are uniform over the indicated intervals unless stated otherwise. Rows marked ( † ) apply only on resets where RSI is not active.
Setting
Reached box
Lifted box
Reached goal
Placed (succ.)
( 0→1 )
( 1→2 )
( 2→3 )
( 3→4 )
Ours
97.1
91.7
82.1
80.7
Ours, 40 s
96.9
92.3
88.0
87.9
WBC (no object)
96.8
85.0
53.5
50.3
WBC (force surrogate)
98.2
91.7
77.3
77.1
Appendix
Table 12: Per-phase completion : % of all episodes that pass each scheduler transition (Appendix D ) at least once (20 s episode length unless noted). The last column equals task success, so the drop between consecutive columns gives the failures in each phase.
Figure 3: Robustness Study (20 s). Task-success ratio (Ours nominal =1 ) under 3 perturbation axes (box mass, external pushes, box dimensions) scaled relative to the training distribution.