Contact transitions allow humanoid robots to redistribute object support across the whole body, enabling manipulation beyond hand-only grasps. Human videos provide a scalable source of such transitions for robots to learn from. However, body occlusion disrupts object pose tracking and corrupts retargeted motion. Moreover, accurately reproducing the object trajectory does not ensure that the robot applies stabilizing forces at the intended contacts, risking slips or drops in later phases. To address these challenges, we propose ReContact, which reconstructs and retargets human-object interactions through an object-centric contact plan. The plan specifies contact phases, contact pairs, anchors, pressing directions, and sticking or sliding modes. Contact priors stabilize object pose reconstruction through occlusion. A residual policy over a frozen whole-body tracker is rewarded for making contact at planned anchors with forces aligned to the specified directions. In simulation, on a thigh-supported regrasp task, ReContact achieves the planned two-palm bottom hold in 83.5 of episodes over three training seeds, versus 0.16% for the best adapted baseline. One policy trained on 247 references re-solved for new placements raises in-workspace tube-transfer success from 20.4% (source reference only) to 97.8%. On a real Unitree G1, policies execute seven contact-rich manipulation and two loco-manipulation tasks.
Figures & tables
Fig. 1: Overview of ReContact. (a) An exocentric video supplies body and object motion; Fig. 2 shows the egocentric route. (b) GPT-6 Astra reviews sparse keyframes against the video, and contact-constrained whole-body optimization turns them into humanoid–object references. (c) The object-centric contact plan, here for the thigh-supported regrasp, lists, per phase, the paired robot–object surfaces with their contact points, contact directions and sticking or sliding modes; adding or removing a pair gives a contact transition. Keyframes of the run shown: contact keyframes where support changes ( ⧫ ) and motion keyframes inside phases ( ◊ ). An objective that does not specify the pressing direction also accepts a squeeze near the lower side edges ( × ). (d) The plan defines the rewards of a residual controller for contact at the planned point and along the planned direction. (e) Re-solving the placement-dependent contact segment and its approach at new object placements gives a task family for one policy.
Method
Data
Contact representation
Body support
Direction
Stick/ slide
Reference generation
HDMI [ 21 ]
Video, MoCap
Hand and foot points in the object frame; per-frame contact flag
Palm anchors in the object frame, fixed within a phase
–
–
–
✓
VICAR [ 20 ]
Video
Annotated hand and foot poses in the world frame
–
–
–
✓
CHORD [ 5 ]
Video, MoCap
Hand contact points and normals as an object-centric wrench space
–
✓
–
–
OmniRetarget [ 19 ]
MoCap
Interaction mesh over body joints, object and terrain points
–
–
–
✓
TABLE I: Comparison of Humanoid Manipulation Methods and How They Specify Contacts and Whether They Generate References
Fig. 2: Egocentric route, shown for a spray bottle and a banana. Recovered hand–object segments are retargeted to the Inspire RH56F1 hand; planning generates the approach and the remaining G1 body motion. Blue paths mark the approach, orange hands the grasp targets. Recovered views use the source camera and crop; the last column shows the composed reference at the same source frame.
Fig. 3: Evaluated tasks in simulation; arrows give the temporal order. T5 shows one policy trained on 247 generated references grasping the tube at four of their positions (tube xy in the world frame).
Method
Complete
SR
Thigh
Bottom L
Bottom R
Normal ↓
Eo↓
Mech. work ↓
(%)
(%) ↑
(%)
(%)
(%)
(deg)
(cm)
(J)
HDMI-style
24.3±20.0
0.08±0.07
0.2±0.3
10.6±16.1
3.5±4.2
47.8±4.6
11.24±1.06
565.2±90.1
ResMimic-style
65.7±5.8
0.01±0.02
20.6±5.6
0.3±0.4
4.5±7.1
55.0±3.0
9.47±1.29
868.0±21.1
CWS adapter
52.3±9.2
0.16±0.24
36.6±25.1
0.6±0.5
65.7±56.2
49.2±2.8
9.25±2.08
787.1±59.9
Ours
89.0±8.6
83.52±5.20
94.4±2.8
80.8±1.8
81.2±7.6
16.3±4.5
5.54±0.64
544.9±20.2
TABLE II: Shared-Reference Controller Comparison in Simulation
Fig. 4: Only ReContact moves the T2 box from thigh support to a two-palm bottom hold. Vertical contact force (normal plus friction) exerted on the box by the right thigh and each palm, divided by the box weight mg (dashed line), in 48 episodes per controller. Lines are medians and bands interquartile ranges over the episodes alive at each reference time ( 0.1 s moving median within each episode); the axis is logarithmic beyond ±mg . Shading marks the palm slide and the terminal bottom hold. Panel titles give two-palm successes.
Fig. 5: The planned contact transition persists across payload variation. (a) Completion of ReContact and the ResMimic-style, HDMI-style and CWS controllers at eight payloads, 3000 episodes per method and mass; shading marks the mass randomization used in training ( 0.074 – 0.135 kg around the 0.1 kg nominal mass). (b) Completed episodes pooled over the payloads, split by the bottom-hold criterion: met by both palms (the T2 success predicate), by one palm, or by neither (side squeeze). Images: terminal holds of completed 0.5 kg episodes (ReContact, CWS, ResMimic-style).
Fig. 6: Contact preservation in retargeted humanoid–object references. (a) Comparison of retargeting methods on a medium-box carry using shared body motion and object tracks. (b) Comparison of object tracks for the tissue-pack transfer and thigh-supported regrasp using the same ReContact retargeting procedure.
Fig. 7: Workspace coverage from one reference (T5 tube-transfer family). Top: success of the task-family policy, smoothed with a 1.5 cm Gaussian kernel, inside the generated workspace (outline) and in the 6 cm band outside it; white points mark the 247 references and the star the source reference’s tube position. Bottom: per-position outcomes of ReContact policies of the same configuration trained on the source reference (local, ±2.5 cm; workspace-wide; no start randomization) and on the family.
References
Start
0 – 10
10 – 20
20 – 30
>30
All
Ext.
Source
None
11.8
0.0
0.0
0.0
1.5
0.1
Source
±2.5 cm
95.5
25.3
0.1
0.0
20.4
8.4
Source
Workspace
12.2
0.0
0.0
0.0
1.6
0.3
Family ( 247 )
±2.5 cm
99.3
98.0
96.6
98.6
97.8
19.5
TABLE III: T5 Tube-Transfer Success (%) of Single-Reference Controls and the Family Policy by Distance (cm) From the Source Tube Position
Fig. 8: Hardware deployment of learned contact skills on a Unitree G1. (a) Walk-then-grasp using locomotion and the bottle-family policy. (b) Table-to-floor box transfer combining forward and reverse thigh-supported regrasps with loaded walking.
Fig. 9: Final hardware states of seven contact-rich manipulation tasks on a Unitree G1.
Contact-rich robotic manipulation studies how robots plan and control physical interactions such as grasping, insertion, pushing, and dexterous object handling.
Current humanoid reinforcement-learning policies excel at free-space motions but struggle with contact-rich tasks, as pure kinematic tracking cannot resolve the physical ambiguities of interacting with objects and uneven terrain. To address this, we introduce SceneBot, a unified motion-tracking framework capable of handling freespace locomotion, terrain traversal, and whole-body manipulation. SceneBot conditions a single policy on both reference motions and per-link contact labels, explicitly defining expected environmental interactions. To overcome the lack of annotated interaction data, we propose a hindsight scene reconstruction approach that infers scene-interaction graphs from retargeted human motion. Trained on 7.5 hours of this reconstructed, contact-rich data, SceneBot successfully generalizes to unseen motions and environments. Our results demonstrate that SceneBot is the first general framework to seamlessly unify free-space and contact-rich behaviors executing complex, long-horizon tasks like carrying a box upstairs and establishing contact conditioning as a powerful interface for humanoid control. All code and data will be open-sourced. More demos and information are available at: https://ericcsr.github.io/scenebot/
Sirui Chen, Shibo Zhao, Zhen Wu +3
Stanford, Amazon FAR United States · Amazon FAR United States · CMU, Amazon FAR United States
Keypoint tracking alone is insufficient for object interaction tasks such as sitting on a chair, wiping a board, or pushing furniture, where the robot can reach the correct pose without making meaningful physical contact with the object. We present CONTACTMIMIC, a learning framework that tracks explicit partlevel binary contact commands alongside keypoint trajectories. CONTACTMIMIC is made possible through the use of contact-following rewards and a trajectory augmentation scheme aimed at breaking the correlations between keypoint trajectories and contact labels. The resulting policy successfully decouples contact behavior from keypoint geometry, and achieves precise physical contact as well as contact-controllability (produce or suppress contact during deployment as desired). Simulation experiments across 10 diverse human-object interaction motions confirm that CONTACTMIMIC exhibits contact controllability that enables it to complete manipulation tasks without task-specific rewards, while also outperforming keypoint-only trackers on contact-relevant tasks. Ablations confirm the necessity of the proposed trajectory augmentation scheme and sim2real deployment validates contact controllability in the real world across 5 different motions. Video results are available on https://lixinyao11.github.io/contactmimic-page/.
Learning from demonstration (LfD) has enabled humanoid robots to acquire diverse whole-body skills, but extending this paradigm to human-object interaction (HOI) is limited by the availability of robot-compatible interaction references. We present HOI-Retarget, a contact-centric retargeting method that transfers HOI onto a humanoid robot for large-scale motion-data generation. Its windowed trajectory optimization uses every labeled contact as a target in the object frame, balancing body tracking, foot support and smoothness under the robot's kinematic limits. The method can augment a single demonstration across object sizes, absorb contacts reconstructed from monocular video, and extend to several robots manipulating one object. We publicly release the code and the retargeted motion dataset.
Jihwan Shin, Adrià López Escoriza, Junzhe He +2
Robotic Systems Lab, ETH Zürich, 8092 Zürich, Switzerland