Organizations: Physical Intelligence Lab, Tongji University, Shanghai, China · Shanghai Innovation Institute, Shanghai, China · AGIBOT, Shanghai, China · Technical University of Munich, Germany
Recent advances in large-scale human motion data, motion retargeting, and motion imitation have enabled increasingly capable whole-body control for individual humanoids. Cooperative carrying is more challenging because successful coordination requires not only plausible individual motions, but also consistent cross-agent and agent-object relations, including shared grasp, relative configuration, and whole-body support. We introduce CoCarry, a cooperative-carrying dataset containing 220 synchronized two-person sequences across 21 task schemes, five shared-object geometries, four terrain types, and approximately 61 minutes of motion. We further propose Contact-Support-Structured Retargeting, which starts from nominal individual motions and jointly refines both humanoids around the same prescribed object trajectory. Cooperation-aware object-local hand anchors preserve compatible four-hand grasp relations, while complementary upper object-interaction and lower support structures retain object-relative geometry and coordinated support. Across all 220 CoCarry sequences retargeted to two A3 humanoids, our method achieves 0.9995 contact preservation, reduces penetration duration by 67.0% and 71.0% relative to GMR and OmniRetarget, respectively, and maintains a 100% retargeting success rate. On CORE4D with Unitree G1 humanoids, it achieves 0.9906 contact preservation with zero measured sliding. We further validate the generated references through physics simulation and multiple real-world cooperative carrying experiments on two physical A3 humanoids, demonstrating their usability for downstream dual-humanoid control.
Figures & tables
Fig. 2: Distribution of the 220 CoCarry sequences across five cooperative-carrying task families. The table reports the number of task schemes and sequences in each family.
Fig. 3: Detailed pipeline of Contact–Support-Structured Retargeting. Starting from synchronized human motions from CoCarry, GMR provides nominal motions for both humanoids. Cooperation-aware hand anchors encode shared grasp relations, while complementary upper object-interaction and lower support structures guide joint refinement around the prescribed object trajectory.
Interaction
Clearance
Support
Output
Method
Contact preservation ↑
Penetration duration ↓
Penetration max. depth (m) ↓
Sliding duration ↓
Max. toe sliding velocity (m/s) ↓
Success rate (%) ↑
GMR
0.7633
0.5840
0.1055
0.1458
0.2978
100.00
OmniRetarget
0.9910
0.6640
0.0505
0.2047
0.1504
97.73
Ours w/o anchors
0.9994
0.6638
0.0501
0.2242
0.0603
100.00
Ours w/o decomposition
0.9993
0.2026
0.0453
0.3600
0.0632
99.55
Ours
0.9995
0.1925
0.0451
0.1913
0.0827
100.00
TABLE I: CoCarry-to-A3 retargeting results and component ablations. Bold denotes the best reported value, with ties retained.
Fig. 4: Condition-wise CoCarry-to-A3 results: (a) contact loss across task families, (b) sliding duration across terrain/support conditions, and (c) penetration duration across object geometries. Bars show means. Lower is better.
Fig. 5: Visual comparison across CoCarry-to-A3 and CORE4D-to-G1 settings. Rows correspond to OmniRetarget, Ours, and GMR, while columns show matched examples illustrating penetration, missing contact, body distortion, and incorrect hand orientation. Colored markers indicate penetration (red), missing contact (orange), misoriented contact (blue), and correctly preserved contact (green).
Method
Contact pres. ↑
Pen. duration ↓
Max. depth (m) ↓
Sliding duration ↓
Max. toe speed (m/s) ↓
GMR
0.6908
0.8386
0.0802
0.1519
0.0115
OmniRetarget
0.7337
0.0008
0.0004
0.0242
0.0019
Ours
0.9906
0.0167
0.0039
0.0000
0.0000
TABLE II: CORE4D-TO-G1 RETARGETING RESULTS.
Fig. 6: Representative downstream execution using two A3 humanoids. (a) Approach, lift, carry, and place keyframes from physics simulation with a free shared object. (b) Representative keyframes from real-world cooperative carrying experiments on two physical A3 humanoids.
Humans are dexterous, but traditional data collection paradigms such as teleoperation, kinesthetic teaching, and wearable exoskeletons force humans to act through the robot embodiment, bottlenecking their dexterity. Retargeting can avoid this. However, existing methods purely translate kinematics and ignore the real-world contact dynamics essential for task success. We present CARD: Contact-Anchored Retargeting for Dexterous Manipulation, a simulator and object-free framework that retargets human demonstrations to robot actions by explicitly preserving contact force targets. CARD unifies human data sources including video, motion capture, and tactile gloves via a shared hand--contact representation and generates explicit contact force targets in addition to kinematics. To maintain these contact targets during hardware open-loop replay, CARD also introduces a tactile tracking controller. For video which does not contain an explicit tactile source, we present Vid-HACO, our video-to-contact model that sets a new state of the art. Across our video, glove, motion-capture data and the GRAB dataset, CARD maintains contact on 43--78% of contact frames, while kinematic and object-aware retargeting methods maintain contact on <10% and 22.5% of the frames respectively. Further, integrating our formulation into physics-based retargeting improves strict task success from 12% to 32% on GRAB dataset. Finally, we show that our tactile tracker is critical to reproduce demonstrated forces during hardware replay. These results establish that contact preservation via CARD is a scalable interface for transferring human dexterity to robots.
Cunxi Dai, Botao He, Francois R Hogan +7
Carnegie Mellon University · Amazon Frontier AI & Robotics (FAR) · University of Maryland, College Park +1
Physical interaction is a key capability for humanoid collaboration, requiring coordinated whole-body motion and precise contact. However, learning diverse dual-humanoid interactions faces two major challenges: (1) human-robot morphological differences disrupt contact interfaces and spatial relationships during retargeting, and (2) jointly learning motor skills and partner coordination makes it difficult to retain control fidelity as interaction diversity grows. In this paper, we propose OmniInteract to address these challenges through surface-aware modeling and scalable residual policy learning. First, we introduce Surface Contact-Aware Retargeting, which leverages dense human-robot surface correspondences to jointly retarget both humanoids, preserving explicit interaction contact interfaces and their near-field geometric context. Second, we propose a Scalable Flow-based Interaction Policy for partner-conditioned residual generation. It factorizes dual-humanoid control into a reusable motor foundation and interaction-specific adaptations: partner-aware experts learn corrections over a frozen whole-body motion prior, while conditional flow matching unifies them in a shared residual action space. This enables scalable skill consolidation while retaining pretrained motor capabilities. Extensive experiments demonstrate that OmniInteract improves retargeting fidelity and dual-humanoid coordination while better retaining interaction performance as skill diversity grows. Real-world deployment further demonstrates successful contact establishment, stable whole-body coordination, and recovery from external perturbations.
Yinghao Li, Tianyu Zhang, Zhuohao Li +4
School of Computer Science and Engineering, Sun Yat-sen University · Shenzhen Loop Area Institute · Beijing Institute of Technology +1
Controllable cooperative humanoid manipulation is a fundamental yet challenging problem for embodied intelligence, due to severe data scarcity, complexities in multi-agent coordination, and limited generalization across objects. In this paper, we present SynAgent, a unified framework that enables scalable and physically plausible cooperative manipulation by leveraging Solo-to-Cooperative Agent Synergy to transfer skills from single-agent human-object interaction to multi-agent human-object-human scenarios. To maintain semantic integrity during motion transfer, we introduce an interaction-preserving retargeting method based on an Interact Mesh constructed via Delaunay tetrahedralization, which faithfully maintains spatial relationships among humans and objects. Building upon this refined data, we propose a single-agent pretraining and adaptation paradigm that bootstraps synergistic collaborative behaviors from abundant single-human data through decentralized training and multi-agent PPO. Finally, we develop a trajectory-conditioned generative policy using a conditional VAE, trained via multi-teacher distillation from motion imitation priors to achieve stable and controllable object-level trajectory execution. Extensive experiments demonstrate that SynAgent significantly outperforms existing baselines in both cooperative imitation and trajectory-conditioned control, while generalizing across diverse object geometries. Codes and data will be available after publication. Project Page: http://yw0208.github.io/synagent
Wei Yao, Haohan Ma, Hongwen Zhang +6
School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing 210094, China · Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China · School of Artificial Intelligence, Beijing Normal University, Beijing 100875, China +2