Full-stack human-robot collaboration (HRC) can become brittle when replacing a planner, partner model, coordination policy, or controller changes the physical meaning of cross-layer signals. We introduce C2C, an object-centric cognition-to-control architecture that preserves these meanings through physical contracts. Rather than standardizing individual modules, C2C standardizes the physical semantics exchanged between them: task intent is represented by a geometrically verified payload path, partner information by a source-independent physical state, learned coordination by bounded 11-D task-space commands, and robot-specific feasibility remains inside whole-body control (WBC). The reference system combines a vision-language model with deterministic geometric verification and multi-agent reinforcement learning (MARL) for partner-aware coordination. Across nine transport scenarios, adaptive MARL variants achieve 77.1-82.1% mean success under unchanged interfaces, compared with 56.5% for a scripted-partner reference. The same contracts further support neural and interpretable hard-tree actors, two- and three-carrier teams, and simulated-to-human partner substitution. Physical Unitree G1-human tests achieve 100% success in spatially confined transport and 80% in super-long-object handling, while a three-carrier system with two G1 humanoids and one human validates the same interface structure on hardware. Together, these results show that C2C converts a tightly coupled HRC stack into a plug-compatible system in which key components can change without redefining the physical collaboration task.
Figures & tables
Fig. 1: Cognition-to-control (C2C) resolves interface entanglement around the shared payload. Cognition outputs a verified object path, partner sensing is mapped to a source-independent physical state, coordination outputs bounded task-space adaptation, and robot-specific feasibility remains inside whole-body control. A module is plug compatible when it can change without changing these interface semantics.
Fig. 2: Reference realization of C2C. The two VLM views produce candidate paths for the same payload, not separate robot paths. A deterministic verifier fuses them into one object reference Pt . The coordination policy receives this reference together with self, partner, object, contact, and range states and outputs the same bounded 11-D task-space command ui,t for every tested policy. The execution layer converts that command to constrained joint torque. During hardware deployment, measured human motion fills the same partner-state fields used by the simulated teammate during training.
Component
Size
Description
Observation space ( o∈R210 )
Global guidance
10
5 anchors (relative planar)
Self state
13
Pose/velocity, CoM, torso, wrists
Partner state
13
Measured/policy partner
Object geometry
18
4 corners, CoM pose/velocity
Contact
4
4 binary contact indicators
TABLE I: Default two-carrier policy input and output.
Fig. 3: (a) Episode return during training for IPPO with a scripted partner, HAPPO, HATRPO, and PPO+PCGrad over 2.0×109 steps, with shading denoting one standard deviation across five seeds. (b) Mean success rate by task category, comparing the robot-script baseline with the same learning-based tactical variants. (c) Physical deployment: success rate, completion time (s), and mean object tilt rate ( ∘ /s) for robot-script IPPO and PPO+PCGrad.
Category
Scenario
Robot-script
HAPPO
HATRPO
PPO+PCGrad
Mean gain Δ
OSP
S11 : Alignment
65.4 ± 7.2
83.6 ± 5.3
87.9 ± 4.5
87.7 ± 3.8
+32.1%
S12 : Turnaround
60.1 ± 8.5
77.6 ± 6.3
81.5 ± 5.5
83.6 ± 4.8
+34.6%
S13 : Corner entry
58.2 ± 9.1
72.7 ± 7.0
75.3 ± 6.0
77.8 ± 5.3
+29.3%
SCT
S21 : Narrow gate
59.2 ± 9.0
80.1 ± 4.8
83.4 ± 4.3
88.6 ± 3.5
+42.0%
S22 : S-shaped path
57.5 ± 8.8
76.3 ± 5.8
82.1 ± 5.0
80.6 ± 4.5
+38.6%
S23 : U-shaped path
55.6 ± 9.5
75.7 ± 6.5
79.2 ± 5.3
81.3 ± 4.8
+41.6%
TABLE II: System-level success rate under the shared C2C interfaces across nine scenarios (mean ± standard deviation over five seeds, 100 episodes per seed). Δ : relative difference of the mean adaptive-MARL success rate from the robot-script reference.
Parameter
Value
Parameter
Value
Learning rate
10−4
Hidden layers
[256,256,128]
Epochs/update
10
Activation
ReLU
Minibatches
16
Initialization
Orthogonal
Entropy weight
0.01
Optimizer
Adam
Discount factor
0.99
Weight decay
10−4
GAE factor
0.95
Gradient clip
10.0
TABLE III: Shared training settings.
Fig. 4: VLM guidance and system stress tasks. Panels (a)–(c) show two carrier-centered views. Blue and orange traces are payload-path proposals from the VLM adapter. Deterministic common-frame collision and clearance checks select the verified path, with the active anchor shown in green and the 36-D range observations in cyan. Panels (d)–(f) show the OSP, SCT, and SLH task families used under the same C2C contracts.
Axis
Substitution
Invariant interface
Evidence
Cognition
VLM → deterministic fallback
Verified payload path Pt
Same downstream path contract
Coordination
HAPPO / HATRPO / PPO+PCGrad
Observation, 11-D action, reward, WBC
SR: 77.1 / 80.8 / 82.1%
Partner
Simulated carrier → human
Partner-state fields and robot action
Physical SR: 100 / 100 / 80%
Policy
MLP → hard tree
N=3 adapter, action, reward, WBC
SR: 74.7 → 70.3%
Team size
Two → three carriers
Guidance, action semantics, reward, WBC
SR: 77.1 → 74.7%
TABLE IV: Summary of contract-preserving substitutions under the C2C interfaces.
Fig. 6: Task-family success rate for two-carrier HAPPO-MLP, three-carrier HAPPO-MLP, and three-carrier HAPPO-Tree. Parentheses give carrier count.
Fig. 7: Physical human–multiple robots collaborative transport with two Unitree G1 humanoids and one human: (a) payload reorientation and (b) passage traversal around laboratory partitions, shown at 20 s intervals.
Task
Robot-script IPPO
PPO+PCGrad
SR
Tilt ( ∘ /s)
Time (s)
SR
Tilt ( ∘ /s)
Time (s)
OSP
100%
–
74.2
100%
–
65.2
SCT
40%
3.2
101.6
100%
2.4
81.5
SLH
40%
3.6
113.6
80%
2.7
86.9
TABLE V: Physical results over five trials per task.
We focus on human-robot collaborative transport, a challenging task of broad relevance spanning logistics, manufacturing, and the home, in which a user and a robot work together to relocate a large or heavy object. To act as an effective partner, the robot should reduce the user's effort by contributing to efficient relocation of the object while remaining physically responsive to them. Prior work often addresses these capabilities separately, producing robots that may move the object efficiently but resist user input, or accommodate the user but depend on continuous guidance. Our key insight is that obstacle-constrained collaborative transport requires integrating predictions of human collaborative behavior with compliant robot control. To this end, we introduce PROACT, a framework for human-robot collaborative transport that incorporates anticipation into compliant whole-body control through a learned model of human collaborative behavior. Trained on a large-scale, real-world dataset of dyadic human transport demonstrations, our transformer architecture distills collaborative behavior into predictions of future object motion. Across 108 real-world trials with a 9-DoF mobile manipulator, PROACT reduces mean interaction work by 59.2% and 20.4%, and mean completion time by 12.9% and 6.9%, relative to compliance-only and MPC baselines, respectively. Footage from our experiments can be found at https://youtu.be/qAGvQfVPjbk.
Elvin Yang, Christoforos Mavrogiannis
Department of Robotics, University of Michigan, Ann Arbor, United States
We study cooperative multi-humanoid pickup and transport of objects with varying size, weight, and geometry, requiring robot teams of different sizes. Our approach uses decentralized object-centric control, where each humanoid is assigned a local attachment region on the shared object and learns to realize pickup and transport through gripperless bimanual pinching. This attachment-based interface provides a common control abstraction spanning single-robot pickup, cooperative multi-robot transport, and robot-to-robot handover, without per-task redesign. We find that policies trained only on single-robot pickup already transfer nontrivially to cooperative settings, suggesting that this abstraction captures much of the structure needed for coordination. At the same time, explicit multi-robot training further improves performance, showing that shared-object coupling introduces coordination dynamics that are beneficial to learn directly. We validate the approach in simulation across varying team sizes and object geometries, and demonstrate sim-to-real transfer on hardware, where the learned controllers enable real humanoids to perform cooperative manipulation tasks.
Bikram Pandit, Mohitvishnu S. Gadde, Aayam Kumar Shrestha +1
Dynamics Robotics and AI Lab (DRAIL), Oregon State University, Corvallis, OR, USA
In human-robot collaboration (HRC), robots must adapt online to dynamic task constraints and evolving human intent. While physical corrections provide a natural, low-latency channel for operators to convey motion-level adjustments, extracting task-level semantic intent from such brief interactions remains challenging. Existing foundation-model-based approaches primarily rely on vision and language inputs and lack mechanisms to interpret physical feedback. Meanwhile, traditional physical human-robot interaction (pHRI) methods leverage physical corrections for trajectory guidance but struggle to infer task-level semantics. To bridge this gap, we propose TATIC, a unified framework that utilizes torque-based contact force estimation and a task-aware Temporal Convolutional Network (TCN) to jointly infer discrete task-level intent and estimate continuous motion-level parameters from brief physical corrections. Task-aligned feature canonicalization ensures robust generalization across diverse layouts, while an intent-driven adaptation scheme translates inferred human intent into robot motion adaptations. Experiments achieve a 0.904 Macro-F1 score in intent recognition and demonstrate successful hardware validation in collaborative disassembly (see experimental video at https://youtu.be/xF8A52qwEc8).
Jiurun Song, Xiao Liang, Minghui Zheng
Department of Mechanical Engineering, Texas A&M University, College Station, TX 77843, USA · Department of Civil and Environmental Engineering, Texas A&M University, College Station, TX 77843, USA