We present a decentralized, agent agnostic, and safety-aware control framework for human-robot collaboration based on Virtual Model Control (VMC). In our approach, both humans and robots are embedded in the same virtual-component-shaped workspace, where motion is the result of the interaction with virtual springs and dampers rather than explicit trajectory planning. A decentralized, force-based stall detector identifies deadlocks, which are resolved through negotiation. This reduces the probability of robots getting stuck in the block placement task from up to 61.2% to zero in our experiments. The framework scales without structural changes thanks to the distributed implementation: in experiments we demonstrate safe collaboration with up to two robots and two humans, and in simulation up to four robots, maintaining inter-agent separation at around 20 cm. Results show that the method shapes robot behavior intuitively by adjusting control parameters and achieves deadlock-free operation across team sizes in all tested scenarios.
Figures & tables
Fig. 1 : Human-robot collaboration in a pick and place task.
Fig. 3 : Multi-robot simulation. Robot 1 and 2 are for two-robot experiment. Robot 1-3 are for three-robot experiment.
Fig. 4 : Robot 1 avoiding Robot 2 (left) and a human (right) with the same VMC avoidance spring. In both cases, Robot 1 retreats as the other agent approaches, then returns.
Spring profile
dmin (cm)
T<Sp (s)
Spring profile 1 (baseline)
21.8±2.0
4.6±1.8
Spring profile 2 (increased fmax )
24.4±4.7
2.3±1.1
Spring profile 3 (increased σ )
24.8±9.1
0.9±0.9
Spring profile 4 (both increased)
30.0±9.8
0.5±0.5
Spring profile 1 and damper
19.9±2.9
4.1±0.5
TABLE I : Minimum separation dminRH and violation duration T<Sp under different avoidance profiles and mechanisms.
Fig. 5 : Two UR5 robots transport cubic blocks (black squares) toward their target cells (dashed arrows). The intersection of trajectories represents the potential conflict region.
Fig. 6 : Top: without conflict resolution layer, robots entered a deadlock due to force balance at t=25 s and did not recover. Bottom: with the conflict resolution, robots resolved the conflict by prioritizing robot 1 at 10 s. Top-Left: robot trajectories (dashed - desired EE position; solid - EE position). Top-Right: balancing forces at t=150s , indicating a stall; fgoal goal-reaching force, favoid cumulative avoidance force, f=fgoal+favoid . Bottom-Left: robot trajectories. Blue - prioritized interval, Red - non-prioritized interval. Bottom-Right: top - stall detected at t=10.87s , bottom - stall resolved at t=10.95s
Fig. 7 : Completion time, safety and interaction metrics across scenarios and team sizes.
TABLE II : Control parameters for each experiment.
Multi-robot collaboration allows robots to efficiently take on a wide range of tasks, from moving a couch through a doorway to assembling structures on a construction site. However, achieving such coordination in mobile multi-robot settings remains challenging: centralized methods conditioned on the combined observations of a team scale poorly with team size, and decentralized methods that train one policy per robot often require explicit alignment procedures or information sharing at inference time to overcome partial observability. Our key insight is that the visuomotor priors of pretrained vision-language-action (VLA) models should enable reactive, decentralized collaboration from each robot's local observations alone, without these inference-time assumptions. We propose CHORUS, a framework that adapts a single VLA backbone to control diverse, multi-robot teams. At inference time, each robot runs an independent copy of CHORUS, conditioned only on its own observations and a robot-identifying prompt. In real-world experiments including mobile tape measurement, library book handovers, and laundry basket lifting, CHORUS achieves a 64% point improvement over decentralized, from-scratch models, improves reactivity to teammate behavior by 40% points, and outperforms centralized baselines. Together, these results show that a shared VLA backbone is capable of achieving decentralized multi-robot collaboration, without per-robot policies or inter-robot communication at inference.
Full-stack human-robot collaboration (HRC) can become brittle when replacing a planner, partner model, coordination policy, or controller changes the physical meaning of cross-layer signals. We introduce C2C, an object-centric cognition-to-control architecture that preserves these meanings through physical contracts. Rather than standardizing individual modules, C2C standardizes the physical semantics exchanged between them: task intent is represented by a geometrically verified payload path, partner information by a source-independent physical state, learned coordination by bounded 11-D task-space commands, and robot-specific feasibility remains inside whole-body control (WBC). The reference system combines a vision-language model with deterministic geometric verification and multi-agent reinforcement learning (MARL) for partner-aware coordination. Across nine transport scenarios, adaptive MARL variants achieve 77.1-82.1% mean success under unchanged interfaces, compared with 56.5% for a scripted-partner reference. The same contracts further support neural and interpretable hard-tree actors, two- and three-carrier teams, and simulated-to-human partner substitution. Physical Unitree G1-human tests achieve 100% success in spatially confined transport and 80% in super-long-object handling, while a three-carrier system with two G1 humanoids and one human validates the same interface structure on hardware. Together, these results show that C2C converts a tightly coupled HRC stack into a plug-compatible system in which key components can change without redefining the physical collaboration task.
Hao Zhang, Yisen Li, Ruize Geng +4
University of Texas at Arlington. · Carnegie Mellon University
Human-robot collaboration (HRC) combines the complementary strengths of humans and robots to improve task efficiency. However, many existing collaborative systems rely on hand-engineered pipelines, limiting their scalability and flexibility for new tasks. In this work, we show that models trained end-to-end with imitation learning, specifically vision-language-action (VLA) models, can support collaborative manipulation, and characterize the key factors affecting their real-world performance. We evaluate two state-of-the-art models and identify a failure mode of action-chunking policies in implicit HRC, where demonstration action leakage (i.e., action chunks crossing latent task transitions) can cause premature assistive behavior. We find that this issue increases with longer execution horizons and occurs in real-world collaborative VLA systems, such as when a robot attempts to hand over a tool before the person is ready. We propose an inference-time steering method to mitigate these erroneous assistive actions while preserving policy performance. Finally, through a 16-participant user study on a long-horizon collaborative assembly task, we show that steering enables a longer execution horizon while mitigating premature assistance, leading to faster collaboration and fewer failures compared to a shorter-horizon policy. Our paper website can be found at https://learning-to-assist.github.io/
Leo Xu, Letian Li, Alex Cuellar +1
University of Wisconsin–Madison · Massachusetts Institute of Technology