When Will You Hand It Over? A Hidden Start Delay Makes Self-Play Handover Robust to Unfamiliar Partners
Organizations: Universität Bremen · University of North Texas · Toyota Motor North America · Budapest University of Technology and Economics
Abstract
Cooperative self-play can teach a humanoid to receive an object without any human data, but the partner it meets after training is never the one it trained with; we show that a fluent self-play handover policy breaks with unfamiliar partners, identify the main cause, and repair it. In a physics-based parcel handover between a Unitree G1 and a learned giver, both trained by alternating PPO self-play, the receiver completes 95% and 90% of handovers with its own partner (two training seeds) but only 31% with scripted partners it has never met. Paired interventions isolate the cause: under a shared reward the learned giver always starts its offer at once, so the receiver learns a single rendezvous, and delaying its own partner by one second cuts success from 95% to 6% (87% to 34% in the second seed). We introduce hidden start-delay randomisation (HSD), a one-line rule on the learned partner that the receiver never observes. After 300 further updates, success with unseen scripted partners rises from 31% to 80% in both seeds, against 35--41% for a budget-matched control. The gain holds on a fresh scenario seed, on eleven held-out partner conditions and on partners that start later than any training delay; it cuts high-force contact three- to five-fold and needs no retraining of the partner.
Figures & tables
| Partner | Supported transfer |
|---|---|
| Own learned partner, unchanged | 95.0 / 87.0 |
| without its position corrections | 94.0 / 88.5 |
| without its tilt corrections | 95.0 / 89.0 |
| constant reach rate (cannot wait for the robot) | 93.5 / 81.5 |
| start delayed by 1 s | 6.0 / 33.5 |
| Scripted partner, unchanged | 30.5 / 30.5 |
| Self-play | Control | HSD | HSD , frozen | |
| Partner | only | (no delay) | partner | |
| Learned partner (development seed) | ||||
| Own partner, random delay 0–1 s | 64.5 / 55.5 | 67.5 / 59.5 | 94.5 / 88.5 | 97.0 / 91.5 |
| Own partner, fixed 1 s delay | 6.0 / 33.5 | 28.0 / 33.5 | 93.5 / 89.0 | 95.0 / 90.5 |
| Own partner, no delay | 95.0 / 87.0 | 97.5 / 91.5 | 95.5 / 93.0 | 92.5 / 89.5 |
| Scripted partners, never trained on (development seed) | ||||
| Self-play only | HSD | HSD + release rule | |
|---|---|---|---|
| Scripted load | 30.5 / 30.0 | 81.0 / 79.0 | 87.5 / 83.5 |
| Scripted follow | 24.5 / 20.0 | 70.0 / 66.0 | 83.0 / 75.0 |
| Scripted settle, with withdrawal | 4.0 / 3.5 | 2.0 / 0.0 | 3.5 / 1.5 |
| Own partner, load / follow style | – | 98.3 / 91.7, 87.3 / 79.4 | 88.3 / 93.3, 88.9 / 88.9 |
| Own partner, settle style | – | 24.7 / 40.3 | 79.2 / 88.3 |
Cooperative MARL
Cooperative multi-agent reinforcement learning (MARL) trains agents to coordinate toward shared goals under partial information and decentralized control.
See all 263 papers