Organizations: School of Computation, Information and Technology, Technical University of Munich, Garching, Germany. · State Key Laboratory for Novel Software Technology and the School of Science and Technology, Nanjing University (Suzhou Campus), China.
Learning a collective policy for state-based Cartesian manipulation across diverse robotic embodiments presents a fundamental challenge due to embodiment-dependent transition dynamics. Collective learning aggregates multi-robot experience to discover shared task structure, yet naive aggregation can average incompatible action modes when different embodiments realize the same Cartesian command through different transition behavior. This leads to strategy ambiguity and performance collapse on dynamically outlying embodiments. To address this, we propose PACE, a two-stage collective learning framework that separates shared task-space intent from embodiment-dependent transition dynamics. The first stage employs a representation layer pre-trained on offline trajectories to extract task-level context from collective data. The second stage introduces a decision layer equipped with a predictive dynamics adapter. By forcing the latent state to predict future transitions, this adapter acts as a latent regularizer that grounds the policy in the transition behavior of the executing robot while preserving real-time inference without online planning. Evaluations across simulated embodiments demonstrate that our method prevents performance collapse on distinct robots. It achieves over 93% overall success and 89% on dynamically out-of-distribution zero-shot simulated robots, outperforming the Transformer behavior cloning (82%) and diffusion policy baselines (66%). Physical KUKA deployment further validates the feasibility of the full state-based pipeline under zero-shot sim-to-real transfer.
Figures & tables
Fig. 1 : Overview of the PACE framework. The left panel shows the diverse multi-embodiment dataset. The central panel details the collective network, which consists of in-context trajectory encoding and dynamics-aware policy learning. The right panel demonstrates the cross-embodiment generalization to unseen simulated and real-world robots.
Fig. 2 : Structure of PACE. The two-stage distillation framework features 1. an offline representation layer with a trajectory transformer encoder, and 2. an online decision layer with a predictive dynamics adapter and collective agent policy.
Fig. 3 : Cross-embodiment zero-shot transfer across diverse manipulation tasks. Physical specifications are in Table I . For more details, refer to our supplementary video.
Robot Model
Mass(kg)
Payload(kg)
Reach(mm)
Seen Embodiments (Training & Testing)
Unitree Z1
4.5
3.0
740
Kinova Gen3
8.2
4.0
902
UFactory xArm7
13.7
3.5
700
Rethink Sawyer
19.0
4.0
1260
UR10e
33.5
12.5
1300
TABLE I: Physical specifications of evaluated robotic manipulators.
Fig. 4 : Performance Analysis. (a) Per-robot success rate. Our method prevents performance collapse on outliers. (b) Success rate versus the number of seen robots ( N ).
Method
Seen (%)
Unseen (%)
Transformer BC
82.66 ± 5.16
81.75 ± 5.99
Diffusion Policy (DP)
66.40 ± 3.85
65.25 ± 4.12
PACE (Ours)
93.17 ± 2.41
93.21 ± 2.81
TABLE II: Generalization Performance. Success rates (%) on seen ( N=5 ) and unseen ( N=4 ) robots.
Fig. 5 : Zero-Shot vs. In-Domain generalization on UR10e across varying training set sizes.
Fig. 6 : t-SNE cluster Visualization. t-SNE projection of context embeddings colored by task ID.
TABLE III : Stage 1 Component Analysis. Success rates (%) with a frozen Stage 2 dynamics adapter.
Fig. 8 : Qualitative demonstration of zero-shot sim-to-real transfer on the unseen KUKA iiwa robot. For more details, refer to our supplementary video.
Task Name
Successes / 20 Eps
Success Rate (%)
Reach
18 / 20
90.0
Push
17 / 20
85.0
Pick-Place
17 / 20
85.0
Button Press
16 / 20
80.0
Drawer Open
14 / 20
70.0
Window Open
13 / 20
65.0
TABLE IV : Success rates and episode counts across manipulation tasks.
School of Computer Science, The University of Sydney, Australia. · College of Computer Science and Technology, Zhejiang University of Technology, China.