CIALLO: Learning Shared Intent Representations for Cross-Embodiment Manipulation
Organizations: Tsinghua University
Abstract
Robot demonstrations contain reusable manipulation knowledge, but transferring this knowledge across embodiments requires representations that bridge differences in control interfaces. Action reconstruction alone does not ensure that similar behaviors across robots share a representation. To address this gap, we introduce CIALLO, an action codec that aligns actions, visual changes, and observation-grounded language in a shared space of short-horizon behavioral intent. An embodiment-conditioned decoder maps this intent back to robot-specific actions, allowing a shared representation to support different control interfaces. After pretraining on eight embodiments, CIALLO adapts to a new robot from one recording of at most 10 seconds by updating only its embodiment embeddings. The calibrated codec then remains frozen while a policy learns to predict intent from observations and instructions. We instantiate this policy with Qwen3.5-4B to form Qwen-CIALLO, whose backbone requires no large-scale robot-action pretraining before policy finetuning. With 20 demonstrations per task, Qwen-CIALLO achieves 76.7% average success across real SO-101 and single-arm Piper robots, compared with 70.0% for fully finetuned . In simulation, we additionally finetune the action decoder, achieving 97.3% success on LIBERO and 52.7% on RoboTwin 2.0. The LIBERO policy also achieves 80.8% on LIBERO-Plus without further finetuning. Together, these results support a shared action codec as a transferable control interface that separates embodiment calibration from task learning.
Figures & tables
| Contrastive terms | A A MAE | V A MAE |
|---|---|---|
| None | 0.03917 | 0.05750 |
| A–V | 0.04042 | 0.05465 |
| A–V, A–T | 0.03809 | 0.05495 |
| A–V, A–T, V–T | 0.03639 | 0.05614 |
| Method | Steps | Spatial | Object | Goal | Long | Avg. |
|---|---|---|---|---|---|---|
| Diffusion Policy † [ 9 ] | 100–250k | 78.3 | 92.5 | 68.3 | 50.5 | 72.4 |
| Octo † [ 14 ] | 100–250k | 78.9 | 85.7 | 84.6 | 51.1 | 75.1 |
| OpenVLA † [ 22 ] | 50–150k | 84.7 | 88.4 | 79.2 | 53.7 | 76.5 |
| OpenVLA-OFT † [ 21 ] | 50–150k | 97.6 | 98.4 | 97.9 | 94.5 | 97.1 |
| GR00T-N1 † [ 31 ] | – | 94.4 | 97.6 | 93.0 | 90.6 | 93.9 |
| † [ 2 ] | – | 96.8 | 98.8 | 95.8 | 85.2 | 94.2 |
| Method | Camera | Robot | Language | Light | Background | Noise | Layout | Total |
|---|---|---|---|---|---|---|---|---|
| 15.8 | 6.6 | 61.0 | 79.6 | 78.5 | 79.4 | 70.4 | 53.6 | |
| UniVLA [ 5 ] | 1.8 | 46.2 | 69.6 | 69.0 | 81.0 | 21.2 | 31.9 | 42.9 |
| OpenVLA | 0.8 | 3.5 | 23.0 | 8.1 | 34.8 | 15.2 | 28.5 | 15.6 |
| OpenVLA-OFT | 56.4 | 31.9 | 79.5 | 88.7 | 93.3 | 75.8 | 74.2 | 69.6 |
| Fast-WAM [ 51 ] | 16.3 | 44.1 | 66.8 | 79.8 | 52.6 | 38.5 | 61.4 | 49.9 |
| DreamWAM [ 50 ] | 53.8 | 63.6 | 94.8 | 96.7 | 71.6 | 67.2 | 80.7 | 74.6 |
| Method | Protocol | Avg. | Method | Protocol | Avg. |
|---|---|---|---|---|---|
| Diffusion Policy | Single Task | 28.0 | Single Task | 46.4 | |
| ACT [ 54 ] | Single Task | 29.7 | StarVLA | Co-Train | 46.5 |
| RDT [ 27 ] | Single Task | 34.5 | Qwen-CIALLO | Co-Train | 52.7 |
| SO-101 | Piper | ||||||
|---|---|---|---|---|---|---|---|
| Method | Water bottle | Pour beer | Wooden fish | Water bottle | Pour beer | Wooden fish | Avg. |
| Qwen-CIALLO | 85 | 80 | 70 | 90 | 95 | 40 | 76.7 |
| 90 | 80 | 35 | 100 | 95 | 20 | 70.0 | |
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
| Source / embodiment | (Hz) | (s) | Scale | |
|---|---|---|---|---|
| LIBERO / Panda | 20 | 10 | 0.5 | |
| Bridge / WidowX | 5 | 5 | 1.0 | |
| Fractal / Google | 3 | 6 | 2.0 | |
| RT2 / Aloha AgileX | 30 | 15 | 0.5 | |
| RT2 / Franka | 30 | 15 | 0.5 | |
| RT2 / ARX X5 | 30 | 15 | 0.5 |
| Top-1 retrieval | Action MAE | ||||||
|---|---|---|---|---|---|---|---|
| Embodiment | Samples | A V | A T | V T | AA | VA | TA |
| Panda | 245 | 94.90 | 89.18 | 95.31 | 0.01363 | 0.06924 | 0.08772 |
| WidowX | 263 | 98.86 | 94.87 | 98.86 | 0.02245 | 0.08469 | 0.11743 |
| Google Robot | 269 | 98.88 | 93.49 | 99.07 | 0.01851 | 0.09376 | 0.11423 |
| Aloha AgileX | 259 | 97.68 | 94.98 | 97.88 | 0.00572 | 0.00840 | 0.01098 |
| Franka Dual | 267 | 95.88 | 89.51 | 95.51 | 0.00691 | 0.01198 | 0.01510 |
| 3 embodiments | 5 embodiments | 8 embodiments | ||||
|---|---|---|---|---|---|---|
| Embodiment | AA | VA | AA | VA | AA | VA |
| Panda | 0.0807 | 0.1002 | 0.0339 | 0.0766 | 0.0136 | 0.0692 |
| WidowX | 0.0920 | 0.1107 | 0.0331 | 0.0986 | 0.0225 | 0.0847 |
| Google Robot | 0.0899 | 0.1129 | 0.0336 | 0.1074 | 0.0185 | 0.0938 |
| Aloha AgileX | – | – | 0.0467 | 0.0493 | 0.0057 | 0.0084 |
| Franka Dual | – | – | 0.0506 | 0.0553 | 0.0069 | 0.0120 |
| Validation loss | Macro MAE | Peak memory (GiB) | |||||
|---|---|---|---|---|---|---|---|
| Total | AA | VA | AA | VA | Allocated | Reserved | |
| 1 | 0.264283 | 0.024807 | 0.036571 | 0.035501 | 0.056521 | 37.412 | 38.494 |
| 2 | 0.261221 | 0.023484 | 0.034908 | 0.036392 | 0.056142 | 37.414 | 38.500 |
| 4 | 0.304822 | 0.033762 | 0.045132 | 0.044961 | 0.065114 | 37.418 | 38.508 |
| Path | Correct | Wrong encoder | Wrong decoder | All wrong |
|---|---|---|---|---|
| AA | 0.00761 | 0.04506 (+492.1%) | 0.19124 (+2413.0%) | 0.16470 (+2064.3%) |
| VA | 0.01853 | 0.02189 (+18.1%) | 0.20082 (+983.8%) | 0.21534 (+1062.1%) |
| TA | 0.02429 | 0.02429 (+0.0%) | 0.20684 (+751.5%) | 0.20684 (+751.5%) |
| Method | Params | FLOPs | Latency |
|---|---|---|---|
| OFT [ 21 ] | 151.12M | 2.42G | 0.37ms |
| (JAX) [ 37 ] | 430.10M | 79.13G | 19.88ms |
| CIALLO | 19.89M | 46.49G | 4.89ms |
| CIALLO (JAX) | 19.89M | 46.49G | 1.46ms |
| Task | Instruction |
|---|---|
| Water bottle | Grab the water bottle and place on the black plate. |
| Pour beer | Pour beer from bottle into the cup. |
| Wooden fish | Pick up the screwdriver and hit the wooden fish. |
Cross-Embodiment Robot Learning
Cross-embodiment robot learning transfers skills and policies across robots with different bodies, sensors, dynamics, and action spaces.
See all 374 papers