ACiT: Proprioceptive-Prior Action Chunking with Inverted Transformers
Organizations: Nanjing University of Science and Technology, Nanjing, China
Abstract
Visuomotor policies condition action prediction on visual and proprioceptive observations. While historical proprioceptive sequences carry rich dynamic priors, current methods treat them as equal to high-dimensional vision, causing the proprioceptive signal to be overwhelmed. To identify what is lost in equal-conditioning fusion, we analyze proprioceptive data and find that strong inter-joint coupling is the key action-determining structure. This finding aligns with the neural sensorimotor loop, in which proprioception first forms a body-centered prior and vision is later calibrated via parietal remapping. Inspired by this, we propose ACiT, Proprioceptive-Prior Action Chunking with Inverted Transformers, which reinforces historical joint-coupling as the primary condition with vision as a later calibration. This design is instantiated through three modules: Inverted State Encoding builds a proprioceptive prior via per-joint tokens and inter-joint self-attention. State-Visual Grounding lets joint tokens query visual evidence, making vision goal- and calibration-only. Inverted Action Chunking decodes per-joint trajectories independently. ACiT improves success rates on most LIBERO, RoboTwin 2.0, and real-world tasks, with the largest gains on those that require dynamics and coordination. The same conditioning design transferred to the VLA backbone pi_0-ACiT yields significant gains with better execution efficiency.
Figures & tables
| Variant | LIBERO | RoboTwin | Real world | |||
|---|---|---|---|---|---|---|
| easy | hard | ID | unseen | dark | ||
| ACiT (full) x | 79.0 x | 62.7 x | 14.0 x | 60.0 x | 34.1 x | 44.1 x |
| equal fusion x | 58.3 | 52.3 | 5.0 | 48.4 | 17.5 | 13.3 |
| w/o joint attn x | 69.3 | 57.0 | 9.8 | 52.5 | 23.3 | 28.3 |
| flat state x | 65.3 | 53.3 | 7.0 | 49.2 | 15.0 | 21.6 |
| flat decode x | 75.0 | 60.0 | 11.8 | 54.2 | 25.8 | 41.6 |
| Task | seed 1000 | seed 1001 | seed 1002 | mean std |
|---|---|---|---|---|
| Lift Pot | 55 | 56 | 54 | |
| Pick Dual | 61 | 63 | 59 | |
| Handover | 69 | 69 | 69 | |
| Move Can | 64 | 60 | 68 | |
| Place Dual Shoes | 41 | 35 | 38 | |
| Shake Bottle | 88 | 89 | 90 |
| Method | Params | Steps | ms/act. | Control (Hz) |
| ACT | 84.0 M | 1 (1) | 0.36 | 2794 |
| DP | 293.2 M | 10 (10) | 2.12 | 471 |
| FM | 293.2 M | 10 (20) | 3.74 | 267 |
| ACiT | 92.6 M | 1 (1) | 0.48 | 2064 |
| 4.03 B | 10 (10) | 5.93 | 169 | |
| -ACiT | 4.05 B | 10 (10) | 4.73 | 211 |
Appendix figures & tables16 assets
Supplementary material from the paper’s appendix.
Appendix
| Task | Success condition |
|---|---|
| Lift Pot | pot above 0.82 m and upright, both TCPs within 3 cm of its handles |
| Pick Dual Bottles | both bottles within 10 cm of their targets and above 0.89 m |
| Handover Block | block within 3 cm of the target pose and right gripper open |
| Move Can Pot | can tipped on its side beside the pot ( 20 cm, 3.5 cm, ), at table height, both grippers open |
| Place Dual Shoes | both shoes at the target pose (5 cm, 0.07 quaternion) and both grippers open |
| Shake Bottle | bottle raised above 0.8 m |
| RoboTwin (easy) | RoboTwin (hard) | LIBERO | ||||||||||||||||
| Variant | Lift | Pick | Hand. | Move | Shoes | Shake | Mean | Lift | Pick | Hand. | Move | Shoes | Shake | Mean | 10 | Obj. | Spa. | Mean |
| ACiT | 55 | 61 | 69 | 64 | 38 | 89 | 62.7 | 4 | 14 | 0 | 19 | 0 | 47 | 14.0 | 50 | 99 | 88 | 79.0 |
| equal fusion | 56 | 50 | 30 | 68 | 22 | 88 | 52.3 | 1 | 2 | 0 | 2 | 0 | 25 | 5.0 | 30 | 73 | 72 | 58.3 |
| w/o joint attn | 39 | 60 | 69 | 62 | 30 | 82 | 57.0 | 3 | 6 | 0 | 12 | 0 | 38 | 9.8 | 38 | 90 | 80 | 69.3 |
| flat state | 48 | 50 | 58 | 56 | 28 | 80 | 53.3 | 2 | 3 | 0 | 7 | 0 | 30 | 7.0 | 35 | 85 | 76 | 65.3 |
| flat decode | 54 | 45 | 79 | 74 | 23 | 85 | 60.0 | 5 | 8 | 1 | 16 | 0 | 41 | 11.8 | 45 | 95 | 85 | 75.0 |
| In-distribution | Unseen | Dark background | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Variant | Pick | Stack | Pot | Hand. | Mean | Pick | Stack | Pot | Hand. | Mean | Pick | Stack | Pot | Hand. | Mean |
| ACiT | 93.3 | 63.3 | 33.3 | 50.0 | 60.0 | 53.3 | 23.3 | 16.7 | 43.3 | 34.1 | 83.3 | 43.3 | 16.7 | 33.3 | 44.1 |
| equal fusion | 76.7 | 56.7 | 23.3 | 36.7 | 48.4 | 40.0 | 6.7 | 6.7 | 16.7 | 17.5 | 30.0 | 13.3 | 0.0 | 10.0 | 13.3 |
| w/o joint attn | 80.0 | 56.7 | 30.0 | 43.3 | 52.5 | 43.3 | 13.3 | 13.3 | 23.3 | 23.3 | 53.3 | 23.3 | 13.3 | 23.3 | 28.3 |
| flat state | 70.0 | 56.7 | 23.3 | 46.7 | 49.2 | 33.3 | 6.7 | 6.7 | 13.3 | 15.0 | 43.3 | 23.3 | 6.7 | 13.3 | 21.6 |
| flat decode | 83.3 | 46.7 | 30.0 | 56.7 | 54.2 | 43.3 | 13.3 | 13.3 | 33.3 | 25.8 | 73.3 | 30.0 | 20.0 | 43.3 | 41.6 |
| RoboTwin (easy) | RoboTwin (hard) | LIBERO | ||||||||||||||||
| Lift | Pick | Hand. | Move | Shoes | Shake | Mean | Lift | Pick | Hand. | Move | Shoes | Shake | Mean | 10 | Obj. | Spa. | Mean | |
| 1 | 52 | 45 | 60 | 58 | 34 | 85 | 55.7 | 2 | 5 | 0 | 12 | 0 | 35 | 9.0 | 46 | 91 | 82 | 73.0 |
| 4 | 54 | 56 | 66 | 62 | 36 | 87 | 60.2 | 3 | 10 | 0 | 16 | 0 | 42 | 11.8 | 51 | 99 | 88 | 79.3 |
| 8 | 55 | 61 | 69 | 64 | 38 | 89 | 62.7 | 4 | 14 | 0 | 19 | 0 | 47 | 14.0 | 50 | 99 | 88 | 79.0 |
| 16 | 54 | 60 | 68 | 63 | 37 | 88 | 61.7 | 4 | 13 | 0 | 18 | 0 | 45 | 13.3 | 49 | 98 | 87 | 78.0 |
| In-distribution | Unseen | Dark background | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Pick | Stack | Pot | Hand. | Mean | Pick | Stack | Pot | Hand. | Mean | Pick | Stack | Pot | Hand. | Mean | |
| 1 | 76.7 | 46.7 | 23.3 | 40.0 | 46.7 | 43.3 | 16.7 | 13.3 | 33.3 | 26.6 | 70.0 | 33.3 | 10.0 | 20.0 | 33.3 |
| 4 | 86.7 | 56.7 | 30.0 | 46.7 | 55.0 | 46.7 | 20.0 | 13.3 | 36.7 | 29.2 | 76.7 | 36.7 | 13.3 | 26.7 | 38.4 |
| 8 | 93.3 | 63.3 | 33.3 | 50.0 | 60.0 | 53.3 | 23.3 | 16.7 | 43.3 | 34.1 | 83.3 | 43.3 | 16.7 | 33.3 | 44.1 |
| 16 | 90.0 | 60.0 | 30.0 | 46.7 | 56.7 | 50.0 | 20.0 | 16.7 | 40.0 | 31.7 | 83.3 | 43.3 | 16.7 | 33.3 | 44.1 |
| Task | PR | within-L | within-R | cross(0) | cross(best) |
|---|---|---|---|---|---|
| Lift Pot | 1.44 | 0.602 | 0.579 | 0.802 | 0.956 |
| Pick Dual Bottles | 2.71 | 0.341 | 0.324 | 0.579 | 0.881 |
| Handover Block | 1.92 | 0.422 | 0.596 | 0.405 | 0.716 |
| Place Dual Shoes | 4.42 | 0.341 | 0.301 | 0.352 | 0.670 |
Robot Policy Learning
Robot policy learning develops controllers that map sensory observations, task goals, and interaction history to actions for embodied robots.
See all 405 papers