Visuomotor policies condition action prediction on visual and proprioceptive observations. While historical proprioceptive sequences carry rich dynamic priors, current methods treat them as equal to high-dimensional vision, causing the proprioceptive signal to be overwhelmed. To identify what is lost in equal-conditioning fusion, we analyze proprioceptive data and find that strong inter-joint coupling is the key action-determining structure. This finding aligns with the neural sensorimotor loop, in which proprioception first forms a body-centered prior and vision is later calibrated via parietal remapping. Inspired by this, we propose ACiT, Proprioceptive-Prior Action Chunking with Inverted Transformers, which reinforces historical joint-coupling as the primary condition with vision as a later calibration. This design is instantiated through three modules: Inverted State Encoding builds a proprioceptive prior via per-joint tokens and inter-joint self-attention. State-Visual Grounding lets joint tokens query visual evidence, making vision goal- and calibration-only. Inverted Action Chunking decodes per-joint trajectories independently. ACiT improves success rates on most LIBERO, RoboTwin 2.0, and real-world tasks, with the largest gains on those that require dynamics and coordination. The same conditioning design transferred to the VLA backbone pi_0-ACiT yields significant gains with better execution efficiency.
Figures & tables
Figure 1: Motivation. (a) Embodiment : a dual-arm robot has 14 joints, each with its own state history, while the action space has a much lower effective dimensionality (PR 1.44–4.42) and strong inter-joint coupling, the state therefore should not be compressed into one shared vector. (b) Biological inspiration : proprioception forms a body-centered prior in S1, vision reaches PPC via V1 only to remap and calibrate the goal, and M1 issues the command. The two modalities are not symmetric, and their order is fixed. (c) Conditioning interface : ACT concatenates a single-frame state with vision into one condition shared by all joints; ACiT keeps the per-joint prior as the primary condition, retrieves visual evidence per joint, and decodes one trajectory per joint.
Figure 2: ACiT architecture, organized by its three modules. (a) Inverted State Encoding : per-joint depthwise 1D convolutions turn each joint’s history into one token, and self-attention among the tokens gives the proprioceptive prior Pt . (b) State-Visual Grounding : the joint tokens query the visual tokens of the multi-view observations and produce grounded tokens Gt . (c) Inverted Action Chunking : Gt is the memory of a transformer decoder whose learnable joint queries output one T -step trajectory per joint.
Figure 3: Real-world platform and tasks. The Dual reBotArm-B601-DM performs four tasks: (a) pick cube, (b) stack cube, (c) lift pot, and (d) handover cube.
Figure 4: Joint attention of ACiT. Top: joint self-attention inside the Inverted State Encoding on two tasks, averaged over five episodes and ten frames, with the left arm (joints 1–7) and the right arm (joints 8–14) marked. Bottom: joint-to-vision cross attention inside State-Visual Grounding, for the gripper of the right arm and for the elbow of the left arm.
Figure 5: Main results on the simulation benchmarks. Left: LIBERO. Right: RoboTwin 2.0. Pale bars denote easy and solid bars denote randomized hard. A number above a pale bar is its easy rate, and a number on a solid bar is its hard rate. ACiT is first in each group.
Figure 6: Closed-loop recovery on the real robot. Top: global view. Bottom: gripper view. After a grasp failure (Step 2), ACiT falls back to the previous state (Step 3) and re-attempts (Step 4), completing the pick (Step 6).
Figure 7: Real-world results. Success rates for ACiT, ACT, DP, and FM on four tasks under in-distribution, unseen-OOD, and dark-background-OOD conditions. ACiT is shown first in each group.
Figure 8: Transferring the conditioning design to a large VLA on RoboTwin 2.0. Success rate of π0 -ACiT (left bar of each task) and π0 (right bar). The pale bar is the easy rate and the solid bar drawn on top of it is the randomized-hard rate, so the pale cap is the drop caused by randomization. Numbers above the pale bars are easy rates, numbers on the solid bars hard rates.
Figure 9: Real-world π0 / π0 -ACiT transfer on the B601 platform. Each task has six bars: π0 -ACiT under in-distribution, unseen-object/background, and dark-background conditions (light to dark orange), followed by π0 under the same conditions (light to dark blue).
Variant
LIBERO
RoboTwin
Real world
easy
hard
ID
unseen
dark
ACiT (full) x
79.0 x
62.7 x
14.0 x
60.0 x
34.1 x
44.1 x
equal fusion x
58.3 −20.7
52.3 −10.4
5.0 −9.0
48.4 −11.6
17.5 −16.6
13.3 −30.8
w/o joint attn x
69.3 −9.7
57.0 −5.7
9.8 −4.2
52.5 −7.5
23.3 −10.8
28.3 −15.8
flat state x
65.3 −13.7
53.3 −9.4
7.0 −7.0
49.2 −10.8
15.0 −19.1
21.6 −22.5
flat decode x
75.0 −4.0
60.0 −2.7
11.8 −2.2
54.2 −5.8
25.8 −8.3
41.6 −2.5
Table 1: Module ablation (mean success rate %). Small numbers give the change relative to the full model, per-task rates in Appendix B.1 .
Task
seed 1000
seed 1001
seed 1002
mean ± std
Lift Pot
55
56
54
55.0±1.0
Pick Dual
61
63
59
61.0±2.0
Handover
69
69
69
69.0±0.0
Move Can
64
60
68
64.0±4.0
Place Dual Shoes
41
35
38
38.0±3.0
Shake Bottle
88
89
90
89.0±1.0
Table 2: Training-seed variance of ACiT (easy split, success rate over 100 episodes, hence integer).
Figure 10: History length of the conditioning input. (a) Simulation: RoboTwin easy and hard, and LIBERO. (b) Real world: in-distribution, unseen and dark-background. All curves peak at H=8 or tie it at 16; the per-task rates are in Appendix B .
Method
Params
Steps
ms/act.
Control (Hz)
ACT
84.0 M
1 (1)
0.36
2794
DP
293.2 M
10 (10)
2.12
471
FM
293.2 M
10 (20)
3.74
267
ACiT
92.6 M
1 (1)
0.48
2064
π0
4.03 B
10 (10)
5.93
169
π0 -ACiT
4.05 B
10 (10)
4.73
211
Table 3: Inference cost on one RTX 3090, at the 240×320 observation resolution used throughout the RoboTwin experiments. Steps counts integrator steps (network evaluations in brackets), and the last two columns are per executed action.
Appendix figures & tables16 assets
Supplementary material from the paper’s appendix.
Appendix
Task
Success condition
Lift Pot
pot above 0.82 m and upright, both TCPs within 3 cm of its handles
Pick Dual Bottles
both bottles within 10 cm of their targets and above 0.89 m
Handover Block
block within 3 cm of the target pose and right gripper open
Move Can Pot
can tipped on its side beside the pot ( < 20 cm, 3.5 cm, 15∘ ), at table height, both grippers open
Place Dual Shoes
both shoes at the target pose (5 cm, 0.07 quaternion) and both grippers open
Shake Bottle
bottle raised above 0.8 m
Appendix
Table 4: Success conditions of the six RoboTwin tasks, as implemented by the benchmark. A rollout counts as a success only when every condition of its task holds at the same step.
Figure 11: The six RoboTwin 2.0 tasks used in Sec. 4.2 , viewed from the head camera. Both arms are visible in each panel; the randomized split varies object poses, colors and the background of the same scenes.
RoboTwin (easy)
RoboTwin (hard)
LIBERO
Variant
Lift
Pick
Hand.
Move
Shoes
Shake
Mean
Lift
Pick
Hand.
Move
Shoes
Shake
Mean
10
Obj.
Spa.
Mean
ACiT
55
61
69
64
38
89
62.7
4
14
0
19
0
47
14.0
50
99
88
79.0
equal fusion
56
50
30
68
22
88
52.3
1
2
0
2
0
25
5.0
30
73
72
58.3
w/o joint attn
39
60
69
62
30
82
57.0
3
6
0
12
0
38
9.8
38
90
80
69.3
flat state
48
50
58
56
28
80
53.3
2
3
0
7
0
30
7.0
35
85
76
65.3
flat decode
54
45
79
74
23
85
60.0
5
8
1
16
0
41
11.8
45
95
85
75.0
Appendix
Table 5: Per-task module ablation on the simulation benchmarks (success rate, %, 100 episodes per task).
In-distribution
Unseen
Dark background
Variant
Pick
Stack
Pot
Hand.
Mean
Pick
Stack
Pot
Hand.
Mean
Pick
Stack
Pot
Hand.
Mean
ACiT
93.3
63.3
33.3
50.0
60.0
53.3
23.3
16.7
43.3
34.1
83.3
43.3
16.7
33.3
44.1
equal fusion
76.7
56.7
23.3
36.7
48.4
40.0
6.7
6.7
16.7
17.5
30.0
13.3
0.0
10.0
13.3
w/o joint attn
80.0
56.7
30.0
43.3
52.5
43.3
13.3
13.3
23.3
23.3
53.3
23.3
13.3
23.3
28.3
flat state
70.0
56.7
23.3
46.7
49.2
33.3
6.7
6.7
13.3
15.0
43.3
23.3
6.7
13.3
21.6
flat decode
83.3
46.7
30.0
56.7
54.2
43.3
13.3
13.3
33.3
25.8
73.3
30.0
20.0
43.3
41.6
Appendix
Table 6: Per-task module ablation on the real-world platform (success rate, %; 30 rollouts per cell).
RoboTwin (easy)
RoboTwin (hard)
LIBERO
H
Lift
Pick
Hand.
Move
Shoes
Shake
Mean
Lift
Pick
Hand.
Move
Shoes
Shake
Mean
10
Obj.
Spa.
Mean
1
52
45
60
58
34
85
55.7
2
5
0
12
0
35
9.0
46
91
82
73.0
4
54
56
66
62
36
87
60.2
3
10
0
16
0
42
11.8
51
99
88
79.3
8
55
61
69
64
38
89
62.7
4
14
0
19
0
47
14.0
50
99
88
79.0
16
54
60
68
63
37
88
61.7
4
13
0
18
0
45
13.3
49
98
87
78.0
Appendix
Table 7: History length on the simulation benchmarks (success rate, %, 100 episodes per task). H is the history length, in frames.
In-distribution
Unseen
Dark background
H
Pick
Stack
Pot
Hand.
Mean
Pick
Stack
Pot
Hand.
Mean
Pick
Stack
Pot
Hand.
Mean
1
76.7
46.7
23.3
40.0
46.7
43.3
16.7
13.3
33.3
26.6
70.0
33.3
10.0
20.0
33.3
4
86.7
56.7
30.0
46.7
55.0
46.7
20.0
13.3
36.7
29.2
76.7
36.7
13.3
26.7
38.4
8
93.3
63.3
33.3
50.0
60.0
53.3
23.3
16.7
43.3
34.1
83.3
43.3
16.7
33.3
44.1
16
90.0
60.0
30.0
46.7
56.7
50.0
20.0
16.7
40.0
31.7
83.3
43.3
16.7
33.3
44.1
Appendix
Table 8: History length on the real-world platform (success rate, %; 30 rollouts per cell).
Figure 12: Out-of-distribution dark background. ACT, DP, FM and ACiT on the four tasks under the background shift.
Figure 13: Out-of-distribution unseen initial state and color, on Pick cube and Stack cube.
Figure 14: Qualitative comparison on Place Dual Shoes, same task and episode under four policies. The outcome and the number of steps are marked on the last frame. Only ACiT leaves both shoes in the box and releases them; ACT keeps a shoe in its closed gripper, and DP and FM stop with one shoe outside the box.
Figure 15: Qualitative comparison on Pick Dual Bottles. ACiT lifts the bottle clear of the table and finishes well inside the step budget, while ACT, DP and FM leave both bottles in place.
Figure 16: Qualitative comparison on Lift Pot. ACiT and DP both raise the pot, with ACiT finishing in fewer steps, while ACT and FM fail to lift it.
Figure 17: Qualitative comparison on Move Can Pot. ACiT places the can against the pot and finishes in 158 steps; ACT never moves it into place, DP does not approach it, and FM knocks it away from its start position.
Figure 18: Qualitative comparison on Shake Bottle. ACiT, ACT and FM shake the bottle, while DP never lifts it and is marked failure.
Figure 19: Qualitative comparison on LIBERO-Spatial. All four policies solve the episode, and ACiT finishes in the fewest steps.
Task
PR
within-L
within-R
cross(0)
cross(best)
Lift Pot
1.44
0.602
0.579
0.802
0.956
Pick Dual Bottles
2.71
0.341
0.324
0.579
0.881
Handover Block
1.92
0.422
0.596
0.405
0.716
Place Dual Shoes
4.42
0.341
0.301
0.352
0.670
Appendix
Table 9: Action-space statistics on the four dual-arm RoboTwin tasks. Move Can Pot and Shake Bottle are single-arm and are not listed, because their cross-arm quantities are undefined.
Figure 20: Action correlation matrix (left) versus the best-matching self-attention head (right) for each dual-arm task. The Spearman correlation over off-diagonal entries is shown above each attention map.