Organizations: COWARobot Co. Ltd, China. · The Hong Kong Polytechnic University, Hong Kong SAR, China. · Tongji University, Shanghai, China. · Shanghai Jiao Tong University, Shanghai, China. · National University of Singapore, Singapore. · The Chinese University of Hong Kong, Hong Kong SAR, China. · Northeastern University, Shenyang, China.
Vision-Language-Action (VLA) models that directly regress joint-space actions avoid the deployment-time cost of inverse kinematics. However, their training objectives remain misaligned with task success, as joint-space losses ignore the unequal physical consequences of joint errors and visual conditioning lacks explicit focus on hand-object interaction regions. To address these issues, we propose a training-phase prior injection framework for fine-grained bimanual manipulation. First, a physics-grounded flow-matching loss maps the implicit clean-action estimate at every noise level through differentiable forward kinematics (FK) and supervises the resulting Tool Center Point (TCP) pose and inter-arm relative pose, with a bounded noise-level weighting that compensates gradient decay. Second, we introduce a global prediction query group distilled from a frozen DINOv3 teacher through future-frame feature prediction, together with a local interaction query supervised by Grounding DINO at the attention level. Both teachers are removed at inference, and the deployment pipeline is that of a standard joint-space flow policy, adding only the learned query pathway. Simulation and real-robot experiments on long-horizon bimanual tasks show that the two priors are complementary across different deployment settings. The full model excels in in-distribution deployment, while the distilled query branch is preferable under strong visual randomization.
Figures & tables
Fig. 2: The proposed loss projects intermediate clean actions into task space via differentiable FK and constrains TCP pose and inter-arm relative pose errors, providing physically supervision beyond joint-space regression.
Fig. 3: Interaction-region-guided attention distillation. Left to right: attention before distillation, Grounding DINO region targets, and attention after distillation. Local queries focus attention on task-relevant objects and grippers.
Method
Clean SR(%) ↑
Randomized SR(%) ↑
TCP err. mean(mm) ↓
TCP err. max(mm) ↓
ACT †
29.7
1.7
–
–
DP †
28.0
0.6
–
–
RDT †
34.5
13.7
–
–
π0 †
46.4
16.3
–
–
π0.5 -SFT
64.13
36.15
5.3
7.7
π0.5 -Global Query
64.26
55.29
5.1
7.2
TABLE I: Simulation results on the RoboTwin 2.0 benchmark. Success rates (SR) are averaged over 50 manipulation tasks with 100 episodes per condition. TCP err. denotes the mean and maximum deviation between the executed end-effector trajectory and the expert trajectory. † Results reported by the RoboTwin 2.0 benchmark [ 9 ] under its official protocol: Aloha AgileX embodiment, 50 clean demonstrations per task, and 100 rollouts per condition. Action Chunking with Transformers (ACT) [ 25 ] and Diffusion Policy (DP) [ 26 ] are trained from scratch; RDT [ 24 ] and π0 [ 4 ] are fine-tuned from released pretrained weights in the single-task setting.
Fig. 4: Real-world manipulation setup and results on the dual-arm PiPER-X robot. Each method is trained with 5 random seeds and each checkpoint is evaluated over 30 independent trials; bars show the mean success rate across seeds and error bars denote one standard deviation.