cs.ROOct 10, 2026

DEFT-VLA: Stage-Localized Force Specialization for Contact-Rich Robotic Manipulation

Authors: Zan Huang, Yibo Qiu, Kan Ding, Xuanping Zhang, Ao Tian, Ronald X Xu, Mingzhai Sun

Organizations: the Suzhou Institute for Advanced Research, University of Science and Technology of China, Suzhou, Jiangsu, China, and also with the School of Biomedical Engineering, Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, Anhui, China. · the School of Robotics, Xi’an Jiaotong-Liverpool University, Suzhou, Jiangsu, China.

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Contact-rich manipulation poses a stage-dependent sensory conflict: vision and proprioception are essential for localization and approach, when force is largely uninformative and potentially distracting, whereas force becomes critical after contact, when irrelevant visual details and correlated robot states may become shortcuts. We propose DEFT-VLA (Dual-Expert Force-specialized Training for Vision-Language-Action Manipulation), which retains the task-adapted base expert for vision-dominated non-contact behavior and introduces a force-aware contact expert for critical contact stages, localizing force-conditioned adaptation to where it is needed. We further introduce a contact-localized force-attending curriculum that applies force-attending post-training only to the contact expert, progressively restoring vision and masking state tokens to suppress shortcuts. Real-robot experiments on pipette tip attachment, bimanual pipette aspiration, and whiteboard wiping show that DEFT-VLA effectively exploits force feedback to improve contact-stage performance while maintaining strong non-contact behavior and robustness to held-out grasp heights. It improves overall task success by 20-40 percentage points over the strongest baseline on each task.

Figures & tables

Fig. 2: Overview of DEFT-VLA. A shared VLM conditions a task-adapted base expert for non-contact behavior and a force-specialized contact expert for critical contact stages. The contact expert is initialized from the base expert and receives an additional force token. During contact-expert post-training, the contact-localized curriculum gradually restores image clarity and the original state tokens to suppress visual and proprioceptive shortcuts. At inference, a lightweight stage estimator fuses task-conditioned visual features, robot state, and causal wrench history, while a contact-stage router selects the active expert. Snowflake and flame symbols denote frozen and trainable modules, respectively.
Fig. 3: Robot platform and sensor configuration. The Mercury B2 dual-arm robot is equipped with two arm-mounted cameras, one third-person camera, and two wrist-mounted six-axis force/torque sensors.
Fig. 4: Three contact-rich manipulation tasks and their stage decomposition. In each row, the first three frames show the vision-dominated preparatory stage handled by the task-adapted base expert, while the last three show the critical contact stage handled by the force-specialized contact expert. The central boundary denotes the transition from the base expert to the contact expert.
Fig. 5: Representative non-contact-stage outcomes. The top and bottom rows show failed and successful visual alignment, respectively. (a) Alignment between the left end-effector and the pipette plunger button in bimanual aspiration. (b) Alignment between the pipette tip cone and the target tip opening in tip attachment.
MethodPipette tip attachmentBimanual pipette aspirationWhiteboard wiping
 Approach & AlignPress & RetractOverallPosition & AlignPress & ReleaseOverallVolume MAE ( μL\mu\mathrm{L} ) ↓\downarrowApproach & GraspContact & WipeOverall
π0.5\pi_{0.5}90%33.3%30%95%5.3%5%120.0100%70%70%
π0.5\pi_{0.5} + Force Token40%25%10%100%20%20%132.5100%45%45%
π0.5\pi_{0.5} + Force Token + Global FAC85%29.4%25%35%14.3%5%420.050%60%30%
DEFT-VLA w/o Contact-Localized FAC80%25%20%100%30%30%130.0100%55%55%
DEFT-VLA (Ours)75%93.3%70%100%50%50%104.0100%90%90%
TABLE I: Stage-wise and overall success rates across three contact-rich manipulation tasks.
Fig. 6: Height generalization protocol. Height labels refer to the gripper’s grasp position along the pipette. (a) Tip attachment at the Normal and held-out Low positions. (b) Bimanual aspiration at the Normal and held-out Low and High positions. Red dashed lines indicate the Normal reference in the magnified views.
MethodPipette tip attachmentBimanual pipette aspiration
π0.5\pi_{0.5}0%0%
π0.5\pi_{0.5} + Force Token10%10%
π0.5\pi_{0.5} + Force Token + Global FAC0%0%
DEFT-VLA w/o Contact-Localized FAC0%40%
DEFT-VLA (Ours)50%50%
TABLE II: Success rates under height generalization.
MethodVisual CurriculumState MaskingSuccess Rate
DEFT-VLA w/o Contact-Localized FAC––20%
DEFT-VLA w/o State Masking✓\checkmark–15%
DEFT-VLA (Ours)✓\checkmark✓\checkmark70%
TABLE III: Ablation of state-token masking on pipette tip attachment.
More in this topic

Contact-Rich Robotic Manipulation

Contact-rich robotic manipulation studies how robots plan and control physical interactions such as grasping, insertion, pushing, and dexterous object handling.

See all 413 papers

Explore similar work

CardsList

  1. ForceVLA2: Unleashing Hybrid Force-Position Control with Force Awareness for Contact-Rich Manipulation

    Mar 16, 2026Yang Li, Zhaxizhuoma, Hongru Jiang +11Contact-Rich Robotic ManipulationForce-Controlled Robotic Manipulation

    Tongji University · Shanghai AI Laboratory · Lumos Robotics +6

  2. VisForce: Visual Grounding of Current and Desired Forces for Goal-Conditioned Dexterous Manipulation

    Sep 22, 2026Jung-Woo Lee, Soo-Chul LimContact-Rich Robotic ManipulationGoal-Conditioned Robotic Manipulation

  3. ForceDelta-VLA: Distilling Force-Conditioned Action Corrections for Contact-Rich Manipulation

    Sep 16, 2026Ju Dong, Yu Fu, Jian Chen +9Contact-Rich Robotic ManipulationForce-Controlled Robotic Manipulation

    TAMS (Technical Aspects of Multimodal Systems), Department of Informatics, University of Hamburg, Hamburg, Germany. · University of Science and Technology of China, Hefei, China. · Technical University of Munich, Germany.