Integrating visuomotor policies or Vision-Language-Action (VLA) models with force/torque (F/T) perception has demonstrated significant progress in imitation learning for robotic manipulation. However, existing force-aware VLA models frequently exhibit limited capability in precise force tracking and rapid successive adjustments. This deficiency stems from the limitations of action-chunk execution strategies and the substantial latency between perception and real-time control. Such limitations can lead to task failures and safety risks, particularly when the execution of an action chunk exerts excessive interaction forces without timely adjustment. To overcome this challenge, we propose the Control-aware Compliance VLA (CC-VLA) framework for reactive control. The CC-VLA model employs a multimodal mixture-of-experts (MoE) to encode force signal sequences and vision-language fused feature. Furthermore, it utilizes a multi-stage training strategy to ensure robust perception within the visual-semantic space and effective force perception under sparse sampling conditions. Additionally, a VLA-guided adaptive compliance controller is designed to facilitate precise position tracking during contact-free motion and optimal force-position tracking for contact-rich tasks. To facilitate high-precision F/T data acquisition, we also implement an adversaria shared teleoperation strategy for contact-rich demonstrations that bolsters system safety and interactivity. Extensive real-world experiments demonstrate that CC-VLA significantly improves success rates in challenging force-perception tasks and enhances force-control precision, while providing multi-level safety and robustness under the tested partial-OOD pose-shift settings.
Figures & tables
Figure 1: The CC-VLA framework overview: the VLA model (slow policy) integrates vision, language, force sequences and proprioceptive states to infer action chunks. Meanwhile, the VLA-guided compliance controller (VG-ACC) ensures effective tracking of both the feedforward force and the desired pose.
Figure 2: Overview of our control-aware VLA model. Built upon the π0 pretrained model, historical force sequence encoder and cross-modal fusion expert are proposed to ensure efficient modality fusion and high-quality action generation via flow based shared expert.
Figure 3: The scheme of data collection based on shared teleoperation method.
Figure 4: Overview of task setups used in evaluation.
Figure 5: Diagram of the adaptive compliance controller, which receives desired action sequences inferred by the control-aware VLA model as input.
Figure 6: Success rates across different methods for three contact-rich manipulation tasks.
Figure 7: Z-axis force profiles recorded during the first five trials of wipe the board task using various methods. Notably, trajectory execution times have been normalized to account for discrepancies in the initial robot pose and board placement. The dashed line indicates the demonstrated reference force of 40 N. Data for methods Pi0 and Pi05 are omitted, as these models triggered the robot’s protective stop mechanism due to excessive interaction forces.
Figure 8: (a) Task outcomes: The red horizontal line indicates a protective stop triggered by excessive interaction forces; the cross mark × denotes incomplete wiping; and the check mark √ signifies successful task completion. (b) Z-axis force profiles observed when executing pose sequences generated solely by control-aware VLA inference. (c)–(d) The corresponding stiffness profiles. VG-ACC denotes the proposed VLA-guided adaptive compliance controller.
DP
Pi0
Pi05
Pi0 wi force
ForceVLA
CC-VLA
WB-Base
83.83%
NA
76.12%
28.10%
35.57%
5.52%
WB-OOD
79.33%
81.20%
82.19%
38.15%
52.33%
8.78%
Table 1: Performance (%) of demonstrated force tracking error across different methods.
wo VG-ACC
wi Klow
wi Khigh
wi Kvari
Base
17.73%
7.92%
13.30%
5.52%
Table 2: Performance (%) of demonstrated force tracking error for ablation experiments.
Vision-Language-Action (VLA) models have demonstrated strong performance across diverse robotic manipulation tasks, yet their predominantly vision-centric perception and position-controlled execution remain insufficient for contact-rich manipulation. Visual observations alone often provide limited evidence of contact onset and interaction magnitude, while position-control policies cannot respond compliantly to rapidly changing contact dynamics. To bridge both the perception and control gaps, we propose TAO-Force, a force-conditioned VLA framework that combines force-aware policy learning with contact-regulated execution. For force-aware perception, TAO-Force introduces Force-conditioned Feature-wise Linear Modulation (F-FiLM) to inject encoded force feedback into the representations of a frozen pretrained visual-language backbone while preserving its semantic priors. For responsive control, it employs a contact-gated fast-slow architecture, with a slow position-control branch tracking nominal trajectories during non-contact phases and a fast admittance-control branch regulating physical interaction during contact phases. Detailed analyses on a force-perception task and real-world evaluations across four contact-rich manipulation tasks validate the effectiveness and robustness of TAO-Force.
Bohan Gan, Xuanzhang Wen, Yongsheng Zhao +8
China Mobile (Hangzhou) Information Technology Co., Ltd., Hangzhou 310023, China
Contact-rich manipulation, requiring robots to regulate not only motion but also how they yield to external forces, has emerged as the next frontier for Vision-Language-Action (VLA) models. However, existing VLAs output purely kinematic commands, degrading performance on real-world contact-rich tasks. In this paper, we introduce CompVLA, a unified VLA framework that jointly predicts motion and stiffness matrix from RGB and language inputs. Our approach augments the conventional architecture with a dedicated Compliance Expert, which outputs time-varying stiffness and virtual displacement profiles executed via geometric impedance control. We demonstrate that CompVLA achieves the highest average success rate across diverse contact-rich tasks, outperforming both vanilla and compliance-aware VLA baselines, with ablations confirming each component is essential.
Jongmin Kim, Junsu Ha, Che-Sang Park +5
Seoul National University · Microsoft Research Asia
Force/torque feedback can substantially improve Vision-Language-Action (VLA) models on contact-rich manipulation, but most existing approaches fuse all modalities at a single operating frequency. This design ignores the mismatched sampling rates of real robot sensors, forcing downsampling of the high-frequency contact cues needed for reactive correction. Combined with common VLM-action-expert (AE) pipelines that execute action chunks largely open loop between expensive VLM updates, unified-frequency fusion often yields delayed responses to impacts, stick-slip, and force spikes. We propose FAVLA, a force-adaptive fast-slow VLA that decouples slow perception planning from fast contact-aware control. FAVLA runs a slow VLM at a fixed low frequency to encode modalities to produce latent representations and to predict near-future force variation. A fast AE then executes at a variable high frequency, conditioning on the latest force sequence data to generate reactive actions. We further introduce a force adapter that injects high-frequency force features into multiple AE layers, and adaptively schedules the AE's execution frequency based on the VLM's predicted force variation. Extensive experiments on contact-rich tasks demonstrate that FAVLA significantly outperforms baselines, achieving superior reactivity and success rates, especially with a smaller contact force during manipulation.
Yao Li, Peiyuan Tang, Wuyang Zhang +7
University of Science and Technology of China, Hefei, China · Xi’an Jiaotong University, Xi’an, China · Central South University, Changsha, China