FAVLA: A Force-Adaptive Fast-Slow VLA model for Contact-Rich Robotic Manipulation
Authors: Yao Li, Peiyuan Tang, Wuyang Zhang, Chengyang Zhu, Yifan Duan, Weikai Shi, Xiaodong Zhang, Zijiang Yang, +2 more
Organizations: University of Science and Technology of China, Hefei, China · Xi’an Jiaotong University, Xi’an, China · Central South University, Changsha, China
Force/torque feedback can substantially improve Vision-Language-Action (VLA) models on contact-rich manipulation, but most existing approaches fuse all modalities at a single operating frequency. This design ignores the mismatched sampling rates of real robot sensors, forcing downsampling of the high-frequency contact cues needed for reactive correction. Combined with common VLM-action-expert (AE) pipelines that execute action chunks largely open loop between expensive VLM updates, unified-frequency fusion often yields delayed responses to impacts, stick-slip, and force spikes. We propose FAVLA, a force-adaptive fast-slow VLA that decouples slow perception planning from fast contact-aware control. FAVLA runs a slow VLM at a fixed low frequency to encode modalities to produce latent representations and to predict near-future force variation. A fast AE then executes at a variable high frequency, conditioning on the latest force sequence data to generate reactive actions. We further introduce a force adapter that injects high-frequency force features into multiple AE layers, and adaptively schedules the AE's execution frequency based on the VLM's predicted force variation. Extensive experiments on contact-rich tasks demonstrate that FAVLA significantly outperforms baselines, achieving superior reactivity and success rates, especially with a smaller contact force during manipulation.
Figures & tables
Figure 1: A comparison between the previous unified-frequency scheme and our force-adaptive fast–slow scheme for the VLA model. (a) Previous methods process the vision, force, and proprioceptive state at a unified input frequency, generating open-loop action chunks that cannot promptly exploit high-frequency force feedback. (b) Our scheme applies a force-adaptive fast-slow scheme. The VLM runs at a slow frequency to encode visual and language context, while the AE runs at a force-adaptive high frequency using the latest force data to generate closed-loop action chunks, enabling reactive robot control.
Figure 2: Overview of the FAVLA . FAVLA integrates a large Slow VLM Backbone for semantic reasoning, and a smaller Fast Action Expert for responsive action control. The model processes multimodal inputs at different frequencies and generates action trajectories via the conditional Flow Matching. Notably, the inference frequency of the Action Expert is adaptively adjusted by the Force-Adaptive Fast-Slow Inference Strategy, ensuring there is a smaller contact force during manipulation.
Figure 3: Our force-adaptive fast-slow inference strategy. The fast AE runs multiple times within each action chunk, conditioned on real-time force. In each cycle, we fix the sampled noise and robot state, then perform the temporal ensemble on overlapped action chunks. The AE execution frequency is adaptively set from the VLM-predicted force variance.
Figure 4: Real-world task scenarios and execution phases. Tasks include (1) USB Insertion, (2) Gear Assembly, (3) Box Flipping, and (4) Board Wiping. These tasks present diverse challenges: tasks 1 and 2 require millimeter-level precision for successful alignment, while tasks 3 and 4 demand real-time force adjustment to handle constantly changing contact dynamics.
Figure 5: Experimental setup for robotic manipulation. (a) Overview of the hardware configuration, including the teleoperation device (3D Space Mouse), 6D-force and vision sensors, and the robotic gripper. (b) The set of experimental tasks.
Figure 6: Comparison of the success rate of our proposed method with baselines on four real-world tasks. Our method significantly outperforms all baselines on four contact-rich tasks.
Method
Gear Assembly
Box Flipping
π0
12.0 N
12.2 N
π0 + Force
13.0 N
13.8 N
TA-VLA
11.3 N
10.0 N
ForceVLA
10.9 N
12.4 N
FAVLA (Ours)
7.7 N
9.9 N
Table 1: Average peak contact forces (N) across contact-rich tasks. Lower values indicate safer and more compliant manipulation.
Configuration
Box Flipping
Board Wiping
Vision-Only
50%
10%
+ Force-Injected AE
65%
60%
+ Force Variance Prediction
70%
60%
+ Force-Adaptive Inference
80 %
70 %
Table 2: Ablation study on proposed components. We validate the effectiveness of each component by incrementally adding it to the vision-only baseline.
Figure 7: Ablation study on frequency ratio. Performance comparison between static frequency ratios ( n AE steps per VLM step) and our proposed adaptive frequency mechanism. Our adaptive approach significantly outperforms all static frequency settings across both tasks.
Figure 8: Visualization of adaptive frequency prediction. (a) Raw force measurements. (b) Computed force variance is used to identify environmental dynamics. (c) Model-predicted frequency ratio, demonstrating increased execution frequency in areas where contact is more likely to occur.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 9: Teleoperation system setup.
Task
Trajectories
Frames
Hours
USB Insertion
80
44,716
0.41 h
Gear Assembly
80
58,038
0.54 h
Box Flipping
50
47,476
0.44 h
Board Wiping
50
48,020
0.45 h
Total
260
198,250
1.84 h
Appendix
Table 3: Dataset statistics for the four manipulation tasks.
Field
Components
Shape
Action (Chunk Size = 32)
Delta End-Effector Pose + Gripper Width
Δx,Δy,Δz,Δϕ,Δθ,Δψ,g
32×7
Observation
End-Effector Pose + Gripper Width
x,y,z,ϕ,θ,ψ,g
7
Force / Torque History (10 Steps)
fx,fy,fz,τx,τy,τz
10×6
Head Image
RGB
360×640×3
Appendix
Table 4: Specification of the observation and action spaces.
Vision-Language-Action (VLA) models have significantly advanced the capabilities of robotic agents in executing diverse tasks; however, they still face challenges in contact-rich manipulation scenarios that require precise physical interactions. To address this limitation, recent studies have attempted to incorporate tactile signals during downstream tasks, enabling pretrained VLAs to interpret tactile feedback. Nevertheless, introducing new modalities during finetuning, which are rarely present in the pretrain stage, may disrupt the pretrained capabilities of VLAs. In addition, the inherently slow inference speed of VLAs hampers real-time responsiveness and limits the effective utilization of tactile feedback for action adjustment. To overcome these challenges, we propose Adaptive Tactile Vision-Language-Action (AT-VLA), which introduces a novel Adaptive Tactile Injection mechanism. This mechanism dynamically determines the appropriate timing and locations for tactile injection, incorporating only when it significantly contributes to action generation, thereby minimizing interference with pretrained representations. Furthermore, to enable rapid and accurate tactile responses, we propose a Tactile Reaction Dual-Stream mechanism, which decouples sensory processing into a slow visual-language stream for low-frequency perceptual reasoning and a fast tactile control stream for high-frequency physical interaction understanding, achieving real-time close-loop responses within 0.04 s. Real-world experiments thoroughly validate the effectiveness of AT-VLA in contact-rich manipulation tasks. The project page is available at: https://sites.google.com/view/at-vla.
Xiaoqi Li, Muhe Cai, Jiadong Xu +5
School of Computer Science, Peking University · PrimeBot · PKU Lab
Vision-language-action (VLA) models have achieved impressive generalization in robotic manipulation, and recent memory-augmented VLAs have relaxed the Markovian assumption by conditioning on past images or language summaries. Vision-based memory approaches address this by conditioning on sampled past image frames, but they are computationally expensive and fundamentally limited when temporal events are visually ambiguous, e.g., pushing a button multiple times with small movements. We propose FM-VLA, a VLA model with force-based memory, enabling temporal context reasoning for non-Markovian, contact-rich manipulation. We encode force histories into compact force memory tokens with a variational autoencoder (VAE) pretrained with force time series reconstruction. By projecting force latent representations and short state history as additional conditioning tokens to the action expert module, we enable VLAs to leverage accumulated contact event history to guide manipulation. We evaluate FM-VLA on three memory-dependent tasks, including finding a hidden block, pressing a button, and wiping a dish for a specific number of times. Our lightweight force memory achieves over 80% success rate with minimal inference overhead, significantly outperforming baseline approaches. Project page: https://qft-333.github.io/FM-VLA-Page/
Ruicheng Li, Qixiu Li, Ruichun Ma +8
Tsinghua University · Microsoft Research · Fudan University +1
Vision-Language-Action (VLA) models have demonstrated strong performance across diverse robotic manipulation tasks, yet their predominantly vision-centric perception and position-controlled execution remain insufficient for contact-rich manipulation. Visual observations alone often provide limited evidence of contact onset and interaction magnitude, while position-control policies cannot respond compliantly to rapidly changing contact dynamics. To bridge both the perception and control gaps, we propose TAO-Force, a force-conditioned VLA framework that combines force-aware policy learning with contact-regulated execution. For force-aware perception, TAO-Force introduces Force-conditioned Feature-wise Linear Modulation (F-FiLM) to inject encoded force feedback into the representations of a frozen pretrained visual-language backbone while preserving its semantic priors. For responsive control, it employs a contact-gated fast-slow architecture, with a slow position-control branch tracking nominal trajectories during non-contact phases and a fast admittance-control branch regulating physical interaction during contact phases. Detailed analyses on a force-perception task and real-world evaluations across four contact-rich manipulation tasks validate the effectiveness and robustness of TAO-Force.
Bohan Gan, Xuanzhang Wen, Yongsheng Zhao +8
China Mobile (Hangzhou) Information Technology Co., Ltd., Hangzhou 310023, China