Enabling Dynamic Tracking in Vision-Language-Action Models via Time-Discrete and Time-Continuous Velocity Feedforward
Authors: Johannes Hechtl, Philipp Schmitt, Georg von Wichert, Wolfram Burgard
Organizations: Siemens Foundational Technologies · Department of Computer Science & Artificial Intelligence, University of Technology Nuremberg, Germany.
While vision-language-action (VLA) models have shown great promise for robot manipulation, their deployment on rigid industrial robots remains challenging due to the inherent trade-off between compliance and responsiveness. Standard Behavior Cloning (BC) approaches predict discrete poses at low frequencies, omitting the velocity and acceleration feedforward terms typically used by low-level compliant controllers. This requires to rely on high stiffness for accurate tracking, thereby sacrificing safe contact dynamics. In this paper, we demonstrate the importance of integrating velocity feedforward terms into VLA policies to resolve this trade-off. We propose two methods for extracting velocity targets from VLAs: a time-discrete finite-difference approximation that serves as a highly effective bridge for existing models, and a continuous Cubic B-Spline action space that natively yields C2 continuous trajectories for high-frequency control. Crucially, both approaches are strictly model-agnostic and compatible with any standard action-chunking architecture, requiring modifications only to teleoperation, data processing, and the low-level controller. We fine-tune the π0.5 model and evaluate both of our approaches on a demanding, contact-rich cube-in-hole task. Our results indicate that incorporating the velocity feedforward term via finite differences significantly improves task execution speed, while the continuous B-Spline approach maintains high overall success rates and provides a foundation for smoother higher-order derivatives without compromising compliance.
Figures & tables
Fig. 1 : Comparison of target trajectory representations (also called action chunks) for low-level controllers. Blue dots indicate time-discrete policy predictions. (a) Baseline: Standard VLA outputs yield stepwise position references without velocity feedforward. (b) Ours-1: Finite-difference approximation assumes linear interpolation between positions, providing piecewise-constant velocities. (c) Ours-2: Our proposed Cubic B-Spline action space generates C2 continuous position and velocity profiles directly from model predictions.
Fig. 2 : Workflow illustrating Cubic B-Spline inference. The VLA outputs control points, which are continuously sampled by the high-frequency controller.
Fig. 3 : Experimental setup. Teleoperation is performed using two OMY-L100 devices.
Fig. 4 : Closeup of the cube-in-hole task. A strict 1\mathrm{m}\mathrm{m}$$ tolerance necessitates compliance.
Fig. 5 : Distribution of episode durations for teleoperation across different controller configurations. We include human performance for comparison.
Frequency
Position (Baseline)
Position & Velocity (Method)
t -statistic
p -value
5\mathrm{H}\mathrm{z}$$
9.58±3.97 ( n=68 )
7.64±3.00 ( n=88 )
−3.45
<0.001
15\mathrm{H}\mathrm{z}$$
8.48±3.04 ( n=83 )
5.85±2.27 ( n=120 )
−7.02
<0.001
60\mathrm{H}\mathrm{z}$$
9.31±3.19 ( n=84 )
5.34±1.41 ( n=122 )
−12.06
<0.001
500\mathrm{H}\mathrm{z}$$
8.37±3.49 ( n=76 )
5.37±1.55 ( n=121 )
−8.20
<0.001
TABLE I : Summary statistics and one-tailed t-test results of Task Completion Times (Seconds) for Position-Only and Velocity-Aware Controllers during Teleoperation. The Velocity Aware Controller leads to significantly shorter Task Execution Times for all tested action frequencies.
Fig. 6 : Cumulative success rate over task execution time for different configurations
Method
Mean (s)
Var.
N
t -stat
p -value ∗
Sig. ( α=0.05 )
Baseline
7.35
1.99
71
–
–
–
Finite Difference
6.05
1.71
70
−5.70
<0.001
Yes
Cubic B-Spline
7.45
4.75
94
0.32
0.626
No
Ablation (Velocity-Feedforward Dataset,
Position Only Rollout)
10.01
6.10
49
7.47
<0.001
Yes
∗ One-tailed p -value testing for a decrease in execution time compared to the baseline. For the ablation we instead test for an increase in execution time.
TABLE II : Task Execution Time Statistics and T-Test Results vs. Baseline
Robot manipulation alternates between low-risk transit phases that call for fast execution and high-risk contact stages that demand slow, precise motion. Yet existing Vision-Language-Action models (VLAs) only inherit a single fixed speed from training demonstrations. Prior efforts to accelerate VLAs through model compression, KV-cache reuse, or reinforcement learning only shift the policy from one fixed speed to another, and leave deceleration almost unexplored. We observe that the magnitude of each predicted action already governs how fast the robot moves, opening a direct route to controllable execution speed. We turn this observation into TempoVLA, a single VLA whose execution speed is controlled by an explicit condition. TempoVLA combines two coupled components. (1) A data-side Variable-Speed Trajectory Augmentation (VSTA) that re-times demonstration to any target speed by merging or splitting actions while preserving its motion semantics. (2) A model-side conditioning mechanism that feeds the speed to the policy. Statistics show that VSTA reaches the requested speed with negligible motion error. Experiments in simulation and on real-world tasks demonstrate that TempoVLA achieves flexible speed control in both directions, while VSTA additionally boosts the default 1× performance via better data utilization. Furthermore, by cooperating with a large multimodal model, TempoVLA realizes dynamic speed control, accelerating through low-risk phases and decelerating for high-risk ones.
Vision-Language-Action (VLA) models are a powerful paradigm for generalist robotic control. However, their high computational cost and limited control frequency hinder real-time robotic manipulation, especially when large vision-language backbones and iterative action heads run at every control step. Existing VLA acceleration methods often optimize individual components or rely on fixed acceleration rules, treating different control steps with largely fixed computation and overlooking the non-uniform reasoning demands of sequential embodied control. Inspired by human motor control, where cognitive and feedback resources concentrate on goal-sensitive stages, we argue that VLA models should learn when to invest full computation and when to reuse prior computation. We propose ElegantVLA, a plug-in phase-adaptive inference framework that accelerates VLA models through intra-model dynamic compute scheduling. ElegantVLA introduces a lightweight scheduler that observes temporal representation similarity, robot-motion cues, and episode progress to jointly allocate computation across the vision encoder, LLM, and action head. For perception-language reasoning, the scheduler selects a five-level Vision-LLM compute mode, from full recomputation to multi-step temporal reuse, based on visual-language representation stability. For action generation, it selects a three-level denoising mode, reusing intermediate denoising states during stable motion while preserving full refinement for goal-sensitive stages. By coordinating these decisions, ElegantVLA offers a general acceleration framework for modern VLA pipelines with explicit action-generation modules, without modifying or retraining the base model. Experiments on GR00T and CogACT achieve up to 2.55x and 3.77x speedup, and on six real-world GR00T tasks ElegantVLA cuts computation by 2.18x while raising control frequency from 13.8 Hz to 26.3 Hz.
Ye Li, Huanan Liu, Kangye Ji +7
Tsinghua University · University of Illinois at Urbana-Champaign
Force/torque feedback can substantially improve Vision-Language-Action (VLA) models on contact-rich manipulation, but most existing approaches fuse all modalities at a single operating frequency. This design ignores the mismatched sampling rates of real robot sensors, forcing downsampling of the high-frequency contact cues needed for reactive correction. Combined with common VLM-action-expert (AE) pipelines that execute action chunks largely open loop between expensive VLM updates, unified-frequency fusion often yields delayed responses to impacts, stick-slip, and force spikes. We propose FAVLA, a force-adaptive fast-slow VLA that decouples slow perception planning from fast contact-aware control. FAVLA runs a slow VLM at a fixed low frequency to encode modalities to produce latent representations and to predict near-future force variation. A fast AE then executes at a variable high frequency, conditioning on the latest force sequence data to generate reactive actions. We further introduce a force adapter that injects high-frequency force features into multiple AE layers, and adaptively schedules the AE's execution frequency based on the VLM's predicted force variation. Extensive experiments on contact-rich tasks demonstrate that FAVLA significantly outperforms baselines, achieving superior reactivity and success rates, especially with a smaller contact force during manipulation.
Yao Li, Peiyuan Tang, Wuyang Zhang +7
University of Science and Technology of China, Hefei, China · Xi’an Jiaotong University, Xi’an, China · Central South University, Changsha, China