Vision-language-action (VLA) models predict chunks of future actions, and when objects move, those chunks go stale before they are executed. We study what makes a pretrained VLA reactive, using \pihalf{} on two dynamic simulation benchmarks and three real-robot tasks. We vary the prediction and execution horizons, compare synchronous and asynchronous inference, test methods that deliver actions sooner or keep successive chunks continuous, and adapt the policy with motion cues and latent-space reinforcement learning. The best prediction horizon is about one second, and the best execution horizon depends on the checkpoint. Short horizons help more with asynchronous execution, and methods that keep the trajectory continuous outperform those that only reduce latency. Adapting the policy repairs targeted failures, but on the conveyor task changing the execution horizon gained as much. Reactivity depends on the execution pipeline as much as on the policy, so both should be evaluated together.
Figures & tables
Figure 1: Dynamic VLA manipulation depends on the interaction between prediction and execution horizons, inference scheduling, chunk continuity, and policy behavior, which together determine whether frequent replanning improves task performance.
Method
Model
Inference
Execution
Action Rep.
Target
RTC
✓
Chunk transition
FASTER
✓
✓
Action generation
DynamicVLA
✓
✓
Execution alignment
ABPolicy
✓
✓
✓
Chunk continuity
PUMA
✓
Motion information
AAC
✓
✓
Execution horizon
Table 1: Intervention levels and goal of evaluated methods.
Finding
Result and explanation
Training: a prediction horizon of about one second works best.
The best prediction horizon is 25 steps on DOM and 30 steps on DOMINO. At their control rates of 25 and 30 Hz, both are one second of motion (Fig. 2 a). With longer chunks, the robot keeps executing a plan made for an object that has since moved on.
Deployment: the best execution horizon depends on the checkpoint, not on replanning more often.
The 1k checkpoint works best at s=5 and the 5k checkpoint at s=15 (Fig. 2 b). The 5k policy barely moves during the first steps of each chunk. If we replan sooner, the robot sees almost the same scene, predicts the same slow start again, and never gets going. Replanning helps only when the new prediction is different from the old one.
Adaptive horizons do not help.
AAC never beats the best fixed horizon and fails on the static and fast belts (Fig. 2 c). It samples several candidate chunks to measure the entropy of the predicted actions, and this extra inference delays the next chunk. Entropy is the wrong trigger here. It is high when the scene is unfamiliar or when several actions are equally good, and in both cases a new image does not change the decision. It is low when the policy is confident, even if the object is about to move away. AAC therefore replans at the wrong moments and pays extra inference time to do so.
Table 2: RQ1: How do prediction and execution horizons affect dynamic manipulation?
Figure 2: RQ1. (a) Success against the prediction horizon in seconds on DOM and DOMINO; rings mark the best horizon. (b) Conveyor-belt success of the 1k and 5k checkpoints across execution horizons. (c) Success of the 5k checkpoint with fixed horizons and AAC per belt condition ( H=20 , asynchronous, 45 trials each).
Finding
Result and explanation
Scheduling: short horizons help more with asynchronous execution.
At s=10 , the synchronous π0.5 and PUMA policies succeed in only 20% of trials, while the asynchronous methods reach 60 – 75% (Fig. 3 c). On DOM the gap grows as the horizon shrinks (Fig. 3 a). In synchronous execution, the robot finishes a chunk and then waits for the policy to compute the next one; during that wait it does not move. A shorter horizon means more chunks, so more waits, and the object moves on during each of them. In asynchronous execution the next chunk is computed while the current one is still running, so a short horizon adds no waiting.
Continuity: methods that preserve continuity lead, and low latency alone does not.
ABPolicy is the best method; RTC matches FASTER despite FASTER’s lower latency. (Fig. 3 c, Table 5 ). A fresh action is only useful if the robot can follow it. When a new chunk starts far from where the old one ended, the robot has to jump, and that costs more than the faster inference saves.
Action rate: on the fast belt, every method improves at 30 Hz.
Raising the action rate from 20 to 30 Hz lifts ABPolicy from 13 to 93% and FASTER from 0 to 33% on the fast belt (Fig. 3 b). The same chunk is finished sooner, so the object has less time to move away before the next observation.
Table 3: RQ2: When does more frequent replanning actually help?
Figure 3: RQ2. (a) Success on the dynamic DOM task with the same policy ( H=25 ) under synchronous and asynchronous execution. (b) Conveyor-belt success of the best configuration of each method on the static, slow, and fast belt, and on the fast belt at a higher action rate; each bar is 15 trials. (c) Conveyor-belt success of every method across execution horizons, with synchronous methods above the rule. Blank cells were not evaluated; ABPolicy ( ∗ ) was evaluated only at s=9 , as its effective execution horizon depends on inference delay.
Finding
Result and explanation
Motion information: PUMA gives mixed results.
PUMA does better than π0.5 on the conveyor but worse on the rolling can (Fig. 4 a). Motion cues help on the conveyor, and asynchronous execution makes them count: run synchronously, PUMA still fails at short horizons like the baseline, but run asynchronously it reaches 60 – 70% at every horizon (Fig. 3 c). Knowing where the object is going is only useful when the policy also acts on fresh chunks.
Latent steering: DSRL repairs targeted failures without touching the VLA.
On the targeted condition, success rises from 32 to 47% on DOM and from 0 to 60% on the hardest conveyor case (Fig. 4 b). The gain depends on the checkpoint and disappears at a large noise magnitude. DSRL only changes the noise that the frozen policy turns into actions, so it can reach a behaviour only if the policy already contains it.
Scope of steering: the gains stay local and match what a change of execution horizon gives for free.
Averaged over all nine conditions, DSRL changes success by 4 to 20 points and some conditions get worse (Fig. 4 c). It lifts the 1k checkpoint to 64 – 69% , but the same checkpoint reaches 71% at s=5 with no training at all (Fig. 2 b). Changing how the policy is executed was worth as much as training it.
Table 4: RQ3: Can temporal information or reward-guided adaptation make the policy itself more reactive?
Figure 4: RQ3. (a) Best success of π0.5 and PUMA on each real-robot task. (b) Success on the targeted condition before and after DSRL, on DOM in percent and on the conveyor belt as success counts over 5 trials before and 15 after; brackets give the checkpoint and the noise magnitude. (c) Mean success over all nine conveyor conditions after DSRL; the dashed line is the 48.9% of the original checkpoints.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Synchronous execution introduces waiting periods, while asynchronous execution overlaps inference with motion but may cause inter-chunk discontinuities.
Figure 6: Manipulation tasks used in our experiments. (a) Dynamic pick-and-place in DOM. (b) Adjust Bottle in DOMINO. (c) Place Object into Basket in DOMINO.
Vision-Language-Action (VLA) models commonly predict action chunks, limiting their ability to react to environmental changes during execution. Existing asynchronous inference methods improve reactivity but typically rely on a fixed inference gap. In this paper, we propose an event-guided dynamic inference strategy that adapts the inference gap according to scene changes observed since the previous inference. Thereby, it simultaneously preserves motion consistency and prompt reactivity. Across static and dynamic real-world settings, our method consistently performs best, averaging 95% success and exceeding the strongest baseline by 55 percentage points. The code will be made publicly available upon acceptance. The project page is available at https://react-when-you-need-to.github.io/.
Yansong Wu, Huaqing Li, Tianding Hou +2
Technical University of Munich, Germany · Mohamed Bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE
Vision-Language-Action (VLA) models generalize across static manipulation but fail when objects move during task execution. They map the current observation to an action and assume the scene is stationary between observation and execution, so at any non-trivial object speed the resulting latency exceeds the time available to grasp. We close this gap with AHEAD (Anticipatory Horizon Extrapolation with Adaptive Dynamics), a predict-then-act wrapper that augments a frozen VLA with a motion-aware latent world model. A small world model trained on manipulation video forecasts future patch tokens in the VLA's feature space, conditioned on per-token velocity and acceleration from optical flow. A language-and-motion saliency mask concentrates prediction on task-relevant patches, and the model rolls forward for an adaptive horizon, halting when prediction uncertainty crosses a threshold. The frozen action decoder then receives the predicted future tokens in place of the current ones. AHEAD adds 4.9M parameters to a frozen 7B OpenVLA and reaches 79 to 97% success across 20 dynamic simulation scenarios where the strongest baseline reaches 31 to 58%. On a physical UFactory xArm 7, AHEAD succeeds on 29/30 to 30/30 on three conveyor and rolling-ball tasks, 23/30 on paddle interception, and 19/30 on projectile catching where every baseline scores 0/30.
Shahram Najam Syed, Arthur Jakobsson, Haoran Hao +1
Robotics Institute, Carnegie Mellon University, Pittsburgh, USA
Vision-Language-Action (VLA) models predominantly adopt action chunking, i.e., predicting and committing to a short horizon of consecutive low-level actions in a single forward pass, to amortize the inference cost of large-scale backbones and reduce per-step latency. However, committing these multi-step predictions to real-world execution requires balancing success rate against inference efficiency, a decision typically governed by fixed execution horizons tuned per task. Such heuristics ignore the state-dependent nature of predictive reliability, leading to brittle performance in dynamic or out-of-distribution settings. In this paper, we introduce A3, an Adaptive Action Acceptance mechanism that reframes dynamic execution commitment as a self-speculative prefix verification problem. A3 first computes a trajectory-wise consensus score of actions via group sampling, then selects a representative draft and prioritizes downstream verification. Specifically, it enforces: (1) consensus-ordered conditional invariance, which validates low-consensus actions by judging whether they remain consistent when re-decoded conditioned on high-consensus actions; and (2) prefix-closed sequential consistency, which guarantees physical rollout integrity by accepting only the longest continuous sequence of verified actions starting from the beginning. Consequently, the execution horizon emerges as the longest verifiable prefix satisfying both internal model logic and sequential execution constraints. Experiments across diverse VLA models and benchmarks demonstrate that A3 eliminates the need for manual horizon tuning while achieving a superior trade-off between execution robustness and inference throughput.
Feng Chen, Xianghui Wang, Yuxuan Chen +4
University of Adelaide · Sichuan University · Shanghai Jiao Tong University +3