Vision-language-action (VLA) models re-encode the full observation through a multi-billion-parameter vision-language model (VLM) at every control step, even though scene semantics evolve at roughly 1 Hz while reactive control requires 20-50 Hz. We present a VLA that decouples the two: a VLM runs as a low-frequency semantic anchor whose cached context is reused across control steps, while a flow-matching action expert operates at high frequency, consuming the cached semantics with a fresh, low-latency visual stream from a compact encoder pretrained on robot video with masked reconstruction, optical flow, and depth. The measured age τ of the cache conditions the expert through adaptive layer normalization. Reusing the cache makes the policy a function of its age, and training covers that function explicitly: cache ages sampled from real episode history, plus an age-ramped pixel corruption that erases the stale snapshot the way deployed staleness corrupts content. On the asynchronous deployment stack, the recipe closes most of the staleness gap: on LIBERO-Spatial, 61.4% to 95.5% at a 2 s semantic refresh (95.9% at 1 s, within 1.6 points of the frequency-matched anchor); averaged over the four LIBERO suites, the deployed policy holds 95.2% at a 1 s refresh and 94.5% at 2 s against a 96.3% matched anchor. A forward-only chunk-boundary correction (soft-nudge), an autograd-free reformulation of real-time-chunking guidance, carries per-control-step re-planning without loss (97.1%) at zero added latency. The same lens exposes a second defect: under an instruction-scene mismatch the policy keeps manipulating the scene, and a chain-of-thought verdict gate trained on zero-cost negative episodes freezes execution exactly on mismatched instructions without hurting feasible success.
Figures & tables
Fig. 1: Leap-VLA overview. Slow system: the VLM runs at fslow over a token sequence that appends, after the prompt and the observation, the chain-of-thought region; it produces the cached context C and the chain-of-thought-supervised anchor features, and its reasoning region is emitted as the structured output shown. Because attention is causal, the prompt-region features the expert consumes are unchanged by the appended block (Sec. III-B ). Fast system: the flow-matching expert runs at ffast , consuming C and fresh fast-stream features through one pluggable token-block interface: image tokens from the frozen motion-aware encoder (dual-frame, robot-pretrained with reconstruction, optical flow, and depth). The measured cache age τ conditions the expert via AdaRMS, proprioception enters as a dedicated input token, and the control rate f is passed to the action head; training samples τ from the physical episode history.
Fig. 2: Fast-stream pretraining objective. A dual-frame pair (ot−δ,ot) is fused into a 6-channel input at the stem; a ∼49% row/column strip mask is applied jointly to both frames (pair-level tube masking), and the visible patches go through the compact encoder. Three lightweight decoders share the encoded features: the reconstruction decoder fills in both frames’ masked patches, the optical-flow decoder predicts the dense ot−δ→ot motion of masked patches, and the depth decoder predicts per-pixel depth for each frame. Panels show a held-out RoboTwin sample; decoder outputs are the training-time ground truth (WAFT flow, Depth Anything V2 depth, and the raw frames).
Model
Goal
Spatial
Object
Long
Avg
π0 [ 4 ]
94.4
98.0
96.8
88.4
94.4
InternVLA-M1 [ 34 ]
93.8
98.0
99.0
92.6
95.9
Leap-VLA (ours)
97.2
97.5
98.0
92.4
96.3
π0.5 [ 5 ]
98.0
98.8
98.2
92.4
96.9
GR00T-N1.7 [ 6 ]
97.5
97.7
98.5
94.4
97.0
OpenVLA-OFT [ 33 ]
97.9
97.6
98.4
94.5
97.1
TABLE I: Published LIBERO four-suite success (%), transcribed from each method’s original publication (as aggregated in the public X-VLA benchmark table). Published rows follow each method’s own evaluation protocol and are not commensurable with one another; the table is public context, not a controlled comparison. The Leap-VLA row reports our fine-tuned model, measured on the asynchronous deployment stack used throughout this paper at the matched-rate operating point (DiT:VLM scheduling ratio 1:1 ; 50 episodes per task; Appendix A ).
Semantic-condition refresh period Tslow
Suite
0.5 s
1 s
2 s
DiT:VLM 1:1
2:1
4:1
Goal
97.2
95.8
95.6
Spatial
97.5
95.9
95.5
Object
98.0
98.0
96.2
Long
92.4
91.2
90.8
TABLE II: Leap-VLA on all four LIBERO suites as the semantic loop slows ( 50 episodes per task, fixed seed; the client re-plans every 0.5 s and the semantic condition is refreshed every 0.5/1/2 s, i.e., on every 1st/2nd/4th re-plan; stale-clock gating on observation timestamps, Appendix A ).
Training variant
Semantic-condition refresh period Tslow
0.2 s (matched)
1 s
2 s
Frequency-matched fine-tuning (reference)
97.5
82.9
61.4
+ asynchronous Δt exposure
–
92.5
91.8
+ measured- Δt conditioning
–
94.5
92.9
+ stale-frame masking curriculum
–
95.9
95.5
TABLE III: Success (%) on LIBERO-Spatial versus semantic-condition refresh period (500 scored episodes per cell). Each row adds one training ingredient to the row above. The frequency-matched reference is fine-tuned at the matched period and never sees asynchronous data. Re-planning is held at Treplan=0.5 s throughout; the matched column evaluates at the fine-tuning refresh period ( 0.2 s).
Semantic-condition refresh period
1 s
2 s
Δ (2 s − 1 s)
Measured context age (s)
0/0.5 alternating
0→1.5 cycle
–
Frequency-matched fine-tuning (reference)
82.9
61.4
−21.5
Asynchronous training (ours)
95.9
95.5
−0.4
per-variant range
[92.5,95.9]
[91.8,95.5]
[−1.6,−0.4]
TABLE IV: Success (%) versus semantic-condition refresh period on the deployment line (LIBERO-Spatial, 500 episodes per cell—the same measured cells as Table III ). Asynchronous training (ours) is the full asynchronous training recipe of that table; the range row spans the three asynchronous training variants. Measured context age follows the deployed-age protocol of Sec. IV-A .
Re-planning period Treplan
Training variant
0.5 s ( R=10 ; default)
0.05 s ( R=1 )
0.05 s ( R=1 ) + soft-nudge
Frequency-matched fine-tuning (reference)
61.4
26.9
39.2
Asynchronous training ( + stale-frame masking)
95.5
91.4
97.1
TABLE V: Success (%) on LIBERO-Spatial at the Tslow=2 s semantic-refresh operating point of Table III , varying the re-planning period Treplan —the clock at which a fresh action chunk is requested, distinct from the semantic refresh (the R=1 columns re-plan on every 50 ms control step and execute a single action per re-plan). Soft-nudge: η=1 , exponential prefix schedule spanning the first 10 entries of the fresh chunk (Sec. IV-D ).
Execution
boundary/intra ↓
peak bnd. ↓
RMS ∣Δa∣↓
path len. ↓
No correction (baseline)
1.907 ± 0.731
0.281 ± 0.096
0.091 ± 0.021
7.29 ± 1.30
RTC guidance (autograd) [ 16 ]
1.194 ± 0.271
0.207 ± 0.076
0.086 ± 0.023
6.96 ± 1.33
Soft-nudge (ours, forward-only)
1.035 ± 0.186
0.181 ± 0.068
0.085 ± 0.024
6.91 ± 1.32
TABLE VI: Chunk-boundary smoothness on the asynchronous stack, base checkpoint: the 200-trajectory LIBERO evaluation. Boundary metric: 6-DoF displacement at the first executed step of each new chunk, normalized by the mean intra-chunk displacement; peak boundary = per-episode maximum; ± is the 95% CI over episodes.
Fig. 3: Chunk-boundary analysis over the seam events of the 200-trajectory evaluation behind Table VI . (a) Displacement at the boundary step, aligned over all boundary events (mean ± 95% CI; maxima 0.120 / 0.078 / 0.070; the double spike reflects the 10-step chunk cadence). (b) Boundary events paired by ordinal position, corrected vs. uncorrected displacement (log–log); the median corrected/uncorrected ratio is 0.60 for RTC guidance and 0.52 for soft-nudge.
Single system (same-frequency)
Leap-VLA (decoupled)
Gain
Action latency per request
73.9 ms
26.7 ms
2.8×
Compute per request (semantics amortized 1:4)
73.9 ms
38.5 ms
1.9×
Max single-stream re-planning rate
13.5 Hz
26 Hz
1.9×
GPU occupancy at the 0.05 s clock
148 % (infeasible)
56 %
feasible only decoupled
TABLE VII: Deployment cost on a single RTX 4090D (batch 1, the stack of Sec. IV-D : bf16 semantic backbone, fp32 fast loop and action expert; p50 over ≥ 30 requests after warm-up): the same model and weights, only the schedule changes, each system at its best measured configuration (denoise loop graph-captured on both sides). Occupancy is the GPU duty cycle while serving the most reactive re-planning clock of Table V (one request per 50 ms control step, at that column’s 2 s semantic refresh).
Model
Camera
Robot
Language
Light
Background
Noise
Layout
Total
OpenVLA [ 3 ]
0.8
3.5
23.0
8.1
34.8
15.2
28.5
15.6
UniVLA [ 44 ]
1.8
46.2
69.6
69.0
81.0
21.2
31.9
43.9
π0 [ 4 ]
13.8
6.0
58.8
85.0
81.4
79.0
68.9
53.6
OpenVLA-OFT w [ 33 ]
10.4
38.7
70.5
76.8
93.6
49.9
69.9
55.8
π0 -Fast [ 43 ]
65.1
21.6
61.0
73.2
73.2
74.4
68.8
61.6
OpenVLA-OFT m [ 33 ]
55.6
21.7
81.0
92.7
91.0
78.6
68.7
67.9
TABLE VIII: LIBERO-plus [ 18 ] leaderboard: success (%) over the full 10,030 perturbed tasks, grouped by perturbation dimension. All rows except Leap-VLA are quoted from the benchmark’s public leaderboard under the same protocol. Bold: best among models trained without LIBERO-plus data; † marks the single entry fine-tuned on LIBERO-plus data (not comparable). Leap-VLA is trained only on the original LIBERO suites and evaluated zero-shot on every perturbation.
Fig. 4: Robustness to environment-level perturbations on LIBERO-plus (difficulty L5), one successful episode per row. Top: heavy visual noise corrupts the observations, yet the behavior is unaffected. Middle: an extreme camera viewpoint. Bottom: an unseen robot embodiment. Under the shared evaluation protocol, our model ranks 1st on Robot (68.4%), Camera (77.2%), and Noise (82.6%) among models trained without LIBERO-plus data (69.2% overall, first tier).
Model
Object
Position
Language
Task
Environment
Avg
π0.5 [ 5 ]
96.0
20.8
95.8
0.8
52.8
53.2
OpenVLA [ 3 ]
93.0
0.0
97.3
0.0
68.0
51.7
Leap-VLA (ours)
90.0
9.5
85.7
20.4
45.1
50.1
X-VLA [ 32 ]
79.0
0.8
96.8
6.8
—
45.8
π0 [ 4 ]
90.5
0.0
90.5
0.0
38.8
44.0
Molmoact [ 45 ]
76.0
1.5
85.8
1.5
—
41.2
TABLE IX: Official-protocol perturbation summary: success (%) averaged over the four LIBERO suites per dimension. Baseline rows are quoted from the official evaluation; “—” marks dimensions not reported for that model (excluded from its average). Leap-VLA is trained only on the original LIBERO suites (Qwen2.5-VL-2B backbone) and evaluated zero-shot on every perturbation; perturbation initial states are regenerated by our harness (same distribution).
Fig. 5: Task-perturbation generalization on LIBERO-PRO (libero_goal). Top: the original task “put the wine bottle on the rack”, which appears in the training data—our model completes it. Bottom: only the goal is changed (“Put the wine bottle on the stove”); the scene and objects are identical. Our model still executes the new instruction, whereas VLA baselines score 0.0–6.8% on this perturbation dimension (ours: 20.4% averaged over the four suites, 8.4–49% per suite). Both rows are successful episodes re-recorded with the evaluation protocol of Table IX (asynchronous deployment: semantic anchor refreshed at each 0.5 s re-plan, fast stream and measured cache age updated every control step); frames are keyframes sampled from the 100 ms-interval agentview replay.
Fig. 6: Real-robot deployment platforms. From left to right: Realman dual-arm workstation, Galaxea dual-arm workstation, Dobot Atom mobile dual-arm humanoid, and our in-house humanoid upper body. All policies run on-board on the same Leap-VLA checkpoint layout; only the embodiment config differs per platform.
Task
our same-freq.
Leap-VLA
Realman
Sort blocks by color
0.94
0.94
Pack workpieces into box
0.74
0.70
4-piece pick-and-place
1.00
1.00
Dobot Atom
Pick-and-place workpiece
0.86
0.80
TABLE X: Real-robot success rate, 50 trials per task. Tasks correspond to the seven rows of Fig. 7 . Both columns use the Leap-VLA architecture (Sec. III ): Same-freq. is trained without staleness awareness and deployed with the VLM at every control step (the compute-unconstrained anchor); Leap-VLA is the full staleness-aware recipe deployed under slowed semantic refresh. The naive-reuse collapse that motivates the recipe is quantified in simulation (Sec. IV-C ).
Fig. 7: Qualitative real-robot rollouts on the four platforms of Fig. 6 . Each row shows five keyframes of one episode (start, approach, grasp, transport, final state) sampled from the onboard camera stream, one task per row: sorting blocks by color and packing workpieces into a box (Realman, rows 1–2); four-piece continuous dual-arm pick-and-place (Realman, row 3); pick-and-place of a workpiece (Dobot Atom, row 4); color-based sorting and word spelling with letter cubes (Galaxea, rows 5–6); placing a bottle into a marked target zone (in-house humanoid, row 7). All keyframes are sampled from successful autonomous segments of the policy rollout; in the interactive task (row 7) a human only hands over and re-stands the bottle between cycles, which is not shown.
Config
Seen mismatch
Unseen mismatch
Feasible success
Success-only baseline
≈ 100% false execution ∗
≈ 100% false execution ∗
–
+ zero-cost negatives
1.29 / 41%
1.31 / 41%
20/20
+ goal-scene positives (no CoT)
2.24 / 66%
2.28 / 70%
20/20
+ positives + CoT verdict (policy only)
0.05 / 0%
0.05 / 0%
20/20
+ positives + CoT verdict (gated)
0.02 / 0%
0.02 / 0%
20/20
TABLE XI: Behavior under instruction–scene mismatch (mismatch modes: 30 pairs × 3 trials; feasible: 10 tasks × 2 trials; 300 steps). Mismatch cells report rollout motion (mean absolute grayscale frame difference) / fraction of active frames; a fully frozen rollout reads 0.02/0%. ∗ Qualitative probe: the success-only policy keeps manipulating the scene on essentially every mismatched rollout (Fig. 8 ); motion metrics were not run for it.
Fig. 8: Left: false execution under instruction–scene mismatch—each row is one rollout at increasing time fractions, instruction quoted above; the instruction refers to absent objects, yet the policy grasps an unrelated plate (top) or knocks over a bottle (bottom). Right: a consistency-gated policy freezes on a feasible instruction (“open the middle drawer of the cabinet”, t=0/50/97%): the negatives-only policy executes correctly, while behind a CoT-verdict gate misjudged as inconsistent the policy freezes for the entire episode (0/20 feasible tasks under gating vs 6/20 without)—a property of scene-imbalanced gate supervision, not of gating itself: after scene-balanced retraining the gate passes 20/20 feasible tasks and matches the ungated policy.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Fig. 9: The eight boundary windows with the largest uncorrected-vs-corrected contrast, one per episode (two LIBERO-Goal tasks, 200-trajectory evaluation; Table VI ). Each panel shows the per-step 6-DoF displacement around one boundary (step 0 = first executed step on the new chunk; each method aligned to its own nearest boundary); the title gives the boundary displacement of all three executions (no-correction/guidance/nudge). Both corrections remove most of the boundary spike; soft-nudge consistently more.
Fig. 10: Per-episode distributions of the four smoothness metrics of Table VI (bars: mean ± 95% CI; dots: individual episodes).
Stale self-leftover ( ∥c∥=0.166 )
Foreign leftover ( ∥c∥=1.237 )
Correction
dtail
dgt
dtail
bridging
dgt
None (reference)
–
0.080
–
–
0.080
RTC inpainting [ 16 ]
0.170
0.086
1.204
2.7%
0.085
RTC guidance [ 16 ]
0.164
0.082
1.208
2.4%
0.092
Soft-nudge η=0.1
0.165
0.081
1.235
0.2%
0.080
Soft-nudge η=0.3
0.165
0.081
1.229
0.7%
0.079
Appendix
TABLE XII: Leftover-conflict probe (base checkpoint, 8 paired samples). Left: stale self-leftover; right: foreign (off-manifold) leftover. Bridging =1−dtail/∥c∥ . No correction reaches the leftover at the conflict row; suffix tracking is preserved throughout.
Checkpoint family
No corr.
RTC guidance
Soft-nudge
Frozen-backbone expert (Goal)
2.87
1.04
–
Deployed expert (Goal)
1.64
1.11
–
Deployed expert + prefix fine-tuning (15k)
2.05
1.04
–
Deployed expert (Table VI protocol)
1.91
1.19
1.04
Appendix
TABLE XIII: Boundary/intra ratio without and with inference-time correction, across checkpoint families. Rows use different suites, episode sets, and window sizes; compare within a row only.
Fig. 11: Environment-perturbation generalization on LIBERO-PRO. Each pair shows the same task in the original environment (top) and in an unseen perturbed environment (bottom): the light table is replaced by a dark wooden one, with background and lighting changed as well. Our model completes the task in both settings—placing the book into the rear compartment of the caddy (rows 1–2) and the tomato sauce into the basket (rows 3–4). Episodes are successful takes re-recorded with the evaluation protocol of Table IX ; frames are keyframes of the 100 ms-interval agentview replay.
Fig. 12: Language-perturbation robustness on LIBERO-PRO. Each pair shows the same task from the same initial state under the original instruction (top) and a paraphrase (bottom): a telegraphic rewording (“put book in rear compartment of caddy”, rows 1–2) and an elliptical one (“place salad dressing in basket”, rows 3–4). The executed behavior is essentially unchanged and both episodes succeed, indicating the policy follows the task semantics rather than surface word patterns.
Fig. 13: Failure analysis on LIBERO-PRO task perturbations (libero_goal). Rows 1–3: under task-perturbed instructions, the characteristic failure is semantic interference—the policy reverts to the behavior of the trained task. Asked to turn the stove off, it turns the burner on (row 1); asked to put the wine bottle on the stove, it carries the bottle back to the trained rack goal (rows 2–3). Row 4: on the unperturbed original task, failures are motor-level instead (the grasp slips and the bottle is dropped). Perturbed-instruction failures therefore reflect instruction generalization rather than manipulation competence. These modes are systematic rather than cherry-picked: 2/2 recorded failures of “turn off the stove” turn it on, and 2/6 recorded failures of the wine-bottle task revert to the rack.
Fig. 14: Representative failures on LIBERO-plus for the two weakest perturbation categories of our model: extreme lighting (Light, 53.5%) and unseen table textures (Background, 55.6%), both at difficulty L5. Rows 1–2 (dim lighting): the policy reaches and manipulates the bowl but fails to complete the placement—the bowl is knocked beside the plate (row 1), or the bowl is never extracted from the cabinet (row 2). Rows 3–4 (dark table texture): the policy manipulates the distractor bowl next to the plate while the target bowl remains on the ramekin (row 3), and the grasp on the can fails so that it never reaches the basket (row 4). All episodes run to the step limit without a success.
Vision-Language-Action models (VLAs) have demonstrated strong task understanding and generalization in robotic manipulation, yet the high computational cost of full-model inference limits their deployment in low-latency, high-frequency closed-loop control. We propose an asynchronous semantic-action decoupling framework that separates semantic understanding from action generation along the internal semantic-action interface of existing VLAs, without redesigning the vision-language backbone or introducing an external planner. A low-frequency understanding module asynchronously updates reusable semantic conditions, while a high-frequency action module continuously outputs control actions without repeatedly invoking the full model. To mitigate the temporal mismatch between stale semantics and the current execution state, we further introduce historical action conditioning and time-misalignment training, which provide short-horizon execution context and improve feedback control robustness under stale semantic conditions. Experiments on LIBERO with π0.5 and UniVLA, together with real-robot deployment using UniVLA, show that the proposed framework achieves up to 35.6 Hz server-side action-module inference throughput and offers a low-intrusion path to high-frequency closed-loop control without running full VLA inference at control rate.
Shenhao Yan, Ge Wang, Qi Liu +7
Northeastern University · Ising AI · CUHK-Shenzhen
Vision-language-action (VLA) policies commonly place a multi-billion-parameter model inside the feedback loop, repeatedly recomputing task context whenever they predict a new action chunk. Yet task understanding changes far more slowly than the robot state. We introduce Fast-VLA, which invokes a frozen 4B VLM once per episode to contextualize the instruction and initial views as fixed, multi-layer key-value task memory. During execution, a lightweight online actor fuses learned time- and arm-indexed queries with current visual and proprioceptive feedback to form action queries. These queries retrieve task context from the fixed memory and predict action chunks without re-running the foundation model. Fast-VLA achieves 92.46% and 92.40% success on RoboTwin 2.0 Clean and Randomized, and 95.95% and 79.11% on LIBERO and LIBERO-Plus evaluation. Against three continually conditioned baselines on the same hardware, it reduces RoboTwin amortized policy inference time by 10.9-17.1x and requires 1.22 GiB of post-prefill online resident memory. Task-memory swaps further show that the actor relies on episode-matched task memory. These results show that task context can be established once while closed-loop control remains responsive throughout execution. Project page: https://garfield-kh.github.io/FastVLA
Kehong Gong, Guanli Hou, Xiaoxu Zheng +6
Huawei Technologies Co., Ltd · Huawei Central Research Institute
Vision-Language-Action (VLA) models are a powerful paradigm for generalist robotic control. However, their high computational cost and limited control frequency hinder real-time robotic manipulation, especially when large vision-language backbones and iterative action heads run at every control step. Existing VLA acceleration methods often optimize individual components or rely on fixed acceleration rules, treating different control steps with largely fixed computation and overlooking the non-uniform reasoning demands of sequential embodied control. Inspired by human motor control, where cognitive and feedback resources concentrate on goal-sensitive stages, we argue that VLA models should learn when to invest full computation and when to reuse prior computation. We propose ElegantVLA, a plug-in phase-adaptive inference framework that accelerates VLA models through intra-model dynamic compute scheduling. ElegantVLA introduces a lightweight scheduler that observes temporal representation similarity, robot-motion cues, and episode progress to jointly allocate computation across the vision encoder, LLM, and action head. For perception-language reasoning, the scheduler selects a five-level Vision-LLM compute mode, from full recomputation to multi-step temporal reuse, based on visual-language representation stability. For action generation, it selects a three-level denoising mode, reusing intermediate denoising states during stable motion while preserving full refinement for goal-sensitive stages. By coordinating these decisions, ElegantVLA offers a general acceleration framework for modern VLA pipelines with explicit action-generation modules, without modifying or retraining the base model. Experiments on GR00T and CogACT achieve up to 2.55x and 3.77x speedup, and on six real-world GR00T tasks ElegantVLA cuts computation by 2.18x while raising control frequency from 13.8 Hz to 26.3 Hz.
Ye Li, Huanan Liu, Kangye Ji +7
Tsinghua University · University of Illinois at Urbana-Champaign