When to Ask for Help: A Decision-Theoretic Account and Measurement Standard for Slow-Module Scheduling in Dual-System Vision-Language-Action Models

Shilin Zhang

cs.ROOct 15, 2026

When to Ask for Help: A Decision-Theoretic Account and Measurement Standard for Slow-Module Scheduling in Dual-System Vision-Language-Action Models

Authors: Shilin Zhang

Organizations: North China University of Technology

Abstract

Dual-system vision-language-action (VLA) models pair a high-frequency fast policy with a slow, expensive module, and published trigger mechanisms for invoking it lack a shared objective, measured costs, and any standard separating schedule quality from module quality. We provide the missing account: invocation is formalized as a calibrated decision problem whose fixed-ratio, fixed-budget, and history-gated schedules are degenerate cases of one quantity, the Value of Intervention (VoI); a renewal baseline proves the optimal schedule periodic under state-independent collapse, with adaptive gains governed by the cost ratio $ρ$ and the dispersion of collapse. Measured systems span three orders of magnitude in $ρ$ (FiS-VLA $0.95$ ; RoboICL about $10^3$ ), and a direct test confirms the low- $ρ$ prediction: calibrated gains vanish under homogeneous collapse and grow with heterogeneity. Fourteen closed-loop arms on LIBERO test the contested middle of the continuum ( $ρ=1.53$ ): the adaptive trigger allocates its queries exactly as the threshold rule predicts, yet the language-rewrite channel cannot convert them into success on either suite, and a revert actuator that undoes a failed rewrite is the only intervention with zero net downside on both LIBERO-Spatial and LIBERO-Goal, where the rewrite deficit deepens from thirteen to forty-two points. A four-measure leak-free protocol cuts apparent early-warning AUC from $0.93$ to $0.62$ on $929$ deployment traces, a detection-limited frontier theorem converts detection quality into a schedulability statement, and a compilation loop turns measured value functions into per-task refresh periods within $2.4\%$ of the closed-loop oracle. Together these results turn slow-module invocation from a design opinion into a measured decision.

Figures & tables

Figure 1: Overview. (a) The dual-system loop: a fast policy

\pi_{f}

acts at control rate conditioned on a slow latent that ages between refreshes; a router invokes the slow module

M_{s}

only when the Value of Intervention exceeds the query cost. (b) The account: one threshold rule subsumes fixed-ratio, fixed-budget, and history-gated scheduling as degenerate cases, and the cost-asymmetry continuum

\rho

spans three orders of magnitude with both ends measured (FiS on hardware, RoboICL from reported API costs). (c) The standard: a four-measure leak-free protocol and a distillation loop that compiles the measured staleness value function into per-task refresh periods.

System	Family	Training	Slow mechanism	Scheduling	$\rho$
FiS-VLA ( Chen et al., 2025 )	trained	SFT + RL	latent refresh	fixed $1{:}4$	$0.95$ (measured)
Hume ( Song et al., 2025 )	trained	Cal-QL head	value best-of- $N$	fixed budget	${\sim}22$ (from rates)
RoboICL ( Liu et al., 2026 )	frozen LLM	none	suffix reuse gate	history heuristic	${\sim}10^{3}$ (from costs)
TUD ( Yang et al., 2026 )	trained	none	generalist re-call	uncertainty trigger	not reported
System Switch ( Bailo, 2026 )	trained	none	confidence defer	confidence trigger	not reported

Table 1: Taxonomy of the source systems plus the concurrent triggers, with the cost asymmetry

\rho

of Eq. ( 12 ) where it is recoverable: measured on hardware for FiS, computed from published control rates for Hume, from published API costs for RoboICL. No trigger mechanism reports

\rho

, the quantity that decides whether it can pay (Section 3.4 ). Entries are restricted to claims verified against the papers’ own sources.

Schedule	Query rate	Success rate	$95\%$ CI
Always query (Hume-like)	1.000	0.921	$\pm.001$
Fixed $1{:}2$	0.500	0.815	$\pm.002$
Fixed $1{:}4$ (FiS-like)	0.250	0.762	$\pm.002$
Fixed $1{:}8$	0.127	0.736	$\pm.003$
Jev-style gate	0.222	0.778	$\pm.002$
Calibrated, $c{=}0.3$	0.282	0.832	$\pm.002$

Table 2: Synthetic testbed (5 seeds). At matched budget

0.25

, the calibrated router outperforms the FiS-like fixed ratio by

6.4

points; at half that budget (

0.127

) it still matches the full-budget fixed

1{:}4

schedule. The calibrated operating point at

0.202

dominates the Jev-style gate on both axes.

Figure 2: Routing frontier (success vs. query rate, mean

\pm

CI over 5 seeds). The calibrated frontier (red) sits below-and-left of the fixed-ratio frontier (orange) across the budget range.

Figure 3: Left: budget-matched success versus uncertainty-signal quality. Right: advantage of calibrated routing over the fixed 1:4 schedule; it decays monotonically but remains positive throughout.

Per-episode statistic	Success ( $n{=}486$ )	Failure ( $n{=}443$ )
Slow decisions (count)	$39.4\pm 1.5$	$69.5\pm 1.1$
Decision density (queries/step)	$0.141\pm.006$	$0.105\pm.005$
Mean dispatched steps (staleness)	$9.8\pm 0.5$	$12.3\pm 0.5$
Late-decision fraction (last 20%)	$0.183\pm.002$	$0.193\pm.001$

Table 3: RoboICL traces: slow-query scheduling by outcome (

95\%

CI). Failed episodes query the slow module less densely and run with greater staleness; the per-episode staleness gap (

+2.5

steps) is significant under label permutation (

p=0.0005

Figure 4: Early-warning AUC as a function of prefix length

K

under four evaluation settings. The permuted-label null is

0.50

for every series; significance vanishes as the observed prefix grows.

Figure 5: Decision-level routing frontier on the 929-episode RoboICL deployment traces (

33{,}443

decision points, leak-free episode-grouped scores, detection rate

a=0.64

). The threshold router dominates every label-blind period at matched call reduction; the binormal envelope of Proposition 7 bounds it from above.

Quantity	Wall-clock	Rate
Slow query (latent refresh)	$52.0$ ms	$19.2$ Hz
Fast action (4-step DDIM)	$54.8$ ms	$18.2$ Hz
one denoising step	$13.7$ ms	$73$ Hz
$c_{\text{slow}}/c_{\text{fast}}$ , per action	$0.95$
$c_{\text{slow}}$ in denoising steps	$3.8$
Fixed $1{:}4$ compute overhead	$+24\%$

Table 4: Latency of the FiS-VLA system on one RTX 4090 (reference code path,

10

–

12

timed runs after warmup; run-to-run spread

53.8

–

61.1

ms for the fast action,

52.0

–

52.1

ms for the slow query). One slow refresh costs about one fast action, or

3.8

denoising steps.

System	Slow call	Fast step	$\rho$
FiS-VLA ( Chen et al., 2025 )	$52.0$ ms local	$54.8$ ms	$0.95$ (meas.)
LIBERO (ours)	$0.46$ – $2.2$ s	$300$ ms	$1.5$ – $7.2$ (meas.)
Hume ( Song et al., 2025 )	VLM at $4$ Hz	$90$ Hz ctrl	${\sim}22$ (rep.)
DualVLN ( Wei et al., 2025 )	VLM consult	fast step	$10$ – $20$ (rep.)
RoboICL ( Liu et al., 2026 )	$36$ – $52$ s API	$40$ ms chunk	${\sim}10^{3}$ (rep.)

Table 5: The cost-asymmetry continuum under one protocol (Eq. ( 12 )). “Meas.” rows are this paper’s runs (Table 4 for FiS-VLA); “rep.” rows read published latencies and control rates. The empirical span is three orders of magnitude, and the LIBERO row shows that co-locating both modules on one GPU already moves a separate-VLM system an order of magnitude down the axis.

Heterogeneity $w$	0	0.05	0.10	0.25
Advantage $\Delta$	$+0.005$	$+0.019$	$+0.022$	$+0.059$
$95\%$ CI	$\pm.004$	$\pm.005$	$\pm.004$	$\pm.005$

Table 6: Budget-matched advantage of calibrated routing over the fixed

1{:}4

schedule as collapse heterogeneity grows (mean-matched; 5 seeds, 100 episodes of 120 steps each). The advantage vanishes in the homogeneous regime and grows monotonically, as Proposition 4 predicts.

Refresh period $k$	1	2	4	8
Action MSE	$2.431$	$2.325$	$2.331$	$2.308$
Per-step latency (ms)	$107.6$	$75.1$	$58.9$	$50.8$

Table 7: The measured frontier of the fine-tuned FiS-VLA on held-out RLBench demonstrations (teacher-forced, 16 episodes from 4 tasks, bf16, one RTX 4090; latency amortised as

c_{\text{slow}}/k+c_{\text{fast}}

with

c_{\text{slow}}=65.3

ms,

c_{\text{fast}}=42.5

ms measured in the same runs). Fidelity is flat in

k

while latency falls

2.1\times

Figure 6: Staleness and drift on the fine-tuned checkpoint, held-out episodes. (a) The staleness-cost curve

\Delta(\delta)

is nonpositive: an aged slow latent costs nothing. (b) The latent itself drifts sub-diffusively, linearly in age rather than as a random walk.

Figure 7: The measured frontier. (a) Action MSE is flat in the refresh period

k

; the shuffled-latent control (

+11\%

) marks the scale at which the latent does matter, an order of magnitude above any staleness effect. (b) Amortised per-step latency falls

2.1\times

over the same range.

Family	$\Delta(\delta)$ shape	$k^{**}$	$k^{*}$	cost gap
tracking	$+b_{0}\,\delta$	5	5	$0.0\%$
drift	$+b_{0}\sqrt{\delta}$	3	5	$2.0\%$
contact	$-c_{0}$ , $\delta\geq 1$	15	15–16	$1.9\%$
flat	$0$	16	16	$0.0\%$
FiS (4 tasks, meas.)	$\leq 0$ , $\delta\leq 8$	—	8 (edge)	—

Table 8: Schedule compilation from the measured value function. Synthetic families (

60

-step episodes,

400

validation episodes per period,

5

seeds):

k^{**}

minimizes the empirical closed-loop cost,

k^{*}

is compiled from probed

\Delta(\delta)

20{,}000

probe steps per age, cost gap is

|C(k^{*})-C(k^{**})|/C(k^{**})

(worst over seeds). The real FiS row repeats the measured result of Section 5.8 : nonpositive

\Delta

compiles every task to the edge of the probed range.

Arm	Channel	Trigger	Success	$95\%$ CI	Calls/ep	Overhead
A	none	—	$17/30$	$[.37,.75]$	$0$	$0\%$
B	rewrite, raw	periodic $25$	$8/30$	$[.12,.46]$	$10.4$	$8.9\%$
B2	rewrite, clean	periodic $25$	$11/30$	$[.20,.56]$	$9.4$	$10.8\%$
C	rewrite, raw	stall	$13/30$	$[.26,.63]$	$10.7$	$9.0\%$
C2	rewrite, clean	stall	$13/30$	$[.26,.63]$	$10.7$	$13.0\%$
A 10	none	—	$52/90$	$[.47,.68]$	$0$	$0\%$

Table 9: LIBERO-Spatial closed-loop arms at

\rho=1.53

(success over

30

episodes per arm with exact

95\%

Clopper–Pearson intervals; the ten-episode extension, superscript

10

, reports the nine tasks common to all arms — ninety paired episodes per arm, see the statistical-reading paragraph). “Overhead” is slow-query wall-clock over measured fast-policy wall-clock; the revert channel’s undos are local and consume no query, so its Calls/ep counts rewrites only. The extension sweeps the rewrite channel over periods

k\in\{10,25,50\}

and a verification screen, the ranking channel over trigger thresholds with a shuffled-selection control that removes the position bias, and adds a revert channel that undoes a rewrite that fails to restore progress. The bottom block replicates arms A, B2 and E4 on LIBERO-Goal, a second task suite fine-tuned and evaluated under the same protocol with an LLM slow endpoint (mean call latency

600

ms); its overhead is slow-call wall-clock over the fast-only arm’s wall-clock on the same tasks.

Vision-Language-Action Models

Vision-language-action models turn visual observations and natural-language instructions into actions for robots and other embodied agents.

See all 1,192 papers

May 4, 2026

Latent Bridge: Feature Delta Prediction for Efficient Dual-System Vision-Language-Action Model Inference

Dual-system Vision-Language-Action (VLA) models achieve state-of-the-art robotic manipulation but are bottlenecked by the VLM backbone, which must execute at every control step while producing temporally redundant features. We propose Latent Bridge, a lightweight model that predicts VLM output deltas between timesteps, enabling the action head to operate on predicted outputs while the expensive VLM backbone is called only periodically. We instantiate Latent Bridge on two architecturally distinct VLAs: GR00T-N1.6 (feature-space bridge) and π0.5 (KV-cache bridge), demonstrating that the approach generalizes across VLA designs. Our task-agnostic DAgger training pipeline transfers across benchmarks without modification. Across four LIBERO suites, 24 RoboCasa kitchen tasks, and the ALOHA sim transfer-cube task, Latent Bridge achieves 95-100% performance retention while reducing VLM calls by 50-75%, yielding 1.65-1.73x net per-episode speedup.

Yudong Liu, Yuan Li, Zijia Tang +12

1Duke University · 2Qualcomm AI Research · University of Florida

Efficient VLM Inference Efficient VLA Model Inference Efficient VLA Models Vision-Language-Action Models VLMs for Robotics

Ask PDF

Oct 4, 2026

Triggering Generalist Reasoning via Predictive Uncertainty for Dual-System VLA

Dual-system Vision-Language-Action (VLA) models improve real-time robotic control by pairing a slow, reasoning-capable generalist with a fast specialist action expert. However, existing methods invoke the generalist at a fixed frequency, ignoring the fact that decision-making complexity varies throughout a rollout. This static strategy wastes computation in easy phases and can delay renewed reasoning when the scene changes unexpectedly. We propose TUD (Triggering generalist reasoning via predictive Uncertainty for Dual-system VLA), an adaptive inference framework that selectively skips unnecessary generalist calls. TUD measures the cross-step dispersion of action re-predictions at the upcoming chunk slot under the cached generalist context, as a predictive uncertainty signal. This signal captures how much the future action plan shifts as new observations arrive and is computed from forwards the architecture already runs, requiring neither manual phase labels nor an auxiliary uncertainty model. On VLA-Arena, it achieves a higher success rate at matched call budgets than alternative uncertainty baselines while maintaining low wall-clock overhead, and more consistently separates successful from failed rollouts. Also, TUD finds a more favorable cost-success trade-off than non-adaptive baselines, tracing an entire operating curve as a single threshold is varied, and substantially reduces VLM calls at matched success rate. The same trade-off appears in our real-robot experiments, where TUD cuts generalist calls by 75% relative to the strongest fixed-interval baseline while achieving an even higher success rate. Our results suggest that predictive uncertainty provides a practical criterion for adaptive reasoning in efficient VLA control.

Hyemin Yang, Wooseong Jeong, Giwon Lee +1

KAIST

Uncertainty Estimation for VLA Models Efficient VLA Model Inference Test-Time Adaptation of VLA Models VLMs for Robotics Reasoning for VLA Models

Ask PDF

Jun 10, 2026

DAM-VLA: Decoupled Asynchronous Multimodal Vision Language Action model

Vision-language-action (VLA) models inherit a shared synchronous clock from vision-language pretraining, processing every input at one rate. This is misaligned with physical interaction, where a high-frequency modality changes at hundreds of hertz, vision evolves more slowly, and language stays constant across an episode. A synchronous VLA oversamples slow modalities, undersamples fast ones, and caps action generation at the lowest effective frequency. We hypothesize that decoupling temporal processing per modality, letting each update and retain information at its own sensor rate, yields stronger representations and more robust control. We present DAM-VLA, which maintains per-modality latent buffers refreshed at sensor rates and read continuously by the action head, integrating new high-frequency modalities through gated cross-attention that leaves the pretrained backbone intact. Across seven contact-rich real-world manipulation tasks, DAM-VLA more than doubles the average success rate of the strongest synchronous baseline (95.2% vs.\ 40.95%) while sustaining smooth, reactive 100,Hz control. Project website: intuitive-robots.github.io/DAM-VLA/

Pankhuri Vanjani, Zhuoyue Li, Jakub Suliga +4

Intuitive Robots Lab, Karlsruhe Institute of Technology (KIT), Germany · NVIDIA · Robotics Institute of Germany

AI Control Robotic Manipulation Vision-Language-Action Models VLMs for Robotics Multimodal Sensor Fusion Temporal Modeling for VLA Models

Ask PDF

Abstract

Vision-Language-Action Models

Explore similar work

Latent Bridge: Feature Delta Prediction for Efficient Dual-System Vision-Language-Action Model Inference

Triggering Generalist Reasoning via Predictive Uncertainty for Dual-System VLA

DAM-VLA: Decoupled Asynchronous Multimodal Vision Language Action model