When to Ask for Help: A Decision-Theoretic Account and Measurement Standard for Slow-Module Scheduling in Dual-System Vision-Language-Action Models

Shilin Zhang

cs.ROOct 15, 2026

When to Ask for Help: A Decision-Theoretic Account and Measurement Standard for Slow-Module Scheduling in Dual-System Vision-Language-Action Models

Authors: Shilin Zhang

Organizations: North China University of Technology

Abstract

Dual-system vision-language-action (VLA) models pair a high-frequency fast policy with a slow, expensive module, and published trigger mechanisms for invoking it lack a shared objective, measured costs, and any standard separating schedule quality from module quality. We provide the missing account: invocation is formalized as a calibrated decision problem whose fixed-ratio, fixed-budget, and history-gated schedules are degenerate cases of one quantity, the Value of Intervention (VoI); a renewal baseline proves the optimal schedule periodic under state-independent collapse, with adaptive gains governed by the cost ratio $ρ$ and the dispersion of collapse. Measured systems span three orders of magnitude in $ρ$ (FiS-VLA $0.95$ ; RoboICL about $10^3$ ), and a direct test confirms the low- $ρ$ prediction: calibrated gains vanish under homogeneous collapse and grow with heterogeneity. Fourteen closed-loop arms on LIBERO test the contested middle of the continuum ( $ρ=1.53$ ): the adaptive trigger allocates its queries exactly as the threshold rule predicts, yet the language-rewrite channel cannot convert them into success on either suite, and a revert actuator that undoes a failed rewrite is the only intervention with zero net downside on both LIBERO-Spatial and LIBERO-Goal, where the rewrite deficit deepens from thirteen to forty-two points. A four-measure leak-free protocol cuts apparent early-warning AUC from $0.93$ to $0.62$ on $929$ deployment traces, a detection-limited frontier theorem converts detection quality into a schedulability statement, and a compilation loop turns measured value functions into per-task refresh periods within $2.4\%$ of the closed-loop oracle. Together these results turn slow-module invocation from a design opinion into a measured decision.

Figures & tables

Figure 1: Overview. (a) The dual-system loop: a fast policy

\pi_{f}

acts at control rate conditioned on a slow latent that ages between refreshes; a router invokes the slow module

M_{s}

only when the Value of Intervention exceeds the query cost. (b) The account: one threshold rule subsumes fixed-ratio, fixed-budget, and history-gated scheduling as degenerate cases, and the cost-asymmetry continuum

\rho

spans three orders of magnitude with both ends measured (FiS on hardware, RoboICL from reported API costs). (c) The standard: a four-measure leak-free protocol and a distillation loop that compiles the measured staleness value function into per-task refresh periods.

System	Family	Training	Slow mechanism	Scheduling	$\rho$
FiS-VLA ( Chen et al., 2025 )	trained	SFT + RL	latent refresh	fixed $1{:}4$	$0.95$ (measured)
Hume ( Song et al., 2025 )	trained	Cal-QL head	value best-of- $N$	fixed budget	${\sim}22$ (from rates)
RoboICL ( Liu et al., 2026 )	frozen LLM	none	suffix reuse gate	history heuristic	${\sim}10^{3}$ (from costs)
TUD ( Yang et al., 2026 )	trained	none	generalist re-call	uncertainty trigger	not reported
System Switch ( Bailo, 2026 )	trained	none	confidence defer	confidence trigger	not reported

Table 1: Taxonomy of the source systems plus the concurrent triggers, with the cost asymmetry

\rho

of Eq. ( 12 ) where it is recoverable: measured on hardware for FiS, computed from published control rates for Hume, from published API costs for RoboICL. No trigger mechanism reports

\rho

, the quantity that decides whether it can pay (Section 3.4 ). Entries are restricted to claims verified against the papers’ own sources.

Schedule	Query rate	Success rate	$95\%$ CI
Always query (Hume-like)	1.000	0.921	$\pm.001$
Fixed $1{:}2$	0.500	0.815	$\pm.002$
Fixed $1{:}4$ (FiS-like)	0.250	0.762	$\pm.002$
Fixed $1{:}8$	0.127	0.736	$\pm.003$
Jev-style gate	0.222	0.778	$\pm.002$
Calibrated, $c{=}0.3$	0.282	0.832	$\pm.002$

Table 2: Synthetic testbed (5 seeds). At matched budget

0.25

, the calibrated router outperforms the FiS-like fixed ratio by

6.4

points; at half that budget (

0.127

) it still matches the full-budget fixed

1{:}4

schedule. The calibrated operating point at

0.202

dominates the Jev-style gate on both axes.

Figure 2: Routing frontier (success vs. query rate, mean

\pm

CI over 5 seeds). The calibrated frontier (red) sits below-and-left of the fixed-ratio frontier (orange) across the budget range.

Figure 3: Left: budget-matched success versus uncertainty-signal quality. Right: advantage of calibrated routing over the fixed 1:4 schedule; it decays monotonically but remains positive throughout.

Per-episode statistic	Success ( $n{=}486$ )	Failure ( $n{=}443$ )
Slow decisions (count)	$39.4\pm 1.5$	$69.5\pm 1.1$
Decision density (queries/step)	$0.141\pm.006$	$0.105\pm.005$
Mean dispatched steps (staleness)	$9.8\pm 0.5$	$12.3\pm 0.5$
Late-decision fraction (last 20%)	$0.183\pm.002$	$0.193\pm.001$

Table 3: RoboICL traces: slow-query scheduling by outcome (

95\%

CI). Failed episodes query the slow module less densely and run with greater staleness; the per-episode staleness gap (

+2.5

steps) is significant under label permutation (

p=0.0005

Figure 4: Early-warning AUC as a function of prefix length

K

under four evaluation settings. The permuted-label null is

0.50

for every series; significance vanishes as the observed prefix grows.

Figure 5: Decision-level routing frontier on the 929-episode RoboICL deployment traces (

33{,}443

decision points, leak-free episode-grouped scores, detection rate

a=0.64

). The threshold router dominates every label-blind period at matched call reduction; the binormal envelope of Proposition 7 bounds it from above.

Quantity	Wall-clock	Rate
Slow query (latent refresh)	$52.0$ ms	$19.2$ Hz
Fast action (4-step DDIM)	$54.8$ ms	$18.2$ Hz
one denoising step	$13.7$ ms	$73$ Hz
$c_{\text{slow}}/c_{\text{fast}}$ , per action	$0.95$
$c_{\text{slow}}$ in denoising steps	$3.8$
Fixed $1{:}4$ compute overhead	$+24\%$

Table 4: Latency of the FiS-VLA system on one RTX 4090 (reference code path,

10

–

12

timed runs after warmup; run-to-run spread

53.8

–

61.1

ms for the fast action,

52.0

–

52.1

ms for the slow query). One slow refresh costs about one fast action, or

3.8

denoising steps.

System	Slow call	Fast step	$\rho$
FiS-VLA ( Chen et al., 2025 )	$52.0$ ms local	$54.8$ ms	$0.95$ (meas.)
LIBERO (ours)	$0.46$ – $2.2$ s	$300$ ms	$1.5$ – $7.2$ (meas.)
Hume ( Song et al., 2025 )	VLM at $4$ Hz	$90$ Hz ctrl	${\sim}22$ (rep.)
DualVLN ( Wei et al., 2025 )	VLM consult	fast step	$10$ – $20$ (rep.)
RoboICL ( Liu et al., 2026 )	$36$ – $52$ s API	$40$ ms chunk	${\sim}10^{3}$ (rep.)

Table 5: The cost-asymmetry continuum under one protocol (Eq. ( 12 )). “Meas.” rows are this paper’s runs (Table 4 for FiS-VLA); “rep.” rows read published latencies and control rates. The empirical span is three orders of magnitude, and the LIBERO row shows that co-locating both modules on one GPU already moves a separate-VLM system an order of magnitude down the axis.

Heterogeneity $w$	0	0.05	0.10	0.25
Advantage $\Delta$	$+0.005$	$+0.019$	$+0.022$	$+0.059$
$95\%$ CI	$\pm.004$	$\pm.005$	$\pm.004$	$\pm.005$

Table 6: Budget-matched advantage of calibrated routing over the fixed

1{:}4

schedule as collapse heterogeneity grows (mean-matched; 5 seeds, 100 episodes of 120 steps each). The advantage vanishes in the homogeneous regime and grows monotonically, as Proposition 4 predicts.

Refresh period $k$	1	2	4	8
Action MSE	$2.431$	$2.325$	$2.331$	$2.308$
Per-step latency (ms)	$107.6$	$75.1$	$58.9$	$50.8$

Table 7: The measured frontier of the fine-tuned FiS-VLA on held-out RLBench demonstrations (teacher-forced, 16 episodes from 4 tasks, bf16, one RTX 4090; latency amortised as

c_{\text{slow}}/k+c_{\text{fast}}

with

c_{\text{slow}}=65.3

ms,

c_{\text{fast}}=42.5

ms measured in the same runs). Fidelity is flat in

k

while latency falls

2.1\times

Figure 6: Staleness and drift on the fine-tuned checkpoint, held-out episodes. (a) The staleness-cost curve

\Delta(\delta)

is nonpositive: an aged slow latent costs nothing. (b) The latent itself drifts sub-diffusively, linearly in age rather than as a random walk.

Figure 7: The measured frontier. (a) Action MSE is flat in the refresh period

k

; the shuffled-latent control (

+11\%

) marks the scale at which the latent does matter, an order of magnitude above any staleness effect. (b) Amortised per-step latency falls

2.1\times

over the same range.

Family	$\Delta(\delta)$ shape	$k^{**}$	$k^{*}$	cost gap
tracking	$+b_{0}\,\delta$	5	5	$0.0\%$
drift	$+b_{0}\sqrt{\delta}$	3	5	$2.0\%$
contact	$-c_{0}$ , $\delta\geq 1$	15	15–16	$1.9\%$
flat	$0$	16	16	$0.0\%$
FiS (4 tasks, meas.)	$\leq 0$ , $\delta\leq 8$	—	8 (edge)	—

Table 8: Schedule compilation from the measured value function. Synthetic families (

60

-step episodes,

400

validation episodes per period,

5

seeds):

k^{**}

minimizes the empirical closed-loop cost,

k^{*}

is compiled from probed

\Delta(\delta)

20{,}000

probe steps per age, cost gap is

|C(k^{*})-C(k^{**})|/C(k^{**})

(worst over seeds). The real FiS row repeats the measured result of Section 5.8 : nonpositive

\Delta

compiles every task to the edge of the probed range.

Arm	Channel	Trigger	Success	$95\%$ CI	Calls/ep	Overhead
A	none	—	$17/30$	$[.37,.75]$	$0$	$0\%$
B	rewrite, raw	periodic $25$	$8/30$	$[.12,.46]$	$10.4$	$8.9\%$
B2	rewrite, clean	periodic $25$	$11/30$	$[.20,.56]$	$9.4$	$10.8\%$
C	rewrite, raw	stall	$13/30$	$[.26,.63]$	$10.7$	$9.0\%$
C2	rewrite, clean	stall	$13/30$	$[.26,.63]$	$10.7$	$13.0\%$
A 10	none	—	$52/90$	$[.47,.68]$	$0$	$0\%$

Table 9: LIBERO-Spatial closed-loop arms at

\rho=1.53

(success over

30

episodes per arm with exact

95\%

Clopper–Pearson intervals; the ten-episode extension, superscript

10

, reports the nine tasks common to all arms — ninety paired episodes per arm, see the statistical-reading paragraph). “Overhead” is slow-query wall-clock over measured fast-policy wall-clock; the revert channel’s undos are local and consume no query, so its Calls/ep counts rewrites only. The extension sweeps the rewrite channel over periods

k\in\{10,25,50\}

and a verification screen, the ranking channel over trigger thresholds with a shuffled-selection control that removes the position bias, and adds a revert channel that undoes a rewrite that fails to restore progress. The bottom block replicates arms A, B2 and E4 on LIBERO-Goal, a second task suite fine-tuned and evaluated under the same protocol with an LLM slow endpoint (mean call latency

600

ms); its overhead is slow-call wall-clock over the fast-only arm’s wall-clock on the same tasks.

Vision-Language-Action Models

Vision-language-action models turn visual observations and natural-language instructions into actions for robots and other embodied agents.

See all 1,192 papers

Latent Bridge: Feature Delta Prediction for Efficient Dual-System Vision-Language-Action Model Inference
May 4, 2026Yudong Liu, Yuan Li, Zijia Tang +12Efficient VLM Inference Efficient VLA Model Inference
1Duke University · 2Qualcomm AI Research · University of Florida
Triggering Generalist Reasoning via Predictive Uncertainty for Dual-System VLA
Oct 4, 2026Hyemin Yang, Wooseong Jeong, Giwon Lee +1Uncertainty Estimation for VLA Models Efficient VLA Model Inference
KAIST
DAM-VLA: Decoupled Asynchronous Multimodal Vision Language Action model
Jun 10, 2026Pankhuri Vanjani, Zhuoyue Li, Jakub Suliga +4AI Control Robotic Manipulation
Intuitive Robots Lab, Karlsruhe Institute of Technology (KIT), Germany · NVIDIA · Robotics Institute of Germany

May 4, 2026J/K move · Enter open · S save

Latent Bridge: Feature Delta Prediction for Efficient Dual-System Vision-Language-Action Model Inference

Yudong Liu, Yuan Li, Zijia Tang, Yuxi Zheng+11

1Duke University · 2Qualcomm AI Research · University of Florida

Efficient VLM Inference Efficient VLA Model Inference Efficient VLA Models Vision-Language-Action Models VLMs for Robotics

Open paper AskPDF arXiv

Dual-system Vision-Language-Action (VLA) models achieve state-of-the-art robotic manipulation but are bottlenecked by the VLM backbone, which must execute at every control step while producing temporally redundant features. We propose Latent Bridge, a lightweight model that predicts VLM output deltas between timesteps, enabling the action head to operate on predicted outputs while the expensive VLM backbone is called only periodically. We instantiate Latent Bridge on two architecturally distinct VLAs: GR00T-N1.6 (feature-space bridge) and π0.5 (KV-cache bridge), demonstrating that the approach generalizes across VLA designs. Our task-agnostic DAgger training pipeline transfers across benchmarks without modification. Across four LIBERO suites, 24 RoboCasa kitchen tasks, and the ALOHA sim transfer-cube task, Latent Bridge achieves 95-100% performance retention while reducing VLM calls by 50-75%, yielding 1.65-1.73x net per-episode speedup.

Abstract

Vision-Language-Action Models

Explore similar work

Latent Bridge: Feature Delta Prediction for Efficient Dual-System Vision-Language-Action Model Inference

Triggering Generalist Reasoning via Predictive Uncertainty for Dual-System VLA

DAM-VLA: Decoupled Asynchronous Multimodal Vision Language Action model