Triggering Generalist Reasoning via Predictive Uncertainty for Dual-System VLA

Hyemin Yang; Wooseong Jeong; Giwon Lee; Kuk-Jin Yoon

cs.CVOct 4, 2026

Triggering Generalist Reasoning via Predictive Uncertainty for Dual-System VLA

Authors: Hyemin Yang, Wooseong Jeong, Giwon Lee, Kuk-Jin Yoon

Organizations: KAIST

Abstract

Dual-system Vision-Language-Action (VLA) models improve real-time robotic control by pairing a slow, reasoning-capable generalist with a fast specialist action expert. However, existing methods invoke the generalist at a fixed frequency, ignoring the fact that decision-making complexity varies throughout a rollout. This static strategy wastes computation in easy phases and can delay renewed reasoning when the scene changes unexpectedly. We propose TUD (Triggering generalist reasoning via predictive Uncertainty for Dual-system VLA), an adaptive inference framework that selectively skips unnecessary generalist calls. TUD measures the cross-step dispersion of action re-predictions at the upcoming chunk slot under the cached generalist context, as a predictive uncertainty signal. This signal captures how much the future action plan shifts as new observations arrive and is computed from forwards the architecture already runs, requiring neither manual phase labels nor an auxiliary uncertainty model. On VLA-Arena, it achieves a higher success rate at matched call budgets than alternative uncertainty baselines while maintaining low wall-clock overhead, and more consistently separates successful from failed rollouts. Also, TUD finds a more favorable cost-success trade-off than non-adaptive baselines, tracing an entire operating curve as a single threshold is varied, and substantially reduces VLM calls at matched success rate. The same trade-off appears in our real-robot experiments, where TUD cuts generalist calls by 75% relative to the strongest fixed-interval baseline while achieving an even higher success rate. Our results suggest that predictive uncertainty provides a practical criterion for adaptive reasoning in efficient VLA control.

Figures & tables

Panel of Figure 1 — Figure 1 : Two properties of existing dual-system VLAs that motivate adaptive generalist calling. (a) Per-task success rate of OpenHelix on CALVIN at varying generalist call interval $\Delta$ ( $N$ on the x-axis); the best $\Delta^{\star}$ differs by task category. (b) Long-horizon rollout performance of Robodual on CALVIN at varying $\Delta$ , averaged over 100 rollouts per $\Delta$ .

	Spec.	Gen.	Dual
Throughput (Hz) $\uparrow$	36.98	2.70	14.79
Latency (s) $\downarrow$	0.216	0.370	-

Table 1 : Per-call cost on Robodual.

Figure 2 : Overview of TUD. The generalist call refreshes the cached context $h_{k}$ at decision points, the specialist forward re-predicts an action chunk against $h_{k}$ at every step, and a sliding buffer of these re-predictions yields the predictive action-plan dispersion $u_{t}$ . The invocation rule thresholds the running mean of $u_{t}$ to skip or invoke the generalist.

Safety (SR / CC)
	Baselines								Ours (TUD)
Task	OpenVLA	OpenVLA-OFT	$\pi_{0}$	$\pi_{0}$ -FAST	UniVLA	SmolVLA	OneTwoVLA	RoboDual	p70	Best
StaticObstacles	0.60 / 0.0	1.00 / 0.0	0.98 / 0.0	1.00 / 0.0	0.84 / 0.0	0.14 / 0.0	0.94 / 0.0	0.68 / 0.0	0.97 / 0.0	0.99 / 0.0
CautiousGrasp	0.80 / 6.6	0.60 / 3.3	0.84 / 3.5	0.64 / 3.3	0.80 / 3.3	0.52 / 2.5	0.72 / 2.5	0.42 / 2.2	0.93 / 3.2	0.93 / 3.2
HazardAvoidance	0.20 / 17.2	0.36 / 9.4	0.74 / 6.4	0.10 / 10.4	0.70 / 5.3	0.16 / 10.4	0.56 / 6.7	0.68 / 5.3	0.85 / 5.5	0.85 / 5.5
StatePreservation	1.00 / 0.0	1.00 / 0.0	0.98 / 0.0	0.60 / 0.0	0.90 / 0.0	0.50 / 0.0	0.84 / 0.0	0.82 / 0.0	0.95 / 0.0	0.97 / 0.0

Table 2 : Per-task success rate (SR) and cumulative cost (CC) on VLA-Arena Level 0. p70 reports TUD at our representative operating point (

p=0.70

), and Best reports TUD at whichever

p

gives the highest result for each individual task.

Panel of Figure 3 — Figure 3 : Cost–success trade-off on VLA-Arena. Left : Safety. Right : Long-Horizon. Top : TUD’s operating curve as the percentile $p$ varies (each marker = one $p$ , e.g., p70 for $p=0.70$ ). Bottom : each non-adaptive baseline at its single operating point.

Forward	ms $\downarrow$	Hz $\uparrow$
Generalist	$105.91$	$9.44$
Specialist	$\phantom{0}13.53$	$73.91$

Table 3 : Per-call latency.

High budget ( $\approx$ 17–18 calls/ep)
Method	Calls	SR (%) $\uparrow$	Wall (s) $\downarrow$
Diff-DAgger	16.5	83.7	31.1
BayesDiff	16.3	83.5	11.9
Visual	17.2	83.0	11.8
AAC-ent.	16.4	82.7	28.7
MC-AleaUQ	19.1	80.2	11.6

Table 4 : VLM re-invocation criteria at matched call budgets on VLA-Arena L0.

Figure 4 : Uncertainty signals on success vs failure rollouts. For each VLA-Arena domain (one block per domain, one sub-panel per signal), we plot each signal in the final 80 steps before the rollout ends, separately for successful (green) and failed (red) episodes. Signals are

z

-normalized per method. Only

u_{t}

shows a consistent gap between success and failure across all four domains.

Figure 5 : Tracking $u_{t}$ within a representative rollout. Long-horizon episode for “Open the top layer of the cabinet” . $x$ -axis: environment time within the rollout. $y$ -axis: $z$ -normalized $u_{t}$ (same normalization as Figure 4 ). Red circles mark the highest- $u_{t}$ peaks plus the two rollout endpoints, with the corresponding RGB frames shown in time order below.

Method	SR $\uparrow$	Calls $\downarrow$	Wall (s) $\downarrow$
Single-system $\pi_{0}$	0.433	49.0	79.5
Fixed $\Delta=9$	0.467	48.5	76.4
Fixed $\Delta=25$	0.433	34.6	76.1
Fixed $\Delta=50$	0.400	25.9	79.2
TUD p50	0.767	24.5	53.4
TUD p70	0.733	20.8	53.5

Table 5 : Real robot results. SR over

30

rollouts per method (

10

per task). Calls and Wall (s) are per-episode generalist invocations and execution time; failures count as

120

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

$E$	$\alpha_{E}$	Interpretation
3	19.80	Tighter bound, but noisier dispersion estimate
5	28.00	Default balance
7	34.29	Smoother dispersion estimate, but looser bound

Appendix

Table 6 : Effect of the prediction-buffer size

E

on the dispersion coefficient

\alpha_{E}=2d_{a}\sqrt{E-1}

with

d_{a}=7

$E$	Minimum	p1	p5	p10
3	1.472	1.654	2.140	2.425
5	2.009	2.210	2.834	3.220
7	2.209	2.742	3.278	3.672
9	2.324	3.054	3.624	4.036
11	2.557	3.224	3.801	4.188

Appendix

Table 7 : Empirical bound-to-deviation ratio

r_{t}

on held-out trajectories. Larger values indicate a more conservative bound.

Factor	Setting	SR $\uparrow$	Calls $\downarrow$
Buffer $E$	3	0.832	27.3
	5	0.850	21.6
	7	0.850	18.5
Measurement $\Delta_{s}$	1	0.848	28.0
	2	0.850	21.6
	4	0.852	51.2

Appendix

Table 8 : Hyperparameter sensitivity of the invocation rule. Bold indicates the default setting. Calls denote generalist invocations per episode.

Specification	CALVIN [ 1 ]	VLA-Arena [ 18 ]
Input Modalities	RGB, Depth, Proprioception, Language	RGB, Language
Visual Observation	Static RGB ( $200\times 200\times 3$ ) Depth Static ( $200\times 200$ ) Gripper RGB ( $84\times 84\times 3$ ) Depth Gripper ( $84\times 84$ ) Tactile Image ( $120\times 160\times 2$ )	Multi-view RGB Cameras (Typically resized to standard inputs, e.g., $224\times 224$ or $256\times 256$ )
Action Space	7-DoF Continuous Control - Relative EE translation (3) - Relative EE rotation (3) - Gripper state (1, discrete)	7-DoF Continuous Control - Absolute EE pos & ori (6) - Gripper state (1, discrete)
Unique Features	Proprioceptive State: 15-dim vector (EE pos/ori, joints, gripper) Control Freq: 30 Hz	Perturbation Inputs: Visual (V0–V4), Language (W0–W4) Task Diversity: 170 tasks (L0, L1, L2)

Appendix

Table 9 : Comparison of detailed specifications between CALVIN and VLA-Arena benchmarks.

Method	Additional model evaluations	Auxiliary computation
TUD ( $u_{t}$ , ours)	None	std over $E$ buffered chunks
GenUQ	$M=4$ sampled action chunks	last-layer Laplace weight samples
MC-AleaUQ	$M=10$ velocity evaluations	re-noising at $t^{\star}=5$
Diff-DAgger	$N=4$ loss evaluations	$(\tau,\epsilon)$ draws, $\tau\sim\mathrm{Beta}(1.5,1)$
BayesDiff	$T=10$ velocity evaluations	variance propagation over $T$ Euler steps
AAC-ent.	$N=20$ sampled action chunks	action entropy

Appendix

Table 10 : Extra computation per uncertainty measurement in our action-chunk setting.

Panel of Figure 6 — Figure 6 : Cost–success trade-off on VLA-Arena level 0. Left : Distractor. Right : Extrapolation. Top : TUD’s operating curve as the percentile $p$ varies (each marker = one $p$ , e.g., p70 for $p=0.70$ ). Bottom : each non-adaptive baseline at its single operating point.

Safety (SR / CC)
	Baselines								Ours (TUD)
Task	OpenVLA	OpenVLA-OFT	$\pi_{0}$	$\pi_{0}$ -FAST	UniVLA	SmolVLA	OneTwoVLA	RoboDual	p70	Best
StaticObstacles	0.60 / 8.2	0.20 / 45.4	0.74 / 8.0	0.40 / 56.0	0.42 / 9.7	0.00 / 8.8	0.58 / 9.3	0.40 / 13.7	0.74 / 14.8	0.77 / 19.0
CautiousGrasp	0.40 / 120.2	0.50 / 6.3	0.06 / 16.4	0.06 / 15.6	0.60 / 32.1	0.28 / 30.7	0.26 / 23.8	0.18 / 48.4	0.16 / 11.8	0.28 / 13.2
HazardAvoidance	0.02 / 22.8	0.00 / 22.9	0.00 / 16.8	0.00 / 15.4	0.12 / 18.3	0.00 / 19.5	0.12 / 18.9	0.14 / 18.9	0.06 / 18.9	0.11 / 18.7
StatePreservation	0.06 / 16.0	0.76 / 7.6	0.64 / 6.4	0.56 / 5.6	0.76 / 7.6	0.18 / 1.8	0.66 / 6.6	0.38 / 3.8	0.59 / 5.9	0.67 / 6.7

Appendix

Table 11 : Per-task results on VLA-Arena Level 1 (Generalization). We report Success Rate (SR) and Cumulative Cost (CC). TUD (p70) is the operating point at

p=0.70

of the online invocation rule (defined in Section 3.3 ) and TUD (Best) is the per-task best operating point.

Panel of Figure 7 — Figure 7 : Cost–success trade-off on VLA-Arena level 1 across four domains (Safety, Long-Horizon, Distractor, and Extrapolation). Top : TUD’s operating curve as the percentile $p$ varies (each marker = one $p$ , e.g., p70 for $p=0.70$ ). Bottom : each non-adaptive baseline at its single operating point.

Panel of Figure 8 — Figure 8 : Additional per-rollout $u_{t}$ trajectories on VLA-Arena. From top to bottom: extrapolation (preposition combinations) failure, extrapolation (unseen objects) success, safety (cautious grasp) success, and distractor (static distractors) success. Same plotting convention as Figure 5 .

Figure 9 : Cost-success trade-off on CALVIN ABC $\to$ D evaluation (100 sequences, max 360 steps, each chaining 5 sub-tasks). The $x$ -axis shows mean VLM calls per sub-task; the $y$ -axis shows completed sub-tasks per chain (out of 5).

Figure 10 : Real robot setup. SO-101 6-DoF arm with third-person and gripper-view RGB cameras.

Panel of Figure 11 — Figure 11 : Test environments. Evaluation scenes for the three task instructions, with two object positions per task.

	pen			drawer			stack			mean
Method	SR	Calls	Wall (s)	SR	Calls	Wall (s)	SR	Calls	Wall (s)	SR	Calls	Wall (s)
Single-system $\pi_{0}$	0.5	42.4	69.8	0.5	47.8	76.7	0.3	56.8	92.0	0.43	49.0	79.5
Fixed $\Delta=9$	0.5	46.8	73.7	0.5	38.9	62.6	0.4	59.7	93.0	0.47	48.5	76.4
Fixed $\Delta=25$	0.4	36.1	79.8	0.6	25.6	56.3	0.3	42.2	92.2	0.43	34.6	76.1
Fixed $\Delta=50$	0.4	25.8	79.0	0.6	19.5	58.3	0.2	32.5	100.1	0.40	25.9	79.2
TUD p50	0.9	19.6	38.9	0.7	18.7	50.6	0.7	35.3	70.6	0.77	24.5	53.4

Appendix

Table 12 : Real robot comparison per task. Metrics are the same as in Table 5 . SR is the success rate; Calls and Wall are the generalist (VLM prefix) call count and the wall-clock time in seconds, both per episode. Each per-task entry is over

10

rollouts; the mean columns average the three tasks and reproduce the values in Table 5 .

Path	Image $\rightarrow$ action	Cycle
Generalist	1077 $\pm$ 109	1107 $\pm$ 104
Specialist	94 $\pm$ 3	120 $\pm$ 3

Appendix

Table 13 : Latency. mean

\pm

SD in ms.

Method	Image $\rightarrow$ action	Cycle
	mean / p95	mean $\pm$ SD
$\pi_{0}$	70 / 1076	95 $\pm$ 256
Fixed $\Delta=9$	115 / 1180	136 $\pm$ 321
Fixed $\Delta=25$	60 / 96	82 $\pm$ 200
Fixed $\Delta=50$	39 / 94	62 $\pm$ 140
TUD p50	59 / 100	81 $\pm$ 202

Appendix

Table 14 : Control-loop jitter (ms).

Explore similar work

Apr 17, 2026cs.RO

ReconVLA: An Uncertainty-Guided and Failure-Aware Vision-Language-Action Framework for Robotic Control

Vision-language-action (VLA) models have emerged as generalist robotic controllers capable of mapping visual observations and natural language instructions to continuous action sequences. However, VLAs provide no calibrated measure of confidence in their action predictions, thus limiting their reliability in real-world settings where uncertainty and failures must be anticipated. To address this problem we introduce ReconVLA, a reliable conformal model that produces uncertainty-guided and failure-aware control signals. Concretely, our approach applies conformal prediction directly to the action token outputs of pretrained VLA policies, yielding calibrated uncertainty estimates that correlate with execution quality and task success. Furthermore, we extend conformal prediction to the robot state space to detect outliers or unsafe states before failures occur, providing a simple yet effective failure detection mechanism that complements the action-level uncertainty. We evaluate ReconVLA in both simulation and real robot experiments across diverse manipulation tasks. Our results show that conformalized action predictions consistently improve failure anticipation, reduce catastrophic errors, and provide a calibrated measure of confidence without retraining or modifying the underlying VLA.

Lingling Chen, Zongyao Lyu, William J. Beksi

Department of Computer Science and Engineering, The University of Texas at Arlington, Arlington, TX, USA

Diffusion-Based Vision-Language-Actions

Ask PDF

Jun 23, 2026cs.CV

Reflective VLA: In-Context Action Consequences Make VLAs Generalize

Most vision-language-action (VLA) models are reactive: they predict the next action from the current instruction and observation, implicitly assuming that the current observation fully specifies the action-relevant state. In embodied control, however, embodiment-specific factors such as camera-to-robot geometry, robot calibration, or systematic actuation bias are often hard to identify from a single observation. As a result, reactive policies cannot reliably disambiguate these factors in general, overfitting to training environments and generalizing poorly at deployment. We propose Reflective VLA, which conditions each decision on a context of observation-action-consequence triplets. Each triplet records not only what the robot observed and executed, but also how the scene changed afterward, exposing the deployment-specific mapping from actions to observed effects. Architecturally, Reflective VLA routes all observation modalities through the VLM under shared attention, so the action expert reasons directly over past triplets and the current observation. A block-causal mask enables parallel multi-frame training without leakage and supports KV-cached real-time inference. On standard LIBERO and SimplerEnv-Bridge, Reflective VLA preserves strong in-distribution performance. Under distribution shift on LIBERO-Plus and the harder LIBERO-Plus-Hard, it improves average success rate by 5.4 and 4.2 percentage points over a matched reactive baseline. Ablations with a matched history-only baseline further show that action consequences -- rather than additional context length alone -- are the key to cross-environment generalization. Project page: https://lianqing11.github.io/reflective-vla-page/

Qing Lian, Kent Yu, Lei Zhang

Futian Laboratory · International Digital Economy Academy (IDEA) · Visincept

Generalist Vision--Language--Action Rdf

Ask PDF

May 28, 2026cs.RO

ElegantVLA: Learning When to Think for Efficient Vision-Language-Action Models

Vision-Language-Action (VLA) models are a powerful paradigm for generalist robotic control. However, their high computational cost and limited control frequency hinder real-time robotic manipulation, especially when large vision-language backbones and iterative action heads run at every control step. Existing VLA acceleration methods often optimize individual components or rely on fixed acceleration rules, treating different control steps with largely fixed computation and overlooking the non-uniform reasoning demands of sequential embodied control. Inspired by human motor control, where cognitive and feedback resources concentrate on goal-sensitive stages, we argue that VLA models should learn when to invest full computation and when to reuse prior computation. We propose ElegantVLA, a plug-in phase-adaptive inference framework that accelerates VLA models through intra-model dynamic compute scheduling. ElegantVLA introduces a lightweight scheduler that observes temporal representation similarity, robot-motion cues, and episode progress to jointly allocate computation across the vision encoder, LLM, and action head. For perception-language reasoning, the scheduler selects a five-level Vision-LLM compute mode, from full recomputation to multi-step temporal reuse, based on visual-language representation stability. For action generation, it selects a three-level denoising mode, reusing intermediate denoising states during stable motion while preserving full refinement for goal-sensitive stages. By coordinating these decisions, ElegantVLA offers a general acceleration framework for modern VLA pipelines with explicit action-generation modules, without modifying or retraining the base model. Experiments on GR00T and CogACT achieve up to 2.55x and 3.77x speedup, and on six real-world GR00T tasks ElegantVLA cuts computation by 2.18x while raising control frequency from 13.8 Hz to 26.3 Hz.

Ye Li, Huanan Liu, Kangye Ji +7

Tsinghua University · University of Illinois at Urbana-Champaign

Diffusion-Based Vision-Language-Actions Action Generation

Ask PDF