Shifting Uncertainty to Critical Moments: Towards Reliable Uncertainty Quantification for VLA Model

Yanchuan Tang; Taowen Wang; Yuefei Chen; Boxuan Zhang; Qiang Guan; Ruixiang Tang

cs.ROMar 18, 2026

Shifting Uncertainty to Critical Moments: Towards Reliable Uncertainty Quantification for VLA Model

Authors: Yanchuan Tang, Taowen Wang, Yuefei Chen, Boxuan Zhang, Qiang Guan, Ruixiang Tang

Organizations: Rutgers University · Kent State University

Abstract

Vision-Language-Action (VLA) models enable general-purpose robotic policies by mapping visual observations and language instructions to low-level actions, but they often lack reliable introspection. A common practice is to compute a token-level uncertainty signal and take its mean over a rollout. However, mean aggregation can dilute short-lived but safety-critical uncertainty spikes in continuous control. In particular, successful rollouts may contain localized high-entropy segments due to benign noise or non-critical micro-adjustments, while failure rollouts can appear low-entropy for most timesteps and only exhibit brief spikes near the onset of failure. We propose a unified uncertainty quantification approach for predicting rollout success versus failure that (1) uses max-based sliding window pooling to preserve transient risk signals, (2) applies motion-aware stability weighting to emphasize high-frequency action oscillations associated with unstable behaviors, and (3) performs DoF-adaptive calibration via Bayesian Optimization to prioritize kinematically critical axes. Experiments on the LIBERO benchmark show that our method substantially improves failure prediction accuracy and yields more reliable signals for failure detection, which can support downstream human-in-the-loop interventions.

Figures & tables

Figure 1: Overview of the proposed uncertainty quantification framework. Standard global averaging often masks failure signals. We propose: (A) Sliding Window Pooling (SW) to capture transient uncertainty spikes; (B) Action Transfer Reweighting (ATR) to prioritize uncertainty during oscillatory actions; and (C) Bayesian Optimization (BO) to learn adaptive weights for kinematically critical DoFs.

Panel of Figure 2 — Figure 2: Empirical evidence of the Averaging Trap on LIBERO-10. The plots show the probability density of the global mean entropy $S_{\text{Avg}}(\tau)$ for success (blue) and failure (red) rollouts. The significant overlap between the two distributions, coupled with near-random AUROC scores (0.51 on train and 0.47 on test), demonstrates that global averaging masks critical failure signals and fails to distinguish between successful and failed executions.

	LIBERO-SPATIAL		LIBERO-OBJECT		LIBERO-GOAL		LIBERO-10
Method (best params)	AUROC $\uparrow$	Acc $\uparrow$	AUROC $\uparrow$	Acc $\uparrow$	AUROC $\uparrow$	Acc $\uparrow$	AUROC $\uparrow$	Acc $\uparrow$
Baseline ( $\text{mean}_{t,d}$ )	0.845	0.892	0.542	0.744	0.641	0.755	0.468	0.622
SW ( $w{=}70$ )	0.908	0.912	0.765	0.744	0.837	0.846	0.699	0.691
ATR ( $\alpha{=}0.9$ )	0.847	0.868	0.567	0.768	0.670	0.819	0.508	0.606
SW+ATR ( $w{=}60,\alpha{=}0.9$ )	0.919	0.948	0.772	0.768	0.841	0.843	0.738	0.711
SW+ATR+BO ( $w^{\star},\alpha^{\star},\beta^{\star}$ )	0.936	0.939	0.793	0.692	0.811	0.807	0.838	0.743

Table 1: Rollout-level failure prediction on LIBERO suites with OpenVLA. We report test AUROC and test Accuracy (threshold chosen by Youden’s index on the train split). Bold denotes the best result and underline denotes the second-best result for each metric within a suite.

Figure 3: Ablation studies on LIBERO-10. (a) SW window size ablation on LIBERO-10. (b) ATR stability contrast $\alpha$ ablation on LIBERO-10. (c) Joint SW+ATR ablation on LIBERO-10; the red star indicates the best-performing $(w,\alpha)$ pair. (d) Effect of DoF-adaptive calibration, comparing SW+ATR with and without Bayesian Optimization across LIBERO suites.

Target Suite	Global Mean	Zero-Shot Transfer	Oracle BO
LIBERO-OBJECT	0.542	0.780	0.793
LIBERO-GOAL	0.641	0.819	0.811
LIBERO-10	0.468	0.715	0.838

Table 2: Cross-suite zero-shot transfer test AUROC. Parameters calibrated solely on the LIBERO-SPATIAL train split transfer robustly to unseen suites.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Suite	$w^{\star}$	$\alpha^{\star}$	$\boldsymbol{\beta}^{\star}$ (7-DoF)
LIBERO-SPATIAL	93	0.95	[1.0, 2.6, 10.0, 1.0, 1.0, 1.0, 10.0]
LIBERO-OBJECT	62	0.05	[2.6, 1.0, 1.2, 1.0, 10.0, 7.5, 10.0]
LIBERO-GOAL	72	0.05	[1.0, 1.0, 4.6, 1.0, 10.0, 1.0, 10.0]
LIBERO-10	100	0.95	[1.6, 6.9, 10.0, 1.0, 1.0, 1.0, 10.0]

Appendix

Table A1 : Best SW+ATR+BO hyperparameters found on the train split of each suite. All

\beta^{\star}

values are rounded to one decimal place for readability.

Figure A1 : DoF importance learned by Bayesian Optimization. We visualize the optimized DoF weights

\beta^{\star}

across LIBERO suites. Gripper and

\Delta z

consistently receive high weights, while

\Delta\text{pitch}

is emphasized in LIBERO-OBJECT and LIBERO-GOAL, supporting DoF-adaptive calibration.

Figure A2 : Per-task success rates (%) of OpenVLA baseline across LIBERO suites. Darker cells indicate higher success. Task difficulty varies significantly both across and within suites, motivating the need for reliable failure prediction via uncertainty quantification.

Figure A3 : Ablation Study on LIBERO-SPATIAL. (a) Impact of window size $w$ on AUROC. (b) Impact of stability contrast $\alpha$ on AUROC. (c) Joint heatmap of AUROC across $w$ and $\alpha$ . (d-e) Detailed metrics (Accuracy, Precision, Recall, F1) for SW and ATR components respectively.

Figure A4 : Ablation Study on LIBERO-OBJECT. (a) Impact of window size $w$ on AUROC. (b) Impact of stability contrast $\alpha$ on AUROC. (c) Joint heatmap of AUROC across $w$ and $\alpha$ . (d-e) Detailed metrics (Accuracy, Precision, Recall, F1) for SW and ATR components respectively.

Figure A5 : Ablation Study on LIBERO-GOAL. (a) Impact of window size $w$ on AUROC. (b) Impact of stability contrast $\alpha$ on AUROC. (c) Joint heatmap of AUROC across $w$ and $\alpha$ . (d-e) Detailed metrics (Accuracy, Precision, Recall, F1) for SW and ATR components respectively.

Figure A6 : Ablation Study on LIBERO-10. (a) Impact of window size $w$ on AUROC. (b) Impact of stability contrast $\alpha$ on AUROC. (c) Joint heatmap of AUROC across $w$ and $\alpha$ . (d-e) Detailed metrics (Accuracy, Precision, Recall, F1) for SW and ATR components respectively.

Uncertainty Estimation for VLA Models

Uncertainty estimation for Vision-Language-Action (VLA) models predicts when robotic actions or task outcomes may be unreliable.

See all 32 papers

Perturbation-Based Epistemic Uncertainty for Failure Detection in Vision-Language-Action Models
Jun 18, 2026Yousung Lee, Dongsoo HarFailure Detection for VLA Models Uncertainty Estimation for VLA Models
Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea.
SentryVLA: Uncertainty-Guided Action Selection and State Monitoring for Vision-Language-Action Robotic Control
Apr 17, 2026Zongyao Lyu, Lingling Chen, William J. BeksiUncertainty Estimation for VLA Models Robot Failure Detection
Department of Computer Science and Engineering, The University of Texas at Arlington, Arlington, TX, USA.
Uncertainty Quantification for Flow-Based Vision-Language-Action Models
Jun 16, 2026Ralf Römer, Maximilian Seeliger, Saida Liu +5Flow Matching Uncertainty Estimation for VLA Models
TU Munich · ETH Zurich · TU of Munich +1

Jun 18, 2026J/K move · Enter open · S save

Perturbation-Based Epistemic Uncertainty for Failure Detection in Vision-Language-Action Models

Yousung Lee, Dongsoo Har

Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea.

Failure Detection for VLA Models Uncertainty Estimation for VLA Models Robotic Manipulation Vision-Language-Action Models Epistemic Uncertainty

Open paper AskPDF arXiv

Vision-Language-Action (VLA) models have shown strong performance in robotic manipulation, but reliable uncertainty quantification remains challenging, particularly under distribution shift. Unlike autoregressive policies, many modern VLA models generate continuous actions through regression or flow-based generation, where explicit predictive probabilities are unavailable. Moreover, stochastic action sampling primarily captures action-generation variability under a fixed model, while failure detection under distribution shift can benefit from capturing uncertainty in the model itself. Motivated by Bayesian perspectives on local model variations, we propose perturbation-based failure detection (PFD), a training-free framework for estimating epistemic uncertainty in VLA models through low-rank weight perturbations. Specifically, we inject random low-rank perturbations into selected transformer weight matrices and estimate epistemic uncertainty from disagreement across perturbed action predictions. Experiments on LIBERO-PRO show that PFD achieves the highest average AUROC and balanced accuracy among the evaluated methods while consistently outperforming stochastic action sampling across distribution shifts. Real-world robot experiments further demonstrate that PFD provides a competitive failure-detection signal under an unseen object shift.

Abstract

Uncertainty Estimation for VLA Models

Explore similar work

Perturbation-Based Epistemic Uncertainty for Failure Detection in Vision-Language-Action Models

SentryVLA: Uncertainty-Guided Action Selection and State Monitoring for Vision-Language-Action Robotic Control

Uncertainty Quantification for Flow-Based Vision-Language-Action Models