Post-hoc Epistemic Uncertainty for Vision-Language-Action Policies

Johannes Keller; Albrecht M. Bloße; Adrian Lindenmeyer; Thomas Neumuth; Daniel Schneider

cs.ROOct 21, 2026

Post-hoc Epistemic Uncertainty for Vision-Language-Action Policies

Authors: Johannes Keller, Albrecht M. Bloße, Adrian Lindenmeyer, Thomas Neumuth, Daniel Schneider

Organizations: Innovation Center Computer Assisted Surgery (ICCAS), Leipzig University, Leipzig, Germany

Abstract

Vision-language-action (VLA) policies need to recognize gaps in their training knowledge to request useful demonstrations or assistance in new situations, especially when failures are costly. Disagreement within an ensemble of independently trained policies can guide these decisions, but requires training and running multiple policies. We propose post-hoc kernel-variance heads that estimate epistemic uncertainty from the embeddings of a single policy. Each head computes the conditional variance of a Gaussian process conditioned on embeddings from the training demonstrations. Fitting requires no regression targets or gradient updates and leaves the policy unchanged. In experiments with SmolVLA on LIBERO-10, the kernel variance heads perform competitively with two-policy ensemble scores in task-level correlation with task success, in demonstration acquisition during active learning, and in referral to a stronger supervisor policy as a proxy for human assistance. For SmolVLA, the head on the vision-language backbone reduces uncertainty-scoring time by up to approximately a factor of 50 compared with ensemble scores, while also substantially reducing setup costs. We also show that the established evaluation protocol on LIBERO-10 primarily rewards recognition of coverage differences between tasks, while execution variability weakens assessment of within-task rankings. Thus, controlled perturbations provide a complementary evaluation: modality-specific heads identify unfamiliar camera, instruction, or state inputs, while fused heads can detect unfamiliar combinations of individually familiar inputs. With a supervisor proxy trained on the evaluated perturbation families, the decoder head improves within-task referral over ensemble disagreement. These findings show that useful uncertainty estimates for adaptive robot learning can be obtained from a single policy.

Figures & tables

Panel of Figure 1 — Figure 1: Uncertainty heads on a frozen SmolVLA policy. Dashed arrows read embeddings without changing the policy. The blue heads estimate the epistemic uncertainty of the whole observation. The orange context-specific heads estimate it for one input each, before the transformer mixes the inputs (raw) or after it (fused).

			This study		Reported
Score	Policies	Type	$-\rho_{\rm S}$	$-r_{\rm P}$	$-\rho_{\rm S}$	$-r_{\rm P}$
VFD	2	E	$0.75\pm 0.12$	$0.64\pm 0.17$	$0.71$	$0.71$
GU	2	E	$0.66\pm 0.19$	$0.68\pm 0.20$	$0.62$	$0.65$
Action-L2	2	E	$0.56\pm 0.07$	$0.50\pm 0.15$	$0.50$	$0.48$
ACE	1	A	$0.43\pm 0.16$	$0.44\pm 0.16$	$0.31$	$0.36$
DECU	2	E	$0.37\pm 0.29$	$0.40\pm 0.08$	$0.31$	$0.23$

Table 1: Task-level calibration: negative Spearman and Pearson correlation between task-mean uncertainty and task success. All scores are evaluated on the same policies, before acquisition and after ten rounds of random acquisition (Section 3.4 ), averaged over these two checkpoints (mean

\pm

SD over three seeds, best learned score in bold). Policies: number of trained policies a score needs. Type E marks disagreement between two policies, A an entropy-based score of one policy (mainly aleatoric) and K a kernel variance. Reported: values of Römer et al. [14] , averaged over 15 rounds of a separate 2,000-update run, so the protocols differ. † Reference that uses task labels, not a usable score.

Figure 2: SAVE learning curves on LIBERO-10. Success of the fine-tuned policy (30 evaluation rollouts per task) as demonstrations are requested, mean and

\pm 1

SD band over three independent runs.

		Success (%)					Overhead
Acquisition	Policies	5	10	15	Mean	New	(min/round)
Random	1	$43.0\pm 1.5$	$50.9\pm 5.1$	$54.7\pm 5.1$	$44.5\pm 2.7$	$46.3\pm 6.7$	–
VFD	2	$47.6\pm 2.8$	$56.7\pm 2.9$	$61.9\pm 2.5$	$49.2\pm 0.6$	$69.3\pm 4.5$	$43.8$
GU	2	$49.6\pm 3.9$	$59.0\pm 2.3$	$64.4\pm 3.1$	$51.0\pm 1.0$	$62.7\pm 2.5$	$44.6$
KV-Ctx	1	$47.4\pm 3.3$	$60.9\pm 3.7$	$65.7\pm 1.8$	$51.2\pm 1.6$	$75.0\pm 0.0$	$1.0$
KV-Dec	1	$46.4\pm 5.6$	$56.4\pm 4.4$	$61.8\pm 1.9$	$48.7\pm 3.2$	$75.0\pm 0.0$	$12.0$

Table 2: Policy success and cost during demonstration acquisition (mean

\pm

SD over three seeds, best in bold). All arms use five queries and 4,000 updates per round. Success: after 5, 10 and 15 rounds, and the normalized area under the linearly interpolated learning curve (Mean). New: unique previously unseen demonstrations acquired after 75 queries. Overhead: median GPU-minutes per round for scoring the candidate pool. For ensemble scores, it also includes fine-tuning the second policy for 4,000 updates per round, but not the one-time pre-training of that policy (30,000 updates). The training of the acquiring policy itself (about 37 minutes per round) is common to all arms and excluded.

Figure 3: Referral to the supervisor at round 10: combined success minus random referral, as a function of the fraction of conditions referred (mean over three seeds). The hindsight oracle refers by the observed signed gain. In the within-task panel it peaks at 14.1 percentage points, outside the plotted range.

	Global	Within-task	Automated
Score	nAUC	nAUC (r10)	fraction
VFD	$0.48\pm 0.06$	$0.07\pm 0.05$	$0.21\pm 0.03$
GU	$0.42\pm 0.05$	$0.06\pm 0.03$	$0.19\pm 0.04$
Action-L2	$0.35\pm 0.01$	$0.06\pm 0.04$	$0.13\pm 0.03$
ACE	$0.29\pm 0.06$	$0.06\pm 0.10$	$0.13\pm 0.03$
DECU	$0.20\pm 0.11$	$0.03\pm 0.01$	$0.11\pm 0.01$

Table 3: Referral from the policies of Table 1 to a stronger supervisor policy (mean

\pm

SD over three seeds, best learned score in bold). Global nAUC and automated fraction average acquisition rounds 0 and 10. Within-task nAUC refers to round 10. Chance level is 0. † Conditions ranked by the demonstration count of their task, a reference that uses task labels and is constant within a task (Table 1 ). ‡ Conditions ranked by the supervisor’s gain observed in one of the three executions of each condition and evaluated on the other two. It shows how much a single observed outcome reveals (Section 3.3 ).

Figure 4: (a) Share of the variance of single-execution success that lies between tasks, between conditions of a task (corrected for the sampling noise of three-execution means), and between executions of a condition, averaged over three seeds. (b) Within-task failure AUROC at round 10: seed values (small dots), mean (large dot) and 95% hierarchical bootstrap interval (bar). Chance is 0.5.

		Raw heads				Fused heads				Whole-policy
	Perturbation	Workspace	Wrist	Instruction	State	Workspace	Wrist	Instruction	State	KV-Ctx	KV-Dec	VFD
Instruction
S1	Strongly paraphrased	0.50	0.50	1.00	0.50	0.53	0.55	0.99	0.98	0.99	1.00	0.95
S2	All object words replaced by synonyms	0.50	0.50	1.00	0.50	0.51	0.52	0.94	0.95	0.93	0.99	0.92
S3	One typo in a non-object word	0.50	0.50	1.00	0.50	0.50	0.51	0.88	0.85	0.86	0.97	0.82
S4	Object nouns replaced by pronouns	0.50	0.50	1.00	0.50	0.63	0.65	1.00	1.00	1.00	1.00	0.97

Table 4: Detection of perturbed starting observations: AUROC of perturbed against clean starting observations within a task for the context and whole-policy heads of Figure 1 and VFD, mean over three seeds. Each of the LIBERO-10 tasks contributes 25 initial states, of which the more expensive VFD was evaluated on only 10. Chance level is 0.5.

		Raw heads				Fused heads				Whole-policy
	Perturbation	Workspace	Wrist	Instruction	State	Workspace	Wrist	Instruction	State	KV-Ctx	KV-Dec	VFD
Instruction
E1	Strongly paraphrased	0.52	0.56	1.00	0.56	0.54	0.52	0.98	0.76	0.97	0.95	0.90
E2	Switched to another task’s instruction	0.59	0.63	0.50	0.68	0.61	0.60	0.79	0.77	0.88	0.93	0.92
E3	Negated (“do not …”)	0.48	0.48	1.00	0.49	0.46	0.46	0.90	0.59	0.83	0.78	0.63
Camera images

Table 5: Detection of perturbations during execution: AUROC of the 50 steps after a perturbation against the same steps of an unperturbed run for the context and whole-policy heads of Figure 1 and VFD with the same initial state and seed, for two initial states per task and averaged over three seeds. Chance level is 0.5.

Score	Within-task nAUC	Global nAUC	Automated	Failure AUROC
KV-Ctx	$0.40\pm 0.05$	$0.35\pm 0.02$	$0.65\pm 0.22$	$0.76\pm 0.03$
KV-Dec	$0.46\pm 0.03$	$0.41\pm 0.02$	$0.76\pm 0.13$	$0.79\pm 0.01$
VFD (two policies)	$0.40\pm 0.03$	$0.39\pm 0.03$	$0.73\pm 0.14$	$0.75\pm 0.00$
Cross-replicate reference	$0.21\pm 0.03$	$0.21\pm 0.03$	$0.53\pm 0.25$	$0.62\pm 0.01$

Table 6: Within-task referral under perturbations to a supervisor trained with them, based on uncertainty scores of clean and perturbed starting observations (mean

\pm

SD over three seeds). Chance level is 0 for nAUC and 0.5 for AUROC. The cross-replicate reference ranks the conditions by the supervisor’s gain observed in one of the three executions of each condition and is evaluated on the other two.

Figure 5: Task success under perturbations. (a) Detection of perturbed starting observations against the loss of success they cause. Each point is one level of a perturbation family (Appendix Table 10 ), placed by KV-Dec AUROC of perturbed against clean starting observations and by success decline relative to clean executions from the same initial states. (b) Success of the supervisor, which is trained on these perturbations, minus that of the policy whose uncertainty is estimated, for each perturbation of the referral experiment (Table 6 ), averaged over tasks and their ten initial states, with seed values (small dots), their mean (large dot) and the 95% hierarchical bootstrap interval (bar). Rows are sorted by the mean, and coloured rows have an interval above zero. Symbols give the perturbed input in both panels.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Head	Embedding	Geometry and kernel
KV-Ctx	Final prefix states: mean per camera and over instruction tokens, plus 8 state coordinates (2,888)	Standardize, PCA512, Ledoit–Wolf whitening, $k$ -NN bandwidth, $\lambda=10^{-3}$
KV-Dec	Action-expert final hidden state at flow step 9 of 10, mean over 50 action tokens (480)	Standardize, full Ledoit–Wolf whitening, median-distance bandwidth, $\lambda=10^{-2}$
Context heads	Raw: mean token embedding of the workspace camera, wrist camera and instruction (960 each), 8 state values. Fused: final prefix states of the same groups and of the state token (960 each)	As KV-Ctx , scores compared as percentiles among clean calibration observations

Appendix

Table 7: Kernel-variance heads in the main experiments. All heads use exact GP conditioning on their support observations and need no regression targets.

	Round 0				Round 10
Score	$-\rho_{\rm S}$	$-r_{\rm P}$	Global	Within	$-\rho_{\rm S}$	$-r_{\rm P}$	Global	Within
VFD	$0.83\pm 0.05$	$0.75\pm 0.05$	$0.65\pm 0.05$	$0.08\pm 0.05$	$0.66\pm 0.27$	$0.54\pm 0.29$	$0.31\pm 0.09$	$0.07\pm 0.05$
GU	$0.81\pm 0.04$	$0.89\pm 0.02$	$0.65\pm 0.03$	$0.05\pm 0.03$	$0.50\pm 0.40$	$0.47\pm 0.40$	$0.20\pm 0.10$	$0.06\pm 0.03$
Action-L2	$0.81\pm 0.04$	$0.66\pm 0.06$	$0.58\pm 0.05$	$0.06\pm 0.06$	$0.32\pm 0.14$	$0.34\pm 0.24$	$0.13\pm 0.07$	$0.06\pm 0.04$
ACE	$0.72\pm 0.11$	$0.67\pm 0.05$	$0.43\pm 0.05$	$0.01\pm 0.08$	$0.15\pm 0.28$	$0.20\pm 0.26$	$0.15\pm 0.09$	$0.06\pm 0.10$
DECU	$0.50\pm 0.39$	$0.44\pm 0.11$	$0.26\pm 0.19$	$-0.03\pm 0.08$	$0.24\pm 0.42$	$0.36\pm 0.26$	$0.14\pm 0.08$	$0.03\pm 0.01$

Appendix

Table 8: Calibration and referral at each common-policy checkpoint (mean

\pm

SD over three seeds).

Difference	$-\rho_{\rm S}$	$-r_{\rm P}$	Global nAUC	Within nAUC	Automated fraction
KV-Ctx $-$ VFD	$+0.020$ $[-0.021,+0.090]$	$+0.176$ $[+0.106,+0.254]$	$+0.022$ $[-0.019,+0.063]$	$-0.041$ $[-0.142,+0.062]$	$+0.044$ $[-0.014,+0.082]$
KV-Ctx $-$ GU	$+0.108$ $[+0.003,+0.225]$	$+0.139$ $[+0.045,+0.244]$	$+0.082$ $[+0.039,+0.122]$	$-0.022$ $[-0.143,+0.092]$	$+0.068$ $[+0.025,+0.117]$
KV-Dec $-$ VFD	$+0.003$ $[-0.036,+0.059]$	$+0.152$ $[+0.112,+0.187]$	$+0.028$ $[-0.005,+0.059]$	$-0.011$ $[-0.079,+0.060]$	$+0.084$ $[+0.030,+0.111]$
KV-Dec $-$ GU	$+0.091$ $[+0.030,+0.129]$	$+0.115$ $[+0.051,+0.179]$	$+0.087$ $[+0.048,+0.124]$	$+0.008$ $[-0.070,+0.084]$	$+0.108$ $[+0.068,+0.149]$

Appendix

Table 9: Paired differences averaged over rounds 0 and 10, with 95% hierarchical bootstrap intervals (2,000 draws over conditions within tasks and over seeds, shared across scores).

Input	Perturbation and levels	Starting observ.	During execution	Referral	Label- preserving
instruction	graded paraphrase: P3; P5	$\checkmark$	$\checkmark$	$\checkmark$	yes
instruction	object synonym: all object words replaced	$\checkmark$		$\checkmark$	yes
instruction	surface variation: ’please ’ prefix; one typo in a non-object word	$\checkmark$		$\checkmark$	yes
instruction	underspecified: object nouns replaced by pronouns (’put it in the basket’)	$\checkmark$			no
instruction	blatant: negated (’do not …’); German translation	$\checkmark$	$\checkmark$		no
vision (both cameras)	lighting change: intensity x0.8; x0.4	$\checkmark$	$\checkmark$	$\checkmark$	yes

Appendix

Table 10: Perturbation library, with the experiments that use each family. Label-preserving: the demonstrated actions remain correct, so the supervisor can be trained on the perturbation. P3 and P5: two of the released paraphrases of each instruction, which range from close (P2) to distant (P5).

Experiment	Difference	Mean over seeds, 95 % hierarchical interval
Referral (Table 6 )		Within-task nAUC	Global nAUC	Automated	Failure AUROC
	KV-Ctx $-$ VFD	$0.00$ $[-0.06,+0.06]$	$-0.05$ $[-0.10,+0.01]$	$-0.08$ $[-0.21,+0.02]$	$+0.01$ $[-0.02,+0.04]$
	KV-Dec $-$ VFD	$+0.06$ $[+0.03,+0.10]$	$+0.02$ $[-0.01,+0.06]$	$+0.03$ $[-0.02,+0.10]$	$+0.03$ $[+0.02,+0.05]$
Detection AUROC		Higher	Not different	Lower	Median difference
Starting observations (Table 4 )	KV-Ctx $-$ VFD	4	11	2	$+0.01$
	KV-Dec $-$ VFD	12	5	0	$+0.04$

Appendix

Table 11: Paired differences between the paper heads and VFD in the context-head experiments: mean over three seeds with 95% hierarchical bootstrap intervals for referral, and the number of perturbations (Table 4 ) or events (Table 5 ) for which the paper head’s AUROC is significantly higher, not significantly different or significantly lower than VFD’s. On starting observations, the comparison uses the ten initial states per task on which VFD is computed. Intervals are unadjusted for multiple comparisons, and perturbation-wise comparisons are exploratory.

Uncertainty Estimation for VLA Models

Uncertainty estimation for Vision-Language-Action (VLA) models predicts when robotic actions or task outcomes may be unreliable.

See all 32 papers

Jun 18, 2026

Perturbation-Based Epistemic Uncertainty for Failure Detection in Vision-Language-Action Models

Vision-Language-Action (VLA) models have shown strong performance in robotic manipulation, but reliable uncertainty quantification remains challenging, particularly under distribution shift. Unlike autoregressive policies, many modern VLA models generate continuous actions through regression or flow-based generation, where explicit predictive probabilities are unavailable. Moreover, stochastic action sampling primarily captures action-generation variability under a fixed model, while failure detection under distribution shift can benefit from capturing uncertainty in the model itself. Motivated by Bayesian perspectives on local model variations, we propose perturbation-based failure detection (PFD), a training-free framework for estimating epistemic uncertainty in VLA models through low-rank weight perturbations. Specifically, we inject random low-rank perturbations into selected transformer weight matrices and estimate epistemic uncertainty from disagreement across perturbed action predictions. Experiments on LIBERO-PRO show that PFD achieves the highest average AUROC and balanced accuracy among the evaluated methods while consistently outperforming stochastic action sampling across distribution shifts. Real-world robot experiments further demonstrate that PFD provides a competitive failure-detection signal under an unseen object shift.

Yousung Lee, Dongsoo Har

Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea.

Failure Detection for VLA Models Uncertainty Estimation for VLA Models Robotic Manipulation Vision-Language-Action Models Epistemic Uncertainty

Ask PDF

Mar 18, 2026

Shifting Uncertainty to Critical Moments: Towards Reliable Uncertainty Quantification for VLA Model

Vision-Language-Action (VLA) models enable general-purpose robotic policies by mapping visual observations and language instructions to low-level actions, but they often lack reliable introspection. A common practice is to compute a token-level uncertainty signal and take its mean over a rollout. However, mean aggregation can dilute short-lived but safety-critical uncertainty spikes in continuous control. In particular, successful rollouts may contain localized high-entropy segments due to benign noise or non-critical micro-adjustments, while failure rollouts can appear low-entropy for most timesteps and only exhibit brief spikes near the onset of failure. We propose a unified uncertainty quantification approach for predicting rollout success versus failure that (1) uses max-based sliding window pooling to preserve transient risk signals, (2) applies motion-aware stability weighting to emphasize high-frequency action oscillations associated with unstable behaviors, and (3) performs DoF-adaptive calibration via Bayesian Optimization to prioritize kinematically critical axes. Experiments on the LIBERO benchmark show that our method substantially improves failure prediction accuracy and yields more reliable signals for failure detection, which can support downstream human-in-the-loop interventions.

Yanchuan Tang, Taowen Wang, Yuefei Chen +3

Rutgers University · Kent State University

Uncertainty Estimation for VLA Models Uncertainty Calibration

Ask PDF

Feb 25, 2026

When to Act, Ask, or Learn: Uncertainty-Aware Policy Steering

Policy steering is an emerging way to adapt robot behaviors at deployment-time: a learned verifier analyzes low-level action samples proposed by a pre-trained policy (e.g., diffusion policy) and selects only those aligned with the task. While Vision-Language Models (VLMs) are promising general-purpose verifiers due to their reasoning capabilities, existing frameworks often assume these models are well-calibrated. In practice, the overconfident judgment from VLM can degrade the steering performance under both high-level semantic uncertainty in task specifications and low-level action uncertainty or incapability of the pre-trained policy. We propose uncertainty-aware policy steering (UPS), a framework that jointly reasons about semantic task uncertainty and low-level action feasibility, and selects an uncertainty resolution strategy: execute a high-confidence action, clarify task ambiguity via natural language queries, or ask for action interventions to correct the low-level policy when it is deemed incapable at the task. We leverage conformal prediction to calibrate the composition of the VLM and the pre-trained base policy, providing statistical assurances that the verifier selects the correct strategy. After collecting interventions during deployment, we employ residual learning to improve the capability of the pre-trained policy, enabling the system to learn continually but with minimal expensive human feedback. We demonstrate our framework through experiments in simulation and on hardware, showing that UPS can disentangle confident, ambiguous, and incapable scenarios and minimizes expensive user interventions compared to uncalibrated baselines and prior human- or robot-gated continual learning approaches. Videos can be found at https://jessie-yuan.github.io/ups/

Jessie Yuan, Yilin Wu, Andrea Bajcsy

Carnegie Mellon University

Robot Policy Adaptation VLMs for Robotics Continual Robot Learning

Ask PDF

Abstract

Uncertainty Estimation for VLA Models

Explore similar work

Perturbation-Based Epistemic Uncertainty for Failure Detection in Vision-Language-Action Models

Shifting Uncertainty to Critical Moments: Towards Reliable Uncertainty Quantification for VLA Model

When to Act, Ask, or Learn: Uncertainty-Aware Policy Steering