What Would Removing This Corruption Change? Failure Attribution for Vision-Language-Action Models

Carson Sobolewski; Anjali Parashar; Fei Chen; Yingke Li; Chuchu Fan

cs.ROOct 19, 2026

What Would Removing This Corruption Change? Failure Attribution for Vision-Language-Action Models

Authors: Carson Sobolewski, Anjali Parashar, Fei Chen, Yingke Li, Chuchu Fan

Organizations: The authors are with the Massachusetts Institute of Technology, USA.

Abstract

Vision-language-action (VLA) models are sensitive to visual corruptions, but knowing that a corruption is typically harmful does not establish whether removing it would benefit the current episode. Furthermore, when several corruptions co-occur, deciding what to correct requires understanding which ones contribute to failure risk and how much their removal would help. We define this problem as failure attribution. Detecting corruption presence alone cannot answer this question, as corruptions can be present but relatively harmless, while directly measuring removal benefits requires costly paired policy rollouts with and without the corruption present. To address this, we introduce a factorized risk model that learns separate representations of the scene and corruptions, allowing us to estimate removal benefits through latent intervention from the current image and task instruction, without paired clean images or additional counterfactual rollouts. We evaluate our approach on OpenVLA and $π_{0.5}$ in simulation and $π_{0.5}$ -DROID on a Franka robot. Our method outperforms monolithic and vision-language-model (VLM) baselines, including achieving approximately 3x lower attribution error than frontier VLMs. We also demonstrate our method adapts to execution context and corruption intensity, and can be used for online failure monitoring on hardware.

Figures & tables

Fig. 2: t-SNE projections of input features (left) and internal policy features (right) for the same held-out initial observations from failed OpenVLA episodes. Colors denote corruption types, and numbered markers indicate the location of one example image per corruption (shown below, with empirical failure rates). Corruption types are more visibly separated in the input features than in the internal policy features.

Fig. 3: Factorized risk model for failure attribution. A frozen image-and-text encoder produces features used to construct scene and corruption latents, $z_{s}$ and $z_{c}$ . INLP and adversarial training discourage corruption information in the scene branch, while presence and anchor losses structure the corruption latent. A shared failure head predicts risk from both latents. Attribution is estimated by zeroing the corresponding corruption block and reevaluating risk ( Equation 2 ). Red labels indicate training objectives.

Fig. 4: Predicted vs. empirical per-corruption attribution for OpenVLA (top) and

\pi_{0.5}

(bottom). Columns compare Ours (SigLIP) with rescaled responsibility scores from VLM (Sol) . Each point is one present corruption within one corruption condition (e.g., the condition light+view contributes one point for light and one for view). Our estimates show closer agreement (diagonal), while the VLM’s scores are less correlated with empirical per-corruption attribution.

Policy	Method	Single $\downarrow$	Multi $\downarrow$	Regret $\downarrow$
OpenVLA	Monolithic	$0.416_{\pm.000}$	$0.265_{\pm.000}$	$0.256_{\pm.025}$
	VLM (Terra)	$0.207$	$0.237$	$0.285$
	VLM (Sol)	$0.246$	$0.308$	$0.098$
	VLM (Opus 5)	$0.245$	$0.292$	$0.107$
	Ours (SAFE)	$0.104_{\pm.012}$	$0.117_{\pm.020}$	$0.029_{\pm.010}$
	Ours (SigLIP)	$\mathbf{0.095_{\pm.025}}$	$\mathbf{0.101_{\pm.016}}$	$\mathbf{0.011_{\pm.001}}$

TABLE I: Failure attribution on LIBERO (2,500 episodes per policy). Attribution MAE (single-/multi-corruption conditions) and regret (multi-corruption failure episodes). Five-seed mean

\pm

standard deviation. Oracle achieves

0.000

for all columns. Bold : best among non-oracle methods.

Fig. 5: Joint versus summed individual attributions for two-corruption combinations under Ours (SigLIP) . The vertical axis shows attribution from removing both corruptions together

\hat{\Delta}_{\{j,k\}}

. The horizontal axis sums their individual attributions. The diagonal marks perfect additivity. Several OpenVLA combinations lie far above the diagonal, indicating greater predicted benefit from joint removal than from the sum of individual removals. The

\pi_{0.5}

estimates lie closer to the diagonal.

Episode position	Nominal	Corrupted
0.0	$0.510_{\pm.149}$	$0.918_{\pm.017}$
0.2	$0.655_{\pm.093}$	$0.918_{\pm.017}$
0.4	$0.485_{\pm.193}$	$0.917_{\pm.017}$
0.6	$0.722_{\pm.062}$	$0.932_{\pm.014}$
0.8	$0.679_{\pm.160}$	$0.927_{\pm.017}$
1.0	$0.723_{\pm.162}$	$0.925_{\pm.018}$

TABLE II: Single-frame failure prediction ROC-AUC at different episode positions on held-out OpenVLA episodes ( Ours (SigLIP) , five-seed mean

\pm

standard deviation). The oracle assigns test-set empirical corruption failure rates and is independent of episode position.

Fig. 6: Example observations from the evaluation environments: LIBERO-Object simulation (left) and the Franka Panda hardware setup for the task “put the cube in the bowl” (right). We evaluate OpenVLA and $\pi_{0.5}$ in simulation and $\pi_{0.5}$ -DROID on hardware.

Variant	Pres.	MAE $\downarrow$	Reg. $\downarrow$	$\\|z_{c}^{(k)}\\|_{\varnothing}$ $\downarrow$	$z_{s}$ leak
Full	$99.4$	$0.092_{\pm.017}$	$0.010_{\pm.004}$	$0.31_{\pm.07}$	$75.1_{\pm 8.2}$
$-$ adversary	$99.5$	$0.080_{\pm.010}$	$0.014_{\pm.003}$	$0.21_{\pm.04}$	$99.8_{\pm 0.1}$
$-$ anchor	$99.4$	$0.089_{\pm.021}$	$0.040_{\pm.031}$	$30.3_{\pm 3.1}$	$78.0_{\pm 9.0}$
$-$ consistency	$99.5$	$0.132_{\pm.015}$	$0.016_{\pm.011}$	$0.06_{\pm.04}$	$72.1_{\pm 6.9}$
$-$ INLP	$99.5$	$0.084_{\pm.011}$	$0.016_{\pm.005}$	$0.35_{\pm.03}$	$82.6_{\pm 10.7}$
$-$ presence	$49.3$	$0.370_{\pm.033}$	$0.245_{\pm.061}$	$0.52_{\pm.14}$	$68.6_{\pm 6.8}$

TABLE III: Loss ablation (OpenVLA, Ours (SigLIP) ; five-seed mean

\pm

standard deviation). Pres.: presence accuracy (%). MAE: single-corruption attribution MAE. Reg.: regret on multi-corruption failure episodes.

\|z_{c}^{(k)}\|_{\varnothing}

: mean norm of block

k

when corruption

k

is absent (

0

ideal).

z_{s}

leak: AUC (%) of predicting presence from

z_{s}

(

50

ideal,

100

full leakage). Ablations use separately trained models.

Fig. 7: Fog attribution across six intensities for OpenVLA with Ours (SigLIP) . Each intensity is applied to the same held-out scenes. Level 4 occurs in the training data, while the others are unseen. Our predictions respond to intensity, while the oracle retains its original test-set lookup value across the sweep.

Corruption	$\bar{\Delta}_{k}$ (emp.)	$\hat{\Delta}_{k}$ (ours)	VLM score
Light	$0.96$	$0.96_{\pm.01}$	$1.00$
Fog	$0.56$	$0.56_{\pm.05}$	$0.70$
Blur	$0.24$	$0.27_{\pm.03}$	$0.45$
Object	$0.04$	$0.03_{\pm.01}$	$0.35$

TABLE IV: Hardware attribution (

\pi_{0.5}

-DROID, “put the cube in the bowl”; five-seed mean

\pm

standard deviation). Empirical per-corruption attribution

\bar{\Delta}_{k}

(all episodes), our estimate

\hat{\Delta}_{k}

(held-out), and VLM (Sol) ’s responsibility score rescaled to

[0,1]

Robustness of VLA Models

Robustness of vision-language-action (VLA) models concerns reliable robot behavior under changes to language, vision, sensors, tasks, and hardware.

See all 221 papers

What Can One Sentence Fix? A Failure Taxonomy and a Minimal Language Correction for Frozen VLA Policies
Aug 30, 2026Owen Kwon, Pablo Ortega-Kral, Arthur Bucker +1Vision-Language-Action Models Test-Time Adaptation of VLA Models
Biomedical Engineering, Carnegie Mellon University, Pittsburgh PA 15213, USA. · Robotics Institute, Carnegie Mellon University Pittsburgh PA 15213, USA.
FailPatch: Failure Residual Patching for Vision-Language-Action Models
Sep 28, 2026Peng Yu, Jiacheng Wang, Ziheng Zhang +6Robot Policy Learning Robot Policy Adaptation
Xi’an Jiaotong University · Dexmal · Nanjing University
Are Vision-Language-Action Models Robust to One-Step Observation Perturbations?
Sep 26, 2026Shojiro Yamabe, Jun SakumaRobustness of VLA Models Safety Evaluation of VLA Models
Science Tokyo · Science Tokyo, RIKEN

Aug 30, 2026J/K move · Enter open · S save

What Can One Sentence Fix? A Failure Taxonomy and a Minimal Language Correction for Frozen VLA Policies

Owen Kwon, Pablo Ortega-Kral, Arthur Bucker, Jean Oh

Biomedical Engineering, Carnegie Mellon University, Pittsburgh PA 15213, USA. · Robotics Institute, Carnegie Mellon University Pittsburgh PA 15213, USA.

Vision-Language-Action Models Test-Time Adaptation of VLA Models Robustness of VLA Models

Open paper AskPDF arXiv

When a deployed vision-language-action (VLA) policy fails, fine-tuning it again for every new setting is costly, yet the person watching it can usually say what went wrong in one sentence. We ask what such a sentence can and cannot fix in a frozen policy. Labelling every failing LIBERO-90 task (out of distribution) for OpenVLA-OFT and pi0.5, we find that the failures sort into four modes and that only one, execution misalignment, where the policy reaches the right target but miscalibrates how far or how hard to move, is within reach of a sentence; the other three lack a target or a plan, not a magnitude. For this mode we define the narrowest human correction: one retrospective sentence per task, grounded by an LLM into an action dimension, direction, magnitude term, and time window, becomes a short bias on the frozen policy's actions, after which the policy completes the rest on its own, episode after episode. Re-executed on its task's failed rollouts, it recovers 65 of all 139 in-distribution and 33 of all 328 out-of-distribution failures in simulation; on a real xArm7 after the robot base is moved, it raises pick-and-place from 10/30 to 30/30 in each of two same-object conditions and from 2/30 to 24/30 with a new object, while two long-horizon tasks stay at 0/30 with and without correction, as the taxonomy predicts. A naively prompted VLM that watches the same failures and writes the same kind of correction recovers none in up to 10 attempts, suggesting that the value lies in the person's calibrated sense of magnitude and timing rather than in language itself. Videos: https://correctvla.github.io/

Abstract

Robustness of VLA Models

Explore similar work

What Can One Sentence Fix? A Failure Taxonomy and a Minimal Language Correction for Frozen VLA Policies

FailPatch: Failure Residual Patching for Vision-Language-Action Models

Are Vision-Language-Action Models Robust to One-Step Observation Perturbations?