Vision-language models (VLMs), such as CLIP, have achieved strong performance across multimodal tasks by aligning visual and textual representations in a shared embedding space. As VLMs are increasingly used for high-stakes domains, failure prediction becomes critical for risk-aware deployment and human intervention. Existing failure prediction methods typically rely on confidence scores or auxiliary classifiers. Although these methods are effective on predicting VLM failures, they provide limited interpretability. In this work, we investigate the use of Sparse Autoencoders (SAEs) for interpretable failure prediction in VLMs. We formulate failure prediction as a classification task over sparse SAE latent activations and introduce a three-stage failure-aware training pipeline that encourages the learned latent directions to remain interpretable while becoming more informative for failure prediction. Our experiments show that the resulting framework outperforms the evaluated baselines in failure prediction. Further analysis suggests that failure-aware training encourages SAE latent directions to capture more class-specific concepts. We also use the SAE to provide a concept-level analysis of how model representations change during failures, revealing a shift from class-specific concepts toward more ambiguous or style-related concepts. Finally, we explore how the learned SAE latent directions can support runtime failure recovery.
Figures & tables
Figure 1: Overview.
Fine-grained prediction
Binary prediction
Dataset
Method
Acc ↑
Macro F1 ↑
Macro Recall ↑
Acc ↑
AUROC ↑
FPR@95 ↓
CIFAR-10
ORCA-B
-
-
-
84.5 ± 0.0
85.2 ± 0.0
52.5 ± 0.0
ORCA-R
-
-
-
85.0 ± 0.0
85.3 ± 0.0
49.3 ± 0.0
SSL
82.9 ± 0.6
60.7 ± 0.8
61.6 ± 0.7
88.4 ± 0.5
87.1 ± 0.5
43.5 ± 1.4
SuperMentor
86.4 ± 0.5
65.2 ± 0.7
62.7 ± 0.6
90.1 ± 0.4
91.7 ± 0.5
38.6 ± 1.2
FailSAE
88.0 ± 0.7
66.6 ± 0.6
66.3 ± 0.4
92.4 ± 0.6
93.9 ± 0.1
32.0 ± 0.9
Table 1: Failure prediction performance comparison across datasets.
CIFAR-10
CIFAR-100
ImageNet-1K
Method
Acc
AUROC
Acc
AUROC
Acc
AUROC
Embeddings
80.7
77.6
68.7
68.2
63.5
64.6
w.o. Stage 3
85.8
82.1
71.0
73.4
66.5
70.2
w.o. Stage 2
87.0
90.8
72.1
80.2
70.1
77.4
FailSAE
88.0
93.9
76.3
85.6
73.0
81.4
Table 2: Ablation study.
Figure 2: Impact of top- K latent masking.
CIFAR-10
CIFAR-100
ImageNet-1K
Method
Acc
AUROC
Acc
AUROC
Acc
AUROC
CLIP-B/32
85.9
87.9
69.0
78.9
66.8
75.0
SigLIP-B/16
84.4
87.7
70.3
79.6
64.4
70.5
CLIP-L/14
85.7
84.1
65.8
75.5
65.3
70.9
CLIP-B/16
88.1
94.0
76.8
85.8
73.0
81.5
Table 3: Performance on various VLMs.
Figure 3: Distribution of SAE latent directions.
Figure 4: Reference images of latent directions.
Ambiguous concepts
Class-specific concepts
Correct
297
703
Error-ID
414 ( +39.4% )
586 ( -16.6% )
Error-OOD
384 ( +29.3% )
616 ( -12.4% )
Error-ADV
373 ( +25.6% )
627 ( -10.8% )
Table 4: Distribution of top activated SAE latent directions. Numbers in parentheses indicate the relative change compared to correct predictions.
Figure 5: Emerging (green arrows) and disappearing (red arrows) SAE latent direction for two classes.
Vision-Language Models (VLMs) are increasingly used in safety-critical applications because of their broad reasoning capabilities and ability to generalize with minimal task-specific engineering. Despite these advantages, they can exhibit catastrophic failures in specific real-world situations, constituting failure modes. We introduce REVELIO, a framework for systematically uncovering interpretable failure modes in VLMs. We define a failure mode as a composition of interpretable, domain-relevant concepts-such as pedestrian proximity or adverse weather conditions-under which a target VLM consistently behaves incorrectly. Identifying such failures requires searching over an exponentially large discrete combinatorial space. To address this challenge, REVELIO combines two search procedures: a diversity-aware beam search that efficiently maps the failure landscape, and a Gaussian-process Thompson Sampling strategy that enables broader exploration of complex failure modes. We apply REVELIO to autonomous driving and indoor robotics domains, uncovering previously unreported vulnerabilities in state-of-the-art VLMs. In driving environments, the models often demonstrate weak spatial grounding and fail to account for major obstructions, leading to recommendations that would result in simulated crashes. In indoor robotics tasks, VLMs either miss safety hazards or behave excessively conservatively, producing false alarms and reducing operational efficiency. By identifying structured and interpretable failure modes, REVELIO offers actionable insights that can support targeted VLM safety improvements.
Recent advances in language model interpretability using sparse autoencoders (SAEs) have yet to effectively translate to the visual domain, mainly due to the difficulty and ambiguity of labeling visual concepts. In this paper, we introduce Visual Interpretability via SAE Transfer Alignment (VISTA), a framework that transfers interpretability from language to vision in a LLaVA-style vision-language model by constraining a visual projector to map visual tokens into an LLM's pre-existing, labeled textual SAE space. This approach enables visual interpretability without training dedicated vision SAEs. By regularizing the projector using the LLM's SAE reconstruction loss, VISTA achieves a threefold increase in the matching rate, which measures how accurately the most activating textual concepts in the SAE space correspond to semantic elements in the image. Using this framework, we further analyze spatial localization properties of different vision encoders and show that DINOv2 features have stronger localization abilities than other encoders. Leveraging this precision, we validate VISTA's cross-modal alignment through fine-grained, localized concept interventions, where specific objects are removed or replaced in the model's perception while preserving the surrounding scene. This results in improvements of 35% in object removal and 47% in object replacement tasks over vision-only baselines, providing causal evidence that visual tokens inhabit the text SAE manifold. These contributions are validated across multiple LLM architectures.
Alexey Kravets, Da Li, Chuan Li +2
University of Bath, UK · Samsung AI Centre Cam- bridge · Lambda, Inc..
Vision-Language-Action (VLA) models have emerged as powerful general-purpose robot policies, but their successes and failures remain difficult to diagnose and understand. In this work, we show that sparse autoencoder (SAE) representations of OpenVLA activations contain outcome-predictive information and can be used to predict task failure before the robot reaches an unrecoverable state. Moreover, to provide insights into what common behaviors may contribute to these outcomes, we identify individual SAE features whose strong activations are associated with specific stages of success or failure, and uncover recurring behavioral patterns by localizing temporal intervals where these features exhibit significant activation changes. We further develop a VLM-based interpretation pipeline to generate behavioral descriptions of these patterns and validate their correspondence to feature activations on held-out intervals.