The use of Vision-Language Models (VLMs) in automated driving applications is becoming increasingly common, with the aim of leveraging their reasoning and generalisation capabilities to handle long-tail scenarios. However, these models often fail on simple visual questions that are highly relevant to automated driving, and the reasons behind these failures remain poorly understood. In this work, we examine the intermediate activations of VLMs and assess the extent to which specific visual concepts are linearly encoded, with the goal of identifying bottlenecks in the flow of visual information. Specifically, we create counterfactual image sets that differ only in a targeted visual concept and then train linear probes to distinguish between them using the activations of five state-of-the-art (SOTA) VLMs, including two training variants for one model. Our results show that concepts such as the presence of an object or agent in a scene are explicitly and linearly encoded, whereas other spatial visual concepts, such as the orientation of an object or agent, are only implicitly encoded by the spatial structure retained by the vision encoder, or are not linearly encoded at all. In parallel, we observe that in certain cases, even when a concept is linearly encoded in the model's activations, the model still fails to answer correctly. This leads us to identify two failure modes. The first is perceptual failure, where the visual information required to answer a question is not linearly encoded in the model's activations. The second is cognitive failure, where the visual information is present but the model fails to align it correctly with language semantics. Finally, we show that increasing the distance of the object in question quickly degrades the linear separability of the corresponding visual concept...
Figures & tables
Figure 1: Framework for tracking visual concept representations across VLM architectures . Left: The model receives counterfactual input pairs that differ only by a specific visual concept (e.g., the presence of a pedestrian). Middle: Linear probes are trained on intermediate activations to detect if the concept is linearly encoded within the Vision Encoder, Projector, or LLM. Right: Comparing probe accuracy to model output reveals two distinct failure modes: Perceptual Failure , where visual information is not linearly encoded in the activations, and Cognitive Failure , where the information is encoded (high probe accuracy) but the model fails to align it with language semantics, resulting in an incorrect answer.
Figure 2: Counterfactual sets of images representing four basic visual concepts : Presence , Count , Spatial Relatioinship , Orientation
VLM
Param (B)
Vision Encoder
Projector
Language Model
FT
Ovis2.5-2B
2.57
SigLIP2-400M
MLP 5 5 5 The Ovis2.5 technical report states that visual features are projected into the LLM input space via a weighted sum over a learned visual embedding table. Mathematically, however, this mechanism is equivalent to a softmax-activated MLP projector with a significantly expanded hidden dimension. (436M)
Qwen3-1.7B
General
InternVL3.5-2B
2.35
InternViT-300M-v2.5
MLP (12.6M)
Qwen3-1.7B 6 6 6 For InternVL3.5, they unbind the embedding and unembedding matrices of Qwen3, resulting actually in a model with 2B parameters instead of 1.7B.
General
Qwen3-VL-2B
2.13
SigLIP2-Large (300M)
MLP (100M)
Qwen3-1.7B
General
VST-3B
3.75
QwenViT (632M)
MLP (36.7M)
Qwen2.5-3B
Spatial
DriveFusionQA
3.75
QwenViT (632M)
MLP (36.7M)
Qwen2.5-3B
AD
Table 1: VLMs used in the experiments . FT indicates whether the model is a general-purpose vision-language model (General) or has been additionally fine-tuned for a specific task/domain, such as spatial reasoning (Spatial) or autonomous driving (AD).
Category
Question
Answers
Presence-1
Is there a pedestrian ahead?
Yes | No
Presence-2
Is there a traffic barrel ahead?
Yes | No
Count-1
How many pedestrians are ahead?
Zero | One | Two | Three | Four
Count-2
How many traffic barrels are ahead?
Zero | One | Two | Three | Four
Spatial-1
Which of the truck’s blinkers is on?
Left | Right
Spatial-2
On which side of the road is the pedestrian walking?
Left | Right
Table 2: Questions used for each data category .
Figure 3: Activation extraction methodology . a) We apply average pooling to all patch vectors to obtain a single vector representation of the intermediate activations. b) We split the image at a selected point and apply average pooling to the left and right regions independently. We then concatenate the resulting vectors to form a single representation of the intermediate activations while retaining minimal spatial structure. c) We apply average pooling to the visual token vectors within the LLM and concatenate the result with the activation of the last token.
Figure 4: Linear separability of visual concepts across models and layers based on average-pooled activations . Linear separability for each model, layer, and distance is measured as the average test-set accuracy across ten linear probes trained under the same setting.
Figure 5: Linear separability of visual concepts across models and layers based on region-pooled activations . Linear separability for each model, layer, and distance is measured as the average test-set accuracy across ten linear probes trained under the same setting.
Figure 6: Cosine similarity between the learned weights of the probes for each data category . The probes considered here were trained on average-pooled activations of the last layer of the model (after layer normalisation) for the shortest-distance samples.
Table 3: Activation steering results . Each row shows how applying the directions learned by the linear probes as steering vectors alters the model’s output, changing the original description to the steered description. The “Steering” column specifies the layer and token positions where the steering vector is applied. Additional details along with quantitative results are provided in Appendix C .
Model
Presence-Real Acc. (%)
Count-Real Acc. (%)
Ovis2.5
82.1 (138/168)
84.6 (44/52)
InternVL3.5
81.5 (137/168)
78.9 (41/52)
Qwen3-VL
89.9 (151/168)
61.5 (32/52)
VST-SFT
97.6 (164/168)
90.4 (47/52)
VST-RL
97.0 (163/168)
75.0 (39/52)
DriveFusionQA
97.6 (164/168)
82.7 (43/52)
Table 4: Probes accuracy on nuScenes data .
Figure 7: Chance-corrected accuracy gap . The gap is calculated between the test accuracy of the probes trained on the last-token activations of the last layer of the model (after layer normalisation) and the test accuracy of the models themselves. Grey indicates the absence of Spatial-2 data at 5 meters.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Model
Category
Accuracy (%)
Gen.
Constr. Gen.
Vis. Enc.
LLM
Ovis2.5
Presence-1
99.0
96.0
94.9
99.5
Presence-2
91.0
81.0
99.1
100.0
Count-1
86.5
87.0
72.8
88.9
Count-2
86.8
86.0
94.6
98.4
Spatial-1
100.0
50.0
57.7
100.0
Appendix
Table 5: Detailed Evaluation Results . We report the accuracy achieved by each model on the test set (Gen.) when prompted with images in which the object of interest is at a distance of 5 meters (or 10 meters for Spatial-2). Additionally, we report the accuracy obtained under constrained generation (Constr. Gen.). Finally, we report the accuracy of linear probes trained on the models’ activations at the last vision encoder layer (Vis. Enc.) and at the last LLM layer (LLM), based on the activations of the last token in the sequence only.
Figure 8: Lower-bound equivalent of Figure 4 .
Figure 9: Lower-bound equivalent of Figure 5 .
Model
Presence-1
Presence-2
Count-1
Count-2
Spatial-1
Spatial-2
Ovis2.5
94.0 (47/50)
100.0 (50/50)
93.8 (45/48)
60.0 (30/50)
76.6 (36/47)
100.0 (48/48)
InternVL3.5
100.0 (50/50)
100.0 (48/48)
75.0 (36/48)
42.6 (20/47)
83.9 (26/31)
87.8 (43/49)
Qwen3-VL
100.0 (50/50)
100.0 (49/49)
73.9 (34/46)
71.4 (35/49)
97.0 (32/33)
100.0 (49/49)
VST-SFT
100.0 (50/50)
100.0 (46/46)
97.9 (47/48)
71.4 (35/49)
97.4 (37/38)
100.0 (49/49)
VST-RL
100.0 (50/50)
100.0 (38/38)
78.3 (36/46)
64.4 (29/45)
100.0 (42/42)
87.8 (43/49)
DriveFusionQA
100.0 (40/40)
100.0 (5/5)
7.9 (3/38)
0.0 (0/5)
88.2 (15/17)
100.0 (45/45)
Appendix
Table 6: Activation steering success rate (%) by model and data category . Percentages are computed over valid steering cases only; counts in parentheses indicate successful cases over valid cases.
Table 7: Logistic regression on logits results . The word clouds show the tokens that correspond to non-zero weights. Increased size means a larger weight value. Blue and red colours mean positive and negative weight values, respectively.
Table 8: Logistic regression on logits results . The word clouds show the tokens that correspond to non-zero weights. Increased size means a larger weight value. Blue and red colours mean positive and negative weight values, respectively.
Vision-Language Models (VLMs) are increasingly proposed for autonomous driving tasks, yet their performance on sequential driving scenes remains poorly characterized, particularly regarding how input configurations affect their capabilities. We introduce VENUSS (VLM Evaluation oN Understanding Sequential Scenes), a framework for systematic sensitivity analysis of VLM performance on sequential driving scenes, establishing baselines for future research. Building upon existing datasets, VENUSS extracts temporal sequences from driving videos, and generates structured evaluations across custom categories. By comparing 25+ existing VLMs across 2,600+ scenarios, we reveal how even top models achieve only 57% accuracy, not matching human performance under similar constraints (65%) and exposing significant capability gaps. Our analysis shows that VLMs excel with static object detection but struggle with understanding vehicle dynamics and temporal relations. VENUSS offers the first systematic sensitivity analysis of VLMs focused on how input image configurations - resolution, frame count, temporal intervals, spatial layouts, and presentation modes - affect performance on sequential driving scenes. Supplementary material available at https://TUM-AVS.github.io/VENUSS/.
Roberto Brusnicki, Mattia Piccinini, Johannes Betz
Professorship of Autonomous Vehicle Systems, TUM School of Engineering and Design, Technical University of Munich, Munich, Germany
High-performing vision language models still produce incorrect answers, yet their failure modes are often difficult to explain. To make model internals more accessible and enable systematic debugging, we introduce VisualScratchpad, an interactive interface for visual concept analysis during inference. We apply sparse autoencoders to the vision encoder and link the resulting visual concepts to text tokens via text-to-image attention, allowing us to examine which visual concepts are both captured by the vision encoder and utilized by the language model. VisualScratchpad also provides a token-latent heatmap view that suggests a sufficient set of latents for effective concept ablation in causal analysis. Through case studies, we reveal three underexplored failure modes: limited cross-modal alignment, misleading visual concepts, and unused hidden cues. Project page: https://hyesulim.github.io/visual_scratchpad_projectpage/
When a Vision-Language Model (VLM) sees a blue banana and answers "yellow", is the problem of perception or arbitration? We explore the question in ten VLMs with various sizes and reveal an Encoding-Grounding Dissociation: models that fail to report what they see (and thus provide a wrong answer) still encode the visual evidence as strongly as models that provide the correct answer. Using Multimodal Arbitration Crossover (MAC) analysis with layer-by-layer Logit Lens probing, we track the competition between visual and prior signals across every layer of each model. We show that visual attributes can be linearly decodable from early layers (AUC > 0.86). The accuracy remains nearly identical for both successful and failed samples. However, the gap in the final-layer logit - not the strength of encoding - better predicts grounding outcomes with a correlation of ρ= 0.847. After having studied when VLMs base their answers on image clues rather than prior knowledge, we want to understand the causal relationships. We establish causality through full-sequence activation patching. The standard last-token interventions in LLM interpretability do not affect VLMs. In contrast, replacing the full token sequence at layers identified by MAC alters 60 to 84% of outputs. Partial-token decomposition shows that image tokens carry almost all of the causal impact, while text tokens have none. Scaling addresses the remaining architectural differences to achieve perfect retention. Moving from diagnosis to intervention, we show that training-free activation steering - both linear and sparse autoencoder-guided - in early layers can improve visual grounding by up to +3.8% with degrading performance in some setups. Overall, these findings lead to a clear conclusion: VLMs already see well, but the challenge is acting on what they see. Targeted interventions can help to bridge this gap.
Farhad Nooralahzadeh, Omid Rohanian, Yi Zhang +2
Institute of Computer Science, Zurich University of Applied Sciences, Switzerland · University of Oxford, UK.