cs.CVMar 6, 2026

Probing Visual Concepts in Lightweight Vision-Language Models for Automated Driving

Authors: Nikos Theodoridis, Reenu Mohandas, Ganesh Sistu, Anthony Scanlan, Ciarán Eising, Tim Brophy

Organizations: Department of Electronic and Computer Engineering University of Limerick · Valeo Vision Systems

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

The use of Vision-Language Models (VLMs) in automated driving applications is becoming increasingly common, with the aim of leveraging their reasoning and generalisation capabilities to handle long-tail scenarios. However, these models often fail on simple visual questions that are highly relevant to automated driving, and the reasons behind these failures remain poorly understood. In this work, we examine the intermediate activations of VLMs and assess the extent to which specific visual concepts are linearly encoded, with the goal of identifying bottlenecks in the flow of visual information. Specifically, we create counterfactual image sets that differ only in a targeted visual concept and then train linear probes to distinguish between them using the activations of five state-of-the-art (SOTA) VLMs, including two training variants for one model. Our results show that concepts such as the presence of an object or agent in a scene are explicitly and linearly encoded, whereas other spatial visual concepts, such as the orientation of an object or agent, are only implicitly encoded by the spatial structure retained by the vision encoder, or are not linearly encoded at all. In parallel, we observe that in certain cases, even when a concept is linearly encoded in the model's activations, the model still fails to answer correctly. This leads us to identify two failure modes. The first is perceptual failure, where the visual information required to answer a question is not linearly encoded in the model's activations. The second is cognitive failure, where the visual information is present but the model fails to align it correctly with language semantics. Finally, we show that increasing the distance of the object in question quickly degrades the linear separability of the corresponding visual concept...

Figures & tables

Figure 1: Framework for tracking visual concept representations across VLM architectures . Left: The model receives counterfactual input pairs that differ only by a specific visual concept (e.g., the presence of a pedestrian). Middle: Linear probes are trained on intermediate activations to detect if the concept is linearly encoded within the Vision Encoder, Projector, or LLM. Right: Comparing probe accuracy to model output reveals two distinct failure modes: Perceptual Failure , where visual information is not linearly encoded in the activations, and Cognitive Failure , where the information is encoded (high probe accuracy) but the model fails to align it with language semantics, resulting in an incorrect answer.
Figure 2: Counterfactual sets of images representing four basic visual concepts : Presence , Count , Spatial Relatioinship , Orientation
VLMParam (B)Vision EncoderProjectorLanguage ModelFT
Ovis2.5-2B2.57SigLIP2-400MMLP 5 5 5 The Ovis2.5 technical report states that visual features are projected into the LLM input space via a weighted sum over a learned visual embedding table. Mathematically, however, this mechanism is equivalent to a softmax-activated MLP projector with a significantly expanded hidden dimension. (436M)Qwen3-1.7BGeneral
InternVL3.5-2B2.35InternViT-300M-v2.5MLP (12.6M)Qwen3-1.7B 6 6 6 For InternVL3.5, they unbind the embedding and unembedding matrices of Qwen3, resulting actually in a model with 2B parameters instead of 1.7B.General
Qwen3-VL-2B2.13SigLIP2-Large (300M)MLP (100M)Qwen3-1.7BGeneral
VST-3B3.75QwenViT (632M)MLP (36.7M)Qwen2.5-3BSpatial
DriveFusionQA3.75QwenViT (632M)MLP (36.7M)Qwen2.5-3BAD
Table 1: VLMs used in the experiments . FT indicates whether the model is a general-purpose vision-language model (General) or has been additionally fine-tuned for a specific task/domain, such as spatial reasoning (Spatial) or autonomous driving (AD).
CategoryQuestionAnswers
Presence-1Is there a pedestrian ahead?Yes | No
Presence-2Is there a traffic barrel ahead?Yes | No
Count-1How many pedestrians are ahead?Zero | One | Two | Three | Four
Count-2How many traffic barrels are ahead?Zero | One | Two | Three | Four
Spatial-1Which of the truck’s blinkers is on?Left | Right
Spatial-2On which side of the road is the pedestrian walking?Left | Right
Table 2: Questions used for each data category .
Figure 3: Activation extraction methodology . a) We apply average pooling to all patch vectors to obtain a single vector representation of the intermediate activations. b) We split the image at a selected point and apply average pooling to the left and right regions independently. We then concatenate the resulting vectors to form a single representation of the intermediate activations while retaining minimal spatial structure. c) We apply average pooling to the visual token vectors within the LLM and concatenate the result with the activation of the last token.
Figure 4: Linear separability of visual concepts across models and layers based on average-pooled activations . Linear separability for each model, layer, and distance is measured as the average test-set accuracy across ten linear probes trained under the same setting.
Figure 5: Linear separability of visual concepts across models and layers based on region-pooled activations . Linear separability for each model, layer, and distance is measured as the average test-set accuracy across ten linear probes trained under the same setting.
Figure 6: Cosine similarity between the learned weights of the probes for each data category . The probes considered here were trained on average-pooled activations of the last layer of the model (after layer normalisation) for the shortest-distance samples.
Table 3: Activation steering results . Each row shows how applying the directions learned by the linear probes as steering vectors alters the model’s output, changing the original description to the steered description. The “Steering” column specifies the layer and token positions where the steering vector is applied. Additional details along with quantitative results are provided in Appendix C .
ModelPresence-Real Acc. (%)Count-Real Acc. (%)
Ovis2.582.1 (138/168)84.6 (44/52)
InternVL3.581.5 (137/168)78.9 (41/52)
Qwen3-VL89.9 (151/168)61.5 (32/52)
VST-SFT97.6 (164/168)90.4 (47/52)
VST-RL97.0 (163/168)75.0 (39/52)
DriveFusionQA97.6 (164/168)82.7 (43/52)
Table 4: Probes accuracy on nuScenes data .
Figure 7: Chance-corrected accuracy gap . The gap is calculated between the test accuracy of the probes trained on the last-token activations of the last layer of the model (after layer normalisation) and the test accuracy of the models themselves. Grey indicates the absence of Spatial-2 data at 5 meters.
Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

ModelCategoryAccuracy (%)
Gen.Constr. Gen.Vis. Enc.LLM
Ovis2.5Presence-199.096.094.999.5
Presence-291.081.099.1100.0
Count-186.587.072.888.9
Count-286.886.094.698.4
Spatial-1100.050.057.7100.0
Appendix
Table 5: Detailed Evaluation Results . We report the accuracy achieved by each model on the test set (Gen.) when prompted with images in which the object of interest is at a distance of 5 meters (or 10 meters for Spatial-2). Additionally, we report the accuracy obtained under constrained generation (Constr. Gen.). Finally, we report the accuracy of linear probes trained on the models’ activations at the last vision encoder layer (Vis. Enc.) and at the last LLM layer (LLM), based on the activations of the last token in the sequence only.
Figure 8: Lower-bound equivalent of Figure 4 .
Figure 9: Lower-bound equivalent of Figure 5 .
ModelPresence-1Presence-2Count-1Count-2Spatial-1Spatial-2
Ovis2.594.0 (47/50)100.0 (50/50)93.8 (45/48)60.0 (30/50)76.6 (36/47)100.0 (48/48)
InternVL3.5100.0 (50/50)100.0 (48/48)75.0 (36/48)42.6 (20/47)83.9 (26/31)87.8 (43/49)
Qwen3-VL100.0 (50/50)100.0 (49/49)73.9 (34/46)71.4 (35/49)97.0 (32/33)100.0 (49/49)
VST-SFT100.0 (50/50)100.0 (46/46)97.9 (47/48)71.4 (35/49)97.4 (37/38)100.0 (49/49)
VST-RL100.0 (50/50)100.0 (38/38)78.3 (36/46)64.4 (29/45)100.0 (42/42)87.8 (43/49)
DriveFusionQA100.0 (40/40)100.0 (5/5)7.9 (3/38)0.0 (0/5)88.2 (15/17)100.0 (45/45)
Appendix
Table 6: Activation steering success rate (%) by model and data category . Percentages are computed over valid steering cases only; counts in parentheses indicate successful cases over valid cases.
Table 7: Logistic regression on logits results . The word clouds show the tokens that correspond to non-zero weights. Increased size means a larger weight value. Blue and red colours mean positive and negative weight values, respectively.
Table 8: Logistic regression on logits results . The word clouds show the tokens that correspond to non-zero weights. Increased size means a larger weight value. Blue and red colours mean positive and negative weight values, respectively.
More in this topic

Vision-Language Models

Vision-language models connect visual inputs with text for retrieval, question answering, generation, and grounded multimodal reasoning.

See all 734 papers

Explore similar work

CardsList

  1. How Well Do Vision-Language Models Understand Sequential Driving Scenes? A Sensitivity Study

    Apr 8, 2026Roberto Brusnicki, Mattia Piccinini, Johannes BetzVisual Evidence Sensitivity in VLMsTemporal Reasoning in VLMs

    Professorship of Autonomous Vehicle Systems, TUM School of Engineering and Design, Technical University of Munich, Munich, Germany

  2. VisualScratchpad: Inference-time Visual Concepts Analysis in Vision Language Models

    Mar 7, 2026Hyesu Lim, Jinho Choi, Taekyung Kim +3VLM InterpretabilityVision-Language Alignment

    KAIST AI · NAVER AI Lab

  3. Arbitration Failure, Not Perceptual Blindness: How Vision-Language Models Resolve Visual-Linguistic Conflicts

    Apr 10, 2026Farhad Nooralahzadeh, Omid Rohanian, Yi Zhang +2Vision-Language GroundingVisual Evidence Sensitivity in VLMs

    Institute of Computer Science, Zurich University of Applied Sciences, Switzerland · University of Oxford, UK.