Vision-language models (VLMs) are powerful but remain opaque black boxes. We introduce the first framework for transparent circuit tracing in VLMs to systematically analyze multimodal reasoning. By utilizing transcoders, attribution graphs, and attention-based methods, we uncover how VLMs hierarchically integrate visual and semantic concepts. We reveal that distinct visual feature circuits can handle mathematical reasoning and support cross-modal associations. Validated through feature steering and circuit patching, our framework proves these circuits are causal and controllable, laying the groundwork for more explainable and reliable VLMs.
Figures & tables
Figure 1: Given an image and a prompt, how can we extract a circuit, as an internal computation graph, of open-source vision language models such as Gemma3-4B [ 34 ] from Google. We introduce the first framework for successful circuit tracing in VLMs, enabling analysis of the internal circuits and association of concepts in underlying multimodal reasoning.
Figure 2: A summary of our method, we first train transcoders for Gemma-3-4b-it on our curated dataset, yielding a replacement model with monosemantic features. Then, we utilize feature activation analysis to obtain information that would provide interpretable information about the features. Then, we generate an attribution graph of a given prompt, and utilize human experts to derive the final circuit.
Figure 3: Transcoder vs SAE; SAEs learn to reconstruct model activations, whereas transcoders imitate MLP sublayers’ input-output behavior.
Name
Content
Size
SmoLIM2-135M-10B [ 2 ]
2048 text tokens
144,000
ImageNet [ 31 ]
1 image; Caption
144,000
Cauldron [ 17 ]
1 image; QA Text
72,000
Table 1: Datasets used for Transcoder Training.
Figure 4: Top: Percentage of dead latents (Dead PCT) across layers for different values of Nlatents . Bottom: Fraction of variance unexplained (FVU) when training transcoders on a text-only dataset compared to our multimodal split.
Figure 5: Feature activations on images with a mix of feature annotations resulting from full images (right) and feature analysis resulting from noisy attention maps (left).
Figure 6: We conduct circuit tracing on the same prompt This is the planet with images of mars and earth. We then conduct activation patching by suppressing the mid-layer visual features of mars in the mars circuit, and set the activation of earth visual features discovered in the earth circuit in the mars circuit
Figure 7: Tracing a purely visual attribution path from an image of Mars reveals internal visual associations (e.g., “space shuttle”) even without supporting cues.
Figure 10: Circuit tracing analysis of prompt ”The number of fingers in the image is ” with the result being 5, despite image containing six fingers.
Figure 11: Circuit tracing analysis of a seaotter, with interesting feature attributions.
Figure 12: Analysis of the mars circuit, revealing a transition from purely visual features to a semantically grounded feature that controls the final output.
Figure 13: Circuit tracing analysis of a simple addition task.