Transparency of neural networks' internal reasoning is at the heart of interpretability research, adding to trust, safety, and understanding of these models. The field of mechanistic interpretability has recently focused on studying task-specific computational graphs, defined by connections (edges) between model components. Such edge-based circuits have been defined in the context of large language models, yet vision-based approaches so far only consider neuron-based circuits. These tell which information is encoded, but not how it is routed through the complex wiring of a neural network. In this work, we investigate whether useful mechanistic circuits can be identified through computational graphs in vision transformers. We propose an effective method for Automatic Visual Circuit Discovery (Vi-CD) that recovers class-specific circuits for classification, identifies circuits underlying typographic attacks in CLIP, and discovers circuits that lend themselves for steering to correct harmful model behavior. Overall, we find that insightful and actionable edge-based circuits can be recovered from vision transformers, adding transparency to the internal computations of these models.
Figures & tables
Figure 2: Vi-CD: Circuit discovery in vision computation graphs. Left: Construction of corrupted inputs by removing class-specific information via a segmentation and inpainting pipeline. Middle: Graph representation of the model, where nodes correspond to residual-stream components (attention heads and MLPs) and edges represent information flow between them. Right: Activation patching, where contributions along edges in the candidate circuits are replaced with clean activations while non-circuit edges retain corrupted activations, allowing identification of faithful circuits.
Figure 3: Transformer circuitry in a 2-layer toy transformer. Left: Red edges are simplified in Vi-CD for scalability: multiple attention-head receiver nodes are collapsed into a single attention-input node. Right: Green edges correspond to input the sender node, Yellow edges correspond to MLPs sender nodes, and Purple edges correspond to attention heads sender nodes.
Figure 4: Vi-CD finds 10x sparser circuits. We report accuracy of the circuit on the target class ( ↑ higher is better) as faithfulness and report different sparsity levels for circuits as edges remaining ( ↓ lower is better) for different circuit extraction methods indicated by colors. We compare linear probe classification performance of (ViT-B)OpenCLIP and of a supervised ViT-B on Imagenet data.
Figure 5: Vi-CD discovers circuits reflecting semantic similarity of classes. Pairwise Jaccard similarity ( ∣A∩B∣/∣A∪B∣ ) of example class circuits of class Pembroke Corgi (left) and Revolver (right) against other classes in OpenCLIP.
Big Text
Small Text
Bezel
Setting
Metric
Base
Steered
Base
Steered
Base
Steered
Clean ImageNet
Top-1 Acc. (%)
57.0
55.8
57.1
55.7
57.1
49.9
Top-5 Acc. (%)
81.9
80.9
81.9
80.9
81.9
75.6
Corrupted ImageNet
Top-1 Acc. (%)
34.7
50.0
34.7
49.1
34.4
45.0
Top-5 Acc. (%)
69.9
75.9
70.1
75.3
69.8
71.2
ASR Top-1 (%)
39.1
2.8
39.4
1.6
39.5
3.1
Table 1: Typographic attacks on images. Average performance of circuit steering against typographic attacks on ImageNet (unsteered base vs. steered with Attack Success Rate ↓ 90%). Circuit-steering defenses are succesful, greatly reducing attack success with only marginal loss in performance on clean images.
Figure 6: Circuit-based steering prevents typographic attacks without harming performance. Left: Attack success rate ( ↓ lower is better). Right: performance retention on accuracy ( ↑ higher is better) . We show metrics against steering strength and the maximal receiver layer in the circuit. Purple region indicates regions where steering is successfully applied. See results for random circuits in Suppl. 0.G .
Base
Assault Rifle
Revolver
Rifle
Recall
R@1 (%)
40.42
42.36 ± 0.42
42.01 ± 0.36
43.47 ± 0.48
R@5 (%)
69.48
68.59 ± 0.46
68.39 ± 0.24
70.22 ± 0.61
R@10 (%)
78.32
78.04 ± 0.38
77.65 ± 0.21
79.41 ± 0.43
Aggregate
R mean (%)
62.74
63.00 ± 0.37
62.68 ± 0.22
64.37 ± 0.47
Safety
RSMS (%)
11.68
5.15 ± 0.24
5.69 ± 0.41
4.86 ± 0.21
Table 2: RoCOCO retrieval under circuit-based steering ( α=0.4 ). R@ k : fraction of queries where the correct item ranks in the top k ( ↑ higher is better). R mean : mean of R@1/5/10 ( ↑ ). RSMS [ 26 ] : R@ k on adversarially manipulated items, measuring retrieval of danger-class images for benign queries ( ↓ lower is better). Mean ± std over three seeds.
Appendix figures & tables18 assets
Supplementary material from the paper’s appendix.
Appendix
Table 3: Example data used for class circuits. Rows show clean and corrupted inputs, while columns correspond to different ImageNet classes. Corrupted inputs remove object-level signal.
Class
OpenCLIP δ
ViT-B δ
Class
OpenCLIP δ
ViT-B δ
tench
1.00e-3
5.11e-2
zebra
4.57e-4
5.51e-2
English springer
7.29e-4
5.72e-2
tabby cat
4.57e-4
2.05e-2
cassette player
7.29e-4
2.86e-2
golden retriever
4.57e-4
3.48e-2
chain saw
6.96e-4
2.66e-2
Brabancon griffon
4.57e-4
5.11e-2
church
6.61e-4
4.50e-2
Pembroke corgi
4.57e-4
6.13e-2
French horn
7.96e-4
3.48e-2
Cardigan corgi
4.57e-4
3.68e-2
Appendix
Table 4: Per-class pruning thresholds δ used for circuit extraction in OpenCLIP and ViT-B.
Figure 7: Types of typographic corruptions. Left to right: Bezel, Multiple Small Texts, and Big Text on Image typographic corruptions.
Index
Corruption Text
Classifier Embedding
Bezel
Big Text
Multiple Small Text
Top-1
Top-5
Top-1
Top-5
Top-1
Top-5
1
whistle
whistle
66.0%
94.4%
42.6%
74.3%
64.1%
85.9%
2
espresso
espresso
79.7%
98.2%
47.1%
75.2%
76.6%
90.3%
3
wing
wing
2.5%
10.3%
1.7%
7.4%
1.6%
8.3%
4
overskirt
overskirt
67.5%
94.7%
34.6%
67.9%
74.7%
91.0%
5
convertible
convertible
10.9%
41.4%
9.2%
32.0%
19.0%
43.7%
Appendix
Table 5: Different typographic attack success rate typographic corruptions. Values report Top-1 and Top-5 success rates, meaning the corruption attack target was listed in the Top-1 or Top-5.
Mean
Medoid
Pre-Normed
Average of Δipre
Most representative Δipre
Post-Normed
Average of Δipost
Most representative Δipost
Appendix
Table 6: Steering Vector Aggregation: Pre- vs Post-Normalization and Mean vs Medoid.
Figure 8: Faceted results for the typographic object “Orange” with Big Text overlay. Each row shows one combination of normalization regime (Pre-/Post-Normed) and aggregation method (Medoid/Mean). The x -axis denotes steering strength α ; the y -axis reports either ImageNet accuracy or attack success rate (ASR). Accuracy: Blue — clean input, circuit edges; Red — corrupted input, circuit edges; Cyan (dashed) — clean input, same-size non-circuit edges; Orange (dashed) — corrupted input, same-size non-circuit edges. ASR: Green — circuit edges; Purple (dashed) — same-size non-circuit edges. Results are averaged across 20 circuits, each evaluated on a randomly sampled batch; shaded regions denote the interquartile range (Q25–Q75).
Big Text on Image
Multiple Small Text
Bezel
Attack Word
Setting
Metric
Base
ASR ↓ 90%
Base
ASR ↓ 90%
Base
ASR ↓ 90%
ambulance
Clean IN
ACC Top-1 (%)
56.9 ±5.7
58.0 ±5.9
57.9 ±6.2
54.9 ±6.0
58.3 ±5.8
49.8 ±5.6
ACC Top-5 (%)
81.1 ±5.0
82.3 ±4.9
81.9 ±4.7
80.8 ±4.6
82.8 ±4.6
75.2 ±5.4
Corr. IN
ACC Top-1 (%)
33.6 ±6.0
48.7 ±6.0
33.3 ±6.1
46.2 ±5.8
32.3 ±6.0
44.9 ±6.6
ACC Top-5 (%)
69.3 ±5.7
75.4 ±4.8
69.5 ±5.7
72.9 ±5.3
68.7 ±5.8
71.0 ±6.2
ASR Top-1 (%)
41.4 ±6.2
2.9 ±2.1
40.6 ±6.4
0.7 ±1.2
41.0 ±6.3
2.4 ±1.9
Appendix
Table 7: Defended ImageNet (IN) performance under text corruptions (Corr.). Results are averaged over 20 circuits, over random batches of size 128. Bold indicates improvement over the unsteered base model. Steered results are reported at ASR ↓ 90%, i.e. the steering strength α at which the attack success rate is reduced by 90% relative to the base model.
Table 8: Faceted panel of Text Overlays under a fixed steering type (Pre-Normed Mean). Column shows multiple small text overlays corruption regime.
Method
R@1
R@5
R@10
R mean
RSMS
Base
40.42
69.48
78.31
62.74
11.68
α=0.3
Assault Rifle Circuit
44.19 ± 0.46
70.42 ± 0.32
79.71 ± 0.26
64.77 ± 0.28
5.66 ± 0.27
Assault Rifle Random
42.28 ± 1.57
69.85 ± 0.96
78.57 ± 0.97
63.57 ± 1.13
9.15 ± 1.06
Revolver Circuit
44.04 ± 0.46
70.72 ± 0.45
79.98 ± 0.27
64.91 ± 0.36
6.15 ± 0.29
Revolver Random
41.77 ± 1.18
69.81 ± 0.79
78.68 ± 0.72
63.42 ± 0.87
9.76 ± 0.96
Appendix
Table 9: RoCOCO retrieval under steering for α∈{0.3,0.4} . R@ k : fraction of queries where the correct item ranks in the top k ( ↑ ). R mean : mean of R@1/5/10 ( ↑ ). RSMS [ 26 ] : retrieval of danger-class images for benign queries ( ↓ ). Mean ± std over three seeds. Bold indicates the better result within each Circuit/Random pair.
Figure 9: RoCOCO steering as a function of steering strength α . Top: R@1 retrieval performance ( ↑ higher is better). Bottom: RSMS ( ↓ lower is better). Solid lines show steering applied along discovered circuit edges; dashed lines show steering along random edges of matched size. Shaded regions denote mean ± std over three seeds.
Figure 10: Overlap of class circuits in CLIP. Each cell shows the Jaccard similarity between the sets of edges present in all runs (frequency =1.0 ) for a pair of classes. Classes are ordered by hierarchical clustering. Dog breeds (bottom-right block) share substantially more core edges with each other than with semantically unrelated classes.
Figure 11: Circuit size and stability per class. Circuit sizes in CLIP and stability across Imagenette [ 17 ] classes. We report the average circuit size and the mean pairwise Jaccard similarity between circuits mined from repeated runs.
Figure 12: Stability of different network components. Stability of edges per edge type, broken down into stable ( > 90% inclusion frequency), borderline (50–90%), and unstable ( < 50%) categories. The y -axis reports the fraction of edges of each type falling into each stability category, averaged over all class circuits. Edges originating from attention heads are disproportionately unstable in both models.
Figure 13: Circuit stability with #samples. Mean within-class pairwise Jaccard similarity between circuits mined from repeated runs, as a function of the number of datapoints used for circuit mining (log scale). Circuit stability increases consistently with dataset size.
Figure 14: Zero-shot classification performance using unions of class-specific circuits.
Figure 15: Circuit class specificity. For each class pair (A, B), edges in each binary circuit run are classified as: appearing in the union of all per-class circuits across runs for class A but not B ( A only ); appearing in the union for class B but not A ( B only ); appearing in the union for both classes ( both A&B ); or appearing in neither class-specific union ( only in binary ). The y -axis reports the mean edge count per binary circuit run, averaged over 8 seeds. The dominant contribution of shared edges across all class pairs indicates strong overlap between the individual class circuits and the explicitly mined binary circuit, suggesting that binary discrimination reuses the same computational pathways as the underlying class-specific circuits.
Figure 16: Ablations of selection criterion. For edge typographic circuits we report accuracy and achieved sparsity for each target class for different selection criteria and backbones. Green dotted line marks 70% sparsity, red dotted line marks 70% accuracy.
Figure 17: Ablations of typographic circuits up to a layer. The x -axis denotes steering strength α and the y -axis the maximal receiver layer included in the circuit (we are ignoring later layers of the circuit). Left column reports performance retention ( ↑ higher is better); right column reports attack success rate (ASR, ↓ lower is better). Rows correspond to: circuit edges (top), random edges of matched size (middle), and random non-circuit edges of matched size (bottom). Circuit-edge steering uniquely achieves simultaneous ASR suppression and performance retention on corrupted samples across steering strengths and receiver layers.
Despite high accuracy, Vision Transformer (ViT) predictions can be driven by spurious cues, raising the need to understand their inner workings before safe deployment. Sparse autoencoders (SAEs) provide a promising lens for decomposing model representations into human-interpretable concepts, yet adapting SAE-based interpretation to ViTs remains challenging due to limited control over concept coverage and subjective, non-scalable feature interpretation. To fill the gaps, motivated by neuroscience-inspired principles, we propose ViSAE, a mechanistic interpretability toolbox for understanding ViT inner workings through concept circuits. ViSAE consists of three components: (1) A probing suite with 64K images and a 16K visually grounded concept vocabulary, improving concept coverage efficiency by 20x over ImageNet and interpretation accuracy by 28.7% over existing concept sets. (2) Top-down concept reading and Bottom-up circuit tracing algorithms that automatically recover ViT inner workings via concept circuits. (3) Applications for auditing and steering ViT behavior. Through concept editing, ViSAE improves the worst-group accuracy on WaterBirds by 48.2%, outperforming existing methods by 23.8%. Our data and code: https://github.com/deep-real/ViSAE.
Tang Li, Yanlin Chen, Mengmeng Ma +1
Department of Computer & Information Science, University of Delaware, Newark, DE, USA · Department of Computer Science, University of Virginia, Charlottesville, VA, USA
Vision transformers (ViTs) have achieved remarkable generalization across visual domains, yet little is known about how they internally represent the structure of the world. To address this gap, we use Cross-Layer Transcoders (CLTs) to read concept circuits from ViTs: directed graphs whose nodes correspond to sparse, interpretable concepts and edges capture concept interactions across layers. Our method yields two complementary views of model behavior. The global concept circuit is input-invariant and can be recovered directly from learned cross-layer weights, exposing the reusable "world knowledge" encoded in the model. The instance concept circuit is input-dependent and identifies the concepts and pathways actually used for a specific prediction, enabling faithful example-level explanations. We demonstrate the utility of concept circuits in three ways: (1) Automatic spurious correlation discovery: leveraging the statistics of our global concept circuits to identify shortcut dependencies within the model. (2) Spurious correlation removal: intervening on the instance concept circuit to steer the model towards correct predictions. Empirical results show that our method outperforms existing counterparts by 11.0% on the Waterbird dataset. (3) Model comparison: contrasting the global concept circuits of different foundation models (e.g., CLIP vs. DINO) to reveal how supervision paradigms shape representational structure. Our code is available at https://github.com/deep-real/VisionCLT
Yanlin Chen, Tang Li, Xi Peng
Department of Computer & Information Sciences, University of Delaware, USA · Department of Computer Science, University of Virginia, USA
Understanding how neural networks arrive at their predictions is essential for debugging, auditing, and deployment. Mechanistic interpretability pursues this goal by identifying circuits--minimal subnetworks responsible for specific behaviors. However, existing circuit discovery methods are brittle: circuits depend strongly on the chosen concept dataset and often fail to transfer out-of-distribution, raising doubts whether they capture the concept or merely dataset-specific artifacts. We introduce Certified Circuits, which provide provable stability guarantees for circuit discovery. Our framework wraps any black-box discovery algorithm with randomized data subsampling to certify that inclusion decisions over circuit components--neurons or edges of the model graph, depending on the base algorithm--are invariant to bounded edit-distance perturbations of the concept dataset. Unstable components are abstained from, yielding circuits that are more compact and more accurate. We validate across three architectures (ResNet, ViT, GPT-2) on vision (ImageNet and four OOD datasets) and language (IOI, IOI-Hard, Greater-Than) tasks. Certified circuits achieve up to 56% higher accuracy and up to 80% fewer components, and remain reliable where baselines degrade. Certified Circuits puts circuit discovery on formal ground by producing mechanistic explanations that are provably stable and better aligned with the target concept. Code: https://github.com/AlaaAnani/certified-circuits.
Alaa Anani, Tobias Lorenz, Bernt Schiele +2
Max Planck Institute for Informatics, Saarland Informatics Campus, Saarbrücken, Germany · CISPA Helmholtz Center for Information Security, Saarbrücken, Germany