Tracking Equivalent Mechanistic Interpretations Across Neural Networks

Alan Sun; Mariya Toneva

cs.LGMar 31, 2026

Tracking Equivalent Mechanistic Interpretations Across Neural Networks

Authors: Alan Sun, Mariya Toneva

Organizations: Carnegie Mellon University · Max Planck Institute for Software Systems

Abstract

Mechanistic interpretability (MI) is an emerging framework for interpreting neural networks. Given a task and model, MI aims to discover a succinct algorithmic process, an interpretation, that explains the model's decision process on that task. However, MI is difficult to scale and generalize. This stems in part from two key challenges: there is no precise notion of a valid interpretation; and, generating interpretations is often an ad hoc process. In this paper, we address these challenges by defining and studying the problem of interpretive equivalence: determining whether two different models share a common interpretation, without requiring an explicit description of what that interpretation is. At the core of our approach, we propose and formalize the principle that two interpretations of a model are equivalent if all of their possible implementations are also equivalent. We develop an algorithm to estimate interpretive equivalence and case study its use on Transformer-based models. To analyze our algorithm, we introduce necessary and sufficient conditions for interpretive equivalence based on models' representation similarity. We provide guarantees that simultaneously relate a model's algorithmic interpretations, circuits, and representations. Our framework lays a foundation for the development of more rigorous evaluation methods of MI and automated, generalizable interpretation discovery methods.

Figures & tables

Figure 1: A high-level overview of our algorithmic approach to interpretive equivalence. Consider models

h_{1},h_{2}

that correspond to possibly unknown interpretations

A_{1},A_{2}

(Left) . To determine whether models

h_{1}

and

h_{2}

are interpretively equivalent, we propose a two-step procedure. First, we sample another model

h^{\star}

that also has interpretation

A_{1}

(Center) . Second, we compare the representation similarity (

d_{\mathrm{repr}}

) between

h_{1},h^{\star}

and

h^{\star},h_{2}

(Right) . If

A_{1},A_{2}

are equivalent, then averaged over all implementations

h^{\star}

, we should not be able to differentiate

d_{\mathrm{repr}}(h_{1},h^{\star})

and

d_{\mathrm{repr}}(h^{\star},h_{2})

Figure 2: (Left) Average congruity between models associated with different interpretations.

\blacksquare

indicates models have different interpretations; whereas

\blacksquare

indicates that models have statistically indistinguishable interpretations. (Center) Congruity between GPT2 and Pythia family of models on the IOI task. groups models based on their actual interpretive differences observed by Tigges et al. (2024) ; Merullo et al. (2024) . (Right) Congruity between GPT2 on next-token prediction (for different token types: all tokens, articles, prepositions, punctuation, parentheses, and terminal punctuation) vs. GPT2 on in-context parts-of-speech identification.

Figure 3:

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: Pipeline for generating implementations for our interpretations (i.e. RASP language models). We first construct RASP programs, then using the procedure introduced in Lindner et al. (2023) , we compile these programs into Transformer models. These Transformers exclusively have hard attention. Moreover, their architecture is minimal (containing only the necessary components to fully implement the given RASP program). We then apply the procedure introduced in Gupta et al. (2024) to translate these Tracr-Compiled Transformers into “real” Transformers: ones whose weight distribution matches those trained with stochastic gradient descent; these translated models also contain more

Figure 5: Activation patching results for each attention head. The coloring indicates the % of performance this individual attention head contributes to POS performance.

Term	Intuitive Explanation	Formal Definition
Causal/Computation Graph	A graph that describes a model’s computational dependencies	Definition 4.1
Variables	Nodes in a model’s computation graph that store latent computation results	Definition 4.1
Circuit	A subset of the model’s computational graph that on its own can fully recreate the model’s input-output behavior	Definition D.1
Mechanistic Explanation	A symbolic algorithm, we can understand, that explains the model’s computation	N/A
Causal Abstraction	Mapping from low-level variables to higher-level ones that preserve low-level causal relationships	Definition D.3
Interpretation	A causal graph much smaller than the model’s computation graph that explains the model’s computation graph	Definition D.4

Appendix

Table 1: Glossary of terms.

Symbol	Meaning
$\Sigma$	An alphabet
$\Sigma^{\star}$	Set of all finite strings constructed from $\Sigma$
$S$	A subset of $\Sigma^{\star}$
$A$	An interpretation
$K$	A circuit
$R$	A representation

Appendix

Table 2: Notations used throughout our formal treatment. This is partially adapted from both Geiger et al. (2025) and Ravfogel et al. (2025) .

Explore similar work

May 14, 2026

When Are Two Networks the Same? Tensor Similarity for Mechanistic Interpretability

Mechanistic interpretability aims to break models into meaningful parts; verifying that two such parts implement the same computation is a prerequisite. Existing similarity measures evaluate either empirical behaviour, leaving them blind to out-of-distribution mechanisms, or basis-dependent parameters, meaning they disregard weight-space symmetries. To address these issues for the class of tensor-based models, we introduce a weight-based metric, tensor similarity, that is invariant to such symmetries. This metric captures global functional equivalence and accounts for cross-layer mechanisms using an efficient recursive algorithm. Empirically, tensor similarity tracks functional training dynamics, such as grokking and backdoor insertion, with higher fidelity than existing metrics. This reduces measuring similarity and verifying faithfulness into a solved algebraic problem rather than one of empirical approximation.

ML Nissen Gonzalez, Melwina Albuquerque, Laurence Wroe +3

MARS & University Heidelberg · MARS · MARS & Stanford University +2

Mechanistic Interpretability Weight-Space Analysis

Ask PDF

Feb 26, 2026

Certified Circuits: Stability Guarantees for Mechanistic Circuits

Understanding how neural networks arrive at their predictions is essential for debugging, auditing, and deployment. Mechanistic interpretability pursues this goal by identifying circuits--minimal subnetworks responsible for specific behaviors. However, existing circuit discovery methods are brittle: circuits depend strongly on the chosen concept dataset and often fail to transfer out-of-distribution, raising doubts whether they capture the concept or merely dataset-specific artifacts. We introduce Certified Circuits, which provide provable stability guarantees for circuit discovery. Our framework wraps any black-box discovery algorithm with randomized data subsampling to certify that inclusion decisions over circuit components--neurons or edges of the model graph, depending on the base algorithm--are invariant to bounded edit-distance perturbations of the concept dataset. Unstable components are abstained from, yielding circuits that are more compact and more accurate. We validate across three architectures (ResNet, ViT, GPT-2) on vision (ImageNet and four OOD datasets) and language (IOI, IOI-Hard, Greater-Than) tasks. Certified circuits achieve up to 56% higher accuracy and up to 80% fewer components, and remain reliable where baselines degrade. Certified Circuits puts circuit discovery on formal ground by producing mechanistic explanations that are provably stable and better aligned with the target concept. Code: https://github.com/AlaaAnani/certified-circuits.

Alaa Anani, Tobias Lorenz, Bernt Schiele +2

Max Planck Institute for Informatics, Saarland Informatics Campus, Saarbrücken, Germany · CISPA Helmholtz Center for Information Security, Saarbrücken, Germany

Circuit Discovery Mechanistic Interpretability Circuits in Language Models

Ask PDF

Jul 8, 2026

Mechanistic Interpretability for Neural Networks: Circuits, Sparse Features and Symbolic Reasoning

This article offers a comprehensive overview of mechanistic interpretability, an emerging field that seeks to reverse-engineer the internal algorithms of modern neural networks. While traditional explainable AI methods often stop at surface-level input-output correlations, this approach directly addresses the opaque "black box" nature of machine learning models, which is essential for ensuring safety and auditability in high-stakes deployments. The paper provides a detailed examination of Transformer circuit analysis, exploring how internal components like the residual stream, attention mechanisms, and induction heads drive complex tasks and in-context learning. It subsequently tackles the core challenge of superposition and polysemanticity, demonstrating how tools like Sparse Autoencoders (SAEs) and transcoders can decompose tangled network activations into distinct, human-interpretable features. Furthermore, the paper explores methods for actively controlling and modifying model behavior through steering vectors and causal interventions. Finally, it connects these mechanistic insights with neurosymbolic AI frameworks designed to translate neural representations into explicit, executable logical rules.

Pranav Sawant, Jakub Krejčí

The University of Texas at Dallas 800 W Campbell Rd, Richardson, TX 75080, United States · VSB—Technical University of Ostrava 708 00, Ostrava, Czech Republic

Transformer Interpretability Mechanistic Interpretability Circuits in Language Models

Ask PDF