Tracking Equivalent Mechanistic Interpretations Across Neural Networks

Alan Sun; Mariya Toneva

cs.LGMar 31, 2026

Tracking Equivalent Mechanistic Interpretations Across Neural Networks

Authors: Alan Sun, Mariya Toneva

Organizations: Carnegie Mellon University · Max Planck Institute for Software Systems

Abstract

Mechanistic interpretability (MI) is an emerging framework for interpreting neural networks. Given a task and model, MI aims to discover a succinct algorithmic process, an interpretation, that explains the model's decision process on that task. However, MI is difficult to scale and generalize. This stems in part from two key challenges: there is no precise notion of a valid interpretation; and, generating interpretations is often an ad hoc process. In this paper, we address these challenges by defining and studying the problem of interpretive equivalence: determining whether two different models share a common interpretation, without requiring an explicit description of what that interpretation is. At the core of our approach, we propose and formalize the principle that two interpretations of a model are equivalent if all of their possible implementations are also equivalent. We develop an algorithm to estimate interpretive equivalence and case study its use on Transformer-based models. To analyze our algorithm, we introduce necessary and sufficient conditions for interpretive equivalence based on models' representation similarity. We provide guarantees that simultaneously relate a model's algorithmic interpretations, circuits, and representations. Our framework lays a foundation for the development of more rigorous evaluation methods of MI and automated, generalizable interpretation discovery methods.

Figures & tables

Figure 1: A high-level overview of our algorithmic approach to interpretive equivalence. Consider models

h_{1},h_{2}

that correspond to possibly unknown interpretations

A_{1},A_{2}

(Left) . To determine whether models

h_{1}

and

h_{2}

are interpretively equivalent, we propose a two-step procedure. First, we sample another model

h^{\star}

that also has interpretation

A_{1}

(Center) . Second, we compare the representation similarity (

d_{\mathrm{repr}}

) between

h_{1},h^{\star}

and

h^{\star},h_{2}

(Right) . If

A_{1},A_{2}

are equivalent, then averaged over all implementations

h^{\star}

, we should not be able to differentiate

d_{\mathrm{repr}}(h_{1},h^{\star})

and

d_{\mathrm{repr}}(h^{\star},h_{2})

Figure 2: (Left) Average congruity between models associated with different interpretations.

\blacksquare

indicates models have different interpretations; whereas

\blacksquare

indicates that models have statistically indistinguishable interpretations. (Center) Congruity between GPT2 and Pythia family of models on the IOI task. groups models based on their actual interpretive differences observed by Tigges et al. (2024) ; Merullo et al. (2024) . (Right) Congruity between GPT2 on next-token prediction (for different token types: all tokens, articles, prepositions, punctuation, parentheses, and terminal punctuation) vs. GPT2 on in-context parts-of-speech identification.

Figure 3:

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: Pipeline for generating implementations for our interpretations (i.e. RASP language models). We first construct RASP programs, then using the procedure introduced in Lindner et al. (2023) , we compile these programs into Transformer models. These Transformers exclusively have hard attention. Moreover, their architecture is minimal (containing only the necessary components to fully implement the given RASP program). We then apply the procedure introduced in Gupta et al. (2024) to translate these Tracr-Compiled Transformers into “real” Transformers: ones whose weight distribution matches those trained with stochastic gradient descent; these translated models also contain more

Figure 5: Activation patching results for each attention head. The coloring indicates the % of performance this individual attention head contributes to POS performance.

Term	Intuitive Explanation	Formal Definition
Causal/Computation Graph	A graph that describes a model’s computational dependencies	Definition 4.1
Variables	Nodes in a model’s computation graph that store latent computation results	Definition 4.1
Circuit	A subset of the model’s computational graph that on its own can fully recreate the model’s input-output behavior	Definition D.1
Mechanistic Explanation	A symbolic algorithm, we can understand, that explains the model’s computation	N/A
Causal Abstraction	Mapping from low-level variables to higher-level ones that preserve low-level causal relationships	Definition D.3
Interpretation	A causal graph much smaller than the model’s computation graph that explains the model’s computation graph	Definition D.4

Appendix

Table 1: Glossary of terms.

Symbol	Meaning
$\Sigma$	An alphabet
$\Sigma^{\star}$	Set of all finite strings constructed from $\Sigma$
$S$	A subset of $\Sigma^{\star}$
$A$	An interpretation
$K$	A circuit
$R$	A representation

Appendix

Table 2: Notations used throughout our formal treatment. This is partially adapted from both Geiger et al. (2025) and Ravfogel et al. (2025) .

Explore similar work

When Are Two Networks the Same? Tensor Similarity for Mechanistic Interpretability
May 14, 2026ML Nissen Gonzalez, Melwina Albuquerque, Laurence Wroe +3Mechanistic Interpretability Weight-Space Analysis
MARS & University Heidelberg · MARS · MARS & Stanford University +2
Certified Circuits: Stability Guarantees for Mechanistic Circuits
Feb 26, 2026Alaa Anani, Tobias Lorenz, Bernt Schiele +2Circuit Discovery Mechanistic Interpretability
Max Planck Institute for Informatics, Saarland Informatics Campus, Saarbrücken, Germany · CISPA Helmholtz Center for Information Security, Saarbrücken, Germany
Mechanistic Interpretability for Neural Networks: Circuits, Sparse Features and Symbolic Reasoning
Jul 8, 2026Pranav Sawant, Jakub KrejčíTransformer Interpretability Mechanistic Interpretability
The University of Texas at Dallas 800 W Campbell Rd, Richardson, TX 75080, United States · VSB—Technical University of Ostrava 708 00, Ostrava, Czech Republic

May 14, 2026J/K move · Enter open · S save

When Are Two Networks the Same? Tensor Similarity for Mechanistic Interpretability

ML Nissen Gonzalez, Melwina Albuquerque, Laurence Wroe, Jacob Meyer Cohen+2

MARS & University Heidelberg · MARS · MARS & Stanford University+2

Mechanistic Interpretability Weight-Space Analysis

Open paper AskPDF arXiv

Mechanistic interpretability aims to break models into meaningful parts; verifying that two such parts implement the same computation is a prerequisite. Existing similarity measures evaluate either empirical behaviour, leaving them blind to out-of-distribution mechanisms, or basis-dependent parameters, meaning they disregard weight-space symmetries. To address these issues for the class of tensor-based models, we introduce a weight-based metric, tensor similarity, that is invariant to such symmetries. This metric captures global functional equivalence and accounts for cross-layer mechanisms using an efficient recursive algorithm. Empirically, tensor similarity tracks functional training dynamics, such as grokking and backdoor insertion, with higher fidelity than existing metrics. This reduces measuring similarity and verifying faithfulness into a solved algebraic problem rather than one of empirical approximation.

Abstract

Explore similar work

When Are Two Networks the Same? Tensor Similarity for Mechanistic Interpretability

Certified Circuits: Stability Guarantees for Mechanistic Circuits

Mechanistic Interpretability for Neural Networks: Circuits, Sparse Features and Symbolic Reasoning