ADAG: Automatically Describing Attribution Graphs

Aryaman Arora; Zhengxuan Wu; Jacob Steinhardt; Sarah Schwettmann

cs.CLApr 8, 2026

ADAG: Automatically Describing Attribution Graphs

Authors: Aryaman Arora, Zhengxuan Wu, Jacob Steinhardt, Sarah Schwettmann

Organizations: Stanford University · Transluce

Abstract

In language model interpretability research, circuit tracing aims to identify which internal features causally contributed to a particular output and how they affected each other, with the goal of explaining the computations underlying some behaviour. However, all prior circuit tracing work has relied on ad-hoc human interpretation of the role that each feature in the circuit plays, via manual inspection of data artifacts such as the dataset examples the component activates on. We introduce ADAG, an end-to-end pipeline for describing these attribution graphs which is fully automated. To achieve this, we introduce attribution profiles which quantify the functional role of a feature via its input and output gradient effects. We then introduce a novel clustering algorithm for grouping features, and an LLM explainer--simulator setup which generates and scores natural-language explanations of the functional role of these feature groups. We run our system on known human-analysed circuit-tracing tasks and recover interpretable circuits, and further show ADAG can find steerable clusters which are responsible for a harmful advice jailbreak in Llama 3.1 8B Instruct.

Figures & tables

Figure 1: An overview of ADAG, our end-to-end circuit interpretation pipeline.

Panel of Figure 2 — Figure 2: Results of MLP non-locality experiments (excluding BOS from all analyses).

Figure 3: Results of description generation experiments for gold supernodes for the texas circuit in the capitals dataset.

Figure 4: Final circuit graph for texas example in the capitals dataset from Llama 3.1 8B Instruct. We show input attribution, output contribution, and neuron descriptions for C44 ‘Dallas Texas’ to the left. Red indicates negative attribution score, and blue the opposite.

Figure 5: Cluster attribution for top 3-base-ASR influencing clusters per our results, compared with reported ASR in Chowdhury et al. (2025) .

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 6 — Figure 6: Benchmarking results for circuit tracing + attribution profile computation for Llama 3.1 8B Instruct, on capitals .

Panel of Figure 7 — Figure 7: Additional locality results for MLP neurons in Llama 3.1 8B Instruct.

Figure 8: Attribution description score for quantile and topk approaches, when sweeping

k

for highlight threshold.

Figure 9: Final circuit graph for texas example in the capitals dataset from Llama 3.1 8B Instruct.

ID	Summary	Input Attribution	Output Contribution
C0	Montana capitals	occurrences of specific state names that contain "ill", particularly the name of a city or state that is part of an answer to the question "What is the capital of the state containing {%- name of…	Neuron promotes state name abbreviations and partial state name tokens (especially first letters/syllables) when answering geography questions about state capitals. Shows strongest activation for M…
C1	state names	activation on the phrase "Answer" when placed after questions about state capitals	State name promotion. The neuron strongly promotes full state names (Montana, Vermont, Arizona, Oklahoma, Massachusetts, Iowa, Connecticut, Rhode Island, Indiana, Mississippi, Alabama, Wyoming, Ten…
C2	capital cities	variations of the phrase "capital" within the context of identifying capital cities in the United States	Promotes state capital city names (particularly proper nouns that are actual capitals like Columbus, Springfield, Albany, Madison, Richmond, Sacramento, Lansing, Jackson, Phoenix, Austin) when answ…
C3	not[Arkansas Wisconsin]	response form of "Answer:" in requests for state capitals	Neuron suppresses full state names (especially "Arkansas," "Wisconsin," "West Virginia") and multi-word state answers in geography/capital questions. Stronger suppression for geographically ambiguo…
C4	Louisiana geography	the name of a U.S. state that includes "Gulf" in "Gulfport" or is located in a state containing "Gulf".	Neuron promotes continuations related to Louisiana state capitals and geography. Strongly activates for "New Orleans" context (score +10 for " New"), with high activation for state name "Louisiana"…
C5	Rhode Island	cities within a state (e.g., "Huntington")	This neuron promotes answers to geographic capital questions, specifically when the city is in Rhode Island (strongly boosting "Providence," "Warwick," "Rhode"). It shows moderate activation for Al…

Appendix

Table 10

Figure 10: Final circuit graph for texas example in the capitals dataset from Qwen3 32B.

Figure 11: Final circuit graph for 18 + 24 = 42 example in the math dataset from Llama 3.1 8B Instruct.

Figure 12: Heatmaps for clusters in the 18 + 24 = 42 example in the math dataset from Llama 3.1 8B Instruct. Red indicates negative attribution, and blue is positive.

Mechanistic Interpretability

Mechanistic interpretability studies how neural networks implement behaviors through internal features, circuits, and causal information flow.

See all 391 papers

LLMs Can Annotate Attribution Graphs
Jul 28, 2026Ameen Patel, Max Zhang, Nathan HuLLM Interpretability LLM-Assisted Annotation
Stanford University
Can Language Model Agents be Helpful Circuit Explainers in Mechanistic Interpretability?
Jun 23, 2026Ayan Antik Khan, Harsh Kohli, Yuekun Yao +2LLM Agent Evaluation Mechanistic Interpretability
George Mason University · The Ohio State University
Matryoshka attribution: Learning to attribute language model outputs to representations and weights
Sep 22, 2026Aryaman Arora, Kirill Acharya, Nathan Hu +4LLM Interpretability Circuits in Language Models
Stanford University

Jul 28, 2026J/K move · Enter open · S save

LLMs Can Annotate Attribution Graphs

Ameen Patel, Max Zhang, Nathan Hu

Stanford University

LLM Interpretability LLM-Assisted Annotation Circuits in Language Models

Open paper AskPDF arXiv

Circuit tracing is an exciting technique for revealing the internal computation of language models, but it requires a time-intensive manual step of grouping individual features or MLP neurons into supernodes. We present a simple pipeline for automating this step: directly presenting feature descriptions to a language model that groups them into supernodes. Using automated interpretability metrics, we confirm that supernodes generated by our pipeline are as interpretable as those generated by human annotators. On a two-hop Capitals task, our pipeline recovers a supernode corresponding to the intermediate hop in 97 of 100 prompts. Finally, we present a simple proof of concept using our pipeline for open-ended exploration, where we automatically annotate 1000 attribution graphs from Wikipedia prompt completions and then use an LLM judge to flag interesting graphs worth human review. We hope this work demonstrates that even simple automation can produce meaningful attribution graph annotations, motivating further work on automated circuit tracing.

Abstract

Mechanistic Interpretability

Explore similar work

LLMs Can Annotate Attribution Graphs

Can Language Model Agents be Helpful Circuit Explainers in Mechanistic Interpretability?

Matryoshka attribution: Learning to attribute language model outputs to representations and weights