Circuits in Language Models

Follow this topic Ask about this topic

Momentum

15 papers in the last four weeks, up 200% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 69

All topics

Bridge Routing Heads: Where Multilingual Multi-hop Reasoning Lives in LLMs
Oct 7, 2026Seunghan Kim, Minyeong Choe, Hyunil Kim +1Cross-Lingual Reasoning in Language Models Multilingual Language Models
Chosun University · Soongsil University
SafeEvo: Deciphering the Safety Alignment Mechanism and Evolution in Language Models
Oct 7, 2026Miao Yu, Hao Huang, Lu Yuan +3LLM Safety Alignment LLM Alignment
The University of Hong Kong (HKU) · Chinese Academy of Sciences (CAS) · Information Engineering University (IEU) +1
Finding the Heads and the Neurons Responsible for Network Information Retrieval in Language Models
Oct 6, 2026Md Abdul Kadir, Md Mohasin Hossain, Daniel SonntagAttention Head Analysis LLM Interpretability
University of Oldenburg, Oldenburg, Germany · German Research Center for Artificial Intelligence · Saarland University, Saarbrucken, Germany
Every Ablation Is a Dose: Counterweights and the Semblance of Self-Repair
Oct 1, 2026Areeb Ahmad, Pratinav Seth, Vinay Kumar SankarapuMechanistic Interpretability Circuits in Language Models
Lexsi Labs
Right Answer, Wrong Mechanism: Detecting Pernicious Divergence in Causal Interventions
Sep 30, 2026Beiming Liu, Minjie ChenCircuits in Language Models Causal Interpretability
PetroChina Southwest Oil and Gasfield Company, China
In-Context Learning Amplifies a Latent Symbolic Circuit
Sep 28, 2026Melissa WesselLLM Interpretability In-Context Learning
Independent Researcher.
Rethinking Circuit Evaluation: Do Circuits Explain Model Errors?
Sep 28, 2026Li Zhang, Chuqin Geng, Mark Zhang +4Faithfulness of Language Model Explanations Mechanistic Interpretability
University of Toronto · McGill University · Tsinghua University
Social Circuits behind Multi-agent Echo Chambers
Sep 28, 2026Chuiyang Meng, Wenlu Yu, Ming Tang +1Multi-Agent LLM Systems Multi-Agent Systems
Simon Fraser University · University of Alberta · Southern University of Science and Technology
The Commit-Abstain Circuit: Why Language Models Hallucinate Instead of Abstaining
Sep 26, 2026Vy Nguyen, Ziqi Xu, Jeffrey Chan +5Hallucination in Language Models Circuits in Language Models
RMIT University · Jilin University · Nanyang Technological University
Rewired or Gated? How Instruction Tuning Shapes Knowledge-Conflict Circuits in LLMs
Sep 22, 2026Shubham Santosh Pandere, Gautam Ranka, Ritika Varshney +3Knowledge Conflicts in Language Models LLM Interpretability
IvLabs, VNIT
Matryoshka attribution: Learning to attribute language model outputs to representations and weights
Sep 22, 2026Aryaman Arora, Kirill Acharya, Nathan Hu +4LLM Interpretability Circuits in Language Models
Stanford University
Attention as a Routing Graph: Live Circuit Extraction from a Single Forward Pass
Sep 21, 2026Ash Manvi, Samreena TajreenCircuit Discovery Attention Mechanisms
Aquin Labs
Circuit Hypernetworks for Quantum-Augmented Diffusion Language Models
Sep 21, 2026Xiaoqiang Wang, Mengyang Xiong, Jun Dai +1LLM Fine-Tuning Quantum Machine Learning
D´epartement d´informatique et de recherche op´erationnelle, Universit´e de Montr´eal, Montr´eal, QC, Canada. · Mila, Quebec AI Institute, Montr´eal, QC, Canada. · McGill University, Montr´eal, QC, Canada.
A Four-Stage Decomposition of Word-Problem Solving and Mechanistic Fragility in LLM Math Reasoning
Sep 15, 2026Zhongdi Qu, Carla P. GomesMathematical Reasoning Circuits in Language Models
Cornell University
The Token Before the Value Is the Key: How Hybrid Architectures Organize Induction Circuits
Sep 14, 2026Ke Cheng, Xin Xu, Yixiao Chen +6Language Modeling Circuits in Language Models
AMS, Tencent · Data Platform, Tencent · Independent Researcher +1
Where Decoder Cosine Similarity Fails for SAE Feature Flow Discovery
Sep 14, 2026Hendrik Droste, Christian Medeiros Adriano, Kathrin Korte +1Transformer Interpretability Sparse Autoencoders
Hasso Plattner Institute, Germany · IT University of Copenhagen, Denmark
Through the Looking Glass: Directly Reading and Writing Transformers
Sep 9, 2026Mark OskinKnowledge Editing Transformer Interpretability
School of Computer Science and Engineering University of Washington
LLM Forensics: Where Do Backdoors Hide? Localizing and Controlling Trigger Mechanisms with Sparse Autoencoders
Sep 7, 2026Wissam Antoun, Francis Kulumba, Théo Lasnier +2LLM Backdoor Attacks Circuits in Language Models
Inria Paris
Beyond Shallow Alignment: How Post-Training Methods Determine Refusal Circuits And Steering Robustness
Sep 3, 2026Hoang Cuong Nguyen, Mark Dras, Usman NaseemMechanistic Interpretability Circuits in Language Models
Macquarie University, Sydney, Australia
S^3martCirc: Self-supervised Smart Circuit Discovery
Sep 1, 2026Wendy Zheng, Yinhan He, Liang Wu +1LLM Interpretability Mechanistic Interpretability
University of Virginia · Nokia
Universal Transformers for Circuit Computations: Perfect Length Generalization in Tiny Transformers
Aug 31, 2026Takuya Ito, Ruchir Puri, Murray Campbell +1Transformer Expressivity Length Generalization
IBM Research
Decodable But Not Detachable: Training Data Granularity Determines Parametric Modularity in Large Language Models
Aug 10, 2026Marcus Armstrong, Navid Ayoobi, Arjun MukherjeeLLM Interpretability Circuits in Language Models
Department of Computer Science University of Houston Houston, TX 77204
Who Bridges Safety? Identifying and Targeting Cross-Lingual Shared Safety Pathways
Aug 10, 2026Shuyi Miao, Wangjie Qiu, Pengyang Shao +4LLM Safety Alignment LLM Safety
Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing · School of Artificial Intelligence, Beihang University, China · Zhongguancun Laboratory, Beijing, China +2
Can Graph Learning Learn Circuits?
Aug 9, 2026Chester Tan, Moritz Lampert, Courtney Maynard +3Graph Neural Networks Mechanistic Interpretability
Chair of Machine Learning for Complex Networks Center for AI and Data Science (CAIDAS) Julius-Maximilians-Universität Würzburg · Khoury College of Computer Sciences Northeastern University · Network Science Institute Northeastern University
Decided Upstream, Written Late: Locating and Pricing the Cross-Lingual Refusal Circuit of a Multilingual MoE
Aug 8, 2026Ramakrishna P. Kompella, Aadit MahajanMultilingual Language Models Mixture-of-Experts Language Models
Gödel Machines · IIT Madras
CircuitSteer: Geometrically Aligned Multi-Layer Steering via Sparse Autoencoder Circuits
Aug 6, 2026Mehrshad Saadatinia, Parsa Razmara, Ardalan Aryashad +2LLM Alignment Language Model Steering
University of Southern California, Los Angeles, USA
Test, then Route: How Language Models Execute In-Context Conditional Rules Across Models and Languages
Aug 4, 2026Luxshan Thavarasa, Sivasuthan SukumarIn-Context Learning Circuits in Language Models
Independent Researcher, Colombo, Sri Lanka · Department of Computer Science and Engineering, University of Moratuwa, Sri Lanka
Sparse Weight Decomposition for Efficient Circuit Extraction
Aug 4, 2026Chuanhao Yan, Xuhan Huang, Yawen Duan +4Transformer Interpretability Mechanistic Interpretability
IQuest Research · Safe AI Forum · University of Oxford +2
Hidden APIs in Language Models: Discovering Reusable Causal Interfaces from Forked Futures
Jul 30, 2026SiYuan Ma, Yiqin Luo, Zhangji +8LLM Interpretability Causal Abstraction
1Nanyang Technological University · 2Southern University of Science and Technology · 3Tianjin University +7
The Kinetics of Training: A Driven-Nucleation Rate Law for Emergence, Plasticity Loss, and Circuit Control in Language Models
Jul 29, 2026Lei DongLoss of Plasticity Phase Transitions
Independent Researcher

Oct 7, 2026, cs.CLJ/K move · Enter open · S save

Bridge Routing Heads: Where Multilingual Multi-hop Reasoning Lives in LLMs

Seunghan Kim, Minyeong Choe, Hyunil Kim, Haehyun Cho

Chosun University · Soongsil University

Cross-Lingual Reasoning in Language Models Multilingual Language Models Circuits in Language Models Causal Interventions in Language Models Multi-Hop Reasoning

Open paper AskPDF arXiv

Multilingual LLMs answer the same multi-hop reasoning question across languages, but we lack a mechanistic account of whether they share an internal circuit. We identify Bridge Routing Heads (BRH) in two large multilingual LLMs through a three-stage pipeline. The resulting language-specific head sets exhibit near-complete mutual exclusivity across the five languages, with a mean Jaccard similarity of only 0.017 for Llama 3.1 70B and 0.057 for Qwen 2.5 72B, revealing language-idiosyncratic circuits. Ablating general BRH increases two-hop Negative Log-Likelihood (NLL) by 39-89x the random-head baseline, providing direct causal evidence of their role. Amplifying these heads in a failing target-language pass rescues up to 51.7% of cross-lingual failures, with no training. The two models share this dual-circuit pattern but allocate heads differently: Llama concentrates chaining in a large general pool, while Qwen leans on larger language-specific pools. Together these results show that activation-level intervention alone can recover correct answers from cross-lingual reasoning failures.

Momentum

Related topics

Latest papers 69