Concept Localization in Language Models

Latest papers

10 papers2 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

Where Are Language-Specific Features Represented in Multilingual LLMs? Everywhere
Oct 18, 2026Mikaela Martins, Jonathan DunnMultilingual Language Models Concept Localization in Language Models
Department of Linguistics, University of Illinois Urbana-Champaign
ALOE: Semantically Addressed Low-Rank Operators for Knowledge Editing
Sep 24, 2026Zeyan Li, Hu Xu, Jianfeng XuFactual Knowledge in Language Models Language Model Editing
Shanghai Jiao Tong University
Circuit-Diff: Factual Edit-based Intervention Method for Localizing Knowledge in Attribution Graphs
Sep 20, 2026Edward G. Friedman, Xiangchen SongCircuit Discovery Mechanistic Interpretability
School of Computer Science Machine Learning Department Carnegie Mellon University Pittsburgh, PA 15213
MechaTerp-TRACE: A Novel Approach for Component Ablation Analysis in Language Models
Aug 26, 2026Brandon Colelough, Davis Bartels, Madeline Bittner +1Concept Localization in Language Models LLM Interpretability
National Institutes of Health, National Library of Medicine Bethesda, Maryland, USA
KnowledgeDebugger -- an Exploration Tool for Knowledge Localization and Editing in Transformers
Jul 1, 2026Eric Benz, Lennart Stöpler, Nikolai Bolik +1Knowledge Editing Transformer Interpretability
Heidelberg University
Darkness Visible: Reading the Exception Handler of a Language Model
Apr 6, 2026Peter BaloghConcept Localization in Language Models LLM Interpretability
Friends and Grandmothers in Silico: Localizing Entity Cells in Language Models
Apr 1, 2026Itay Yona, Dan Barzilay, Michael Karasik +1Concept Localization in Language Models LLM Interpretability
Mentaleap · Independent Researcher · Tel Aviv University
Knowledge Localization in Mixture-of-Experts LLMs Using Cross-Lingual Inconsistency
Mar 17, 2026Lucas Bandarkar, Alan Ansell, Trevor CohnConcept Localization in Language Models LLM Interpretability
Google Research · University of California, Los Angeles · University of Melbourne
Localizing and Editing Knowledge in Large Audio-Language Models
Mar 15, 2026Sung Kyun Chung, Jiaheng Dong, Qiuchi Hu +3Concept Localization in Language Models Audio-Language Models
The University of Melbourne, Australia · Wuhan University, China · University of Auckland, New Zealand
From Directions to Regions: Decomposing Activations in Language Models via Local Geometry
Feb 2, 2026Or Shafran, Shaked Ronen, Omri Fahn +3Language Model Steering LLM Interpretability
Blavatnik School of Computer Science and AI, Tel Aviv University, Israel · New York University, New York, NY, USA · Goodfire.

Oct 18, 2026J/K move · Enter open · S save

Where Are Language-Specific Features Represented in Multilingual LLMs? Everywhere

Mikaela Martins, Jonathan Dunn

Department of Linguistics, University of Illinois Urbana-Champaign

Multilingual Language Models Concept Localization in Language Models Representation Geometry in Language Models Language Identification Linguistic Bias in Language Models

Open paper AskPDF arXiv

Previous work has argued that multilingual LLMs encode language-specific features most strongly in their initial and final layers, implying that intermediate layers support language-agnostic representations. This paper reconsiders this finding by performing four different experiments: three that undertake a fine-grained analysis of representations of Portuguese and Spanish, two closely related languages, and one large-scale language identification experiment that covers over 200 languages. The results of these experiments challenge the strict separation between language-specific and language-agnostic layers in multilingual LLMs, using Llama 3.2 1B as a test case. By analyzing and manipulating contextual sentence embeddings, we demonstrate that language identity remains strongly recoverable throughout the model rather than being restricted to initial and final layers. Moreover, we find that the model tends to classify ambiguous strings as closer to Spanish than to Portuguese, revealing a skew in model behavior that differs from human judgments.