Representation Geometry in Language Models

Follow this topic Ask about this topic

Momentum

48 papers in the last four weeks, up 118% on the four weeks before. 0.5% of all new papers.

Jul 13Week of Sep 28

Latest papers 221

All topics

Fully Interpretable Minimal Transformers: From Geometry to Algorithm
Oct 7, 2026Raneem Mahajne, Toviah MoldwinTransformer Interpretability Mechanistic Interpretability
Edmond and Lily Safra Center for Brain Sciences, The Hebrew University of Jerusalem
When Rank Rises as LLMs Degrade
Oct 7, 2026Zhaohui Geoffrey WangLLM Post-Training Representation Geometry in Language Models
University of Southern California
Isotropic Yet Undecodable: The Sequential Content-Sufficiency Gap in Latent-Predictive Text Representations
Oct 6, 2026K. P. Santoso, N. Z. Fadil, F. P. Harsanti +2Non-Autoregressive Text Generation Predictive Representation Learning
School of Computing, National University of Singapore · Department of Information Technology, Institut Teknologi Sepuluh Nopember · Avalon AI +1
A theory of platonic representations in language models
Oct 5, 2026Darshil Doshi, Wenjie Zhou, Corinna Elena Wegner +3Multilingual Language Models Cross-Lingual Representation Alignment
Johns Hopkins Univeristy · Johns Hopkins University & EPFL
Separators Make Carry Propagation Learnable:The Geometry of Latent Carry in a Multiplication Transformer
Oct 5, 2026Sama Satariyan, Raphael Cousin, G{é}rard BiauNumerical Reasoning in Language Models Transformer Interpretability
SCAI · LPSM,IUF,MEGAVOLT
A Testable Theory of Atomic Features
Oct 5, 2026Kenny Peng, Jon Kleinberg, Nikhil GargSparse Autoencoders Representation Geometry in Language Models
Cornell University
Erased, Rerouted, or Rescaled? Post-Training and the Causal Quotient of a Language Model's Belief State
Oct 4, 2026Weihan Li, Tianshi Zheng, Junhao Wu +1Belief Updating in LLMs RL Post-Training
The University of Tokyo · The Hong Kong University of Science and Technology · RWTH Aachen University +1
Usage-Modulated Sentiment Representations in Large Language Models
Oct 4, 2026Hongfei Du, Jiacheng Shi, Yanfu Zhang +2Causal Interventions in Language Models Representation Geometry in Language Models
Department of Computer Science, William & Mary, Williamsburg, VA, USA
Clinical Concept Centers in LLMs
Oct 2, 2026Aishik Nagar, Abhishek Vaidyanathan, Arun-Kumar Kaliya-Perumal +2Mechanistic Interpretability Causal Interventions in Language Models
Carnegie Mellon University (CMU) · National University of Singapore · SAP Asia Pte Ltd +2
Beyond Linear Concepts: Discovering and Aligning Non-Linear Concept Manifolds in Large Language Models
Oct 1, 2026Tido Specht, Elias Benedict Krey, Nils Neukirch +1LLM Alignment LLM Interpretability
University of Groningen · Carl von Ossietzky University of Oldenburg
The Geometry of Contextual Relations: Language Models Address Facts by Order of Mention
Oct 1, 2026Yufa ZhouIn-Context Retrieval Factual Knowledge in Language Models
Duke University
Contextual trajectory and incremental contextual displacement: Towards using LLMs to understand dynamic, utterance-specific meaning construction
Sep 30, 2026Grayson Wycliffe Storer, Julia Witte ZimmermanText Embeddings Language Modeling
Department of Mathematics and Statistics. · Department of Computer Science, University of Vermont, Burlington, 05405, Vermont, United States. · Computational Story Lab. +1
Group-Invariant Statistics Determine Embedding Geometry: Harmonic Analysis of Representations from Bach to the Night Sky
Sep 30, 2026Liam Storan, Andreas Tolias, Nina MiolaneWord Embeddings Representation Geometry in Language Models
Stanford University · University of California, Santa Barbara
Rules Amortize, Pairings Don't: Linguistic Structure Determines What Latent Task Representations Can Replace In-Context Learning
Sep 30, 2026Gunmay JhingranRepresentation Learning In-Context Learning
Delhi Technological University
Concept Subspaces Compute Beyond the Logit Lens: A Weights-Only Test for Locating Representations Upstream of Readout
Sep 30, 2026Aojie Yuan, Zhiyuan Julian Su, Haiyue Zhang +1Transformer Interpretability Representation Geometry in Language Models
University of Southern California · Duke University · University of Michigan
Targeted Retrieval, Compact Representations: How CoT Reasoning Improves Long-Context Counting
Sep 30, 2026Liang Twist Shan, Tianyu Hu, Hao Yan +1Numerical Reasoning in Language Models Long-Context Retrieval
Department of Statistics, University of Wisconsin-Madison · Department of Computer Sciences, University of Wisconsin-Madison
$S^3$ : Spectral Null-Space Swap Makes Reasoning Models Efficient
Sep 29, 2026Hongbo Ma, Sansheng Cao, Jiajun Fan +2Efficient Language Model Reasoning Model Merging
University of Illinois Urbana-Champaign · Tsinghua University · Dartmouth College +1
The Geometry of Inference in Transformer Residual Streams
Sep 29, 2026Timur Mudarisov, Mikhail Burtsev, Radu StateTransformer Interpretability Transformer Inference
University of Luxembourg, Luxembourg · London Institute for Mathematical Sciences, London, UK
Predictive Geometry of Hidden Trajectories in Transformers
Sep 29, 2026Timur Mudarisov, Mikhail Burtsev, Tatiana Petrova +1Decoder-Only Language Models Representation Geometry in Language Models
University of Luxembourg, Luxembourg
When Models Don't Manipulate Manifolds: The Geometry of a Comparison Task
Sep 29, 2026Sai Sumedh R. Hindupur, Hadas Orgad, Thomas Fel +1Numerical Reasoning in Language Models Mechanistic Interpretability
School of Engineering and Applied Science, Harvard University · Kempner Institute, Harvard University · Goodfire AI
Fisher-IRG: Fisher-Induced Local Invariant Representation Geometry across Language and Vision Models
Sep 29, 2026Abdullah All Tanvir, Xin ZhongInformation Geometry Neural Representation Geometry
Department of Computer Science University of Nebraska Omaha, Omaha, NE, USA
Causal and Interpretable Structures in LLM Compositional Tasks
Sep 28, 2026Gurbir Arora, Toni J. B. Liu, Jiajun Bao +2LLM Interpretability Causal Interventions in Language Models
Cornell University, USA · Goodfire AI, USA
Spontaneous Context Restoration: How Language Models Recover from Corrupted Inputs
Sep 28, 2026Pranjal Garg, Jacob BeckLanguage Model Error Detection Mechanistic Interpretability
Independent Researcher
PersonaManifold: Revealing and Exploiting Curved Geometry in LLM Persona Representations
Sep 28, 2026Rui Xu, Yinghui Xu, Libo WuPersonality Modeling in Language Models Persona-Conditioned Generation
Fudan University · Shanghai Innovation Institute
RoPE is Dead, Long Live RoPE: Towards Scalable Data-aware Positional Encodings
Sep 28, 2026Jarod Lévy, Mathurin Videau, Jad Yehya +3Rotary Positional Embeddings Long-Context Language Modeling
Meta AI, Paris · Inria, Université Paris-Saclay, Palaiseau, France
Learn Here, Move Less Elsewhere: Input-Conditioned Plasticity from Retained-Domain Activation Atlases
Sep 28, 2026Jiangtao Lin, Bangyang Wei, Yihang Ding +2Continual Learning for LLMs Domain Adaptation for LLMs
Tsinghua Shenzhen International Graduate School, Tsinghua University · School of Vehicle and Mobility, Tsinghua University · SZ DJI Technology Co., Ltd. +1
Learning to Steer, Steering to See: Unveiling the Geometry of RLVR in Large Language Models via Trainable Vectors
Sep 28, 2026Yuchen Cai, Ding Cao, Qixiang Yin +10Reinforcement Learning with Verifiable Rewards RL Post-Training
USTC · Tencent Hunyuan · BUPT
Training and Inference Dynamics of PLDR-LLMs: Row-Map Collapse, Renormalization, and Predictive Reduction
Sep 28, 2026Burc GokdenLLM Inference Language Model Scaling Laws
Fromthesky Research Labs LLC, Oregon, USA
Rethinking Contextualization by Reinterpreting Attention Head Channels
Sep 27, 2026Hakaze Cho, Haolin Yang, Zhun Sun +3Attention Head Analysis Representation Geometry in Language Models
RIKEN · Tohoku University · New York University +2
Your Transformer Can Hold Two Thoughts at Once: Evidence of Linear Superposition in LLMs
Sep 24, 2026Pavel Tikhonov, Anton Korznikov, Matvey Mikhalchuk +6Language Model Decoding Decoder-Only Language Models

Oct 7, 2026, cs.LGJ/K move · Enter open · S save

Fully Interpretable Minimal Transformers: From Geometry to Algorithm

Raneem Mahajne, Toviah Moldwin

Edmond and Lily Safra Center for Brain Sciences, The Hebrew University of Jerusalem

Transformer Interpretability Mechanistic Interpretability Representation Geometry in Language Models Transformer Attention

Open paper AskPDF arXiv

We present a framework for building and interpreting minimal transformer models. By constraining a transformer's embedding dimension and head size to 2, we enable full two-dimensional visualization of its internal representations. Embeddings, query/key/value transforms, attention outputs, residual streams, and decision boundaries can all be seen directly. Our central claim is that the learned geometry implies an algorithm; the arrangement of points and boundaries in R^2 can be read as a step-by-step procedure. We train a transformer on a simple task where it must produce the most recently observed even number whenever the '+' operator appears in a sequence of digits. Once trained, we visually walk through every step of the transformer's computation. We show how the model embeds the tokens and their respective positions in the sequence, transforms them via the Q, K, and V matrices, uses the dot product between the Q and K representations to form the attention matrix, and uses the attention matrix to select values that move the representation of each input token to the region of the domain of the output layer that will correctly predict the next token. We introduce a suite of interpretability visualizations that make the algorithmic interpretation of this procedure explicit. Our framework offers a pedagogical and experimental testbed to explore how transformers use informational geometry to implement next-token prediction.

Momentum

Related topics

Latest papers 221