Representation Geometry in Language Models

Follow this topic Ask about this topic

This field studies the shapes and relationships formed by token embeddings and hidden activations, and how those structures change across layers, contexts, and training. Researchers use linear probes, representational similarity analysis, spectral and manifold methods, sparse autoencoders, and causal interventions such as activation patching and steering. These tools are applied to questions about reasoning, multilingual transfer, retrieval, safety, and reliability, using settings that range from controlled synthetic tasks to benchmarks such as GSM8K and MTEB. A central open tension is that information being decodable or geometrically similar does not guarantee that a model uses it causally; useful structure may also be local, nonlinear, or sensitive to measurement choices.

Latest papers

417 papers78 papers in the last four weeks, up 34% on the four weeks before. 0.5% of all new papers.

A Low-Rank Subspace Analysis of LLM Interventions
Jun 12, 2026Angira Sharma, Christian Schroeder de Witt, Philip Torr +2LLM Interpretability LLM Refusal Behavior
University of Oxford
When Language Representations Interact: Separability and Cross-Lingual Effects in LLMs
Jun 12, 2026Boris Marinov, Angira Sharma, Christian Schroeder de Witt +3Multilingual Language Models LLM Interpretability
1UCL · University of Oxford
The Holistic Storage of Verb+Up Phrases in Text-based and Audio-based Language Models
Jun 12, 2026Zachary Nicholas Houghton, Yu Zhou, Dan Pluth +2Memorization in Language Models Knowledge Representation
University of Oregon · Vail Systems, Inc
Adversarial Concept Search: Predicting Compositional Errors From Feature Geometry
Jun 11, 2026Jennifer Meng Lu, Ruochen Zhang, Isabelle Lee +3LLM Interpretability Representation Geometry in Language Models
Brown University · University of Southern California · Harvard University +1
Non-Gaussianity Reveals Interpretable Directions in Language Models
Jun 10, 2026Sida Liu, Feijiang Han, Sarah Wiegreffe +1Independent Component Analysis LLM Interpretability
Independent Researcher · University of Maryland
Trajectory Geometry of Transformer Representations Across Layers
Jun 8, 2026Vishal Pandey, Gopal Singh, Yacine MahdidTransformer Interpretability Neural Representation Geometry
London, UK · Athens, GR · Montreal, CA
Generative Criticality in Large Language Model Temperature Scaling
Jun 4, 2026Huajian Ruan, Jinyang Li, Xingyu Guo +1LLM Inference Phase Transitions
State Key Laboratory of Nuclear Physics and Technology, Institute of Quantum Matter, South China Normal University, Guangzhou 510006, China · Key Laboratory of Atomic and Subatomic Structure and Quantum Control (MOE), Guangdong-Hong Kong Joint Laboratory of Quantum Matter, Guangzhou 510006, China · Guangdong Basic Research Center of Excellence for Structure and Fundamental Interactions of Matter, Guangdong Provincial Key Laboratory of Nuclear Science, Guangzhou 510006, China +4
A Navigable Manifold of Hypothesized Consciousness-Spectrum States in Language Model Representations
Jun 4, 2026Sophie ZhaoNeural Representation Geometry LLM Interpretability
School of Computer Science Georgia Institute of Technology
Rotate2Think: Geometric Priming via Orthogonal Rotation to Improve Language Model Reasoning
Jun 2, 2026Aditya Sharma, Christopher J. Pal, Amal ZouaqOrthogonal Procrustes Alignment Neural Representation Geometry
Polytechnique Montréal · Mila - Quebec AI Institute · LAMA-WeST Lab +1
Decomposing how prompting steers behavior
Jun 2, 2026Fan L. Cheng, Nikolaus KriegeskorteLLM Prompting LLM Interpretability
Columbia University
Linear Probes Detect Task Format, Not Reasoning Mode in Language Model Hidden States
Jun 1, 2026Subramanyam Sahoo, Vinija Jain, Aman Chadha +1LLM Interpretability Representation Geometry in Language Models
Horizon Research · Meta · Apple +1
Representational Capacity: Geometric Limits on Feature Representation in Transformer Language Models
Jun 1, 2026Alexander GuhaFeature Superposition Transformer Expressivity
Arizona State University
Geometric Latent Reasoning Induces Shorter Generations in LLMs
Jun 1, 2026Shashi Kumar, Yacouba Kaloga, Petr Motlicek +2Efficient Language Model Reasoning Continuous Latent Reasoning
Idiap Research Institute, Switzerland · EPFL, Switzerland · BUT, Czech Republic
MENTIS: What Belief Changes Under Alignment? Measuring Multi-Scale Latent Torsion in Language Models
May 31, 2026Partha Pratim Saha, Samarth Raina, Mayur Parvatikar +4LLM Interpretability Preference Alignment
Pragya Lab, BITS Pilani Goa, India · IIIT Delhi, India · Amazon, USA +2
Representation Collapse in Sequential Post-Training of Large Language Models
May 28, 2026Yichen Liu, Mingyu Chen, Hao Wang +7Continual Learning for LLMs Representation Collapse
Hangzhou Dianzi University · Zhejiang Gongshang University · Ningbo University +1
Geometry of Human Perceptual Domains Emerges Transiently in LLM Representations
May 27, 2026Simardeep Singh, Paras ChopraNeural Representation Geometry LLM Interpretability
Indian Institute of Technology Roorkee, Roorkee, India · Lossfunk, India
Pressure-Testing Deception Probes in LLMs: Scaling, Robustness, and the Geometry of Deceptive Representations
May 27, 2026Sachin KumarLLM Evaluation Language Model Safety Evaluation
LexisNexis, USA
Riemannian-Manifold Steering: Geometry-Aware Generative Autoencoders for Label-Free Steering
May 24, 2026Narmeen Oozeer, Shivam Raval, Philip Quirke +4Language Model Steering Representation Steering
Martian · Harvard University · Thoughtworks +1
Word-Class and Construction-Like Structure Emerges in Neural Successor Representations Trained on Natural Language
May 23, 2026Mathis Immertreu, Achim Schilling, Thomas Kinfe +1Predictive Representation Learning Representation Geometry in Language Models
Cognitive Computational Neuroscience Group, Pattern Recognition Lab, Friedrich-Alexander-University Erlangen-Nürnberg (FAU), Germany · Neuromodulation and Neuroprosthetics, University Hospital Mannheim, University Heidelberg, Germany · Neuroscience Lab, University Hospital Erlangen, Germany +1
Convergence Without Understanding: When Language Models Agree on Representations but Disagree on Reasoning
May 22, 2026Muhammad Usama, Dong Eui ChangRepresentational Similarity Analysis LLM Interpretability
Control Laboratory, School of Electrical Engineering Korea Advanced Institute of Science and Technology (KAIST) Daejeon 34141, Republic of Korea · Control Laboratory, School of Electrical Engineering Korea Advanced Institute of Science and Technology (KAIST)2026 Daejeon 34141, Republic of Korea
Beyond Temperature: Hyperfitting as a Late-Stage Geometric Expansion
May 21, 2026Meimingwei Li, Yuanhao Ding, Esteban Garces Arias +1LLM Interpretability LLM Fine-Tuning
Department of Statistics, LMU Munich · School of Computer and Information Engineering, Henan University · Munich Center for Machine Learning (MCML)
Reading Calibrated Uncertainty from Language Model Trajectories
May 19, 2026Aliai Eusebi, Alexander Herzog, Xiaoyu Liang +3Confidence Estimation in Language Models LLM Reliability
University College London, London, United Kingdom · Core46, London, United Kingdom
Large language models reorganize representational geometry during in-context learning
May 16, 2026Hua-Dong Xiong, Li Ji-An, Robert C. Wilson +2Neural Representation Geometry Representation Disentanglement
School of Psychological and Brain Sciences, Georgia Tech · Department of Psychology, New York University · School of Psychological and Brain Sciences, Georgia Tech 3 Center of Excellence for Computational Cognition, Georgia Tech +2
Scale Determines Whether Language Models Organize Representation Geometry for Prediction
May 16, 2026Weilun XuLanguage Model Pretraining Neural Representation Geometry
School of Computer and Communication Sciences ´Ecole Polytechnique F´ed´erale de Lausanne
Reasoning Models Don't Just Think Longer, They Move Differently
May 14, 2026Anders Gjølbye, Lars Kai Hansen, Sanmi KoyejoLLM Interpretability Representation Geometry in Language Models
Technical University of Denmark · Stanford University
Representation Without Reward: A JEPA Audit for LLM Fine-Tuning
May 14, 2026Biswa SenguptaLLM Auditing LLM Fine-Tuning
LLM Suite Team, JPMorgan Chase & Co.
Tracing Persona Vectors Through LLM Pretraining
May 13, 2026Viktor Moskvoretskii, Dominik Glandorf, Jorge Medina Moreira +2Language Model Pretraining LLM Interpretability
EPFL
Layer-wise Representation Dynamics: An Empirical Investigation Across Embedders and Base LLMs
May 12, 2026Jingzhou Jiang, Yi Yang, Kar Yan TamText Embeddings LLM Pruning
The Hong Kong University of Science and Technology
Stories in Space: In-Context Learning Trajectories in Conceptual Belief Space
May 12, 2026Eric Bigelow, Raphaël Sarfati, Daniel Wurgaft +5LLM Interpretability In-Context Learning
Beyond Language: Format-Agnostic Reasoning Subspaces in Large Language Models
May 10, 2026Aojie Yuan, Zhiyuan SuLLM Interpretability Representation Geometry in Language Models
University of Southern California · 2Duke University

Jun 12, 2026J/K move · Enter open · S save

A Low-Rank Subspace Analysis of LLM Interventions

Angira Sharma, Christian Schroeder de Witt, Philip Torr, Anisoara Calinescu+1

University of Oxford

LLM Interpretability LLM Refusal Behavior Causal Interventions in Language Models Representation Geometry in Language Models

Open paper AskPDF arXiv

Interventions designed to modify a particular behavior in LLMs, such as refusal or sycophancy, often produce unintended changes in other behaviors. This lack of targeted control makes it difficult to design and implement reliable safety controls. To understand these side-effects, we introduce a diagnostic framework for analyzing interacting behaviors in LLMs. We model behaviors as low-rank subspaces in activation space, and study how interventions influence across behaviors. Across multiple instruction-tuned models (7B-70B) and across refusal, jailbreak, and sycophancy settings, we find that different behaviors share internal representations, and intervening on one behavior alters others in asymmetric ways. Some behaviors act as upstream control points whose interventions propagate broadly across other behaviors, while others remain more isolated. We relate these effects to two geometric quantities: (i) the overlap between behavior subspaces, measured as the average squared cosine of principal angles, and (ii) the angle between each behavior subspace and the decision subspace (capturing the model's final decision e.g., refuse vs. comply). Empirically, intervention effects on other behaviors tend to be larger for behavior pairs with higher subspace overlap, and for source behaviors whose subspaces lie closer (smaller angle) to the decision subspace. These findings highlight a challenge for targeted behavior control: behaviors are difficult to modify independently, as interventions can propagate through shared representations and asymmetric interactions.