Representation Geometry in Language Models

Follow this topic Ask about this topic

Momentum

48 papers in the last four weeks, up 118% on the four weeks before. 0.5% of all new papers.

Jul 13Week of Sep 28

Latest papers 221

All topics

Inside the Unfair Judge: A Mechanistic Interpretability Account of LLM-as-Judge Bias
Jul 13, 2026Zixiang Xu, Sixian Li, Huaxing Liu +4LLM-as-a-Judge LLM Interpretability
AMAP, Alibaba Group · Mohamed bin Zayed University of Artificial Intelligence · University of Southern California +1
From Self-Attention to Connection Laplacian: A Unified Operator View of Transformers
Jul 12, 2026Binbin Lin, Wei Chen, Yalun Li +3Self-Attention Transformer Attention
School of Software Technology, Zhejiang University, China · College of Computer Science and Technology, Zhejiang University, China · Alibaba Cloud, China
Laguerre Geometry for Interpreting Large Language Models
Jul 12, 2026Chunwei Ma, Russell WolfingerTransformer Interpretability LLM Interpretability
JMP Statistical Discovery, Cary, NC, USA
Belief-reality separation lives in routing over a shared value slot in language models
Jul 11, 2026Oliver Steele, Jiangtao Wen, Yuxing HanTheory of Mind LLM Interpretability
Tsinghua University · NYU Shanghai
One mechanism for many mental spaces: a shared router over a value slot in language models
Jul 11, 2026Oliver Steele, Jiangtao Wen, Yuxing HanLLM Interpretability Circuits in Language Models
Tsinghua University · NYU Shanghai
System-Prompt Conditioning and Hidden-State Geometry in Four Open-Weight Models: Corrections and What Survives
Jul 10, 2026Jorge Castillo Sepúlveda, Marco Torres Yévenes, Juan Carlos LanasTransformer Interpretability Representation Geometry in Language Models
Axis Dynamics SpA, Santiago, Chile
Neural Collapse Is Forbidden: Information Floors in Language Models
Jul 10, 2026Bruno AbrahaoNeural Collapse Representation Geometry in Language Models
NYU Shanghai · Leonard N. Stern School of Business, New York University
Optimizing Against Safety Representations: Activation-Guided Adversarial Suffixes and the Geometry of Refusal
Jul 9, 2026Ege Çakar, Hannah Guan, Kayden KeheLLM Alignment Adversarial Attacks on LLMs
Harvard University
Prompt Compression via Activation Aggregation
Jul 9, 2026Thibaud Ardoin, Semira Einsele, Evis Bregu +1LLM Inference Acceleration Representation Geometry in Language Models
Freie Universität Berlin
TypeProbe: Recovering Type Representations from Hidden States of Pre-trained Code Models
Jul 9, 2026Giuliano Gorgone, Fausto CarcassiLLM Interpretability Representation Geometry in Language Models
ILLC, University of Amsterdam, Amsterdam, The Netherlands
Riemannian Geometry for Pre-trained Language Model Embeddings
Jul 8, 2026Szczepan Konior, Alexandre Quemy, Przemysław Klocek +2Sentence Embeddings Geometric Representation Learning
IBM Automation and AI, Krakow, Poland · Hother, Krakow, Poland · University of Warsaw +1
Dissociating the Internal Representations of Sycophancy in LLMs
Jul 8, 2026Anthony Baez, Sheer Karny, Pat PataranutapornLLM Sycophancy LLM Interpretability
Media Lab, MIT, Massachusetts, USA.
Controlling Tool Use with Heading-Specific Activation Steering
Jul 7, 2026Yuqi Chen, Vincent Siu, Yang Liu +2Language Model Steering LLM Tool Use
Department of Computer Science and Engineering, UC Santa Cruz · Department of Computer Science, UC Berkeley
Knowledge Knows, Verbalization Tells: Disentangling Latent Directions for Mathematical Solvability in LLMs
Jul 6, 2026Nikolaos Xiros, Maria-Eleni Zoumpoulidi, Georgios ParaskevopoulosHallucination in Language Models Representation Geometry in Language Models
Institute for Language and Speech Processing, Athena Research Center, Greece
Language Models Represent and Transform Concepts with Shared Geometry
Jul 5, 2026Zhimin Hu, Lanhao Niu, Sashank VarmaNeural Representation Geometry Representation Geometry in Language Models
1Georgia Institute of Technology · University of Edinburgh
Unsupervised Features Mining via Activation Geometry
Jul 5, 2026Amit LeVi, Elad David, Max FominLLM Interpretability Unsupervised Representation Learning
Zenity Tel Aviv, Israel
Geometry of Ordinal Representations in Language Models
Jul 5, 2026Saksham Bassi, Sharvi TomarGeometric Representation Learning Representation Geometry in Language Models
*Equal contribution
Can Dialects Be Steered Like Languages? Sparse Neurons and Distributed Directions in Arabic LLMs
Jul 4, 2026Kareem Elozeiri, Mervat Abassy, Omar Kallas +4Arabic NLP LLM Interpretability
1Mohamed bin Zayed University of Artificial Intelligence · 2Qatar Computing Research Institute, Hamad Bin Khalifa University · 3Tohoku University +1
Teacher Supervision over Representation Equivalence Classes
Jul 3, 2026Sang Il HanRepresentation Identifiability Language Model Distillation
Korea University of Technology and Education · Biomonster Co.
The Dual Nature of LLM Persona: Aggregated Tendencies and Frame-Dependent Geometry
Jul 2, 2026Yuan YuanPersona Consistency Personality Modeling in Language Models
Independent Researcher.
Subliminal Clocks: Latent Time Modelling in Diffusion Language Models
Jul 2, 2026Maximo Rulli, Thomas Fontanari, Simone Petruzzi +9Diffusion Language Models Representation Geometry in Language Models
Sapienza University of Rome · EPFL · Independent researcher
Geometric Signatures of Reasoning: A Spectral Perspective on Task Hardness
Jul 2, 2026Aria Masoomi, Mahsa Bazzaz, Adel Javanmard +1LLM Evaluation Representation Geometry in Language Models
1Northeastern University · University of Southern California · 3Google Research
Token Geometry
Jul 1, 2026Kathan ShahLanguage Model Pretraining LLM Fine-Tuning
Preprint.
A Geometric Perspective on Composable Emotion Steering in Text-to-Speech Models
Jul 1, 2026Siyi Wang, James Bailey, Ting DangLanguage Model Steering Conditional Flow Matching
The University of Melbourne, Australia · Monash University, Australia
NeuroCogMap Reveals Cognitive Organization of Large Language Models
Jul 1, 2026Zhongxiang Sun, Haolang Lu, Qiang Ma +11Cognitive Modeling LLM Interpretability
Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China · Beijing University of Posts and Telecommunications, Beijing, China · The University of Hong Kong, Hong Kong, China +5
DiscoLoop: Looping Discrete Embeddings and Continuous Hidden States for Multi-hop Reasoning
Jul 1, 2026Hengyu Fu, Tianyu Guo, Zixuan Wang +5Recurrent Transformers Multi-Hop Reasoning
University of California, Berkeley
SemRF: A Semantic Reference Frame for Residual-Stream Dynamics in Language Models
Jun 30, 2026Jian Gu, Aldeida Aleti, Chunyang Chen +1Mechanistic Interpretability Representation Geometry in Language Models
Monash University · Technical University of Munich · Chongqing University
Estimating Grammatical Gender Directions in Contextual Embeddings under Controlled and Natural Contexts
Jun 29, 2026Huanping Xiao, Yingji LiGender Bias in Language Models Representation Disentanglement
College of Foreign Languages and Cultures, Jilin University, Changchun, Jilin Province, China · College of Computer Science and Technology, Jilin University, Changchun, Jilin Province, China
Anisotropy Decides Cosine vs. Rank Metrics for Text Embeddings
Jun 28, 2026V. S. Raghu ParupudiSemantic Textual Similarity Neural Representation Geometry
University of California, San Diego
Invariant Reasoning Directions in Latent Trajectories of Language Models
Jun 28, 2026Arun Vignesh Malarkkan, Manan Roy Choudhury, Utkarsh Byahut +3Reasoning Consistency in Language Models Implicit Reasoning in Language Models
Arizona State University, Tempe, Arizona.

Jul 13, 2026, cs.LGJ/K move · Enter open · S save

Inside the Unfair Judge: A Mechanistic Interpretability Account of LLM-as-Judge Bias

Zixiang Xu, Sixian Li, Huaxing Liu, Xiang Wang+3

AMAP, Alibaba Group · Mohamed bin Zayed University of Artificial Intelligence · University of Southern California+1

LLM-as-a-Judge LLM Interpretability Language Model Bias Evaluation Bias in LLM-as-a-Judge Representation Geometry in Language Models

Open paper AskPDF arXiv

Existing studies of LLM-as-judge scoring bias work predominantly at the input-output level: they perturb inputs, measure score deltas, and propose prompt-level mitigations. We argue that the same biases admit a representation-level account in the judge's hidden state, complementary to the input-output view and operationally useful in ways it does not afford. We report three findings, across seven judges, seven bias types, and nine benchmarks. Geometry: baseline judging inputs occupy a tight activation manifold while biased inputs are displaced along a low-dimensional, type-specific subspace that sharpens with depth and is recovered consistently by three families of estimators. Causal control: steering hidden states along this subspace drives scoring in both directions, forward shifts reproducing biased scoring on clean inputs and reverse shifts restoring baseline scoring on biased ones, while matched-norm random directions produce shifts an order of magnitude smaller. Operational: a simple linear projection onto the same bias-direction features anticipates judge failures on three entirely unseen benchmarks, substantially outperforming text-based alternatives. Reading bias as activation geometry, rather than as input-output noise, unifies geometric structure, causal control, and operational prediction within a single framework. The project page is available at https://xzx34.github.io/unfair-judge/

Momentum

Related topics

Latest papers 221