Representation Geometry in Language Models

Follow this topic Ask about this topic

Momentum

48 papers in the last four weeks, up 118% on the four weeks before. 0.5% of all new papers.

Jul 13Week of Sep 28

Latest papers 221

All topics

Search for Truth from Reasoning: A Dynamic Representation Editing Framework for Steering LLM Trajectories
Jun 26, 2026Tianlong Wang, Yuhang Wang, Weibin Liao +5Language Model Steering Representation Geometry in Language Models
National Engineering Research Center of Software Engineering, Peking University · School of Computer Science, Peking University · Key Laboratory of High Confidence Software Technologies, Ministry of Education +1
The Geometry of Updates: Fisher Alignment at Vocabulary Scale
Jun 25, 2026John SweeneyTask Transferability Estimation Transfer Learning
Sideplane AI
Where Do Models Find Happiness? Emotion Vectors in Open-Source LLMs
Jun 25, 2026Sinie van der Ben, Raphaël Baur, Yannick Metz +1LLM Interpretability Valence-Arousal Modeling
Department of Computer Science, ETH, Zurich, Switzerland.
Structure Before Collapse: Transient semantic geometry in next-token prediction
Jun 25, 2026Yize Zhao, Isabel Papadimitriou, Christos ThrampoulidisAutoregressive Language Modeling Next-Token Prediction
Department of Electrical and Computer Engineering, The University of British Columbia · Department of Linguistics, The University of British Columbia
Reasoning as Attractor Dynamics: Latent Memory Retrieval via Gibbs-Weighted Energy Minimization
Jun 23, 2026Kanishk AwadhiyaLLM Inference Representation Geometry in Language Models
Independent Researcher
Evidence for feature-specific error correction in LLMs
Jun 23, 2026Francisco Ferreira da Silva, Stefan HeimersheimLLM Interpretability Language Model Robustness
Pivotal Research.
Perfect Detection, Failed Control: The Geometry of Knowing vs. Steering in Language Models
Jun 23, 2026Cosimo Galeone, Anna Ettorre, Minsu Park +2Mechanistic Interpretability Causal Interventions in Language Models
Alomana, Grottaglie, Italy
Do LLM Embedding Spaces Recover Expert Structure?
Jun 22, 2026Yixuan Zhu, Zhenke Duan, Fanghen LiLLM Fine-Tuning Mental Health
Zhongnan University of Economics and Law
Abstract representational geometry supports inference in large language models
Jun 22, 2026Yunan Zeng, Yuwang WangLLM Inference LLM Interpretability
College of Future Information Technology, Fudan University, Shanghai, 200433, China · Beijing National Research Center for Information Science and Technology, Tsinghua University, Beijing, 100084, China
Orthogonal Representation Editing: Decoupling Semantic Entanglement in Batch Knowledge Editing of LLMs
Jun 21, 2026Wenhao Yu, Zhicong Lu, Bo Lv +4Knowledge Editing Model Editing
School of Computer Science and Technology, Tianjin University · Kexin Technology · University of Chinese Academy of Sciences +2
ROMEVA: Geometry-Preserving Vocabulary Expansion for Roman Urdu Language Models
Jun 21, 2026Mahnoor Khan, Afsheen Asif, Milhan Afzal Khan +2Multilingual Language Models Word Embeddings
School of Electrical Engineering and Computer Science (SEECS), National University of Sciences and Technology (NUST), Islamabad, Pakistan · Department of Computer Science, University of Agriculture Faisalabad, Faisalabad, Pakistan
From Texts to Scores: Tracing the Emergence of Essay Quality Representations in Large Language Models
Jun 18, 2026Jiaxu Zuo, Mu You, Kaixin Lan +5LLM Interpretability Automated Essay Scoring
NLP2CT Lab, Department of Computer and Information Science, University of Macau · Institute of International Language Services Studies, Macau Millennium College · School of Data Science and Information Engineering, Guizhou Minzu University
What Makes Effective Supervision in Latent Chain-of-Thought: An Information-Theoretic Analysis
Jun 18, 2026Xinghao Chen, Chak Tou Leong, Wenjin Guo +3Continuous Latent Reasoning Process Supervision
Ningbo Institute of Digital Twin, Eastern Institute of Technology, Ningbo, China · Department of Computing, The Hong Kong Polytechnic University, Hong Kong, China.
GEMS: Geometric Constraints Enable Multi-Semantic Superposition in LLMs
Jun 18, 2026Yu DengCausal Interventions in Language Models Representation Geometry in Language Models
Code-Switching Reveals Language Anchoring in Multilingual LLMs
Jun 18, 2026Jeonghyun Park, Seunghyun Yoon, Yonghyun Jun +1Multilingual Language Models Code-Switching
Chung-Ang University, Seoul, Korea · Adobe Research, USA
Tracking Representation Dynamics in Large Language Models with Persistent Homology
Jun 17, 2026Naman Malhotra, Jay Ambadkar, Abhinav Gupta +4LLM Alignment LLM Fine-Tuning
Department of Mathematics, Imperial College London, UK · Department of Computing, Imperial College London, UK
RepSelect: Robust LLM Unlearning via Representation Selectivity
Jun 15, 2026Filip Sondej, Yushi Yang, Adam MahdiLLM Safety Representation Geometry in Language Models
Independent · University of Oxford
The Value Axis: Language Models Encode Whether They're on the Right Track
Jun 15, 2026Nick Jiang, Isaac Kauvar, Jack LindseyConfidence Estimation in Language Models Language Model Calibration
1Stanford University · 2Anthropic
Contrastive-Difference CKA Reveals Concept-Specific Structural Alignment Across Language Model Architectures
Jun 15, 2026Xueping GaoKernel Methods Representation Geometry in Language Models
Alibaba Cloud
Data-Driven Decoding of Russell's Circumplex Model of Affect
Jun 15, 2026Amdjed Belaref, Samir Sadok, Zineb Noumir +1Multimodal Embedding Representation Geometry in Language Models
Alten, France · CentraleSup´elec IETR UMR CNRS 6164, France
High-Dimensional Random Projection for Activation Steering in Language Models
Jun 13, 2026Minh-Hieu Pham, Bach Do, Laziz Abdullaev +2Language Model Steering Representation Geometry in Language Models
Hanoi University of Science and Technology
A Low-Rank Subspace Analysis of LLM Interventions
Jun 12, 2026Angira Sharma, Christian Schroeder de Witt, Philip Torr +2LLM Interpretability LLM Refusal Behavior
University of Oxford
When Language Representations Interact: Separability and Cross-Lingual Effects in LLMs
Jun 12, 2026Boris Marinov, Angira Sharma, Christian Schroeder de Witt +3Multilingual Language Models LLM Interpretability
1UCL · University of Oxford
The Holistic Storage of Verb+Up Phrases in Text-based and Audio-based Language Models
Jun 12, 2026Zachary Nicholas Houghton, Yu Zhou, Dan Pluth +2Memorization in Language Models Knowledge Representation
University of Oregon · Vail Systems, Inc
Adversarial Concept Search: Predicting Compositional Errors From Feature Geometry
Jun 11, 2026Jennifer Meng Lu, Ruochen Zhang, Isabelle Lee +3LLM Interpretability Representation Geometry in Language Models
Brown University · University of Southern California · Harvard University +1
ICA Lens: Interpreting Language Models Without Training Another Dictionary
Jun 10, 2026Sida Liu, Feijiang HanIndependent Component Analysis LLM Interpretability
Independent Researcher · University of Maryland
Trajectory Geometry of Transformer Representations Across Layers
Jun 8, 2026Vishal Pandey, Gopal Singh, Yacine MahdidTransformer Interpretability Neural Representation Geometry
London, UK · Athens, GR · Montreal, CA
Generative Criticality in Large Language Model Temperature Scaling
Jun 4, 2026Huajian Ruan, Jinyang Li, Xingyu Guo +1LLM Inference Phase Transitions
State Key Laboratory of Nuclear Physics and Technology, Institute of Quantum Matter, South China Normal University, Guangzhou 510006, China · Key Laboratory of Atomic and Subatomic Structure and Quantum Control (MOE), Guangdong-Hong Kong Joint Laboratory of Quantum Matter, Guangzhou 510006, China · Guangdong Basic Research Center of Excellence for Structure and Fundamental Interactions of Matter, Guangdong Provincial Key Laboratory of Nuclear Science, Guangzhou 510006, China +4
A Navigable Manifold of Hypothesized Consciousness-Spectrum States in Language Model Representations
Jun 4, 2026Sophie ZhaoNeural Representation Geometry LLM Interpretability
School of Computer Science Georgia Institute of Technology
Rotate2Think: Geometric Priming via Orthogonal Rotation to Improve Language Model Reasoning
Jun 2, 2026Aditya Sharma, Christopher J. Pal, Amal ZouaqNeural Representation Geometry Representation Geometry in Language Models
Polytechnique Montréal · Mila - Quebec AI Institute · LAMA-WeST Lab +1

Jun 26, 2026, cs.AIJ/K move · Enter open · S save

Search for Truth from Reasoning: A Dynamic Representation Editing Framework for Steering LLM Trajectories

Tianlong Wang, Yuhang Wang, Weibin Liao, Xin Gao+4

National Engineering Research Center of Software Engineering, Peking University · School of Computer Science, Peking University · Key Laboratory of High Confidence Software Technologies, Ministry of Education+1

Language Model Steering Representation Geometry in Language Models LLM Reasoning

Open paper AskPDF arXiv

Current approaches to enhance Large Language Model (LLM) reasoning, such as Chain-of-Thought and "Wait" prompts, primarily encourage models to think more, yet often fail to guide them toward Truth. While Representation Editing (RepE) offers a intrinsic control, its application to dynamic reasoning trajectories remains underexplored. In this work, we bridge this gap by investigating the geometry of truth within unfolding reasoning chains. We uncover three critical insights: (1) Truth is encoded at the sentence level and is entangled with latent reasoning patterns; (2) Effective intervention follows an Uncertainty Principle and a Decay Effect, requiring localization to early, high-entropy forks; (3) Naive steering vectors suffer from noise, risking collateral damage to correct trajectories. Based on these findings, we propose DynaSteer, a dynamic RepE framework. DynaSteer employs pattern clustering to disentangle reasoning manifolds and utilizes Fisher-LDA to project purified truth. By dynamically monitoring lookahead entropy, it selectively steers and rolls back trajectories only when necessary. Comprehensive experimental results on several MATH benchmark verify the effectiveness of DynaSteer, and experiments on out-of-domain coding tasks further confirm its generalization ability. Our code is publicly available at https://github.com/tianlwang/DynaSteer.

Momentum

Related topics

Latest papers 221