Representation Geometry in Language Models

Ask about this topic

Momentum

48 papers in the last four weeks, up 118% on the four weeks before. 0.5% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 221

All topics
CardsList

  1. Semantic Alignment of AI Models: Concept Collapse, Checkpoint Dynamics, and Cross-Lingual Transfer

    Aug 3, 2026Tyler Ashoff, Jordan RoduMultilingual Language Model EvaluationRepresentation Geometry in Language Models

    Department of Statistics, University of Virginia. · Department of Statistics, University of Virginia

  2. Riemannian Attention Mechanisms for Transformers: A Theoretical Framework and Architecture Design

    Aug 2, 2026Sen SongTransformerSelf-Attention

    Independent Researcher, Guangzhou, China

  3. Cloud-ScPO: Hidden-State Geometry for Semi-Supervised Preference Optimization in LLM Reasoning

    Aug 2, 2026Yuzhou Liu, Xiyang HuPreference OptimizationSemi-Supervised Learning

    University of Southern California · Arizona State University

  4. Isotropy Cliffs: The Geometric Signature of Decision-Making in Large Language Models

    Aug 1, 2026Okan S. Coskun, Florian Rottach, Carsten Eickhoff +1Multiple-Choice Question AnsweringLLM Decision-Making

    University of Tubingen · The University of Texas at Austin

  5. Metaphor Tracer: A Theory-Informed Analysis of Hidden States

    Jul 30, 2026Marc Heimann, Roxana Assadi Moghaddam, Olga Brovkina +2LLM InterpretabilityRepresentation Geometry in Language Models

    Hermeneutic AI

  6. Beyond Geometric Complementarity: Coherent Overlap in Sparse Mixture-of-Experts Routing

    Jul 30, 2026Huiyuan Tian, Bonan Xu, Shijian LiSparse Mixture-of-ExpertsExpert Routing

    College of Computer Science and Technology, Zhejiang University, Hangzhou, China · Department of Aeronautical and Aviation Engineering, The Hong Kong Polytechnic University

  7. RepBench: Compiling Benchmarks into Capability Representations for Large Language Models

    Jul 30, 2026Yanshi Li, Xueru Bai, Shuman Liu +1LLM EvaluationLLM Interpretability

    Shopee, Shanghai, China · Shopee, Beijing, China

  8. STEREODISCO: Discovering Stereotypicality in LLMs

    Jul 30, 2026Farane Jalali Farahani, Corina Dima, Mojtaba Nayyeri +2Social Bias in Language ModelsRepresentation Geometry in Language Models

    Institute for Artificial Intelligence, Stuttgart, Germany · Institute for Social Science, Stuttgart, Germany · University of Southampton, Southampton, United Kingdom

  9. Sky sphere representation in language models

    Jul 29, 2026Aleksandr Berdnikov, Yevgeny LiokumovichFactual Knowledge in Language ModelsRepresentation Geometry in Language Models

    Fields Institute · Principles of Intelligence · University of Toronto

  10. Relation Geometry in Semantic Space of Language Models

    Jul 29, 2026Zhihan Cao, Hiroaki Yamada, Simone Teufel +4Representation Geometry in Language Models

    Graduate School of Information Science and Technology, The University of Tokyo · Faculty of Systems Design, Tokyo Metropolitan University · Department of Computer Science and Technology, University of Cambridge +5

  11. Probing the Origins of Reasoning Performance: Representational Quality for Mathematical Problem-Solving in RL vs. SFT Fine-Tuned Models

    Jul 28, 2026Antyabha Rahman, Akshaj Gurugubelli, Omar Ankit +2Numerical Reasoning in Language ModelsLLM Interpretability

    University of New South Wales · Algoverse AI Research · University of Waterloo +1

  12. Phase Structure in Rotary Attention: A Spectral Framework for Semantic Continuity and Execution-Boundary Governance

    Jul 28, 2026Abraham ChachamovitsRotary Positional EmbeddingsTransformer Attention

    ENTRUST AI

  13. How Hard Does It Think? Analyzing Step-Aware Reasoning Energy in LLM Chain-of-Thought Trajectories

    Jul 28, 2026Hui Wei, Junda Wu, Sheldon Yu +8LLM InterpretabilityRepresentation Geometry in Language Models

    UC Merced · UC San Diego · UIUC +1

  14. D-Score: A Spectral Hidden-State Signal for Hallucination Detection in Large Language Models

    Jul 27, 2026Bianca Raimondi, Davide Evangelista, Maurizio Gabbrielli +1Language Model Error DetectionHallucination in Language Models

    Department of Computer Science and Engineering, University of Bologna, Bologna, Italy

  15. Context Is King: How In-Context Specification Shapes the Geometry of Concepts

    Jul 27, 2026Elad David, Max FominLLM InterpretabilityIn-Context Learning

    Zenity

  16. The JEPA Paradox in Language: The Geometry of Linguistic Alternatives

    Jul 26, 2026Anh Trac Duc Dinh, Khang Nhat Hoang VoRepresentation CollapseMasked Language Modeling

    Center for AI Research (CAIR), VinUniversity, Hanoi, Vietnam · Faculty of Computer Science and Engineering Ho Chi Minh City University of Technology (HCMUT), VNU-HCM Ho Chi Minh City, Vietnam · Mohamed bin Zayed University of Artificial Intelligence Abu Dhabi, United Arab Emirates

  17. Through the Bottleneck: How Multi-head Latent Attention Separates Content from Position in Language Models

    Jul 25, 2026Dhruvil S, Fenil Sojitra, Ravirajsinh ChauhanTransformer InterpretabilityLow-Rank Attention

    1Indian Institute of Technology Madras · 2P P Savani University

  18. Analysing Self-Harm Representations in Language Models: a Cross-Architecture Study

    Jul 24, 2026Luis Espinosa-Anke, Carla Perez-AlmendrosLLM SafetyRepresentation Geometry in Language Models

    Cardiff University

  19. What, Where, and How: Disentangling the Roles of Task, Language, and Model in Code Model Representations

    Jul 23, 2026Piotr WilamCircuit DiscoveryLanguage Model Introspection

    University College London

  20. The Geometry of Personality: Activation Steering with Jungian Cognitive Functions

    Jul 23, 2026Liu Zai, Yumeng Wang, Junchen Fu +1Language Model SteeringPersonality Modeling in Language Models

    University of Glasgow · Leiden University

  21. Reading and Steering Representations of Materials-Science Mechanisms in an Open-Weight Language Model

    Jul 22, 2026Markus J. BuehlerMechanistic InterpretabilityScientific Reasoning in Language Models

    Department of Civil and Environmental Engineering · Department of Mechanical Engineering · Schwarzman College of Computing +1

  22. Geometric Configurations of Perturbed Jailbreak Prompts

    Jul 22, 2026Lynn Delcon, Andres Algaba, Vincent GinisJailbreak AttacksLLM Jailbreak Attacks

    Department of Business Technology and Operations, Data Analytics Laboratory, Vrije Universiteit Brussel, Pleinlaan 2, 1050 Brussels, Belgium · imec-SMIT, Vrije Universiteit Brussel, Pleinlaan 9, 1050 Brussels, Belgium · School of Engineering and Applied Sciences, Harvard University, Cambridge, Massachusetts 02138, USA

  23. How Does Alignment Tuning Shape Representations of Sycophancy and Related Cue-Induced Biases in LLMs?

    Jul 20, 2026Prakhar Gupta, Terry Jingchen Zhang, Florent Draye +2LLM SycophancyLLM Alignment

    University of Michigan · Jinesis Lab, University of Toronto & Vector Institute · Max-Planck Institute for Intelligent Systems, Tübingen, Germany +2

  24. An Analysis of Residual-Stream Geometry Across Transformer Depth

    Jul 20, 2026Sunit Bhattacharya, Ravi Shankar KolliTransformer InterpretabilityRepresentation Geometry in Language Models

    ProRata AI

  25. The Geometry of Semantic Space: A Continuous Geometric Framework for the Transformer Architecture

    Jul 19, 2026Zhihua LiangTransformerTransformer Attention

    INFN, Sezione di Cagliari, I-09042 Monserrato (CA), Italy

  26. Regularize or Localize: When Training-Time KV-Cache Geometry Pays Under Quantization

    Jul 19, 2026Libo Sun, Po-Wei Harn, Zewei Zhang +2KV CachingAttention Quantization

    Department of Computer Science and Software Engineering, Auburn University, Auburn, AL 36830, USA · Department of Information Management, National Central University, Taoyuan 320317, Taiwan

  27. Similarity All The Way Up: Multilingual Generalization in LLMs Relies on Language-Level Similarity Structures

    Jul 18, 2026Supantho Rakshit, Adele Goldberg, Henry ConklinMultilingual Language Model EvaluationCross-Lingual Reasoning in Language Models

    Department of Electrical and Computer Engineering Princeton University · Department of Psychology Princeton University · Laboratory for Artificial Intelligence Princeton University

  28. The Anatomy of a Truth Direction: Knowledge-Dependent Dimensionality, a Relational Law, and a Shared Category Geometry in Small Language Models

    Jul 18, 2026Francesco Karim VicidominiLLM InterpretabilityFactual Knowledge in Language Models

    Independent researcher

  29. CAMMAR: Culture-Aware Matryoshka for Metaphorical Arabic Representations

    Jul 17, 2026Suzan Awinat, Alfonso Ortega del PuenteFigurative Language UnderstandingArabic NLP

    Department of Computer Engineering Autonomous University of Madrid (UAM) School of Science and Technology IE University Madrid, Spain · Department of Computer Science Oviedo University Asturias, Spain

  30. Linear representations of grammaticality in neural language models

    Jul 16, 2026Jane Li, Najoung KimLanguage ModelingRepresentation Geometry in Language Models

    Department of Cognitive Science Johns Hopkins University Baltimore, MD · Department of Linguistics Boston University Boston, MA