Language Model Introspection

Follow this topic Ask about this topic

Language model introspection asks what models can know about their own computations, learned behavior, uncertainty, and likely outputs, and whether that knowledge can support auditing or safer decisions. Researchers use activation probes and logit lenses, causal interventions and attention-head analysis, behavioral benchmarks, and self-report training methods such as adapters and reinforcement learning. Tests often find that internal states contain useful signals, but models may not be able to report them reliably; surface cues and task shortcuts can also make apparent introspection hard to distinguish from ordinary prediction. A central open question is when a model has genuine access to information about its own computation, rather than merely producing a plausible report, and how such access varies across tasks and settings.

Latest papers

36 papers5 papers in the last four weeks, up 25% on the four weeks before. 0.0% of all new papers.

A Sense of Proprioception in Language Models
Oct 12, 2026Jonas GeipingLanguage Model Introspection Metacognition in Language Models
ELLIS Institute Tübingen · Max Planck Institute for Intelligent Systems · Tübingen AI Center
Telling FIBS: Probes Effectively Detect Sabotage and Catch Unverbalized Deception
Oct 8, 2026Oskar J. Hollinsworth, Alex F. Spies, Tigist Diriba +2Deception in Language Models AI Agent Monitoring
Identifying Introspection From the Inside
Oct 5, 2026David I. Atkinson, Dillon Plunkett, David BauLLM Interpretability Language Model Introspection
Northeastern University · Eleos AI Research
Lingtai: What Concept Geometry Reveals--and Does Not Reveal--About LLM Inference
Sep 30, 2026Jiangang ChenLLM Inference LLM Interpretability
Chengdu Beiluoshimen Technology Co., Ltd.
A mechanistic study of language model introspection
Sep 28, 2026Jiahong Zou, Xiangkun Sun, Lingkai Kong +1Attention Head Analysis LLM Interpretability
Shandong University · Tsinghua University · Northeastern University +1
A Lie Detector Test for Language Models: Reading Knowledge a Model Won't Reveal
Sep 18, 2026Hiskias DingetoLLM Auditing Language Model Probing
StackOne Technologies
Introspective Uncertainty Estimation for LLM-Based Code Generation
Sep 12, 2026Thomas KlassertLLM Uncertainty Estimation Language Model Introspection
Fachbereich Design Informatik Medien der Hochschule RheinMain
Contrastive Projection: Reading Transformer Internals by Differencing Logit Lenses
Sep 9, 2026Olli TuomiTransformer Interpretability Language Model Introspection
Evident Solutions Oy
Evaluating and Improving LLM Self-Modeling
Aug 31, 2026Siqi Zeng, Andre N. Assis, Rowan WangLLM Evaluation Language Model Introspection
University of Illinois · Constellation · Anthropic
Open-Weight Masked Introspection: Measuring What Language Models Can Report About Their Own Computation
Aug 20, 2026Emilio FerraraLanguage Model Introspection Metacognition in Language Models
University of Southern California
"Many Are My Names": The Anatomy of the Assistant and Its Personas via Sparse Autoencoders
Aug 8, 2026Adelaide Danilov, Aria Nourbakhsh, Oleksandr Marchenko Breneur +1Large Language Model-Based Role-Play Simulation Personality Modeling in Language Models
Department of Computer Science, Faculty of Science, Technology and Medicine, University of Luxembourg
Looking in the Mirror: Introspecting Side-Effect Misalignments Induced by Fine-Tuning
Aug 5, 2026Kotaro Yoshida, Laura Gomezjurado Gonzalez, Yukinori Yamamoto +3LLM Auditing Language Model Introspection
Institute of Science Tokyo · 2ZOZO Research · 3Stanford University +2
Sample More, Reflect Less: Self-Refine and Reflexion Lose to Repeated Sampling at Equal Token Cost, from 1.5B to 7B
Jul 30, 2026Iliya MirzaeiLanguage Model Self-Assessment LLM Self-Refinement
Department of Computer Science Stony Brook University
What, Where, and How: Disentangling the Roles of Task, Language, and Model in Code Model Representations
Jul 23, 2026Piotr WilamCircuit Discovery Language Model Introspection
University College London
Operational Proto-Introspection in Looped Language Models: Process-Quality Taps, Executable Branching, and the Readout-Control Boundary
Jul 20, 2026Jan KirinLanguage Model Introspection Language Model Self-Assessment
Graded Entity-Familiarity Readouts in Language Models: Polish Adaptation, Cross-Language Robustness, and Refusal Steering
Jul 15, 2026Grzegorz BrzezinkaMultilingual Language Model Evaluation Language Model Introspection
Prosit AS
Self-Reference in Large Language Models: The Introspection Threshold for Recursive Self-Improvement
Jul 5, 2026Jiang Zhang, Bing Yuan, Qian ZhangLanguage Model Introspection Language Model Self-Improvement
School of Systems Science, Beijing Normal University · Swarma Research
Revealing Hidden Model Behaviors with Task-Specific Self-Reports
Jul 3, 2026Taras Kutsyk, Bartosz ZielińskiLLM Auditing Language Model Introspection
1Jagiellonian University, Faculty of Mathematics and Computer Science · 2Jagiellonian University, Doctoral School of Exact and Natural Sciences
Introspective Coupling: Self-Explanation Training Tracks Behavioral Change Despite Fixed Supervision
Jun 30, 2026Zifan Carl Guo, Laura Ruis, Jacob Andreas +1Counterfactual Explanations Faithfulness of Language Model Explanations
MIT EECS
Non-Gaussianity Reveals Interpretable Directions in Language Models
Jun 10, 2026Sida Liu, Feijiang Han, Sarah Wiegreffe +1Independent Component Analysis LLM Interpretability
Independent Researcher · University of Maryland
Can LLMs Introspect? A Reality Check
May 25, 2026Shashwat Singh, Tal Linzen, Shauli RavfogelLLM Evaluation LLM Interpretability
Center for Data Science New York University
From Simulation to Enaction: Post-trained language models recognize and react to their own generations
May 25, 2026Asvin G., Jack LindseyLanguage Model Introspection LLM Uncertainty Estimation
Institute for Advanced Study, Princeton · Anthropic
Tool Calling is Linearly Readable and Steerable in Language Models
May 8, 2026Zekun Wu, Ze Wang, Seonglae Cho +4AI Agent Reliability Tool-Augmented Language Model Agents
University College London · Holistic AI · Imperial College London
TokenScope: Token-Level Explainability and Interpretability for Code-Oriented Tasks in Large Language Models
Apr 30, 2026Amirreza Esmaeili, Fatemeh FardLLM Interpretability Code Generation
University of British Columbia Kelowna, British Columbia, Canada
Language models recognize dropout and Gaussian noise applied to their activations
Apr 19, 2026Damiano Fornasiere, Mirko Bronzi, Spencer Kitts +3LLM Interpretability Language Model Introspection
LawZero
Introspection Adapters: Training LLMs to Report Their Learned Behaviors
Apr 18, 2026Keshav Shenoy, Li Yang, Abhay Sheshadri +4LLM Auditing Language Model Introspection
Anthropic Fellows · Ashwood Centre on AI Science and Policy, University of Cambridge · Anthropic
Masked by Consensus: Disentangling Privileged Knowledge in LLM Correctness
Apr 14, 2026Tomer Ashuach, Shai Gretz, Yoav Katz +2Language Model Introspection Language Model Self-Assessment
Technion – Israel Institute of Technology · IBM Research · Kempner Institute, Harvard University
ProbeLogits: Kernel-Level LLM Inference Primitives for AI-Native Operating Systems
Apr 13, 2026Daeyeon SonAI Safety LLM Guardrails
Independent Researcher Republic of Korea
Do Language Models Know When They'll Refuse? Probing Introspective Awareness of Safety Boundaries
Mar 31, 2026Tanay GondilLanguage Model Introspection Refusal Behavior in Language Models
Purdue University
InnerPond: Fostering Inter-Self Dialogue with a Multi-Agent Approach for Introspection
Mar 29, 2026Hayeon Jeon, Dakyeom Ahn, Sunyu Pang +5Language Model Introspection Multi-Agent LLM Systems
hci+d lab. Seoul National University Seoul, Republic of Korea · Department of Communication Seoul National University Seoul, Republic of Korea

Oct 12, 2026J/K move · Enter open · S save

A Sense of Proprioception in Language Models

Jonas Geiping

ELLIS Institute Tübingen · Max Planck Institute for Intelligent Systems · Tübingen AI Center

Language Model Introspection Metacognition in Language Models

Open paper AskPDF arXiv

In humans, proprioception corresponds to the sense of self-movement or position, e.g. understanding the position of one's arm. This sense is often less well-understood and in popular understanding sometimes conflated with introspection. But, notably, proprioception is a separate sense, with dedicated receptors and verifiable outcomes. On the other hand though, language models cannot observe their own internal states during training (and for example answer the question which experts activated in a certain layer of an MoE) and so cannot accurately report on them. Yet, such information is, by virtue of being internal to the model, both accessible, and readily constructed into verifiable signals. We argue that this provides the basis for a digital analog of proprioception. As such, we train models through SFT and on-policy training to equip them with this ability. We show that modern models can be trained to understand a range of subtypes of proprioception, such as routing decision awareness, token entropy, decision depth or logit lens information, without affecting model capabilities. We then run these models through a broad evaluation of situational awareness, legibility, and epiphenomenality, to evaluate in what ways access to proprioception changes model behavior.