Metacognition in Language Models

Ask about this topic

Overview

The field asks whether language models can tell what they know, detect errors or changes in their internal processing, and use such assessments to decide when to revise, abstain, retrieve information, or spend more computation. Studies measure confidence calibration, error detection, self-prediction, and tool-use decisions on question-answering, reasoning, and agent benchmarks; methods include verbal confidence and token probabilities, hidden-state probes and causal interventions, and reinforcement-trained controllers. Structured reflection and adaptive control can improve accuracy or efficiency in some settings, but confidence and self-reports often fail to track correctness or transfer reliably. A central open tension is whether models have usable access to their internal states, or whether apparent metacognition reflects surface cues, task shortcuts, or external scaffolding.

Latest papers

CardsList

128 papers18 papers in the last four weeks, up 50% on the four weeks before. 0.1% of all new papers.

  1. Before You Interpret the Profile: Validity Scaling for LLM Metacognitive Self-Report

    Apr 20, 2026Jon-Paul CacioliPsychometric ValidationLLM Uncertainty Estimation

    Independent Researcher, Melbourne, Australia

  2. Beyond Meta-Reasoning: Metacognitive Consolidation for Self-Improving LLM Reasoning

    Apr 19, 2026Ziqing Zhuang, Linhai Zhang, Jiasheng Si +2Continual Learning for LLMsMeta-Learning

    Southeast University · King’s College London · Qilu University of Technology (Shandong Academy of Sciences) +1

  3. MEDLEY-BENCH: Scale Buys Evaluation but Not Control in AI Metacognition

    Apr 17, 2026Farhad Abtahi, Abdolamir Karbalaie, Eduardo Illueca-Fernandez +1Belief RevisionLLM Evaluation

    Department of Clinical Science, Intervention and Technology (CLINTEC), Karolinska Institutet, 17177 Stockholm, Sweden · Department of Clinical Physiology, Karolinska University Hospital, 17176 Stockholm, Sweden · Department of Biomedical Engineering and Health Systems, KTH Royal Institute of Technology, 14157 Huddinge, Sweden +2

  4. The Metacognitive Monitoring Battery: A Cross-Domain Benchmark for LLM Self-Monitoring

    Apr 17, 2026Jon-Paul CacioliLLM EvaluationLanguage Model Calibration

    Independent Researcher Melbourne, Australia

  5. MIRROR: A Hierarchical Benchmark for Metacognitive Calibration in Large Language Models

    Apr 15, 2026Jason Z WangMetacognition in Language ModelsLanguage Model Self-Assessment

    Independent

  6. Quantisation Reshapes the Metacognitive Geometry of Language Models

    Apr 10, 2026Jon-Paul CacioliLanguage Model Self-AssessmentMetacognition in Language Models

    Independent Researcher, Melbourne, Australia

  7. Act Wisely: Cultivating Meta-Cognitive Tool Use in Agentic Multimodal Models

    Apr 9, 2026Shilin Yan, Jintao Tong, Hongwei Xue +6Tool UseMultimodal Agents

    Accio Team, Alibaba Group · Huazhong University of Science and Technology

  8. PRISM-MCTS: Learning from Reasoning Trajectories with Metacognitive Reflection

    Apr 7, 2026Siyuan Cheng, Bozhong Tian, YanChao Hao +1Inference-Time SearchEfficient Language Model Reasoning

    Tencent PCG

  9. LLMs Should Express Uncertainty Explicitly

    Apr 7, 2026Junyu Guo, Shangding Gu, Ming Jin +2Language Model Self-AssessmentLLM Uncertainty Estimation

    University of California, Berkeley · Virginia Tech

  10. Verbalizing LLMs' assumptions to explain and control sycophancy

    Apr 3, 2026Myra Cheng, Isabel Sieh, Humishka Zope +7LLM SycophancyHuman-AI Interaction

    Stanford University · University of Oxford · The University of Texas at Austin

  11. Internal Knowledge Without External Expression: Probing the Generalization Boundary of a Classical Chinese Language Model

    Mar 31, 2026Jiuting Chen, Yuan Lian, Hao Wu +4Metacognition in Language ModelsFactual Knowledge in Language Models

    Eaglewood Japan Co., Ltd.

  12. CoT2-Meta: Budgeted Metacognitive Control for Test-Time Reasoning

    Mar 30, 2026Siyuan Ma, Bo Gao, Zikai Xiao +6Efficient Language Model ReasoningMetacognition in Language Models

    Nanyang Technological University · Carnegie Mellon University · Zhejiang University +5

  13. Simulating Human Cognition: Heartbeat-Driven Autonomous Thinking Activity Scheduling for LLM-based AI systems

    Mar 28, 2026Hong SuLLM Agent Workflow OptimizationCognitive Architectures for AI Agents

    School of Computer Science, Chengdu University of Information Technology, Chengdu, China

  14. Selective Deficits in LLM Mental Self-Modeling in a Behavior-Based Test of Theory of Mind

    Mar 27, 2026Christopher AckermanWorking Memory in Language ModelsTheory of Mind

  15. Do LLMs Know What They Know? Measuring Metacognitive Efficiency with Signal Detection Theory

    Mar 26, 2026Jon-Paul CacioliConfidence Estimation in Language ModelsLanguage Model Calibration

    Independent Researcher, Melbourne, Australia

  16. Supervising Ralph Wiggum: Exploring a Metacognitive Co-Regulation Agentic AI Loop for Engineering Design

    Mar 25, 2026Zeda Xu, Nikolas Martelaro, Christopher McCombLLM Agent Workflow OptimizationMetacognition in Language Models

    Department of Mechanical Engineering, Carnegie Mellon University, Pittsburgh, PA 15213 · Human-Computer Interaction Institute, Carnegie Mellon University, Pittsburgh, PA, 15213 · Department of Mechanical Engineering, Carnegie Mellon University, Pittsburgh, PA, 15213

  17. Causal Evidence that Language Models use Confidence to Drive Behavior

    Mar 23, 2026Dharshan Kumaran, Nathaniel Daw, Simon Osindero +2Metacognition in Language ModelsLLM Abstention

    Google DeepMind · Princeton University

  18. Mechanisms of Introspective Awareness

    Mar 22, 2026Uzay Macar, Li Yang, Atticus Wang +3Language Model IntrospectionMetacognition in Language Models

    1Anthropic Fellows Program · 2MIT · 3Constellation +1

  19. How do LLMs Compute Verbal Confidence

    Mar 18, 2026Dharshan Kumaran, Arthur Conmy, Federico Barbero +3Verbalized Confidence EstimationConfidence Estimation in Language Models

    Google DeepMind

  20. Me, Myself, and ππ : Evaluating and Explaining LLM Introspection

    Mar 17, 2026Atharv Naphade, Samarth Bhargav, Sean Lim +1Language Model IntrospectionSelf-Reflection in Language Model Agents

    Carnegie Mellon University

  21. GRPO and Reflection Reward for Mathematical Reasoning in Large Language Models

    Mar 14, 2026Zhijie WangSelf-Reflection in Language Model AgentsLLM Reasoning

    School of Computer Science and Engineering, Sun Yat-sen University(SYSU), Guangzhou, China

  22. Quality-Driven Agentic Reasoning for LLM-Assisted Software Design: Questions-of-Thoughts (QoT) as a Time-Series Self-QA Chain

    Mar 10, 2026Yen-Ku Liu, Yun-Cheng TsaiAI-Assisted Software DevelopmentSoftware Engineering

    Department of Technology Application and Human Resource Development, National Taiwan Normal University, Taipei, Taiwan

  23. Rescaling Confidence: What Scale Design Reveals About LLM Metacognition

    Mar 10, 2026Yuyang Dai, Yuxia WangConfidence Estimation in Language ModelsLanguage Model Calibration

    INSAIT

  24. Recursive Language Models Meet Uncertainty: The Surprising Effectiveness of Self-Reflective Program Search for Long Context

    Mar 7, 2026Keivan Alizadeh, Parshin Shojaee, Minsik Cho +1Long-Context Language Model InferenceRecursive Language Models

    Apple

  25. Emergent Introspection in AI is Content-Agnostic

    Mar 5, 2026Harvey Lederman, Kyle MahowaldLanguage Model IntrospectionMetacognition in Language Models

    Department of Philosophy, The University of Texas at Austin · Department of Linguistics, The University of Texas at Austin

  26. Architectural Proprioception in State Space Models: Thermodynamic Training Induces Anticipatory Halt Detection

    Mar 4, 2026Jay NoonState Space ModelsMetacognition in Language Models

  27. The Tool-Overuse Illusion: Why Does LLM Prefer External Tools over Internal Knowledge?

    Mar 3, 2026Yirong Zeng, Shen You, Yufei Liu +9LLM Tool UseRL for Tool Use

    Harbin Institute of Technology, SCIR · Huawei Technologies Co., Ltd · Peking University

  28. Know What You Know: Metacognitive Entropy Calibration for Verifiable RL Reasoning

    Feb 26, 2026Qiannian Zhao, Chen Yang, Jinhao Jing +5RL for Language Model ReasoningLarge Reasoning Models

  29. Distilling Metacognitive Behaviors through Agentic Editing

    Feb 26, 2026Ik-hwan Kim, Hyeongrok Han, Mingi Jung +5Efficient Language Model ReasoningMetacognition in Language Models

    Dept. of ECE, Seoul Nat’l Univ., Seoul, Korea · AI Center, Samsung Elec., Korea · AIIS, ASRI, INMC, ISRC, Interdisc. Prog. in AI, Seoul Nat’l Univ., Seoul, Korea

  30. Latent Introspection: Models Can Detect Prior Concept Injections

    Feb 23, 2026Theia Pearson-Vogel, Martin Vanek, Raymond Douglas +1Language Model IntrospectionMetacognition in Language Models

    ACS Research, CTS, Charles University