Language Model Self-Assessment

Overview

Language model self-assessment asks whether a model can judge its own answer quality, uncertainty, capabilities, errors, and behavior—and use those judgments to decide when to answer, abstain, verify, or revise. Methods range from verbal confidence, token probabilities, and agreement across samples to hidden-state probes, structured self-questioning, and training with reinforcement learning or self-generated rubrics. Work evaluates these signals on factual and specialized question answering, math and code reasoning, safety tasks, and agent interactions, measuring calibration, selective risk, and whether reflection improves outcomes. Self-assessment can support routing and refinement, but may be sensitive to prompts, miss shared errors, and fail to track objective correctness or actual behavior, making external or environment-grounded checks important.

Latest papers

CardsList

100 papers11 papers in the last four weeks, down 27% on the four weeks before. 0.1% of all new papers.

  1. Done Is Not Correct: Measuring Silent Failures and Self-Verification Calibration When LLM Agents Take CAD Actions

    Nov 6, 2026Carson Rodrigues, Clive Rodrigues, Aravind Reddy GText-to-CAD GenerationLLM Agent Reliability

  2. Circe: Training-Free LLM Self-Evaluation via Component Interaction Consistency

    Nov 5, 2026Shuyi Miao, Zheng Li, Xu Xie +4Language Model Self-AssessmentLLM Evaluation

  3. LLMs Don't Know What They Know: Benchmarking Temporal Self-Knowledge for Financial Backtesting

    Nov 4, 2026Lingfei Qian, Yan Wang, Xueqing Peng +12Financial Language Model EvaluationLanguage Model Self-Assessment

  4. Calibration-Constrained Diagnostics for Evaluating LLM Self-Evaluation

    Nov 1, 2026Jae Oh Woo, Mengdie Flora Wang, Rahul Ghosh +2Language Model Self-AssessmentConfidence Estimation in Language Models

  5. On the Limits of Test-Time Compute for Self-Evaluation

    Oct 30, 2026Anirudh Buvanesh, Khaled Rouissi, Sujin Yun +6Language Model Self-AssessmentLanguage Model Self-Verification

  6. The Quine Test: How Much of a Model's Self Survives Its Own Description?

    Oct 28, 2026Bibin Babu, Ivan P. Yamshchikov, Aleksei ShpilmanLanguage Model IntrospectionLanguage Model Self-Assessment

  7. LADDER: Diagnosing Domain-Level Self-Assessment Under Noisy Feedback

    Oct 26, 2026Zhenshuai Yin, Jianshu Liang, Xuyue Xiong +6Language Model Self-AssessmentMetacognition in Language Models

  8. Disentangling Question, Response, and Verifier Languages in Multilingual P(True) Self-Evaluation

    Oct 23, 2026Jewon Yeom, Jaewon Sok, Seonghyeon Park +3Language Model Self-AssessmentMultilingual Language Model Evaluation

  9. Structural and Representational Agency in LLM-Driven Timbre Generation and Interpolation

    Oct 8, 2026Joseph Cameron, Alan BlackwellAudio GenerationLanguage Model Self-Assessment

  10. Identifying Introspection From the Inside

    Oct 5, 2026David I. Atkinson, Dillon Plunkett, David BauLLM InterpretabilityLanguage Model Introspection

  11. Self-Evaluating Recursive Agents

    Oct 4, 2026TianYi Lyu, Xiaozhe Li, Yang Li +7LLM Agent TrainingRecursive Language Models

  12. Representational Control over Self-Report & Behavior Coherence in LLM Risk-Taking

    Oct 2, 2026Rafal Kocielnik, Peiyang Song, Pengrui Han +4Language Model Self-AssessmentLLM Interpretability

  13. LUMOS: Tracing Parametric Knowledge from Training Data to Behavioral Outputs in LLMs

    Oct 2, 2026Seoyeon Ye, Gayoung Kim, Jiyoung Hong +2Factual Knowledge in Language ModelsLLM Evaluation

  14. Also Small Models Can Reasonably Self-Evaluate Their Confidence

    Sep 30, 2026Idil Kapikiran, Thomas Decker, Thomas RunklerConfidence Estimation in Language ModelsSmall Language Models

  15. Scoring the Wrong Question: Readout Failures in Constrained-Option Evaluation

    Sep 27, 2026Jiaxuan Guo, Kejia Zhang, Shuo Xin +3LLM EvaluationConfidence Estimation in Language Models

  16. Self-Reports Do Not Identify Self-Models: An Identifiability Test for Counterfactual Reports

    Sep 26, 2026Phongsakon Mark Konrad, Toygar Tanyel, Serkan AyvazLanguage Model Self-AssessmentMetacognition in Language Models

  17. Audio LLMs Know When They Can't Hear You

    Sep 24, 2026Amirhosein Javadi, Richa Dixit, Mehrdad Farajtabar +3Audio-Language Model EvaluationLanguage Model Self-Assessment

  18. When Should LLMs Abstain? Chain-of-Self-Questioning for Selective Risk Control

    Sep 15, 2026Ali ŞenolSelective PredictionLLM Prompting