Language Model Calibration

Ask about this topic

Momentum

43 papers in the last four weeks, up 330% on the four weeks before. 0.4% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 247

All topics
CardsList

  1. What LLM Forecasters Know but Don't Say: Probing Internal Representations for Calibration and Faithfulness

    Jul 9, 2026Raphaël Sarfati, Pratyush Ranjan Tiwari, Siddharth Boppana +3CoT FaithfulnessLanguage Model Calibration

    goodfire · eternis

  2. Future Confidence Distillation in Large Language Models

    Jul 8, 2026Sahil KaleConfidence Estimation in Language ModelsLanguage Model Calibration

    University of California, Los Angeles Los Angeles, USA

  3. On the effectiveness of reward functions in reinforcement learning for confidence calibration of large language models

    Jul 5, 2026Chee Heng Tan, Zhuoyi Lin, Mehul Motani +1Reward HackingConfidence Estimation in Language Models

    School of Computing National University of Singapore · Institute of Advanced Intelligence and Computing Agency for Science, Technology and Research · Department of Electrical & Computer Engineering National University of Singapore

  4. Consistent but Miscalibrated: Evaluating LLM Limitations for Risk Communication in Natural Language

    Jul 4, 2026Diego Cerda-Mardini, Sarath Chandar, Sreenath MadathilLLM EvaluationProbability Calibration

    Faculty of Dental Medicine and Oral Health Sciences, McGill University, Montréal, QC, Canada · Chandar Research Lab, Polytechnique Montréal, Montréal, H3T 1J4, Canada · Mila – Québec Artificial Intelligence Institute, Montréal, H2S 3H1, Canada +2

  5. Will Scaling Improve Social Simulation with LLMs?

    Jul 2, 2026Caleb Ziems, William Held, Su Doga Karaca +3Opinion DynamicsLanguage Model Scaling Laws

    Stanford University · Open Athena

  6. Bayesian Sparse Low-Rank Adaptation for Large Language Model Uncertainty Estimation

    Jul 2, 2026Jijie Zhang, Zhe Ren, Quan Zhang +1Low-Rank AdaptationLanguage Model Calibration

    School of Artificial Intelligence, Jilin University · Michigan State University

  7. Scaling with Confidence: Calibrating Confidence of LLMs for Adaptive Test Time Scaling

    Jul 2, 2026Xuqing Yang, Yi Yuan, Shanzhe Lei +1LLM Inference EfficiencyRL for Language Models

    Shanghai Jiao Tong University · Southeast University · Shanghai AI Laboratory

  8. Beyond Activation Alignment:The Alignment-Diversity Tradeoff in Task-Aware LLM Quantization

    Jul 1, 2026Fei Wang, Chao Xue, Taoran Liu +3Mixed-Precision QuantizationLLM Quantization

    South China University of Technology · Ocean University of China · Shenzhen Campus of Sun Yat-sen University +1

  9. Harnessing the Latent Space: From Steering Vectors to Model Calibrators for Control and Trust

    Jun 30, 2026Nishant SubramaniLanguage Model SteeringLLM Interpretability

    Carnegie Mellon University, Language Technologies Institute

  10. Reinforcement Learning with Metacognitive Feedback Elicits Faithful Uncertainty Expression in LLMs

    Jun 30, 2026Gabrielle Kaili-May Liu, Avi Caciularu, Gal Yona +2RL for Language ModelsReinforcement Learning

    Yale University · Google Research

  11. Calibrating the Evaluator: Does Probability Calibration Mitigate Preference Coupling in LLM Agent Feedback Loops?

    Jun 30, 2026Zewen LiuLLM-as-a-JudgePairwise Preference Evaluation

    Qilu Institute of Technology, School of Software Engineering Tai’an, Shandong, China

  12. The Calibration Turn in AI-Assisted Research: A Conceptual and Methodological Framework for Evidence-Licensed Claims

    Jun 30, 2026Hongmin LiAI-Assisted Scientific ResearchLanguage Model Calibration

    School of Life Science and Technology, Institute of Science Tokyo, 2-12-1 Ookayama, Meguro-ku, Tokyo 152-8550, Japan · Department of Computational Biology and Medical Sciences, Graduate School of Frontier Sciences, The University of Tokyo, 5-1-5 Kashiwanoha, Kashiwa-shi, Chiba 277-8561, Japan

  13. An Empirical Study of Security Calibration in Large Language Models for Code

    Jun 30, 2026Mohammed Latif Siddiq, Md. Nafiu Rahman, Joanna C. S. SantosLLM SecurityLanguage Model Calibration

    University of Notre Dame Notre Dame, IN, USA · Brac University Dhaka, Bangladesh

  14. When Calibration Rankings Reverse: Accuracy-Controlled Evaluation for Fair Comparison of LLMs

    Jun 29, 2026Zhichao Yang, Caiqi Zhang, Ruihan Yang +3LLM EvaluationConfidence Estimation in Language Models

    Fudan University · University of Cambridge

  15. Using Large Language Models as Low-Cost Statistical Estimators for Human-Response Data

    Jun 29, 2026Haobo YangLanguage Model CalibrationStatistical Learning Theory

    Department of Computer Science and Engineering, SUSTech University

  16. Know Before You Fetch: Calibrated Retrieval-Budget Allocation for Retrieval-Augmented Generation

    Jun 29, 2026Zhe Dong, Fang Qin, Manish Shah +1Retrieval-Augmented GenerationSelective Prediction

    University of Maine at Presque Isle · Stanford University · Independent Researcher

  17. KbSD: Knowledge Boundary aware Self-Distillation for Behavioral Calibration in Agentic Search

    Jun 29, 2026Tao Feng, Xinke Jiang, Chao WuLanguage Model CalibrationAgentic RAG

    Zhejiang University, Hangzhou, China · Peking University, Beijing, China

  18. BaRA: Bayesian Adaptive Rank Allocation for Parameter-Efficient Fine-Tuning

    Jun 28, 2026Zhibin Duan, Yuhong Wang, Jiahong Fu +3Bayesian Neural NetworksLow-Rank Adaptation

    National Key Lab of Radar Signal Processing, Xidian University, Xi’an, Shaanxi 710071, China · School of Mathematics and Statistics, Xi’an Jiaotong University, Xi’an, Shaanxi, 710049, China

  19. The strength of clinical evidence is recoverable from language model representations but not from their stated grades

    Jun 27, 2026Soroosh Tayebi ArastehLLM InterpretabilityLanguage Model Calibration

    Lab for AI in Medicine, RWTH Aachen University, Aachen, Germany · Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Aachen · Department of Urology, Stanford University, Stanford, CA, USA +1

  20. Beyond the Mean: Three-Axis Fidelity for Aligning LLM-Based Survey Simulators from Small Pilot Data

    Jun 27, 2026Eun Cheol Choi, Youngrae Kim, Prabhu Pugalenthi +2LLM AlignmentLLM Fine-Tuning

    University of Southern California, CA, USA.

  21. Just how sure are you? Improving Verbalized Uncertainty Calibration in Medical VQA

    Jun 25, 2026Eren Senoglu, Federico Toschi, Nicolo Brunello +2Medical VQALanguage Model Calibration

    Politecnico di Milano

  22. ParaPairAudioBench: Paralinguistic Pairwise Audio Benchmark for LALM-as-a-Judge

    Jun 23, 2026Jisu Jeon, Seungyeon Jwa, Joosung Lee +6Audio-Language Model EvaluationPairwise Comparison

    Hongik University, South Korea · Seoul National University, South Korea · NAVER Cloud, South Korea +1

  23. Don't Go Breaking My LLM: The Impact of Pruning Attention Layers on Explanation Faithfulness and Confidence Calibration

    Jun 23, 2026Pietro Tropeano, Maria Maistro, Tuukka Ruotsalo +1LLM PruningLLM Interpretability

    University of Copenhagen Copenhagen, Denmark · LUT University Lahti, Finland

  24. CALIBER: Calibrating Confidence Before and After Reasoning in Language Models

    Jun 23, 2026Conor Finlay, Joshua Kurien, Saurabh Dash +2Confidence Estimation in Language ModelsLanguage Model Calibration

    1Cohere Labs

  25. Bayesian Adaptation Gym: A Benchmark for the Bayesian Low-Rank Adaptation of Multi-Modal Language Models

    Jun 20, 2026Colin Samplawski, Ramneet Kaur, Manoj Acharya +2Distribution Shift RobustnessMultimodal Large Language Models

    Neuro-Symbolic Computing and Intelligence Research Group, Computer Science Laboratory, SRI International

  26. The Score Granularity Gap in Black-Box LLM Classification: A Comparative Study of Confidence Constructions

    Jun 20, 2026Ao Sun, Tian Sun, Jiaxing GengLLM EvaluationSelective Prediction

  27. Latent Confidence Alignment for LLM Self-Assessment

    Jun 20, 2026Ting-Yu Chen, Tingting Yu, Pei-Cing Huang +3LLM EvaluationConfidence Estimation in Language Models

    Department of Information Management National Sun Yat-Sen University Kaohsiung, Taiwan · Kaohsiung Medical University Hospital Kaohsiung Medical University Kaohsiung, Taiwan

  28. CalVerT: Augmenting Agents with Calibrated Verifier Telemetry Improves Action and Learning in Knowledge-Intensive Tasks

    Jun 19, 2026Ashwin Vinod, Ying Ding, Elias Stengel-EskinLLM Answer VerificationLLM Agent Verification

    The University of Texas at Austin

  29. Calibration Without Comprehension: Diagnosing the Limits of Fine-Tuning LLMs for Vulnerability Detection in Systems Software

    Jun 18, 2026Arastoo Zibaeirad, Marco VieiraLLM EvaluationSoftware Vulnerability Detection

    University of North Carolina at Charlotte, Charlotte, NC, USA

  30. Confidence Calibration for Multimodal LLMs: An Empirical Study through Medical VQA

    Jun 18, 2026Yuetian Du, Yucheng Wang, Ming Kong +4Multimodal QALanguage Model Calibration

    College of Computer Science and Technology, Zhejiang University · School of Computer Science and Technology, Xidian University · Zhihui Medical Technology (Shanghai) Co., Ltd., Shanghai, China +1