Language Model Calibration

Follow this topic Ask about this topic

Momentum

43 papers in the last four weeks, up 330% on the four weeks before. 0.4% of all new papers.

Jul 13Week of Sep 28

Latest papers 247

All topics

Case-Based Calibration of Adaptive Reasoning and Execution for LLM Tool Use
May 14, 2026Renning Pang, Tian Lan, Leyuan Liu +3Language Model Calibration LLM Tool Use
University of Electronic Science and Technology of China, Chengdu 611731, China
Inducing Artificial Uncertainty in Language Models
May 13, 2026Sophia Hager, Simon Zeng, Nicholas AndrewsConfidence Estimation in Language Models Language Model Calibration
Johns Hopkins University · Microsoft
Discovery of Hidden Miscalibration Regimes
May 13, 2026Katarzyna Kobalczyk, Mihaela van der SchaarPost-Hoc Calibration Language Model Calibration
University of Cambridge
Training Large Language Models to Predict Clinical Events
May 12, 2026Benjamin Turtel, Paul Wilczewski, Kris SkotheimHealthcare Language Model Calibration
Lightning Rod Labs
ORCE: Order-Aware Alignment of Verbalized Confidence in Large Language Models
May 12, 2026Chen Li, Xiaoling Hu, Songzhu Zheng +2Confidence Estimation in Language Models Language Model Calibration
Stony Brook University, NY, USA · Massachusetts General Hospital and Harvard Medical School, MA, USA · Morgan Stanley, NY, USA
OGLS-SD: On-Policy Self-Distillation with Outcome-Guided Logit Steering for LLM Reasoning
May 12, 2026Yuxiao Yang, Xiaoyun Wang, Weitong ZhangLanguage Model Calibration Language Model Distillation
UNC Chapel Hill · NVIDIA
Assessing and Mitigating Miscalibration in LLM-Based Social Science Measurement
May 12, 2026Jinyuan Wang, Ningyuan Deng, Yi YangLanguage Model Calibration LLM Reliability
The Hong Kong University of Science and Technology
Probabilistic Calibration Is a Trainable Capability in Language Models
May 12, 2026Davide Baldelli, Sruthi Kuriakose, Maryam Hashemzadeh +2LLM Fine-Tuning Controllable Text Generation
Chandar Research Lab · Mila – Quebec AI Institute · LAMA-WeST Lab +3
ROMER: Expert Replacement and Router Calibration for Robust MoE LLMs on Analog Compute-in-Memory Systems
May 12, 2026Wenyong Zhou, Yuannuo Feng, Yizhe Chen +6Compute-in-Memory Mixture-of-Experts Language Models
The Department of Electrical and Computer Engineering, The University of Hong Kong, Hong Kong · The School of Integrated Circuit Science and Engineering, Beihang University, Beijing, China
Sampling More, Getting Less: Calibration is the Diversity Bottleneck in LLMs
May 11, 2026Amin Banayeeanzade, Qingchuan Yang, Dhruv Tarsadiya +6LLM Evaluation Language Model Decoding
♠University of Southern California · ♣Capital One
Training-Free Cultural Alignment of Large Language Models via Persona Disagreement
May 11, 2026Huynh Trung Kiet, Dao Sy Duy Minh, Tuan Nguyen +5LLM Alignment Cultural Alignment
Faculty of Information and Technology, University of Science, Vietnam National University, Ho Chi Minh City, Vietnam. · Department of Computer Science, University of Warwick, Coventry, United Kingdom. · School of Computing, Engineering and Digital Technologies, Teesside University, United Kingdom.
ConQuR: Corner Aligned Activation Quantization via Optimized Rotations for LLMs
May 11, 2026Chayne Thrash, Ali Abbasi, Soheil KolouriRotation-Based Quantization LLM Inference Acceleration
Department of Computer Science Vanderbilt University
Auditing Multimodal LLM Raters: Central Tendency Bias in Clinical Ordinal Scoring
May 11, 2026Jiaqing Zhang, Sandeep Elluri, Bhanu Cherukuvada +7LLM Auditing Multimodal Large Language Models
Department of Electrical & Computer Engineering · Department of Computer and Information Science and Engineering · Department of Clinical and Health Psychology +1
Task-Aware Calibration: Provably Optimal Decoding in LLMs
May 11, 2026Tim Tomov, Dominik Fuchsgruber, Rajeev Verma +1Language Model Decoding MBR Decoding
School of Computation, Information & Technology, Technical University of Munich · Munich Data Science Institute · Munich Center for Machine Learning +1
UCCI: Calibrated Uncertainty for Cost-Optimal LLM Cascade Routing
May 11, 2026Varun KotteCost-Aware Inference Language Model Calibration
Independent Researcher.
The Metacognitive Probe: Five Behavioural Calibration Diagnostics for LLMs
May 11, 2026Rafael C. T. OliveiraLanguage Model Calibration LLM Uncertainty Estimation
Independent Researcher
The Silent Vote: Improving Zero-Shot LLM Reliability by Aggregating Semantic Neighborhoods
May 10, 2026Sanket Badhe, Priyanka Tiwari, Deep ShahConstrained Decoding Zero-Shot Text Classification
Google
Calibrate, Don't Curate: Label-Efficient Estimation from Noisy LLM Judges
May 10, 2026Yanran LiLLM Evaluation LLM-as-a-Judge
Department of Biostatistics Columbia University
NICE FACT: Diagnosing and Calibrating VLMs in Quantitative Reasoning for Kinematic Physics
May 8, 2026Jian Lan, Zhicheng Liu, Xinpeng Wang +5VLM Evaluation Physical Reasoning
University of Munich (LMU), Germany 2Munich Center of Machine Learning3 Sichuan University
A Semantic-Sampling Framework for Evaluating Calibration in Open-Ended Question Answering
May 8, 2026Zhanliang Wang, Jiancong Xiao, Ruochen Jin +3LLM Evaluation Confidence Estimation in Language Models
University of Pennsylvania, Philadelphia, PA · Dartmouth College, Hanover, NH
Beyond Confidence: Rethinking Self-Assessments for Performance Prediction in LLMs
May 8, 2026Sree Bhattacharyya, Samarth Khanna, Leona Chen +3LLM Evaluation Confidence Estimation in Language Models
Department of Informatics and Intelligent Systems, College of Information Sciences, The Pennsylvania State University. · Department of Computer Science and Engineering, College of Engineering, The Pennsylvania State University.
Bayesian Fine-tuning in Projected Subspaces
May 8, 2026Viktar Dubovik, Patryk Marszałek, Jacek Tabor +1Bayesian Neural Networks Fine-Tuning
Jagiellonian University, Kraków, Poland · Doctoral School of Exact and Natural Sciences, Jagiellonian University, Kraków, Poland
Confidence-Aware Alignment Makes Reasoning LLMs More Reliable
May 8, 2026Kejia Chen, Jiawen Zhang, Yihong Wu +5LLM Alignment Direct Preference Optimization
Zhejiang University · Université de Montréal · Sun Yat-sen University +1
Bias and Uncertainty in LLM-as-a-Judge Estimation
May 7, 2026James FiedlerLLM Evaluation LLM-as-a-Judge
Indeed Inc.
A Reproducible Optimisation Protocol for Calibrating Prompt-Based Large Language Model Workflows in Evidence Synthesis
May 7, 2026Teo SusnjakLarge Language Model-Guided Optimization Language Model Calibration
School of Mathematical and Computational Sciences Massey University, Auckland, New Zealand
Log-Likelihood, Simpson's Paradox, and the Detection of Machine-Generated Text
May 7, 2026Tom Kempton, Viktor Drobnyi, Maeve Madigan +1AI-Generated Text Detection Language Model Calibration
Department of Mathematics University of Manchester · Risk and Security AI Lab Visa Inc.
Saliency-Aware Regularized Quantization Calibration for Large Language Models
May 7, 2026Yanlong Zhao, Xiaoyuan Cheng, Huihang Liu +6LLM Quantization Language Model Calibration
University of Science and Technology of China · University College London · 3Shanghai University of Finance and Economics +5
BitCal-TTS: Bit-Calibrated Test-Time Scaling for Quantized Reasoning Models
May 7, 2026Sai Babu Patarlapalli, Surya Teja AvvaruLLM Inference 4-Bit Quantization
Distilling Bayesian Belief States into Language Models for Auditable Negotiation
May 6, 2026Zongqi Cui, Baihan LinMulti-Agent Negotiation Opponent Modeling
Emory University · Icahn School of Medicine at Mount Sinai
EQUITRIAGE: A Fairness Audit of Gender Bias in LLM-Based Emergency Department Triage
May 5, 2026Richard J. Young, Alice M. MatthewsGender Bias in Language Models Clinical Triage
University of Nevada, Las Vegas, Lee Business School, Las Vegas, NV, USA · DeepNeuro AI, Las Vegas, NV, USA · Concorde Career Colleges, Dept. of Cardiovascular and Medical Diagnostic Sonography

May 14, 2026, cs.AIJ/K move · Enter open · S save

Case-Based Calibration of Adaptive Reasoning and Execution for LLM Tool Use

Renning Pang, Tian Lan, Leyuan Liu, Piao Tong+2

University of Electronic Science and Technology of China, Chengdu 611731, China

Language Model Calibration LLM Tool Use RL for Tool Use

Open paper AskPDF arXiv

Tool use extends large language models beyond parametric knowledge, but reliable execution requires balancing appropriate reasoning depth with strict structural validity. We approach this problem from a case-based perspective to present CAST, a case-driven framework that treats historical execution trajectories as structured cases. Instead of reusing raw exemplar outputs, CAST extracts case-derived signals to identify complexity profiles for estimating optimal reasoning strategies, alongside failure profiles to map likely structural breakdowns. The framework translates this knowledge into a fine-grained reward design and adaptive reasoning, enabling the model to autonomously internalize case-based strategies during reinforcement learning. Experiments on BFCLv2 and ToolBench demonstrate that CAST improves both schema-faithful execution and task-level tool-use success while reducing unnecessary deliberation. The approach achieves up to 5.85 percentage points gain in overall execution accuracy and reduces average reasoning length by 26%, significantly mitigating high-impact structural errors. Ultimately, this demonstrates how historical execution cases can provide reusable adaptation knowledge for calibrated tool use.

Momentum

Related topics

Latest papers 247