LLM Uncertainty Estimation

LLM: Large Language Model

Large language model uncertainty estimation studies how to tell when an answer, reasoning step, or action may be unreliable, and whether the cause is missing knowledge, ambiguous input, or random variation. Methods include sampling and semantic agreement, token probabilities and entropy, confidence calibration, probes of hidden representations, and Bayesian or evidential models; conformal prediction adds statistical guarantees for abstention or prediction sets. These signals are used to decide when to answer, ask for clarification, retrieve evidence, spend more computation, or defer to a person, with tests spanning factual QA, reasoning, medicine, code, and multimodal tasks. A central challenge is that no signal works reliably across settings: confidence can be miscalibrated or reflect commitment rather than correctness, and agreement may preserve shared errors, so methods need task-specific validation and robust evaluation under ambiguity and distribution shift.

Latest papers

403 papers38 papers in the last four weeks, up 9% on the four weeks before. 0.2% of all new papers.

Aligning LLMs with Biomedical Knowledge using Balanced Fine-Tuning
Nov 26, 2025Zhenchao Tang, Fang Wang, Haohuai He +13LLM Alignment LLM Fine-Tuning
Tencent AI for Life Sciences Lab, Shenzhen, China. · Sun Yat-sen University, Shenzhen, China. · The Hong Kong Polytechnic University, Hong Kong SAR, China. +2
MedBayes-Lite: A Clinical Uncertainty Governance Layer for Risk-Aware Medical Decision Support
Nov 20, 2025Elias Hossain, Md Mehedi Hasan Nipu, Maleeha Sheikh +5Selective Prediction Healthcare
College of Engineering and Computer Science, University of Central Florida, Orlando, FL 32816, USA · Department of Computer Science and Engineering, North South University, Dhaka 1229, Bangladesh · Department of Electrical and Computer Engineering, Purdue University Fort Wayne, Fort Wayne, IN 46805, USA +4
Sequential Bayesian Evaluation of Large Language Model Behavior
Nov 4, 2025Saatvik Kher, Shang Wu, Rachel Longjohn +2LLM Evaluation Language Model Generation Evaluation
Department of Computer Science, University of California, Irvine · Department of Statistics, University of California, Irvine
When Guessing is Rewarded: Rethinking Language Model Evaluation with Distributional Uncertainty Scoring
Oct 5, 2025Thomas F BurnsLLM Evaluation Language Model Generation Evaluation
Aleph Alpha Research
Evidence for Limited Metacognition in LLMs
Sep 25, 2025Christopher AckermanLLM Evaluation LLM Uncertainty Estimation
Geometric Uncertainty for Detecting and Correcting Hallucinations in LLMs
Sep 17, 2025Edward Phillips, Sean Wu, Soheila Molaei +3Uncertainty Quantification LLM Hallucination Mitigation
Department of Engineering Science, University of Oxford · GlaxoSmithKline · Oxford Suzhou Centre for Advanced Research
PiCSAR: Probabilistic Confidence Selection And Ranking for Reasoning Chains
Aug 29, 2025Joshua Ong Jun Leang, Zheng Zhao, Aryo Pradipta Gema +7Confidence Estimation in Language Models Best-of-N Sampling
Imperial College London · University of Edinburgh · UCL
Toward Efficient Uncertainty in LLMs through Evidential Knowledge Distillation
Jul 24, 2025Lakshmana Sri Harsha Nemani, P. K. Srijith, Tomasz KuśmierczykLLM Uncertainty Estimation Uncertainty-Aware Knowledge Distillation
Department of Computer Science and Engineering, Indian Institute of Technology Hyderabad, India · Jagiellonian University, Faculty of Mathematics and Computer Science, Kraków, Poland
LLM Abstention Can Be a Prompt Artifact, in Addition to Genuine Uncertainty
Jul 22, 2025Zipeng Ling, Shuliang Liu, Yuehao Tang +8LLM Abstention LLM Uncertainty Estimation
The Hong Kong University of Science and Technology (Guangzhou) · University of Alberta · Alberta Machine Intelligence Institute +5
On the Effect of Uncertainty on Layer-wise Inference Dynamics
Jul 9, 2025Sunwoo Kim, Haneul Yoo, Alice OhLLM Inference LLM Interpretability
School of Computing, KAIST, Daejeon, South Korea.
Popular but Wrong: Understanding and Mitigating LLM Overconfidence through Knowledge Popularity
May 23, 2025Shiyu Ni, Keping Bi, Jiafeng Guo +1LLM Uncertainty Estimation LLM Hallucination Mitigation
CAS Key Lab of Network Data Science and Technology, ICT, CAS · State Key Laboratory of AI Safety · University of Chinese Academy of Sciences
Label-Confidence-Aware Uncertainty Estimation in Natural Language Generation
Dec 10, 2024Qinhong Lin, Yinglun Feng, Yuhao Zhang +2Uncertainty Quantification LLM Uncertainty Estimation
School of Cyberspace Security, Beijing University of Posts and Telecommunications, Beijing, China · QuanCheng Laboratory, Jinan, China
Prompting the Unknown: Understanding Response Uncertainty in Large Language Models
Jul 20, 2024Ze Yu Zhang, Arun Verma, Finale Doshi-Velez +1LLM Prompting LLM Reliability
Department of Computer Science, National University of Singapore, Republic of Singapore · Singapore-MIT Alliance for Research and Technology Centre · Harvard University

Nov 26, 2025J/K move · Enter open · S save

Aligning LLMs with Biomedical Knowledge using Balanced Fine-Tuning

Zhenchao Tang, Fang Wang, Haohuai He, Jiale Zhou+12

Tencent AI for Life Sciences Lab, Shenzhen, China. · Sun Yat-sen University, Shenzhen, China. · The Hong Kong Polytechnic University, Hong Kong SAR, China.+2

LLM Alignment LLM Fine-Tuning LLM Uncertainty Estimation LLM Post-Training Medical LLMs

Open paper AskPDF arXiv

Engineering LLMs to accelerate life sciences research requires a robust alignment with biomedical knowledge. We observe that biomedical text exhibits a fundamentally different uncertainty structure from general text: dense low-confidence runs encode epistemic knowledge gaps (dense causal chains, rare entities) rather than the sparse aleatoric stylistic variation typical of general text. Based on this discovery, we propose Balanced Fine-Tuning (BFT), a dual-scale post-training method that combines group-normalized token reweighting with sequence-level reallocation toward knowledge-dense samples exhibiting dense epistemic uncertainty. Across medical evaluation, biological reasoning, sparse-reward RL, and biological representation tasks, BFT provides more consistent gains than SFT and DFT under a shared training setup. When replacing the default closed-source backbones in GeneAgent (GPT-4o) and VCWorld (Gemini-2.5-Flash), the BFT-aligned 70B model delivers stronger performance across biological process reasoning and chemical perturbation prediction. Critically, all BFT variants further improve after subsequent GRPO with sparse rewards, while SFT and DFT degrade, suggesting that epistemic-aware post-training provides a more robust policy initialization. Beyond text generation, BFT-aligned LLMs produce more accurate and professional biomedical profile texts; after encoding these profiles with a text embedding model, the resulting representations support gene-level, cell-level, and perturbation-response tasks, suggesting that BFT-enhanced generation can facilitate biological representation and, in turn, broader biomedical downstream tasks.