LLM Reliability

LLM: Large Language Model

It asks not only whether a model gets an answer right, but whether it knows when it may be wrong, follows constraints, uses evidence faithfully, and behaves consistently across repeated or multi-turn use. Researchers test these properties with adversarial and perturbation benchmarks, repeated-run and cross-language evaluations, domain-specific tasks, and audits of LLM judges and tool-using agents. Common approaches include estimating uncertainty from logits, hidden representations, or response agreement; selective abstention and conformal calibration; retrieval and symbolic verification; and training methods that reward grounded answers, correction, or hesitation. The central tension is that benchmark accuracy and verbal confidence often fail to predict reliability on individual cases, long tasks, or shifted inputs, while stronger checks and abstention trade coverage, latency, and cost for safer decisions.

Latest papers

802 papers116 papers in the last four weeks, up 18% on the four weeks before. 0.7% of all new papers.

A Semantic-Sampling Framework for Evaluating Calibration in Open-Ended Question Answering
May 8, 2026Zhanliang Wang, Jiancong Xiao, Ruochen Jin +3LLM Evaluation Confidence Estimation in Language Models
University of Pennsylvania, Philadelphia, PA · Dartmouth College, Hanover, NH
Sanity Checks for Long-Form Hallucination Detection
May 8, 2026Geigh Zollicoffer, Minh Vu, Hongli Zhan +2LLM Evaluation Hallucination Detection
Los Alamos National Laboratory · The University of Texas at Austin · University of British Columbia
Beyond Confidence: Rethinking Self-Assessments for Performance Prediction in LLMs
May 8, 2026Sree Bhattacharyya, Samarth Khanna, Leona Chen +3LLM Evaluation Confidence Estimation in Language Models
Department of Informatics and Intelligent Systems, College of Information Sciences, The Pennsylvania State University. · Department of Computer Science and Engineering, College of Engineering, The Pennsylvania State University.
Reliable Chain-of-Thought via Prefix Consistency
May 8, 2026Naoto Iwase, Yuki Ichihara, Mohammad Atif Quamar +1Self-Consistency Decoding Reasoning Consistency in Language Models
Nagoya University · Nara Institute of Science and Technology · Mohamed bin Zayed University of Artificial Intelligence +1
Confidence-Aware Alignment Makes Reasoning LLMs More Reliable
May 8, 2026Kejia Chen, Jiawen Zhang, Yihong Wu +5LLM Alignment Direct Preference Optimization
Zhejiang University · Université de Montréal · Sun Yat-sen University +1
Limits of Reliability and Scaling in Language Models
May 8, 2026Subhabrata MajumdarLanguage Model Scaling Laws Autoregressive Language Modeling
Indian Institute of Management Bangalore Bengaluru, India
Hallucination Detection via Activations of Open-Weight Proxy Analyzers
May 8, 2026Akshita Singh, Prabesh Paudel, Siddhartha RoyHallucination in Language Models LLM Reliability
Khoury College of Computer Sciences Northeastern University
The Text Uncanny Valley: Non-Monotonic Performance Degradation in LLM Information Retrieval
May 8, 2026Zekai Tong, Ruiyao Xu, Aryan Shrivastava +2LLM Evaluation LLM Reliability
University of Chicago · 2Northwestern University
Delulu: A Verified Multi-Lingual Benchmark for Code Hallucination Detection in Fill-in-the-Middle Tasks
May 7, 2026Mahdi Erfanian, Nelson Daniel Troncoso, Aashna Garg +4Language Model Error Detection Language Model Generation Evaluation
University of Illinois Chicago · Microsoft
Towards Generation-Efficient Uncertainty Estimation in Large Language Models
May 7, 2026Mingcheng Zhu, Yu Liu, Tingting ZhuLLM Inference Efficiency Uncertainty Quantification
University of Oxford
Knowing but Not Correcting: Routine Task Requests Suppress Factual Correction in LLMs
May 7, 2026Zixuan Chen, Hao Lin, Zizhe Chen +6LLM Self-Correction LLM Reliability
1Shanghai Jiao Tong University · 2The Chinese University of Hong Kong · 3Ant Group
Hallucination as an Anomaly: Dynamic Intervention via Probabilistic Circuits
May 7, 2026Erik Nielsen, Elia Cunegatti, Marcus Vukojevic +1Probabilistic Circuits LLM Hallucination Mitigation
Department of Information Engineering and Computer Science University of Trento Trento, Italy
ReFlect: An Effective Harness System for Complex Long-Horizon LLM Reasoning
May 7, 2026Fan HuangLLM Self-Correction Long-Horizon LLM Reasoning
Indiana University Bloomington
Detecting Hallucinations in Large Language Models via Internal Attention Divergence Signals
May 6, 2026Gijs van DijkLanguage Model Error Detection Confidence Estimation in Language Models
Utrecht University
NoisyCoconut: Counterfactual Consensus via Latent Space Reasoning
May 6, 2026Michael Jerge, David EvansSelective Prediction LLM Inference
University of Virginia
Paraphrase-Induced Output-Mode Collapse: When LLMs Break Character Under Semantically Equivalent Inputs
May 6, 2026Aofan Liu, Jingxiang MengPrompt Sensitivity LLM Evaluation
1Peking University. · University of Chicago.
MCJudgeBench: A Benchmark for Constraint-Level Judge Evaluation in Multi-Constraint Instruction Following
May 5, 2026Jaeyun Lee, Junyoung Koh, Zeynel Tok +2LLM Evaluation LLM-as-a-Judge
University of Oxford · Yonsei University
Two Calls, Two Moments, and the Vote-Accuracy Curve of Repeated LLM Inference
May 5, 2026Yi LiuLLM Inference Test-Time Scaling
York University
Geometric Deviation as an Unsupervised Pre-Generation Reliability Signal: Probing LLM Representations for Answerability
May 4, 2026Yucheng DuRepresentation Geometry LLM Reliability
University of Southern California
A multilingual hallucination benchmark: MultiWikiQHalluA
May 4, 2026Freja Thoresen, Dan Saattrup SmartMultilingual Language Models Multilingual Language Model Evaluation
Alexandra Institute · Rued Langgaards Vej 7, 2300 København
Towards Dependable Retrieval-Augmented Generation Using Factual Confidence Prediction
May 4, 2026Florian Geissler, Francesco Carella, Laura Fieback +1Retrieval-Augmented Generation Factual Consistency Evaluation
Fraunhofer Institute for Cognitive Systems (IKS), Munich, Germany · Volkswagen AG, Wolfsburg, Germany
HalluScan: A Systematic Benchmark for Detecting and Mitigating Hallucinations in Instruction-Following LLMs
May 4, 2026Ahmed CherifLLM Evaluation LLM Hallucination Mitigation
1Sofrecom Tunisia, Orange Innovation, Tunis, 1053, Tunisia.
Zero-Shot Confidence Estimation for Small LLMs: When Supervised Baselines Aren't Worth Training
May 4, 2026Luong N. NguyenConfidence Estimation in Language Models Language Model Self-Assessment
What Single-Prompt Accuracy Misses: A Multi-Variant Reliability Audit of Language Models
May 3, 2026Ranit Karmakar, Jayita ChatterjeeLLM Evaluation Language Model Calibration
Harvard University, Boston, MA 02115 · Boston, MA 02134
Language models fail at extended rule following
May 3, 2026Tianxiang Dai, Jonathan FanLLM Evaluation LLM Reliability
Department of Electrical Engineering, Stanford University, Stanford, CA 94305.
Only Say What You Know: Calibration-Aware Generation for Long-Form Factuality
May 3, 2026Wen Luo, Guangyue Peng, Liang Wang +7Long-Form Document Generation LLM Hallucination Mitigation
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University Research Asia · MSRA · Microsoft
Hallucinations Undermine Trust; Metacognition is a Way Forward
May 2, 2026Gal Yona, Mor Geva, Yossi MatiasLLM Hallucination Mitigation Hallucination in Language Models
Google Research · Tel Aviv University
CLEAR: Revealing How Noise and Ambiguity Degrade Reliability in LLMs for Medicine
May 1, 2026Kevin H. Guo, Chao Yan, Avinash Baidya +5Healthcare LLM Reliability
Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, USA · Intuit AI Research, Mountain View, CA, USA · Department of Electrical and Computer Engineering, Vanderbilt University, Nashville, TN, USA +2
"I Don't Know" -- Towards Appropriate Trust with Certainty-Aware Retrieval Augmented Generation
May 1, 2026Daan Di Scala, Maaike de Boer, Pınar YolumRetrieval-Augmented Generation LLM Reliability
TNO Netherlands Organisation for Applied Scientific Research, Department Data Science · Utrecht University, Department of Information and Computing Sciences
Models Recall What They Violate: Constraint Adherence in Multi-Turn LLM Ideation
Apr 30, 2026Garvin KruthofLLM Evaluation LLM Alignment
Technical University of Munich

May 8, 2026J/K move · Enter open · S save

A Semantic-Sampling Framework for Evaluating Calibration in Open-Ended Question Answering

Zhanliang Wang, Jiancong Xiao, Ruochen Jin, Shu Yang+2

University of Pennsylvania, Philadelphia, PA · Dartmouth College, Hanover, NH

LLM Evaluation Confidence Estimation in Language Models Language Model Calibration Model Calibration LLM Reliability LLM Uncertainty Estimation

Open paper AskPDF arXiv

Calibration measures whether a model's predicted confidence aligns with its empirical accuracy, and is central to the reliable deployment of large language models (LLMs) in high-stakes domains such as medicine and law. While much recent work focuses on improving LLM calibration, the equally important question of how to evaluate it in realistic settings remains underdeveloped. Open-ended question answering (QA), the most common deployment setting for modern LLMs, is where existing evaluation methods fall short: logit-based metrics need restricted output formats and internal probabilities; verbalized confidence is self-reported and often overconfident; and sampling-based methods rely on task-specific extraction rules without a clear finite-sample target. We introduce Sem-ECE (Semantic-Sampling Expected Calibration Error), a calibration evaluation framework for open-ended QA that samples answers from the model, groups them into semantic classes, and uses the resulting frequencies as confidence. We study two estimators within this framework: Sem

_1

-ECE, the same-sample self-consistency score, and Sem

_2

-ECE, a held-out variant that separates answer selection from confidence evaluation. We prove both are asymptotically unbiased, and further show that they agree on easy questions but diverge on hard ones with Sem

_2

achieving strictly smaller calibration error, so their gap also serves as a diagnostic for question difficulty. Experiments on three open-ended QA benchmarks across five leading commercial LLMs match our theoretical predictions and show that Sem-ECE outperforms verbalized confidence and existing sampling-based methods, while complementing logit-based evaluation when internal probabilities are unavailable.