LLM Reliability

LLM: Large Language Model

It asks not only whether a model gets an answer right, but whether it knows when it may be wrong, follows constraints, uses evidence faithfully, and behaves consistently across repeated or multi-turn use. Researchers test these properties with adversarial and perturbation benchmarks, repeated-run and cross-language evaluations, domain-specific tasks, and audits of LLM judges and tool-using agents. Common approaches include estimating uncertainty from logits, hidden representations, or response agreement; selective abstention and conformal calibration; retrieval and symbolic verification; and training methods that reward grounded answers, correction, or hesitation. The central tension is that benchmark accuracy and verbal confidence often fail to predict reliability on individual cases, long tasks, or shifted inputs, while stronger checks and abstention trade coverage, latency, and cost for safer decisions.

Latest papers

802 papers116 papers in the last four weeks, up 18% on the four weeks before. 0.7% of all new papers.

Geometry-Calibrated Conformal Abstention for Language Models
Apr 30, 2026Rui Xu, Yi Chen, Sihong Xie +1Uncertainty Quantification Selective Prediction
Information Hub, AI Thrust Hong Kong University of Science and Technology (Guangzhou) Guangzhou, Guangdong, China
Belief-Guided Inference Control for Large Language Model Services via Verifiable Observations
Apr 30, 2026Wenhao Yuan, Chenchen Lin, Jian Chen +3LLM Inference Cost-Aware Inference
The University of Hong Kong · Sun Yat-sen University
Know2Guess: A Contamination-Aware Multi-Zone Benchmark for Knowledge-Boundary Evaluation in Large Language Models
Apr 30, 2026Renwei Meng, Bowen Zhang, Jian Wang +4LLM Evaluation LLM Auditing
Anhui University, Hefei, China
MoRFI: Monotonic Sparse Autoencoder Feature Identification
Apr 29, 2026Dimitris Dimakopoulos, Shay B. Cohen, Ioannis KonstasLLM Hallucination Mitigation LLM Interpretability
University of Edinburgh, UK · Heriot-Watt University, UK
From Architecture to Output: Structural Origins of Hallucination in Large Language Models and the Amplifying Role of Data
Apr 29, 2026Md. Rejaul Korim Sadi, Toufiqur Rahman Tasin, Golam Mostofa NaeemHallucination in Language Models LLM Reliability
*Dept. of Computer Science and Engineering, Metropolitan University, Sylhet, Bangladesh
HIVE: Hidden-Evidence Verification for Hallucination Detection in Diffusion Large Language Models
Apr 28, 2026Guoshenghui Zhao, Tan Yu, Weijie ZhaoHallucination Detection Hallucination in Language Models
Rochester Institute of Technology · NVIDIA Corporation
Large language models eroding science understanding: an empirical study of malignment
Apr 28, 2026Harry Collins, Hartmut Grote, Paul Newbury +2LLM Evaluation Knowledge Conflicts in Language Models
i · ii · iii
Authority Inversion in LLM-Mediated Ubiquitous Systems: When Models Trust Users Over Sensors
Apr 28, 2026Long Zhang, Zi-bo Qin, Wei-neng ChenKnowledge Conflicts in Language Models LLM Auditing
School of Computer Science and Engineering, South China University of Technology, Guangzhou, China
An Information-Geometric Framework for Stability Analysis of Large Language Models under Entropic Stress
Apr 27, 2026Hikmat Karimov, Rahid Zahid AlekberliLLM Evaluation LLM Reliability
Institute of Defense Technologies and Cybersecurity„ Azerbaijan Technical University, Azerbaijan
Distilling Self-Consistency into Verbal Confidence: A Pre-Registered Negative Result and Post-Hoc Rescue on Gemma 3 4B
Apr 27, 2026Jon-Paul CacioliSupervised Fine-Tuning Confidence Estimation in Language Models
Independent Researcher, Melbourne, Australia
Continual Calibration: Coverage Can Collapse Before Accuracy in Lifelong LLM Fine-Tuning
Apr 27, 2026Ibne Farabi Shihab, Sanjeda Akter, Anuj SharmaContinual Learning for LLMs Conformal Prediction
Department of Computer Science, Iowa State University · Department of Civil, Construction & Environmental Engineering, Iowa State University
Process Supervision of Confidence Margin for Calibrated LLM Reasoning
Apr 25, 2026Liaoyaqi Wang, Chunsheng Zuo, William Jurayj +2Confidence Estimation in Language Models RL for Language Model Reasoning
Johns Hopkins University
AutoPyVerifier: Learning Compact Executable Verifiers for Large Language Model Outputs
Apr 24, 2026Pouya Pezeshkpour, Estevam HruschkaLLM Evaluation LLM Answer Verification
Megagon Labs
Calibration Drift Under Reasoning: How Chain-of-Thought Budgets Induce Overconfidence in Large Language Models
Apr 24, 2026Prakul Sunil Hiremath, Harshit R. HiremathConfidence Estimation in Language Models CoT Reasoning
Department of Computer Science and Engineering Visvesvaraya Technological University, Belagavi · Department of Computer Science and Business System SG Balekundri Institute of Technology, Belagavi
Verbal Confidence Saturation in 3-9B Open-Weight Instruction-Tuned LLMs: A Pre-Registered Psychometric Validity Screen
Apr 24, 2026Jon-Paul CacioliLLM Evaluation Confidence Estimation in Language Models
Independent Researcher, Melbourne, Australia
How Large Language Models Balance Internal Knowledge with User and Document Assertions
Apr 24, 2026Shuowei Li, Haoxin Li, Wenda Chu +1LLM Sycophancy Knowledge Augmentation for Language Models
Santa Clara University · Nanyang Technological University · California Institute of Technology
When Correct Beliefs Collapse: Epistemic Resilience of LLMs under Clinical Pressure
Apr 23, 2026Boyu Xiao, Xiuqi Tian, Xuwen Song +4LLM Sycophancy Healthcare
Research Center for Social Computing and Interactive Robotics, Harbin Institute of Technology, China
Differentiable Conformal Training for LLM Reasoning Factuality
Apr 22, 2026Nathan Hittesdorf, Marco Salzetta, Lu ChengLLM Hallucination Mitigation Conformal Prediction
Department of Computer Science, University of Illinois at Chicago, Chicago, United States · Department of Physics, University of Illinois at Urbana-Champaign, Urbana, United States
Are LLM Uncertainty and Correctness Encoded by the Same Features? A Functional Dissociation via Sparse Autoencoders
Apr 21, 2026Het Patel, Tiejin Chen, Hua Wei +2LLM Interpretability LLM Reliability
University of California, Riverside, CA · Arizona State University, Tempe, AZ
CogniConsole: Externalizing Inference-Time Control as a Formal Abstraction for Reliable LLM Interactions
Apr 21, 2026Vanessa Figueiredo, Wilter FranceschiAI Agent Reliability Language Model-Based Control
Orbital Sea Regina, SK, Canada
Binary Agreement for Efficient Unsupervised Confidence Calibration of Reasoning LLMs
Apr 21, 2026Thomas Zollo, Jimmy Wang, Richard ZemelConfidence Estimation in Language Models Language Model Calibration
Columbia University
Talking to a Know-It-All GPT or a Second-Guesser Claude? How Repair reveals unreliable Multi-Turn Behavior in LLMs
Apr 21, 2026Clara Lachenmaier, Hannah Bultmann, Sina ZarrießLLM Self-Correction Human-AI Interaction
Computational Linguistics, Department of Linguistics Bielefeld University, Germany
AI scientists produce results without reasoning scientifically
Apr 20, 2026Martiño Ríos-García, Nawaf Alampara, Chandan Gupta +5AI Agent Reliability Scientific Reasoning
Laboratory of Organic and Macromolecular Chemistry (IOMC), Friedrich Schiller University Jena, Humboldstr. 10, 07743 Jena, Germany · Department of Civil Engineering, Indian Institute of Technology Delhi, Hauz Khas, New Delhi 110016, India · School of Interdisciplinary Research, Indian Institute of Technology Delhi, Hauz Khas, New Delhi 110016, India +3
Remask, Don't Replace: Token-to-Mask Refinement in Diffusion Large Language Models
Apr 20, 2026Lin YaoLLM Self-Correction Multi-Token Prediction
School of Computer Science, Shanghai Jiao Tong University, Shanghai, 200240, China · Zhongguancun Academy, Beijing, 100097, China
Correction and Corruption: A Two-Rate View of Error Flow in LLM Protocols
Apr 20, 2026Fernando ReitichLLM Evaluation LLM Auditing
Imens, LLC · Weston, Florida, USA
BEACON: Behavioral Entropy Aggregation for Cross-Model Hallucination Detection in Large Language Models
Apr 20, 2026Naveen Bera, Pulijala Sai Nikhila, Kondaguduru Abhiram +6LLM Evaluation Hallucination Detection
LLM Lens
Screen Before You Interpret: A Portable Validity Protocol for Benchmark-Based LLM Confidence Signals
Apr 20, 2026Jon-Paul CacioliLLM Evaluation Psychometric Validation
Independent Researcher, Melbourne, Australia
The Geometric Canary: Predicting Steerability and Detecting Drift via Representational Stability
Apr 20, 2026Prashant C. RajuLanguage Model Steering LLM Reliability
Calibrated? Not for Everyone: How Sexual Orientation and Religious Markers Distort LLM Accuracy and Confidence in Medical QA
Apr 19, 2026Alberto Testoni, Iacer CalixtoIntersectional Bias in Language Models Healthcare
Department of Medical Informatics, Amsterdam University Medical Center, University of Amsterdam, Amsterdam, The Netherlands. · Amsterdam Public Health, Methodology, Amsterdam, The Netherlands.
Beyond "I Don't Know": Evaluating LLM Self-Awareness in Discriminating Data and Model Uncertainty
Apr 19, 2026Jingyi Ren, Ante Wang, Yunghwei Lai +5LLM Evaluation Selective Prediction
Dept. of Comp. Sci. & Tech., Institute for AI, Tsinghua University, Beijing, China · Institute for AI Industry Research (AIR), Tsinghua University, Beijing, China

Apr 30, 2026J/K move · Enter open · S save

Geometry-Calibrated Conformal Abstention for Language Models

Rui Xu, Yi Chen, Sihong Xie, Hui Xiong

Information Hub, AI Thrust Hong Kong University of Science and Technology (Guangzhou) Guangzhou, Guangdong, China

Uncertainty Quantification Selective Prediction Confidence Estimation in Language Models Hallucination in Language Models Conformal Prediction LLM Abstention+4

Open paper AskPDF arXiv

When language models lack relevant knowledge for a given query, they frequently generate plausible responses that can be hallucinations, rather than admitting being agnostic about the answer. Retraining models to reward admitting ignorance can lead to overly conservative behaviors and poor generalization due to scarce evaluation benchmarks. We propose a post hoc framework, Conformal Abstention (CA), adapted from conformal prediction (CP) to determine whether to abstain from answering a query. CA provides finite-sample guarantees on both the probability of participation (i.e., not abstaining) and the probability that the generated response is correct. Importantly, the abstention decision relies on prediction confidence rather than the non-conformity scores used in CP, which are intractable for open-ended generation. To better align prediction confidence with the model's ignorance, we introduce a calibration strategy using representation geometry within the model to measure knowledge involvement in shaping the response. Experiments demonstrate that we improve selective answering significantly with 75 percent conditional correctness.