LLM Reliability

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

78 papers in the last four weeks, up 105% on the four weeks before. 0.8% of all new papers.

Jul 13Week of Sep 28

Latest papers 507

All topics

Semantic Self-Distillation for Language Model Uncertainty
Feb 4, 2026Edward Phillips, Sean Wu, Fredrik K. Gustafsson +2Uncertainty Quantification Confidence Estimation in Language Models
Department of Engineering Science, University of Oxford · Oxford Suzhou Centre for Advanced Research, University of Oxford, Suzhou
HALT: Hallucination Assessment via Log-probs as Time series
Feb 2, 2026Ahmad Shapiro, Karan Taneja, Ashok GoelLanguage Model Error Detection Hallucination Detection
Georgia Institute of Technology · Microsoft
Uncertainty Localization in LLM Reasoning via Embedding Perturbations
Feb 2, 2026Qihao Wen, Jiahao Wang, Yang Nan +3LLM Reliability LLM Uncertainty Estimation
University of Arizona · Michigan State University
MGSM-Pro: A Simple Strategy for Robust Multilingual Mathematical Reasoning Evaluation
Jan 29, 2026Tianyi Xu, Kosei Uemura, Alfred Malengo Kondoro +6Numerical Reasoning in Language Models Mathematical Reasoning Benchmarks
McGill University · Mila-Quebec AI Institute · University of Toronto +7
Bayesian-LoRA: Probabilistic Low-Rank Adaptation of Large Language Models
Jan 28, 2026Moule Lin, Shuhao Guan, Andrea Patane +2Bayesian Neural Networks Confidence Estimation in Language Models
School of Computer Science and Statistics, Trinity College Dublin, Dublin, Ireland and Lero the Research Ireland Centre for Software, Ireland · School of Computer Science, University College Dublin, Dublin, Ireland
AdversaRiskQA: An Adversarial Factuality Benchmark for High-Risk Domains
Jan 21, 2026Adam Szelestey, Sofie van Engelen, Tianhao Huang +3Factual Consistency Evaluation LLM Reliability
Eindhoven University of Technology Eindhoven, Netherlands
Vulnerability of LLMs' Stated Beliefs? LLMs Belief Resistance Check Through Strategic Persuasive Conversation Interventions
Jan 20, 2026Fan Huang, Haewoon Kwak, Jisun AnLLM Evaluation Fine-Tuning
Indiana University Bloomington, United States
Relational Linearity is a Predictor of Hallucinations
Jan 16, 2026Yuetian Lu, Yihong Liu, Sebastian Gerstner +3LLM Interpretability Hallucination in Language Models
Center for Information and Language Processing (CIS), LMU Munich · Ubiquitous Knowledge Processing Lab (UKP Lab), Technical University of Darmstadt · Munich Center for Machine Learning (MCML)
NOVA: NOise-aware Verbal Confidence CAlibration for Robust Large Language Models in RAG Systems
Jan 16, 2026Jiayu Liu, Rui Wang, Qing Zong +9Retrieval-Augmented Generation Confidence Estimation in Language Models
HKUST · UIUC · Northwestern University
Calibration Is Not Enough: Evaluating Confidence Estimation Under Language Variations
Jan 12, 2026Yuxi Xia, Dennis Ulmer, Terra Blevins +3LLM Evaluation Confidence Estimation in Language Models
Faculty of Computer Science, UniVie Doctoral School Computer Science · Faculty of Philological and Cultural Studies, University of Vienna, Austria · ILLC, University of Amsterdam, Netherlands +2
Not All Needles Are Found: How Fact Distribution and Don't Make It Up Prompts Shape Retrieval, Reasoning, and Hallucination in Long-Context LLMs
Jan 5, 2026Amirali Ebrahimzadeh, Seyyed M. SaliliLong-Context Retrieval LLM Hallucination Mitigation
Department of Electrical Engineering & Computer Science, University of Michigan, Ann Arbor, MI 48109, USA · Independent Researcher, Ann Arbor, MI 48106, USA
Honesty over Accuracy: Trustworthy Language Models through Reinforced Hesitation
Nov 14, 2025Mohamad Amin Mohamadi, Tianhao Wang, Zhiyuan LiRL for Language Models LLM Hallucination Mitigation
Toyota Technological Institute at Chicago · University of California, San Diego
A Quantitative Study of Sustained Focus in Large Language Models via Repetitive Deterministic Prediction Tasks
Nov 2, 2025Wanda Hou, Leon Zhou, Hong-Ye Hu +3LLM Evaluation LLM Reliability
EdenCode Inc. · Stevenson School · Harvard University +2
Reasoning Up the Instruction Ladder for Controllable Language Models
Oct 30, 2025Zishuo Zheng, Vidhisha Balachandran, Chan Young Park +2LLM Safety Alignment Instruction Following
Department of Computer Science and Engineering, The Ohio State University · Microsoft Research · Allen Institute for AI
When Guessing is Rewarded: Rethinking Language Model Evaluation with Distributional Uncertainty Scoring
Oct 5, 2025Thomas F BurnsLLM Evaluation Language Model Generation Evaluation
Aleph Alpha Research
Rethinking Reward Models for Multi-Domain Test-Time Scaling
Oct 1, 2025Dong Bok Lee, Seanie Lee, Sangwoo Park +12Reward Modeling LLM Evaluation
KAIST · TH Nürnberg · Microsoft Research Asia
Training-free Truthfulness Detection via Sparse MLP Value Vectors
Sep 22, 2025Runheng Liu, Heyan Huang, Xingchen Xiao +2LLM Reliability LLM Hallucination Detection
Beijing Institute of Technology Beijing, China
Clotho: Measuring Task-Specific Pre-Generation Test Adequacy for LLM Inputs
Sep 22, 2025Juyeon Yoon, Somin Kim, Robert Feldt +1LLM Evaluation LLM Reliability
Korea Advanced Institute of Science and Technology, Daejeon, Republic of Korea · Chalmers University of Technology, Gothenburg, Sweden
Geometric Uncertainty for Detecting and Correcting Hallucinations in LLMs
Sep 17, 2025Edward Phillips, Sean Wu, Soheila Molaei +3Uncertainty Quantification LLM Hallucination Mitigation
Department of Engineering Science, University of Oxford · GlaxoSmithKline · Oxford Suzhou Centre for Advanced Research
Addressing Benchmarking Gaps in Large Language Models for Health and Medicine with Dynamic Red-Teaming
Jul 30, 2025Jiazhen Pan, Bailiang Jian, Paul Hager +20LLM Safety Benchmarks Language Model Safety Evaluation
Cherise · School of Computation, Information and Technology, Technical University of Munich (TUM), Munich, Germany · Department of Engineering Science, University of Oxford, Oxford, United Kingdom +9
Post-Training Large Language Models via Reinforcement Learning from Self-Feedback
Jul 29, 2025Carel van Niekerk, Renato Vukovic, Benjamin Ruppik +3RL for Language Models RL for Language Model Reasoning
Heinrich Heine University Düsseldorf, Germany
The Trilemma of Truth in Large Language Models
Jun 30, 2025Germans Savcisens, Tina Eliassi-RadLLM Interpretability LLM Reliability
Khoury College of Computer Sciences Northeastern University, Boston, MA, USA · Network Science Institute Northeastern University, Boston, MA, USA · Santa Fe Institute, Santa Fe, NM, USA
Popular but Wrong: Understanding and Mitigating LLM Overconfidence through Knowledge Popularity
May 23, 2025Shiyu Ni, Keping Bi, Jiafeng Guo +1LLM Hallucination Mitigation Confidence Estimation in Language Models
CAS Key Lab of Network Data Science and Technology, ICT, CAS · State Key Laboratory of AI Safety · University of Chinese Academy of Sciences
HoT: Highlighted Chain of Thought for Referencing Supporting Facts from Inputs
Mar 3, 2025Tin Nguyen, Logan Bolton, Mohammad Reza Taesiri +2LLM Grounding LLM Answer Verification
Auburn University · University of Alberta
Prompting the Unknown: Understanding Response Uncertainty in Large Language Models
Jul 20, 2024Ze Yu Zhang, Arun Verma, Finale Doshi-Velez +1LLM Prompting LLM Reliability
Department of Computer Science, National University of Singapore, Republic of Singapore · Singapore-MIT Alliance for Research and Technology Centre · Harvard University
VeriSim: A Configurable Framework for Stress-Testing Medical AI Under Patient Communication Noise
Date pendingSina Mansouri, Mohit Marvania, Vibhavari Ashok Shihorkar +5Healthcare LLM Reliability
Department of Computer Science, George Mason University · Department of Health Administration and Policy, College of Public Health, George Mason University · Department of Science, George Mason University
ActMap: Single-Pass Uncertainty Quantification from Generation-Time Activation Maps
Date pendingJacopo Dardini (University of Bologna), Roberta Calegari (University of Bologna)Uncertainty Quantification Confidence Estimation in Language Models
University of Bologna, Italy

Feb 4, 2026, cs.CLJ/K move · Enter open · S save

Semantic Self-Distillation for Language Model Uncertainty

Edward Phillips, Sean Wu, Fredrik K. Gustafsson, Boyan Gao+1

Department of Engineering Science, University of Oxford · Oxford Suzhou Centre for Advanced Research, University of Oxford, Suzhou

Uncertainty Quantification Confidence Estimation in Language Models LLM Reliability Language Model Distillation LLM Uncertainty Estimation

Open paper AskPDF arXiv

Large language models present challenges for principled uncertainty quantification, in part due to their complexity and the diversity of their outputs. Semantic dispersion, or the variance in the meaning of sampled answers, has been proposed as a useful proxy for model uncertainty, but the associated computational cost prohibits its use in latency-critical applications. We show that sampled semantic distributions can be distilled into lightweight student models which estimate a prompt-conditioned density before the language model generates an answer token. The student model predicts a semantic distribution over possible answers; the entropy of this distribution provides a prompt-level uncertainty signal, and the probability density allows answer-level reliability evaluation. Across experiments on TriviaQA and MMLU, we find our student models perform competitively relative to the teacher's sampled semantic dispersion on a hallucination prediction task, whilst offering additional uncertainty primitives for out-of-domain detection and multiple-choice answer selection. We term this technique Semantic Self-Distillation (SSD), which can serve as a general framework for distilling predictive uncertainty in complex output spaces beyond language.

Momentum

Related topics

Latest papers 507