LLM Reliability

LLM: Large Language Model

It asks not only whether a model gets an answer right, but whether it knows when it may be wrong, follows constraints, uses evidence faithfully, and behaves consistently across repeated or multi-turn use. Researchers test these properties with adversarial and perturbation benchmarks, repeated-run and cross-language evaluations, domain-specific tasks, and audits of LLM judges and tool-using agents. Common approaches include estimating uncertainty from logits, hidden representations, or response agreement; selective abstention and conformal calibration; retrieval and symbolic verification; and training methods that reward grounded answers, correction, or hesitation. The central tension is that benchmark accuracy and verbal confidence often fail to predict reliability on individual cases, long tasks, or shifted inputs, while stronger checks and abstention trade coverage, latency, and cost for safer decisions.

Latest papers

802 papers116 papers in the last four weeks, up 18% on the four weeks before. 0.7% of all new papers.

Instinct vs. Reflection: Unifying Token and Verbalized Confidence in Multimodal Large Models
Apr 19, 2026Yunkai Dang, Yifan Jiang, Yizhu Jiang +3Selective Prediction Confidence Estimation in Language Models
School of Artificial Intelligence, Nanjing University.
Complementing Self-Consistency with Cross-Model Disagreement for Uncertainty Quantification
Apr 18, 2026Kimia Hamidieh, Veronika Thost, Walter Gerych +2Selective Prediction Confidence Estimation in Language Models
1MIT · MIT-IBM Watson AI Lab · 3WPI +1
Abstain-R1: Calibrated Abstention and Post-Refusal Clarification via Verifiable RL
Apr 18, 2026Skylar Zhai, Jingcheng Liang, Dongyeop KangLLM Hallucination Mitigation RL for Language Model Reasoning
University of Minnesota
Evaluating Tool-Using Language Agents: Judge Reliability, Propagation Cascades, and Runtime Mitigation in AgentProp-Bench
Apr 17, 2026Bhaskar GurramAI Agent Reliability Tool-Augmented Language Model Agents
Zasti Inc., Ashburn, VA, USA
No-Worse Context-Aware Decoding: Preventing Neutral Regression in Context-Conditioned Generation
Apr 17, 2026Yufei Tao, Ameeta AgrawalLanguage Model Decoding LLM Reliability
Department of Computer Science, Portland State University, USA
Beyond Surface Statistics: Robust Conformal Prediction for LLMs via Internal Representations
Apr 17, 2026Yanli Wang, Peng Kuang, Xiaoyu Han +2Language Model Robustness Conformal Prediction
Imperial College London · Zhejiang University · University of Illinois Urbana-Champaign +1
LLMs Corrupt Your Documents When You Delegate
Apr 17, 2026Philippe Laban, Tobias Schnabel, Jennifer NevilleLLM Reliability Long-Horizon LLM Agents
Microsoft Research
Diagnosing LLM Judge Reliability: Conformal Prediction Sets and Transitivity Violations
Apr 16, 2026Manan Gupta, Dhruv KumarLLM Evaluation LLM-as-a-Judge
1BITS Pilani, Pilani Campus, India
IUQ: Interrogative Uncertainty Quantification for Long-Form Large Language Model Generation
Apr 16, 2026Haozhi Fan, Jinhao Duan, Kaidi XuUncertainty Quantification Language Model Generation Evaluation
University of Pennsylvania · 2UNC Chapel Hill · 3City University of Hong Kong
Pushing the Boundaries of Multiple Choice Evaluation to One Hundred Options
Apr 16, 2026Nahyun Lee, Guijin SonLLM Evaluation Multiple-Choice Question Answering
Chung-Ang University · Seoul National University
Correct Prediction, Wrong Steps? Consensus Reasoning Knowledge Graph for Robust Chain-of-Thought Synthesis
Apr 15, 2026Zipeng Ling, Shuliang Liu, Seonil Son +4Reasoning Trace Analysis Logical Reasoning
The Hong Kong University of Science and Technology (Guangzhou) · University of Alberta · Alberta Machine Intelligence Institute +3
ERRORQUAKE: Heavy-Tailed Error Severity Distributions in Open-Weight Large Language Models
Apr 15, 2026Jason Z WangLLM Evaluation Open-Weight Language Models
Independent
The Cognitive Circuit Breaker: A Systems Engineering Framework for Intrinsic AI Reliability
Apr 15, 2026Jonathan PanLLM Reliability LLM Hallucination Detection
Home Team Science and Technology Agency Singapore
Numerical Instability and Chaos: Quantifying the Unpredictability of Large Language Models
Apr 14, 2026Chashi Mahiul Islam, Alan Villarreal, Mao Nishino +2LLM Reliability
Department of Computer Science, Florida State University, Tallahassee, USA · Department of Mathematics, Florida State University, Tallahassee, USA
Is Vibe Coding the Future? An Empirical Assessment of LLM Generated Codes for Construction Safety
Apr 14, 2026S M Jamil UddinConstruction Safety LLM Code Generation
Assistant Professor, Department of Construction Management, Colorado State University
Phantom Paradoxes: Language Models Treat Unresolved Circular Truth Dependence as Contradiction
Apr 13, 2026Ji Ho BaeLLM Interpretability LLM Reliability
JRTI
Leveraging Weighted Syntactic and Semantic Context Assessment Summary (wSSAS) Towards Text Categorization Using LLMs
Apr 13, 2026Shreeya Verma Kathuria, Nitin Mayande, Sharookh Daruwalla +2Text Clustering Text Classification
Tellagence Inc. · Villanova School of Business, Villanova University
When AI Tells You What You Want to Hear: Sycophantic Behavior of Large Language Models in Dementia Care Settings
Apr 13, 2026Christian KolbDementia Care LLM Sycophancy
From Answers to Arguments: Toward Trustworthy Clinical Diagnostic Reasoning with Toulmin-Guided Curriculum Goal-Conditioned Learning
Apr 13, 2026Chen Zhan, Xiaoyu Tan, Gengchen Ma +3Clinical Reasoning LLM Reliability
School of Electronic and Electrical Engineering, Shanghai University of Engineering Science, Shanghai, 201620, China. · Tencent Youtu Lab, Shanghai, 200232, China.
SciPredict: Can LLMs Predict the Outcomes of Scientific Experiments in Natural Sciences?
Apr 12, 2026Udari Madhushani Sehwag, Elaine Lau, Haniyeh Ehsani Oskouie +14AI-Assisted Scientific Research Scientific Reasoning in Language Models
Scale AI · University of California, Los Angeles · Human Frontier Collective, Scale AI +2
Evaluating Small Open LLMs for Medical Question Answering: A Practical Framework
Apr 12, 2026Avi-ad Avraam BuskilaLLM Evaluation LLM Reliability
Department of Information Science and Applied Artificial Intelligence, Bar-Ilan University, Ramat-Gan, Israel
VeriSim: A Configurable Framework for Stress-Testing Medical AI Under Patient Communication Noise
Apr 12, 2026Sina Mansouri, Mohit Marvania, Vibhavari Ashok Shihorkar +5Healthcare LLM Reliability
Department of Computer Science, George Mason University · Department of Health Administration and Policy, College of Public Health, George Mason University · Department of Science, George Mason University
VeriTrans: Fine-Tuned LLM-Assisted NL-to-PL Translation via a Deterministic Neuro-Symbolic Pipeline
Apr 11, 2026Xuan Liu, Dheeraj Kodakandla, Kushagra Srivastva +1Boolean Satisfiability LLM Reliability
The Pennsylvania State University, University Park, Pennsylvania, USA
From GPT-3 to GPT-5: Mapping their capabilities, scope, limitations, and consequences
Apr 11, 2026Hina Afridi, Habib Ullah, Sultan Daud Khan +1Language Modeling Generative AI Evaluation
MediaFutures, Department of Information Science and Media Studies, University of Bergen, Bergen, Norway. · Faculty of Science and Technology, Norwegian University of Life Sciences, Ås, Norway. · Faculty of Computing and Information Technology, Sohar University, Sohar, Oman. +2
Like a Hammer, It Can Build, It Can Break: Large Language Model Uses, Perceptions, and Adoption in Cybersecurity Operations on Reddit
Apr 11, 2026Souradip Nath, Chih-Yi Huang, Aditi Ganapathi +3LLMs for Cybersecurity Cybersecurity
Arizona State University · Technische Universität Berlin
Building Trust in the Skies: A Knowledge-Grounded LLM-based Framework for Aviation Safety
Apr 10, 2026Anirudh Iyengar, Alisa Tiselska, Dumindu Samaraweera +1Aviation Safety LLM Reliability
Department of Mathematics, Embry-Riddle Aeronautical University, Daytona Beach, FL
HCRE: LLM-based Hierarchical Classification for Cross-Document Relation Extraction with a Prediction-then-Verification Strategy
Apr 9, 2026Guoqi Ma, Liang Zhang, Hongyao Tu +6Relation Extraction Hierarchical Classification
School of Informatics, Xiamen University · Li Auto Inc. · Alibaba International Digital Commerce Group
Guardian-as-an-Advisor: Advancing Next-Generation Guardian Models for Trustworthy LLMs
Apr 8, 2026Yue Huang, Haomin Zhuang, Jiayi Ye +6LLM Guardrails LLM Safety Alignment
University of Notre Dame · University of California, Los Angeles · MIT-IBM Watson AI Lab +1
AGSC: Adaptive Granularity and Semantic Clustering for Uncertainty Quantification in Long-text Generation
Apr 8, 2026Guanran Luo, Wentao Qiu, Wanru Zhao +4Semantic Clustering Uncertainty Quantification
School of Informatics, Xiamen University
The Illusion of Stochasticity in LLMs
Apr 8, 2026Xiangming Gu, Soham De, Michalis Titsias +3LLM Reliability LLM Agent Reliability
Google DeepMind · National University of Singapore

Apr 19, 2026J/K move · Enter open · S save

Instinct vs. Reflection: Unifying Token and Verbalized Confidence in Multimodal Large Models

Yunkai Dang, Yifan Jiang, Yizhu Jiang, Anqi Chen+2

School of Artificial Intelligence, Nanjing University.

Selective Prediction Confidence Estimation in Language Models Multimodal Large Language Models Language Model Calibration LLM Reliability LLM Uncertainty Estimation+1

Open paper AskPDF arXiv

Multimodal Large Language Models (MLLMs) have demonstrated exceptional capabilities in various perception and reasoning tasks. Despite this success, ensuring their reliability in practical deployment necessitates robust confidence estimation. Prior works have predominantly focused on text-only LLMs, often relying on computationally expensive self-consistency sampling. In this paper, we extend this to multimodal settings and conduct a comprehensive evaluation of MLLMs' response confidence estimation. Our analysis reveals a significant instinct-reflection misalignment: the model's implicit token-level support frequently diverges from its verbal self-assessment confidence. To address this misalignment, we propose a monotone confidence fusion framework to merge dual-channel signals and cross-channel consistency to estimate correctness. Subsequently, an order-preserving mean alignment step is applied to correct global bias, which improves calibration while preserving the risk-coverage trade-off for selective prediction. Experiments on diverse open-source and closed-source MLLMs show that our method consistently yields more reliable confidence estimates and improves both calibration and failure prediction. Code will be available at https://github.com/Yunkaidang/Instinct-vs.-Reflection.