LLM Abstention

LLM: Large Language Model

Large language model abstention studies how systems can recognize missing knowledge, insufficient evidence, or unreliable reasoning and choose whether to answer, clarify, defer, or stop. Methods include confidence and consistency signals, evidence checks and internal-state probes, post-hoc selection thresholds, and training with supervised or reinforcement-learning rewards for calibrated abstention. Benchmarks test factual question answering, retrieval-augmented generation, clinical decisions, and tool-using agents, measuring both answer quality and appropriate abstention. A central tension is balancing coverage against the risk of unsupported answers: models may recognize uncertainty but still commit, or abstain so often that useful answers are lost, especially when settings change.

Latest papers

99 papers20 papers in the last four weeks, up 54% on the four weeks before. 0.1% of all new papers.

Trainable Boundary-Keeping Where No Formal Answer Exists: A Controlled Knock-In Study of Reward Training in Evidence-Sufficiency Judgment
Oct 26, 2026YaJie YinLLM Abstention Refusal Behavior in Language Models
Reinforcement Learning Done Right for Explicit Confidence Targets
Oct 26, 2026Zefeng He, Yichen He, Yuan Lin +2LLM Abstention Selective Prediction
ByteDance Seed · Shanghai Jiao Tong University · NTU
Commit or Abstain? System One Decision Models, LLMs, and Missing Legal Authority
Oct 21, 2026M. Mikail DemirLegal Citation Verification LLM Abstention
Department of Information Science, University at Albany, SUNY, USA
Effects of Model Scale and Quantization on Social Bias and Abstention in Question Answering
Oct 21, 2026Kuniko Paxton, Ifeoluwa Wuraola, Dhavalkumar ThakkerSocial Bias in Language Models Language Model Bias Evaluation
University of Hull Cottingham Road, Hull, HU6 7RX, United Kingdom
DeclineCraft: A RAG Abstention Benchmark for Jointly Measuring Combined Retrieval and Parametric Knowledge
Oct 20, 2026Atharva Thakur, Abulhair Saparov, Bruno RibeiroRAG Evaluation LLM Abstention
Purdue University
SOMA: Continuous Monitoring and Intervention for System Two Reasoning with System One Models
Oct 19, 2026Ruizhi Qian, Lei Shi, Tianyu Hu +3Adaptive Reasoning in Language Models Metacognition in Language Models
University of Central Florida
Training Language Models to Fail Gracefully
Oct 14, 2026Daman Arora, Xuecheng Liu, Himanshu Gaurav Singh +6RL for Language Models LLM Abstention
Carnegie Mellon University · UC Berkeley
How Reliable Are System-One Decisions in Medicine? A Comparative Study of Jev, Jev-like Models, and LLMs
Oct 13, 2026Yanrui Du, Danyang Zhao, Sendong Zhao +6Clinical Decision-Making LLM Reliability
Harbin Institute of Technology · Ant Group
Dissociating Answerability from Extraction in LLM Question Answering over Radiology Reports
Oct 13, 2026Priyanshu Sinha, Kelsey McDonaldMedical QA Radiology Report Evaluation
Labcorp, Durham, NC
Strong yet Controllable LLM Abstention without Learning to Abstain
Oct 12, 2026Junwei Zhang, Yifei Wang, Wanli Yang +6Factual QA LLM Abstention
State Key Laboratory of AI Safety, Institute of Computing Technology, CAS · University of Chinese Academy of Sciences · Beijing Institute of Technology
ScienceBuddy-PV: A Verifier-Governed Recursive Improvement Protocol for Panvascular Intervention Research
Oct 10, 2026Lingsen You, Yujun Guo, Xinyu Zhong +4LLM Abstention Selective Prediction
Department of Cardiology, Zhongshan Hospital, Fudan University, Shanghai Institute of Cardiovascular Diseases, State Key Laboratory of Cardiovascular Diseases, NHC Key Laboratory of Ischemic Heart Diseases China, National Clinical Research Center for Interventional Medicine, Key Laboratory of Viral Heart Diseases, Chinese Academy of Medical Sciences, Shanghai 200032, China. · Institutes of Biomedical Sciences, Fudan University, Shanghai 200030, China. · College of Biomedical Engineering, Fudan University, Shanghai 200438, China. +2
Can Language Models Learn When to Abstain? Policy Learning and Calibrated Answer Selection
Oct 9, 2026Juan Terven, Diana-Margarita Córdova-Esparza, Ruben Gomez +3Selective QA LLM Abstention
CICATA Qro., Instituto Politécnico Nacional, México · Facultad de Informática, Universidad Autónoma de Querétaro, México
Certified by Abstention: Distribution-Free Guarantees for Chain-of-Thought Verifiers at Small Calibration Budgets
Oct 7, 2026Arjun BalajiSelective Prediction Conformal Prediction
Columbia University
Arctic Questions, Missing Answers: A Dataset and Benchmark for LLM Abstention in Arctic Science
Oct 7, 2026Benjamin Wilcox, Dawei Gao, Pradeeban Kathiravelu +3LLM Abstention LLM Evaluation
University of North Texas, Denton, TX · University of Alaska Anchorage, Anchorage, AK
Knowing When Not to Answer: Cross-Domain and Multi-Turn Generalization of Latent Underspecification Signals
Oct 6, 2026Jerzy Kamiński, Ilya Galyukshev, Artem Kuznetsov +6Multi-Turn Dialogue Evaluation Language Model Probing
ITMO University, Saint Petersburg, Russia
Penalty-Framed No-Valid-Option MCQA: Analyzing LLM Abstention under Invalid Choices
Oct 6, 2026Jinhyeok Kim, Hye-Young JungLLM Evaluation Multiple-Choice Question Answering
Applied Artificial Intelligence Hanyang University, Republic of Korea · Mathematical Data Science Hanyang University, Republic of Korea
Rethinking Faithfulness in LLMs: A Pairwise Context-Sensitive Perspective
Oct 6, 2026Zizhuo Zhang, Xiong Peng, Jingwei Sun +3Contextual Faithfulness LLM Evaluation
TMLR Group, Department of Computer Science, Hong Kong Baptist University · Huawei Noah’s Ark Lab.
Unanimously Wrong: Certified Abstention from How Medical LLM Consensus Forms
Oct 6, 2026Xiaoyang Wang, Tianrui Wang, Christopher C. YangLLM Abstention Selective Prediction
Drexel University · University of Michigan, Ann Arbor
Decide, Ask, or Defer: Clinical LLMs under Incomplete Evidence
Oct 3, 2026Mingzhan Yang, Weili WuClinical Decision-Making Clinical Language Model Evaluation
Department of Computer Science University of Texas at Dallas Richardson, TX 75080
A Citation-Grounded Benchmark for Trustworthy Earnings Call Transcript Analysis with Large Language Models
Oct 1, 2026Yingzhu Zhao, Vlad Pandelea, Han Yuan +4LLM Evaluation Financial QA
Global Decision Science, American Express · Singapore Decision Science Center of Excellence, American Express, 1 Marina Boulevard, 018989, Singapore.
When the Right Answer Is Missing: An Arithmetic-Dependent Rejection Bottleneck in Jev
Sep 30, 2026Jike Zhong, Ming Li, Yuxiang LaiNumerical Reasoning in Language Models Selective Prediction
University of Southern California · University of Florida · Emory University
When Does an Image Determine the Answer? Benchmarking Visual Answerability across Charts and Scenes
Sep 28, 2026Sungguk Cha, Mintae Kim, Youngsub Han +2VLM Evaluation Visual Question Answering
LG Uplus, Seoul, South Korea
Risk-Controlled Selective LLM Answering by Pricing Label-Free Checks
Sep 27, 2026Dongyub Jude Lee, Jungseob Lee, Chanjun Park +2LLM Answer Verification Selective Prediction
Zoom Communications · Korea University · Soongsil University +1
The Commit-Abstain Circuit: Why Language Models Hallucinate Instead of Abstaining
Sep 26, 2026Vy Nguyen, Ziqi Xu, Jeffrey Chan +5Hallucination in Language Models Circuits in Language Models
RMIT University · Jilin University · Nanyang Technological University
Regime Boundary Alignment for Evidence-Gated Question Answering
Sep 25, 2026Zeyan Li, Qirong Guo, SIyuan Qiu +3Evidence-Grounded QA LLM Grounding
Shanghai Jiao Tong University · The Hong Kong University of Science and Technology (Guangzhou)
ChunkRank: Model-Aware Text Chunking and Abstention-Aware Answer Selection for LLM Pipelines
Sep 24, 2026Amit Nautiyal, Ayush Bhatt, Gaurav NautiyalDocument Chunking Question Answering
Meet, Compare, or Abstain: LatWeave for Deterministic Multi-Hop Question Answering on Knowledge Lattices
Sep 23, 2026Yuze Ren, Shaoheng Fan, Tao Wang +2Multi-Hop QA Question Answering
ZenSmart Technology (Beijing) Co., Ltd., China
Sieve and Sage: Efficient Distraction Filtering for Reliable RALM Abstention
Sep 17, 2026Jongbin Won, Sung Geun An, Jay-yoon LeeLLM Abstention RAG Robustness
Graduate School of Data Science / Seoul National University
Selection Is Retrieval, Abstention Is Not: On-Device Tool Routing over 70 Korean-English Actions
Sep 16, 2026Janghoon LeeTool-Augmented Language Model Agents On-Device Language Model Inference
Redrob
When Should LLMs Abstain? Chain-of-Self-Questioning for Selective Risk Control
Sep 15, 2026Ali ŞenolSelective Prediction LLM Prompting
Department of Computer Engineering, Tarsus University, Tarsus, Türkiye

Oct 26, 2026J/K move · Enter open · S save

Trainable Boundary-Keeping Where No Formal Answer Exists: A Controlled Knock-In Study of Reward Training in Evidence-Sufficiency Judgment

YaJie Yin

LLM Abstention Refusal Behavior in Language Models RL for Language Model Reasoning Evidence-Grounded Reasoning

Open paper AskPDF arXiv

Reasoning-tuned and reward-trained models are often assumed to verify more reliably because they reason longer or receive correctness signals. We test this in evidence-sufficiency judgment: deciding whether available information actually supports a causal claim, a domain with no decidable correctness reward. Using a pre-registered, same-base, controlled knock-in design, we compare a boundary-keeping recipe (SFT then GRPO) with its own baseline, a scrambled-pairing placebo, and a general-purpose reasoning model under an action-pressure protocol with no evaluation invitation. All load-bearing outcomes are rated by two separate model-rating sessions masked to arm labels; automatic judges only trigger. The trained arm raises spontaneous correct refusal from 25% to 100% (40 per arm; Wilson 95% CI lower bound 91.2%, non-overlapping with the baseline interval [14.2%, 40.2%]), while the scrambled placebo collapses (5% spontaneous / 7.5% invited correct; 91.1% false refusal). The effect is boundary-specific rather than blanket refusal: automatic false refusal on 281 well-formed claims is 1.1% (Wilson upper bound 3.09%), and a targeted 18-item arm-(c) recheck (3 flagged plus 15 accepted) yields 0/18 and 1/18 false refusals in two sessions. The dissociation is the main result: final boundary-keeping is trainable without an inference-time verifier, while observed active numeric/calculation checking in the H3 assay remains near zero; SFT saturation makes the GRPO increment not separately identifiable. A pre-registered mechanism adjudication favors reward internalization of final-action status (H2, best-supported, not proven). The scope is a controlled 14B same-family study: we do not claim proof, cross-family generality, or independent human-expert validation; the rating sessions are model sessions, not human reviewers.