Selective Prediction

Follow this topic Ask about this topic

Selective prediction studies how models should decide whether to make a prediction, abstain, or escalate an uncertain case to a person or another system. Common approaches use confidence and uncertainty scores, ensembles or verifiers, and calibration methods such as conformal prediction to set risk–coverage trade-offs. Applications range from image classification and clinical triage to language-model question answering and agent actions, where benchmarks increasingly assess accepted-case errors, coverage, review costs, and subgroup outcomes. A central challenge is that useful confidence rankings do not by themselves guarantee safe decisions: calibration and risk guarantees can depend on data assumptions, shift across domains or groups, and sharply reduce coverage.

Latest papers

292 papers58 papers in the last four weeks, up 38% on the four weeks before. 0.3% of all new papers.

Aligning Language Models with Selective Prediction
Jul 3, 2026Gaoxiang Luo, Yifan Wu, Sinian Zhang +2RL for Language Models LLM Alignment
Department of Computer Science and Engineering · 2Bioinformatics and Computational Biology Program · 3Division of Biostatistics and Health Data Science
Denoised Conformal Alignment for Reliable Selection of Conditional Average Treatment Effect Predictions
Jul 3, 2026Xinyun Lu, Haoang Chi, Zhiheng ZhangCausal Effect Estimation Selective Prediction
Shanghai University of Finance and Economics · Shanghai Jiao Tong University · National University of Defense Technology, Changsha, China +2
Robustness Meets Uncertainty: Evidential Adversarial Training for Robust Selective Classification
Jul 3, 2026Nicolas Sournac, Ahmed Baha Ben Jmaa, Bertrand BraeckeveldtAdversarial Training Evidential Deep Learning
Multitel Research & Innovation Center, Artificial Intelligence Department, Belgium · TRAIL – Trusted AI Labs, Belgium
Efficient Waste Sorting for Circular Economy: A Confidence-guided comparison between One-Vs-All and One-Vs-Rest Classification Strategies with Human-in-the-Loop for Automated Waste Sorting
Jul 2, 2026Mohammed Fahad Ali, Dominique Briechle, Marit Briechle-Mathiszig +2Selective Prediction Multiclass Classification
Institute for Software and Systems Engineering Clausthal University of Technology Germany
Beyond Supervised Clarification: Input Rewriting with LLMs for Dialogue Discourse Parsing
Jul 2, 2026Yiming Liu, Ziyue Zhang, Zhichao Xu +4Selective Prediction LLM Prompting
University of Oklahoma · University of Utah · Lawrence Berkeley National Laboratory +1
Evaluating Large Language Model Raters for German Open-Response Clinical Questions: A Physician-Annotated Benchmark Study of Agreement, Evaluator Bias, and Abstention
Jul 1, 2026William Philipp, Finn Fassbender, Daniel Fister +12Multilingual Language Model Evaluation LLM-as-a-Judge
Section for Clinical Research-IT, Institute of Medical Biometry and Statistics, University of Luebeck, Luebeck, Germany · Medical Faculty, University of Tübingen, Tübingen, Germany · German Medical Students’ Association (bvmd), Berlin, Germany +9
Uncertainty-Aware Last-Layer Adaptation of RETFound for Referable Diabetic Retinopathy Screening Under Dataset Shift
Jun 30, 2026Karim MardhaniBayesian Neural Networks Distribution Shift Robustness
University of Colorado Boulder Boulder, CO, USA
Teaching LLMs to Recommend and Defer in Underrepresented Epilepsy Care
Jun 30, 2026Shreyas Rajesh, Kartik Sharma, Tonmoy Monsoor +8Prompt Learning Selective Prediction
Electrical and Computer Engineering, University of California, Los Angeles · Makerere University · St. Francis Hospital Nsambya +1
Know Before You Fetch: Calibrated Retrieval-Budget Allocation for Retrieval-Augmented Generation
Jun 29, 2026Zhe Dong, Fang Qin, Manish Shah +1Retrieval-Augmented Generation Selective Prediction
University of Maine at Presque Isle · Stanford University · Independent Researcher
Budgeted Act-or-Defer Multi-Agent LLM Deliberation with Local Reliability Bounds
Jun 28, 2026Mengdie Flora Wang, Haochen Xie, Guanghui Wang +2AI Agent Reliability Multi-Agent LLM Systems
1AWS Generative AI Innovation Center, USA · General Motors (GM), USA
HKVLM: Faithful Query--Region Binding for Frozen-Detector Visual Grounding
Jun 27, 2026Bo MaSelective Prediction Cross-Modal Alignment
Auckland University of Technology
Risk-Aware Selective Multimodal Driver Monitoring with Driver-State World Modeling
Jun 25, 2026Daosheng Qiu, Haozhuang Chi, Hao Su +4Selective Prediction World Models
Hubei University, Wuhan, China · Nanyang Technological University, Singapore · Osaka University, Osaka, Japan
Beyond Logprobs: A Multi-Signal Confidence Engine for LLM-Based Document Field Extraction
Jun 23, 2026Nitesh KumarSelective Prediction Document Information Extraction
1Perfios Software Solutions Pvt. Ltd.
Selective Time Series Forecasting via Metalearning
Jun 22, 2026Ricardo Inácio, Vitor Cerqueira, Marília Barandas +1Selective Prediction Time Series Forecasting
Faculdade de Engenharia da Universidade do Porto, Porto, Portugal · University of Coimbra, Portugal · Fraunhofer Portugal AICOS, Portugal
Interpretable Uncertainty Routing Separating Emotion Ambiguity from Distribution Shift in Facial Expression Recognition
Jun 21, 2026Keito Inoshita, Takato UenoUncertainty Quantification Selective Prediction
Faculty of Business and Commerce, Kansai University, Suita, Japan · Data Science and AI Innovation Research Promotion Center, Shiga University, Hikone, Japan · Graduate School of Data Science, Shiga University, Hikone, Japan
The Score Granularity Gap in Black-Box LLM Classification: A Comparative Study of Confidence Constructions
Jun 20, 2026Ao Sun, Tian Sun, Jiaxing GengLLM Evaluation Selective Prediction
Rejections Based on Predictive Uncertainty Enable Reliable Routine Soil Spectroscopy
Jun 19, 2026Jonas Schmidinger, Robin Gebbers, Marc-Olivier Gasser +3Uncertainty Quantification Selective Prediction
Joint Lab Artificial Intelligence and Data Science, Osnabrück University, Osnabrück, Germany. · Department of Agromechatronics, Leibniz Institute for Agricultural Engineering and Bioeconomy (ATB), Potsdam, Germany. · Research and Development Institute for the Agri-environment (IRDA), Qu´ebec City, Canada. +1
GroundControl: Anticipating Navigation Failures in Vision-Language Agents via Trajectory-Consistent Uncertainty Estimates
Jun 18, 2026Nastaran Darabi, Divake Kumar, Sina Tayebati +2Uncertainty Estimation for VLA Models Selective Prediction
University of Illinois at Chicago (UIC)
Confidence-Aware Automated Assessment of Student-Drawn Scientific Models
Jun 18, 2026Luyang Fang, Yingchuan Zhang, Jongchan Park +3Selective Prediction Vision Transformer
AI4STEM Education Center, Athens, GA, USA · Department of Statistics, University of Georgia, Athens, GA, USA
Abstention-Aware Personalized Object Rearrangement via Uncertainty-Guided LLM Assistance
Jun 15, 2026Sam Collin, Ali AyubSelective Prediction Robot Task Planning
Concordia University, Montreal, Quebec H3G 1M8, Canada
Look Again Before You Abstain:Budgeted Conformal Evidence Acquisition for Reliable Vision-Language Model
Jun 15, 2026Jian Xu, Delu Zeng, John Paisley +1VLM Hallucination Mitigation Active Perception
RIKEN iTHEMS · RIKEN AIP · South China University of Technology +1
Adapt or Calibrate? How Local Labels Buy Certified Coverage in Medical VQA
Jun 14, 2026Reza Khanmohammadi, Kundan Thind, Mohammad M. GhassemiVLM Evaluation Confidence Estimation for VLMs
Michigan State University East Lansing, MI, USA · Henry Ford Health Detroit, MI, USA
When Correct Edges Cannot Be Verified: A Provenance Gap in Incomplete KGQA and a Provenance-Favoring Completion Policy
Jun 14, 2026Yongqi Kang, Yu Fu, Yong ZhaoData Provenance Selective Prediction
Sichuan University
Mitigating Visual Hallucinations in Multimodal Systems through Retrieval-Augmented Reliability-Aware Inference
Jun 14, 2026Pratheswaran Hariharan, Haiping Xu, Donghui YanProbability Calibration Selective Prediction
Computer and Information Science University of Massachusetts, Dartmouth, MA · Mathematics and Data Science University of Massachusetts, Dartmouth, MA
False Sense of Safety in Selective Signal Classification: Auditing Bound Tightness and Exchangeability for Risk Control
Jun 13, 2026Jingwen Zhou, Mingzhe WangSelective Prediction Anomaly Detection
Xidian University, Xi’an, China
Capability Minimization as a Safety Primitive: Risk-Aware Causal Gating for Least-Privilege LLM Agents
Jun 11, 2026Laxmipriya Ganesh Iyer, Rahul Suresh BabuSelective Prediction Causal Inference
Independent Researcher
Towards Provably Fair Machine Learning: Bayesian Approaches For Consistent and Transparent Predictions
Jun 10, 2026Owen O'Neill, Fintan CostelloSelective Prediction Bayesian Inference
University College Dublin, Ireland
Density Ridge Selective Prediction for LLM and VLM Hallucination Detection under Calibration Label Scarcity
Jun 8, 2026Nina I. ShamsiHallucination Detection in VLMs Vision-Language Models
Northeastern University Boston, United States
Evidence Acquisition and Stopping via Lattice Distillation
Jun 8, 2026Yixuan Zhang, Yang Song, Jiayi Zhang +4Epidemic Forecasting Selective Prediction
Section of Health Data Science and AI, Department of Public Health, University of Copenhagen, Copenhagen, Denmark · Section of Health Data Science & AI, Department of Public Health, University of Copenhagen, Copenhagen, Denmark · Rutgers University, New Brunswick, NJ, USA +1
A Joint Finite-Sample Certificate for Adaptive Selective Conformal Risk Control
Jun 7, 2026Xiaoli Yu, Jiamiao LiuSelective Prediction Conformal Risk Control
School of Cyber Security and Information Law Chongqing University of Posts and Telecommunications Chongqing 400065, China · Department of Information, Xinqiao Hospital Army Medical University (Third Military Medical University) Chongqing 400037, China

Jul 3, 2026J/K move · Enter open · S save

Aligning Language Models with Selective Prediction

Gaoxiang Luo, Yifan Wu, Sinian Zhang, Aryan Deshwal+1

Department of Computer Science and Engineering · 2Bioinformatics and Computational Biology Program · 3Division of Biostatistics and Health Data Science

RL for Language Models LLM Alignment Selective Prediction LLM Reliability Language Model Post-Training

Open paper AskPDF arXiv

Large language models (LLMs) are increasingly deployed as critical decision-making components in high-stakes real-world AI systems, rendering LLM reliability a foremost practical concern. In this paper, we focus on enhancing LLM reliability through selective prediction (SP), a strategy that allows an LLM to only predict for inputs where it is likely to be correct (i.e., coverage) and hence reduce the error rate (i.e., risk) on that portion of inputs -- flagging the remaining inputs for future human discretion. In other words, SP improves LLM reliability by balancing the risk-coverage trade-off and enabling seamless human-AI collaboration. To integrate SP into LLMs, we focus on the LLM post-training alignment stage and propose to align LLMs with SP performance metrics, in contrast with existing LLM alignment methods that focus primarily on correctness or calibration metrics. Specifically, we propose a novel alignment framework, Reinforcement Learning for Selection Reward (RLSR), which targets the area under the risk-coverage curve (AURC) -- a popular SP performance metric -- as its alignment objective. RLSR achieves substantially better risk-coverage trade-off compared to multiple alignment baselines on both in-domain and out-of-domain tasks.