Selective Prediction

Follow this topic Ask about this topic

Selective prediction studies how models should decide whether to make a prediction, abstain, or escalate an uncertain case to a person or another system. Common approaches use confidence and uncertainty scores, ensembles or verifiers, and calibration methods such as conformal prediction to set risk–coverage trade-offs. Applications range from image classification and clinical triage to language-model question answering and agent actions, where benchmarks increasingly assess accepted-case errors, coverage, review costs, and subgroup outcomes. A central challenge is that useful confidence rankings do not by themselves guarantee safe decisions: calibration and risk guarantees can depend on data assumptions, shift across domains or groups, and sharply reduce coverage.

Latest papers

292 papers58 papers in the last four weeks, up 38% on the four weeks before. 0.3% of all new papers.

High-Precision APT Malware Attribution with Out-of-Scope Resilience
Jun 2, 2026Peter Williams, Adam Sobey, Erisa KarafiliSelective Prediction Malware Classification
University of Southampton, Southampton, UK · The Alan Turing Institute, London, UK
Reliable Multilingual Orthopedic Decision Support from Clinical Narratives: Language-Aware Adaptation and Verification-Guided Deferral
May 29, 2026Danish Ali, Li Xiaojian, Sundas Iqbal +1Multilingual Language Models Selective Prediction
School of Computer Science, Wuhan University, China · School of Software, Nanjing University of Information Science and Technology (NUIST), China · Department of Orthopedic, Bahawal Victoria Hospital, Pakistan
AbstainGNN: Teaching Graph Neural Networks to Abstain for Graph Classification
May 29, 2026Xixun Lin, Zhiheng Zhou, Zhengyin Zhang +9Graph Neural Networks Selective Prediction
Institute of Information Engineering, Chinese Academy of Sciences · School of Mathematics and Statistics, Shandong University · China University of Mining and Technology-Beijing +6
Diversity Matters: Revisiting Test-Time Compute in Vision-Language Models
May 29, 2026Yijie Tong, Yifan Hou, Shaobo Cui +2VLM Evaluation Vision-Language Models
ETH Zürich · Shanghai Jiao Tong University · EPFL
BOKBO (Best of K Bad Options): Calibrated Abstention for VLA Policies
May 28, 2026Anya Singh, Cabrel Happi, Jai Relan +2Selective Prediction Conformal Prediction
Relling Systems
Improving Selective Classification with Pairwise Queries for Binary Classification
May 28, 2026Harsh Vardhan, Sunav Choudhary, Natwar Modani +1Pairwise Comparison Selective Prediction
CSE, UCSD, 9500 Gilman Drive, La Jolla, 92092, CA, USA. · Adobe Research, 345 Park Ave, San Jose, 95110, CA, USA. · Adobe Research, Marathahalli, Bengaluru, 560087, Karnataka, India. +1
CFCamo: A Counterfactual Detect-or-Abstain Framework for Camouflaged Object Detection
May 28, 2026Suhang Li, Osamu Yoshie, Yuya IeiriSelective Prediction Object Hallucination in VLMs
The authors are with the Graduate School of Information, Production and Systems, Waseda University, Fukuoka, Japan.
Wait! There's a Way Out: A Decision Mechanism for Forecasting Conversational Derailment
May 28, 2026Laerdon Kim, Vivian Nguyen, Cristian Danescu-Niculescu-MizilSelective Prediction
Cornell University
Risk-Controlled Lean-as-Judge for Natural-Language Mathematical Reasoning
May 27, 2026Pauline Bourigault, Xiaotong Ji, Matthieu Zimmer +2LLM Answer Verification Selective Prediction
Imperial College London · Huawei Noah’s Ark Lab · UCL Centre for AI
Auditable Decision Models with Learned Abstention and Real-Time Steering
May 26, 2026Sankaranarayanan Palamadai ChandrasekaranAI-Assisted Decision Making AI Accountability
Simple Machine Mind
DeepSciVerify: Verifying Scientific Claim--Citation Alignment via LLM-Driven Evidence Escalation
May 26, 2026Shaghayegh Sadeghi, Khashayar Khajavi, Rise Adhikari +1Selective Prediction Citation Verification
FirstPrinciples · School of Computing Science, Simon Fraser University
Trust but Verify: Prover-Verifier Deliberation for Selective LLM Prediction
May 24, 2026João Sedoc, Baotong Zhang, Dean FosterLLM Answer Verification Selective Prediction
New York University
Ternary Decision Trees with Locally-Adaptive Uncertainty Zones
May 21, 2026William SmitsUncertainty Quantification Selective Prediction
Avathon, Austin, TX, USA
Reliable Automated Triage in Spanish Clinical Notes: A Hybrid Framework for Risk-Aware HIV Suspicion Identification
May 20, 2026Rodrigo Morales-Sánchez, Soto Montalvo, Raquel MartínezSelective Prediction Healthcare
Dept. Lenguajes y Sistemas Informáticos, ETSI Informática, Universidad Nacional de Educación a Distancia (UNED) · Dept. Informática y Estadística, ETSI Informática, Universidad Rey Juan Carlos (URJC)
SAVER: Selective As-Needed Vision Evidence for Multimodal Information Extraction
May 20, 2026Miaobo Hu, Shuhao Hu, Bokun Wang +5Relation Extraction Named Entity Recognition
Institute of Information Engineering, Chinese Academy of Sciences, Beijing 100092, China · School of Cyber Security, University of Chinese Academy of Sciences, Beijing 100049, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China
BalanceRAG: Joint Risk Calibration for Cascaded Retrieval-Augmented Generation
May 19, 2026Zijun Jia, Yuanchang Ye, Sen Jia +6Retrieval-Augmented Generation Selective Prediction
Beihang University · Zhejiang University of Finance & Economics · Shenzhen Institute of Advanced Technology +1
Conformal Selective Acting: Anytime-Valid Risk Control for RLVR-Trained LLMs
May 18, 2026Hamed Khosravi, Xiaoming HuoSelective Prediction Anytime-Valid Inference
Georgia Institute of Technology
Knowing When Not to Predict: Self Supervised Learning and Abstention for Safer DR Screening
May 18, 2026Muskaan Chopra, Lorenz Sparrenberg, Jan H. Terheyden +1Selective Prediction Self-Supervised Pre-Training
Rheinische Friedrich-Wilhelms-Universität Bonn, Bonn, Germany · Lamarr Institute for Machine Learning and Artificial Intelligence, Germany · University Hospital Bonn - Department of Ophthalmology, Germany +1
Confidence-Gated Robot Autonomy: When Does Uncertainty Actually Help?
May 18, 2026Johannes A. Gaus, Jhon P. F. Charaja, Daniel HaeufleUncertainty Quantification Selective Prediction
Hertie Institute for Clinical Brain Research & Center for Integrative Neuroscience, University of Tübingen, Germany
Task Abstention for Large Language Models in Code Generation
May 16, 2026Yanke Zhou, Yuhao Tan, Senrong Xu +4Selective Prediction LLM Hallucination Mitigation
State Key Laboratory of Novel Software Technology Nanjing University · School of Computing and Mathematical Sciences University of London
When Evidence Conflicts: Uncertainty and Order Effects in Retrieval-Augmented Biomedical Question Answering
May 13, 2026Yikun Han, Mengfei Lan, Halil KilicogluSelective Prediction Healthcare
University of Illinois Urbana-Champaign
Pause and Reflect: Conformal Aggregation for Chain-of-Thought Reasoning
May 13, 2026Yu Gu, Zijun Yu, Vahid Partovi Nia +1Selective Prediction Self-Consistency Decoding
Department of Mathematics and Statistics McGill University Montréal, QC, Canada · Department of Mathematics and Industrial Engineering Polytechnique de Montréal Montréal, QC, Canada
Fair and Calibrated Toxicity Detection with Robust Training and Abstention
May 13, 2026Mokshit SuranaProbability Calibration Group Distributionally Robust Optimization
University of Illinois Chicago
Low-Cost Labels, Reliable Choices: Rollout-Calibrated Hyper-Heuristics for Job Shop Scheduling
May 11, 2026Junhao Wei, Yanxiao Li, Yifu Zhao +8Selective Prediction Metaheuristic Optimization
Faculty of Applied Sciences, Macao Polytechnic University, Macao, 999078, China · College of Animal Science and Technology, Zhongkai University of Agriculture and Engineering, Guangzhou, 510225, China · 4Macao Polytechnic University, Macao, 999078, China
Post-hoc Selective Classification for Reliable Synthetic Image Detection
May 9, 2026Kaixiang Zheng, Jacob H. SeidmanUncertainty Quantification Distribution Shift
University of Waterloo · Reality Defender
Flexible Routing via Uncertainty Decomposition
May 8, 2026Charlotte Peale, Siddartha Devic, Parikshit Gopalan +2Uncertainty Quantification Adaptive Model Routing
1Stanford University · University of Southern California · 3Apple
SGC-RML: A reliable and interpretable longitudinal assessment for PD in real-world DNS
May 8, 2026Wenbin Wei, Ruixiang Gao, Suyuan Yao +3Selective Prediction Unified Multimodal Models
∗ Equal contribution. · † Corresponding author.
A Regime Theory of Controller Class Selection for LLM Action Decisions
May 7, 2026Zhaoyang Jiang, Zhizhong Fu, Yunsoo Kim +4Selective Prediction Language Model-Based Control
University of Glasgow · UESTC, Chengdu, China · University College London
NoisyCoconut: Counterfactual Consensus via Latent Space Reasoning
May 6, 2026Michael Jerge, David EvansSelective Prediction LLM Inference
University of Virginia
Ask Before You Diagnose: Safe-Psych, a Sequential Evaluation Benchmark for LLMs in Psychiatry
May 6, 2026Oriana Presacan, Andreea Grama, Larisa Irimină +6Selective Prediction Healthcare
National University of Science and Technology Politehnica Bucharest · Psychiatric Hospital Doctor Gheorghe Preda · Oslo Metropolitan University +3

Jun 2, 2026J/K move · Enter open · S save

High-Precision APT Malware Attribution with Out-of-Scope Resilience

Peter Williams, Adam Sobey, Erisa Karafili

University of Southampton, Southampton, UK · The Alan Turing Institute, London, UK

Selective Prediction Malware Classification Malware Analysis OOD Detection

Open paper AskPDF arXiv

Early attribution of Advanced Persistent Threat (APT) activity can help defenders prioritise investigation, select countermeasures, and reduce the impact of an intrusion. Malware provides useful attribution evidence, but automated APT malware attribution remains difficult in practice. Existing approaches are typically trained and evaluated as closed-set classifiers over a limited number of known APT groups. In operational environments, however, classifiers are likely to encounter samples from groups not represented during training. Closed-set classifiers are then forced to assign such samples to known groups, producing unsupported and potentially misleading attributions. We present a high-precision APT malware attribution method based on ranked binary classifiers with explicit abstention. Rather than training a single multi-class classifier, our approach trains and tunes two binary classifiers per APT group, ranks the classifiers by validation performance, and applies them sequentially. A sample is attributed only when a classifier provides sufficient evidence; otherwise, it abstains. We evaluate the method on the APT Malware dataset and on a larger combined dataset designed to stress-test out-of-scope behaviour. On the APT Malware dataset, the method achieves higher precision than previously published results on the same dataset. In the most challenging setting, where 87% of test samples came from 60 APT groups excluded from training, the method abstained on 94% of out-of-scope samples while maintaining 92% precision and 95% selective accuracy on the samples it classified.