Cost-Sensitive Learning

Follow this topic Ask about this topic

Cost-sensitive learning studies how models should change their predictions when some mistakes or actions are more costly than others. Common approaches reweight examples or losses, adapt penalties to instance difficulty or error severity, and optimize decisions such as test acquisition, abstention, or expert referral. These methods are used in settings including imbalanced classification, medical diagnosis, fraud detection, forecasting, and sequential decision-making, where errors and resources have unequal consequences. A central challenge is specifying costs that reflect real priorities: emphasizing costly cases can reduce rare severe errors, but may trade off performance elsewhere, and lower error costs do not always mean better ranking or calibration.

Latest papers

42 papers10 papers in the last four weeks, up 150% on the four weeks before. 0.1% of all new papers.

Cost-Aware Abstention for LLM-Based IT Ticket Classification
Oct 22, 2026Sachin GuptaSelective Prediction Text Classification
Independent Researcher United States
Circuits of LLM Classifiers
Oct 21, 2026Hadi Khalaf, Sajani Vithana, Kunal Talreja +3Large Language Model-Based Text Classification Language Model Ensembles
Harvard School of Engineering and Applied Science · Kempner Institute for the Study of Natural & Artificial Intelligence · School of Electrical and Computer Engineering, Georgia Institute of Technology +2
Decision-Target Reversal under Informative Label Missingness
Oct 18, 2026Farinborz Setoudehtazangi, Geoffrey J. McLachlanLearning with Missing Data Cost-Sensitive Learning
Dipartimento di Scienze Statistiche, Università di Padova, Padova, Italy · School of Mathematics and Physics, The University of Queensland, Brisbane, Australia
Towards optimal selection schedules for online active learning
Oct 16, 2026Hugo Cui, Francesca Mignacco, Francesco MoriActive Learning Linear Classification
Université Paris-Saclay, CNRS, Laboratoire de mathématiques d’Orsay, 91405, Orsay, France · Joseph Henry Laboratories of Physics and Lewis–Sigler Institute, Princeton University, Princeton NJ 08544, USA · Initiative for the Theoretical Sciences, The Graduate Center, City University of New York, 365 Fifth Ave, New York NY 10016, USA +1
The Lagrangian Structure of Cost-Penalized Active Feature Acquisition
Oct 15, 2026Joseph BinghamActive Feature Acquisition Constrained Optimization
Faculty of Computer Science Faculty of Biology Technion – Israel Institute of Technology Haifa 3200003, Israel
ML-CLAPPS: A Probabilistic Framework for Imbalanced Multilabel Learning under Class Prioritization
Oct 11, 2026Priel Hazan, Nir Almog, Noam Keidar +2Multi-Label Classification Class-Imbalanced Learning
Technion – Israel Institute of Technology, Haifa, Israel
Beyond Second-Order Information: Paid Logistic Regression with Noisy Features
Oct 9, 2026Sanghwa Kim, Kyoungseok Jang, Se-Young YunOnline Learning Logistic Regression
Kim Jaechul Graduate School of AI, KAIST Seoul 02455, Republic of Korea · Department of AI, Chung-Ang University Soeul 06974, Republic of Korea
A Query Is Not a Commitment: Learning to Correct Expert Answers in Online Deferral
Oct 5, 2026Yannis Montreuil, Axel Carlier, Lai Xing Ng +1Cost-Sensitive Learning Learning to Defer
School of Computing, National University of Singapore · IRIT, Toulouse INP, France · Institute for Infocomm Research, A*STAR, Singapore
When Is Coarse Supervision Worth It? Cost-Aware Learning under Unknown Aggregation
Sep 29, 2026Jianyu Xu, Smriti Jha, Aarti Singh +1Cost-Sensitive Learning Statistical Learning Theory
University of North Carolina at Charlotte, Charlotte, NC 28223 · Carnegie Mellon University, Pittsburgh, PA 15213
Introducing the CZAR Loss: A Tailored Objective Function for Financial Log-Return Predictions
Sep 28, 2026Joel Pfeffer, J. M. Diederik Kruijssen, Florian Stecker +1Financial Forecasting Time Series Forecasting
Allora Foundation · LJMU · Liverpool John Moores University
Cost-Sensitive Online Window Size Selection for Portfolio Management
Sep 24, 2026Yi-Chen Liu, Chung-Han HsiehCost-Sensitive Learning Regret Minimization
Interdisciplinary Program of Management and Technology, National Tsing Hua University · Department of Quantitative Finance, National Tsing Hua University, Taiwan.
Misaligned Clinical Risk Classification and Cost Asymmetry in Open-Weight Large Language Models
Sep 21, 2026Star S. D. Liu, Xiyu Ding, Robert B. Barrett +3Cost-Sensitive Learning LLM Interpretability
Biomedical Informatics and Data Science, Johns Hopkins University School of Medicine, Baltimore, MD, USA · Microsoft, Redmond, WA, USA · National Institutes of Health, Bethesda, MD, USA
Adaptive Cost-Sensitive Machine Learning for Autonomous Robot Navigation Failure Prediction: When Not All Errors Are Equal
Sep 4, 2026Rifa FerzanaCost-Sensitive Learning Safe Robot Navigation
School of Informatics, University of Edinburgh, Edinburgh, UK
When and What to Teach: Budget-Aware Online Adaptation for Web Agents
Aug 31, 2026Jianwei Zhang, Sihan Cao, Pengcheng Zheng +7Online Knowledge Distillation LLM Agent Training
University of Electronic Science and Technology of China · Institute of Software Chinese Academy of Sciences · Xi’an University of Architecture and Technology
Cost-Sensitive Conformal Prediction and Human-in-the-Loop Abstention for Imbalanced High-Stakes Decision Support: A Multi-Domain Benchmark
Jul 29, 2026Manpreet Singh, Akshatha Srikantha, Shyamal LakhanpalClass-Imbalanced Learning Cost-Sensitive Learning
Cost-Aware Recovery-Pathway Identification and Bayesian Optimization for Autonomous Materials Discovery
Jul 26, 2026Debajyoti Ray, Niranjan SrinivasCost-Sensitive Learning Bayesian Optimization
Coactive Science
Asymmetric Peak-Aware Loss for Peak-Critical Time Series Forecasting
Jul 16, 2026Theivaprakasham Hari, Yanan Xin, Winnie Daamen +2Multivariate Time Series Forecasting Long-Term Time Series Forecasting
Department of Transport and Planning, Faculty of Civil Engineering and Geosciences Delft University of Technology, Delft, The Netherlands · Mobility Innovation Centre Delft University of Technology, Delft, The Netherlands
Removable Defects: The Economics and Limits of Deliberate Deficiency
Jul 13, 2026Cheng QianCost-Sensitive Learning Selective Prediction
Decision-Aware Training for Sample-Based Generative Models
Jul 1, 2026Kornelius Raeth, Nicole LudwigCost-Sensitive Learning Probabilistic Forecasting
T ubingen AI Center, University of T ubingen, Germany · University of Augsburg, Germany
Deep Neural Networks with Ordinal Loss for Medical Applications
Jun 24, 2026Tal Dvora, Rotem Haba, Gonen SingerCost-Sensitive Learning Ordinal Regression
Bar-Ilan University Ramat Gan, Israel
SDM-Q: Cost-Aware Staged Decision-Making for Multi-Omics Classification with Deep Q-Learning
May 29, 2026Nan Mu, Yangfan Xiao, Ling Wang +3Cost-Sensitive Learning Deep Q-Learning
College of Computer Science, Sichuan Normal University, Chengdu, Sichuan 610101, China. · Department of Mathematics, College of Science and Mathematics, Kennesaw State University, Kennesaw, GA 30144, USA. · Department of Computer Science, College of Computing and Software Engineering, Kennesaw State University, Marietta, GA 30060, USA.
Scalable Decision-Focused Learning through Cost-Sensitive Regression
May 18, 2026Noah Schutte, Senne Berden, Tias Guns +2Cost-Sensitive Learning Decision-Focused Learning
Delft University of Technology · KU Leuven · Independent Researcher
Instance-Level Costs for Nuanced Classifier Evaluation
May 4, 2026Kabir Kang, Stephen MussmannCost-Sensitive Learning Classification
School of Computer Science, Georgia Institute of Technology, Atlanta, GA, USA.
Optimized Deferral for Imbalanced Settings
Apr 30, 2026Corinna Cortes, Anqi Mao, Mehryar Mohri +1Cost-Sensitive Learning Learning to Defer
Google Research, New York, NY · Courant Institute of Mathematical Sciences, New York, NY
Risk-Aware Robust Learning: Reducing Clinical Risk under Label Noise in Medical Image Classification
Apr 26, 2026Maycon R. S. Pereira, Filipe R. CordeiroCost-Sensitive Learning Medical Image Classification
Visual Computing Lab, Departamento de Computac¸˜ao, Universidade Federal Rural de Pernambuco (UFRPE), Brasil
Risk-Calibrated Learning: Minimizing Fatal Errors in Medical AI
Apr 14, 2026Abolfazl Mohammadi-Seif, Ricardo Baeza-YatesMedical Image Classification AI Safety
Department of Engineering, Universitat Pompeu Fabra, Barcelona, Spain
Cost-optimal Sequential Testing via Doubly Robust Q-learning
Apr 13, 2026Doudou Zhou, Yiran Zhang, Dian Jin +3Q-Learning Cost-Sensitive Learning
Department of Statistics and Data Science, National University of Singapore · Public Health Sciences Division, Fred Hutch · Department of Biomedical Data Science, Stanford University School of Medicine +1
GraphDx: A Cost-Aware Knowledge-Enhanced Multi-Agent Framework for Sequential Diagnosis
Apr 8, 2026Shaoting Tan, Ning Liu, Yuntao Du +7Clinical Decision-Making Multi-Agent LLM Systems
Shandong University · Nankai University · East China Normal University +1
Cost-sensitive retraining via posterior learning debt
Apr 7, 2026Harrison KatzBayesian Decision Theory Regret Minimization
Finance Data Science & Strategy, Airbnb Inc., San Francisco, CA, USA
Severity-Aware Weighted Loss for Arabic Medical Text Generation
Apr 7, 2026Ahmed Alansary, Molham Mohamed, Ali HamdiMedical LLMs Text Generation
October University for Modern Sciences & Arts, Egypt

Oct 22, 2026J/K move · Enter open · S save

Cost-Aware Abstention for LLM-Based IT Ticket Classification

Sachin Gupta

Independent Researcher United States

Selective Prediction Text Classification Cost-Sensitive Learning Language Model Calibration

Open paper AskPDF arXiv

Wrong automatic routing of an IT support ticket can cost more than human triage. We evaluate cost-aware abstention as an offline, cost-conditional readiness assessment: select a routing policy on calibration data under explicit costs, then evaluate that frozen policy once on held-out test data. On a public 47,837-ticket, eight-class corpus, we compare four trained non-LLM classifiers with four large language models under zero- and few-shot prompting. Under the illustrative cost model (c_ok, c_triage, c_wrong) = (0, 1, 5), TF-IDF with logistic regression and a threshold of 0.60 selected on calibration reduces test cost from 0.763 to 0.472, a 38.1% relative reduction (paired ratio-bootstrap 95% CI [35.7%, 40.4%]), while lowering wrong routes from 15.3% to 3.5%. Frozen calibration-selected policies also beat both always-route and always-defer on held-out test data in 14 of 16 settings in a post-hoc cost-parameter analysis. The strongest prompted configuration reaches 0.477 accuracy on a stratified 1,000-ticket test subset, versus 0.879 for fine-tuned DistilBERT on the same tickets. Isotonic lowers expected calibration error without reliably improving discrimination: GPT-4o-mini few-shot has one confidence value before and after recalibration and correctness AUROC 0.500. We therefore report correctness AUROC and tie-aware risk-coverage area alongside ECE. At the analytically derived break-even probability 0.80, seven prompted configurations defer all tickets; Haiku few-shot routes only 19/1,000. An LLM-only agreement control and margin/entropy rejection baselines further separate calibration from discrimination. The findings are scoped to this dataset, prompt design, model endpoints, and illustrative costs; the work is an evaluation framework, not a new abstention algorithm or universal deployment certification.