Cost-Sensitive Learning

Ask about this topic

Overview

Cost-sensitive learning studies how models should change their predictions when some mistakes or actions are more costly than others. Common approaches reweight examples or losses, adapt penalties to instance difficulty or error severity, and optimize decisions such as test acquisition, abstention, or expert referral. These methods are used in settings including imbalanced classification, medical diagnosis, fraud detection, forecasting, and sequential decision-making, where errors and resources have unequal consequences. A central challenge is specifying costs that reflect real priorities: emphasizing costly cases can reduce rare severe errors, but may trade off performance elsewhere, and lower error costs do not always mean better ranking or calibration.

Latest papers

CardsList

42 papers10 papers in the last four weeks, up 150% on the four weeks before. 0.1% of all new papers.

Apr 5, 2026

Learning an Interpretable Risk Scoring System for Maximizing Decision Net Benefit

Risk scoring systems are widely used in high-stakes domains to assist decision-making. However, existing approaches often focus on optimizing predictive accuracy or likelihood-based criteria, which may not align with the main goal of maximizing utility. In this paper, we propose a novel risk scoring system that directly optimizes net benefit over a range of decision thresholds. The model is formulated as a sparse integer linear programming problem which enables the construction of a transparent scoring system with integer coefficients, and hence, facilitates interpretation and practical application. We also establish fundamental relationships among net benefit, discrimination, and calibration. Specifically, we derive bounds relating the area under the net benefit curve to a ROC functional, both evaluated on a fixed threshold grid, and show that post-processing can achieve moderate calibration on the training data without decreasing the area under the net benefit curve on that grid. We evaluated our method on multiple public datasets as well as on a large-scale credit risk dataset. This computational study demonstrated that our interpretable method can effectively achieve high net benefit while maintaining competitive discrimination and calibration performance.

Wenhao Chi, Ş. İlker Birbil

Yau Mathematical Sciences Center, Tsinghua University · Amsterdam Business School, University of Amsterdam

Cost-Sensitive LearningInterpretable ML
AskPDF
Mar 25, 2026

Cost-Sensitive Neighborhood Aggregation for Heterophilous Graphs: When Does Per-Edge Routing Help?

Recent work distinguishes two heterophily regimes: adversarial, where cross-class edges dilute class signal and harm classification, and informative, where the heterophilous structure itself carries useful signal. We ask: when does per-edge message routing help, and when is a uniform spectral channel sufficient? To operationalize this question we introduce Cost-Sensitive Neighborhood Aggregation (CSNA), a GNN layer that computes pairwise distance in a learned projection and uses it to soft-route each message through concordant and discordant channels with independent transformations. Under a contextual stochastic block model we show that mean aggregation can reverse the label-aligned signal direction under heterophily, and that cost-sensitive weighting with w+/w−>q/pw_+/w_- > q/p preserves the correct sign. On six benchmarks with uniform tuning, CSNA is competitive with state-of-the-art methods on adversarial-heterophily datasets (Texas, Wisconsin, Cornell, Actor) but underperforms on informative-heterophily datasets (Chameleon, Squirrel) -- precisely the regime where per-edge routing has no useful decomposition to exploit. The pattern is itself the finding: the cost function's ability to separate edge types serves as a diagnostic for the heterophily regime, revealing when fine-grained routing adds value over uniform channels and when it does not. Code is available at https://github.com/eyal-weiss/CSNA-public .

Eyal Weiss

Computer Science Faculty, Technion — Israel Institute of Technology

Graph Neural NetworksGraph HeterophilyNode ClassificationCost-Sensitive Learning
AskPDF
Mar 15, 2026

Consistent Learning-to-Defer with Expert-Conditional Advice

We introduce learning-to-defer with advice, a new setting in which a system learns both which expert should handle an input and which information that expert should receive. The policy acts before observing advice and minimizes prediction loss plus expert and acquisition fees. Training may reveal all advice sources or only a subset on each record. We show that composing routing and advice losses can be inconsistent and that augmented training can incur avoidable estimation variance. We propose reference conditional classification, which jointly learns a fixed expert's error probability and each action's error conditional on that expert's success or failure. The reference outcome is observed on every labeled record, so records with missing advice still inform the estimates. We prove a consistency bound for bounded task losses, show that limited training preserves the full population objective, and establish variance reductions for the analyzed fits. On FEVER and Sensitive, full training has the lowest mean dev loss at every evaluated fee, while limited training improves on full training at most matched acquisition budgets.

Yannis Montreuil, Leïna Montreuil, Axel Carlier +2

School of Computing National University of Singapore Singapore, 117418 · Département de Mathématiques Sorbonne University Paris, 75005, France · Fédération ENAC ISAE-SUPAERO ONERA Université de Toulouse Toulouse, 31555, France +1

Learning to DeferCost-Sensitive Learning
AskPDF
Mar 14, 2026

Every Error has Its Magnitude: Asymmetric Mistake Severity Training for Multiclass Multiple Instance Learning

Multiple Instance Learning (MIL) has emerged as a promising paradigm for Whole Slide Image (WSI) diagnosis, offering effective learning with limited annotations. However, existing MIL frameworks overlook diagnostic priorities and fail to differentiate the severity of misclassifications in multiclass, leaving clinically critical errors unaddressed. We propose a mistake-severity-aware training strategy that organizes diagnostic classes into a hierarchical structure, with each level optimized using a severity-weighted cross-entropy loss that penalizes high-severity misclassifications more strongly. Additionally, hierarchical consistency is enforced through probabilistic alignment, a semantic feature remix applied to the instance bag to robustly train class priority and accommodate clinical cases involving multiple symptoms. An asymmetric Mikel's Wheel-based metric is also introduced to quantify the severity of errors specific to medical fields. Experiments on challenging public and real-world in-house datasets demonstrate that our approach significantly mitigates critical errors in MIL diagnosis compared to existing methods. We present additional experimental results on natural domain data to demonstrate the generalizability of our proposed method beyond medical contexts.

Sungrae Hong, Jiwon Jeong, Jisu Shin +4

Korea Advanced Institute of Science and Technology, Daejeon, South Korea · Seegene Medical Foundation, Seoul, South Korea

Multiple Instance LearningCost-Sensitive LearningHistopathology Image ClassificationHierarchical Classification
AskPDF
Mar 13, 2026

FraudFox: Adaptable Fraud Detection in the Real World

The proposed method (FraudFox) provides solutions to adversarial attacks in a resource constrained environment. We focus on questions like the following: How suspicious is Smith', trying to buy \$500 shoes, on Monday 3am? How to merge the risk scores, from a handful of risk-assessment modules (oracles') in an adversarial environment? More importantly, given historical data (orders, prices, and what-happened afterwards), and business goals/restrictions, which transactions, like the Smith' transaction above, which ones should we pass', versus send to human investigators? The business restrictions could be: at most $x$ investigations are feasible', or at most $yy lost due to fraud'. These are the two research problems we focus on, in this work. One approach to address the first problem (`oracle-weighting'), is by using Extended Kalman Filters with dynamic importance weights, to automatically and continuously update our weights for each 'oracle'. For the second problem, we show how to derive an optimal decision surface, and how to compute the Pareto optimal set, to allow what-if questions. An important consideration is adaptation: Fraudsters will change their behavior, according to our past decisions; thus, we need to adapt accordingly. The resulting system, \method, is scalable, adaptable to changing fraudster behavior, effective, and already in production at Amazon. FraudFox augments a fraud prevention sub-system and has led to significant performance gains.

Matthew Butler, Yi Fan, Christos Faloutsos

Amazon.com, Inc., 410 Terry Ave N, Seattle WA 98109, U.S.A. · Carnegie Mellon University, 5000 Forbes Avenue, Pittsburgh, U.S.A.

Financial Fraud DetectionEnsemble LearningCost-Sensitive Learning
AskPDF
Mar 11, 2026

Relaxed Efficient Acquisition of Context and Temporal Features

In many biomedical applications, measurements are not freely available at inference time: each laboratory test, imaging modality, or assessment incurs financial cost, time burden, or patient risk. Longitudinal active feature acquisition (LAFA) seeks to optimize predictive performance under such constraints by adaptively selecting measurements over time, yet the problem remains inherently challenging due to temporally coupled decisions (missed early measurements cannot be revisited, and acquisition choices influence all downstream predictions). Moreover, real-world clinical workflows typically begin with an initial onboarding phase, during which relatively stable contextual descriptors (e.g., demographics or baseline characteristics) are collected once and subsequently condition longitudinal decision-making. Despite its practical importance, the efficient selection of onboarding context has not been studied jointly with temporally adaptive acquisition. We therefore propose REACT (Relaxed Efficient Acquisition of Context and Temporal features), an end-to-end differentiable framework that simultaneously optimizes (i) selection of onboarding contextual descriptors and (ii) adaptive feature--time acquisition plans for longitudinal measurements under cost constraints. REACT employs a Gumbel--Sigmoid relaxation with straight-through estimation to enable gradient-based optimization over discrete acquisition masks, allowing direct backpropagation from prediction loss and acquisition cost. Across real-world longitudinal health and behavioral datasets, REACT achieves improved predictive performance at lower acquisition costs compared to existing longitudinal acquisition baselines, demonstrating the benefit of modeling onboarding and temporally coupled acquisition within a unified optimization framework.

Yunni Qu, Dzung Dinh, Grant King +5

The University of North Carolina at Chapel Hill · University of North Carolina at Chapel Hill, Chapel Hill, NC, USA · University of Michigan +3

Active Feature AcquisitionFeature SelectionCost-Sensitive Learning
AskPDF
Mar 4, 2026

Algorithmic Compliance and Regulatory Loss in Digital Assets

We study the deployment performance of machine learning based enforcement systems used in cryptocurrency anti money laundering (AML). Using forward looking and rolling evaluations on Bitcoin transaction data, we show that strong static classification metrics substantially overstate real world regulatory effectiveness. Temporal nonstationarity induces pronounced instability in cost sensitive enforcement thresholds, generating large and persistent excess regulatory losses relative to dynamically optimal benchmarks. The core failure arises from miscalibration of decision rules rather than from declining predictive accuracy per se. These findings underscore the fragility of fixed AML enforcement policies in evolving digital asset markets and motivate loss-based evaluation frameworks for regulatory oversight.

Khem Raj Bhatt, Krishna Sharma

Anti-Money LaunderingAnti-Money Laundering DetectionModel CalibrationConcept DriftCost-Sensitive LearningFinancial Fraud Detection+1
AskPDF
Feb 18, 2026

Prediction of Major Solar Flares Using Interpretable Class-dependent Reward Framework with Active Region Magnetograms and Domain Knowledge

In this work, we develop, for the first time, a supervised classification framework with class-dependent rewards (CDR) to predict ≥\geqMM flares within 24 hr. We construct multiple datasets, covering knowledge-informed features and line-of sight (LOS) magnetograms. We also apply three deep learning models (CNN, CNN-BiLSTM, and Transformer) and three CDR counterparts (CDR-CNN, CDR-CNN-BiLSTM, and CDR-Transformer). First, we analyze the importance of LOS magnetic field parameters with the Transformer, then compare its performance using LOS-only, vector-only, and combined magnetic field parameters. Second, we compare flare prediction performance based on CDR models versus deep learning counterparts. Third, we perform sensitivity analysis on reward engineering for CDR models. Fourth, we use the SHAP method for model interpretability. Finally, we conduct performance comparison between our models and NASA/CCMC. The main findings are: (1)Among LOS feature combinations, R_VALUE and AREA_ACR consistently yield the best results. (2)Transformer achieves better performance with combined LOS and vector magnetic field data than with either alone. (3)Models using knowledge-informed features outperform those using magnetograms. (4)While CNN and CNN-BiLSTM outperform their CDR counterparts on magnetograms, CDR-Transformer is slightly superior to its deep learning counterpart when using knowledge-informed features. Among all models, CDR-Transformer achieves the best performance. (5)The predictive performance of the CDR models is not overly sensitive to the reward choices.(6)Through SHAP analysis, the CDR model tends to regard TOTUSJH as more important, while the Transformer tends to prioritize R_VALUE more.(7)Under identical prediction time and active region (AR) number, the CDR-Transformer shows superior predictive capabilities compared to NASA/CCMC.

Zixian Wu, Xuebao Li, Yanfang Zheng +14

School of Computer Science, Jiangsu University of Science and Technology, Zhenjiang 212100, People′s Republic of China · State Key Laboratory of Space Weather, Chinese Academy of Sciences, Beijing 100190, People′s Republic of China · School of software, Southeast University, Nanjing, People′s Republic of China +4

Solar PhysicsCost-Sensitive Learning
AskPDF
Jan 30, 2026

Learning-to-Defer in Non-Stationary Time Series via Switching State-Space Models

Learning-to-defer (L2D) lets a predictor decide, at each round, whether to issue its own forecast or pay for an expert's. In non-stationary time series this decision must keep adapting, although deployment reveals only the consulted expert's forecast while a historical archive records every expert with the target. L2D-SLDS learns from this archive a switching state-space model of the target and all expert forecasts, whose shared and expert-specific states describe how experts move together and apart. Its predictive law supplies the internal forecast and the expected cost of every consultation, which a greedy router minimizes, and one consultation also updates the beliefs about unconsulted and unavailable experts. We prove sublinear regret against a changing conditional-risk oracle without exploration, when the candidate models are accurate and either the archive separates them or live feedback reveals cost differences. On three real datasets, L2D-SLDS has the lowest cost among eight bandit routers and adapts its consultation rate to the fee.

Yannis Montreuil, Letian Yu, Axel Carlier +2

School of Computing National University of Singapore · Université de Toulouse Fédération ENAC ISAE-SUPAERO ONERA · Institute for Infocomm Research A*STAR, Singapore

Cost-Sensitive LearningAdaptive Model RoutingLearning to DeferDecision-Focused LearningNon-Stationary Time Series ForecastingState Space Models
AskPDF
Sep 17, 2025

Risk-Calibrated Bayesian Streaming Intrusion Detection with SRE-Aligned Decisions

[Corrected v2: an audit found that the score, threshold, and latency descriptions below are not what the shared codebase implements, and that the evaluation streams are assembled constructions. See the correction note on the title page and the corrected companion work, arXiv:2605.24696 (corrected v3), artifact doi:10.5281/zenodo.22673735.] We present a risk-calibrated approach to streaming intrusion detection that couples Bayesian Online Changepoint Detection (BOCPD) with decision thresholds aligned to Site Reliability Engineering (SRE) error budgets. BOCPD provides run-length posteriors that adapt to distribution shift and concept drift; we map these posteriors to alert decisions by optimizing expected operational cost under false-positive and false-negative budgets. We detail the hazard model, conjugate updates, and an O(1)-per-event implementation. A concrete SRE example shows how a 99.9% availability SLO (43.2 minutes per month error budget) yields a probability threshold near 0.91 when missed incidents are 10x more costly than false alarms. We evaluate on the full UNSW-NB15 and CIC-IDS2017 benchmarks with chronological splits, comparing against strong unsupervised baselines (ECOD, COPOD, and LOF). Metrics include PR-AUC, ROC-AUC, Brier score, calibration reliability diagrams, and detection latency measured in events. Results indicate improved precision-recall at mid to high recall and better probability calibration relative to baselines. We release implementation details, hyperparameters, and ablations for hazard sensitivity and computational footprint. Code and reproducibility materials will be made available upon publication; datasets and implementation are available from the corresponding author upon reasonable request.

Michel A. Youssef

Independent Researcher · Independent Researcher, Lebanon

Cost-Sensitive LearningProbability CalibrationOnline Anomaly DetectionNetwork Intrusion Detection
AskPDF
Jun 2, 2025

Model-agnostic Mitigation Strategies of Data Imbalance for Regression

Data imbalance persists as a pervasive challenge in regression tasks, introducing bias in model performance and undermining predictive reliability. This is particularly detrimental in applications aimed at predicting rare events that fall outside of the domain of the bulk of the training data. In this study, we review the current state-of-the-art regarding sampling-based methods and cost-sensitive learning. Additionally, we propose novel approaches to mitigate model bias. To better assess the importance of data, we introduce the density-distance and density-ratio relevance functions, which effectively integrate empirical frequency of data with domain-specific preferences, offering enhanced interpretability for end-users. Furthermore, we present advanced mitigation techniques (cSMOGN and crbSMOGN), which build upon and improve existing sampling methods. In a quantitative evaluation, we benchmark state-of-the-art methods on 10 synthetic and 42 real-world datasets, using neural networks, XGBoosting trees and Random Forest models. Our analysis shows that while most strategies improve performance on rare samples, they degrade it on frequent ones. The trade-off becomes larger the more the performance on rare samples is increased. However, to reduce this effect we demonstrate that constructing an ensemble of models -- one trained with imbalance mitigation and another without -- can be used. The key findings underscore the superior performance of our novel crbSMOGN sampling technique with the density-ratio relevance function for neural networks, outperforming state-of-the-art methods.

Jelke Wibbeke, Sebastian Rohjans, Andreas Rauh

Department of Computing Science Carl von Ossietzky Universität Oldenburg · Energy Division, OFFIS - Institute of Information Technology, Oldenburg, Germany · Department for Civil Engineering, Geoinformation and Health Technology Jade University of Applied Science

Cost-Sensitive Learning
AskPDF
Sep 19, 2024

iCost: A Novel Instance-Complexity-Based Cost-Sensitive Learning Framework

Class imbalance poses a significant challenge in classification tasks, often causing standard learning algorithms to become biased toward the majority class. Cost-sensitive learning (CSL) addresses this issue by assigning higher penalties to minority-class misclassifications. However, conventional CSL typically applies a uniform penalty to all minority-class instances, ignoring the fact that minority samples may differ substantially in terms of local safety, overlap, boundary ambiguity, and outlier-like behavior. Uniform penalization can therefore introduce undue bias, increasing the number of misclassifications. In this study, we propose iCost, an instance-complexity-aware CSL framework that assigns adaptive penalties to minority-class samples according to their estimated learning difficulty. This fine-grained penalization strategy ensures fairer weighting, reduces unwarranted bias, and improves overall classification performance. Two complementary complexity estimation strategies are introduced: Neighbor-iCost, based on local neighborhood composition, and Gini-iCost, based on Gini-impurity-based feature-space partitioning. Extensive experiments on 65 binary and 10 multiclass imbalanced datasets show that iCost outperforms conventional CSL by a clear margin and remains highly competitive with widely used resampling methods. To support reproducibility and practical adoption, the proposed algorithm has been released as a scikit-learn-compatible Python package through PyPI. This work offers a fresh perspective on imbalanced learning by integrating instance-level data complexity into the learning process, opening new avenues for developing adaptive, complexity-aware strategies for imbalanced classification.

Asif Newaz, Asif Ur Rahman Adib, Taskeed Jabid

Department of Electrical and Electronic Engineering, Islamic University of Technology (IUT), Gazipur, Bangladesh · Department of Electrical and Electronic Engineering, Ahsanullah University of Science and Technology (AUST), Dhaka, Bangladesh · Department of Computer Science and Engineering, East West University, Dhaka, Bangladesh

Class-Imbalanced LearningCost-Sensitive LearningAdaptive Loss Weighting
AskPDF