AUC Metric

The Area Under the Receiver Operating Characteristic Curve (AUC) is a widely used metric for evaluating the performance of binary classifiers, and increasingly, multi-class classifiers. Current research focuses on improving AUC calculation and interpretation, particularly addressing challenges in multi-class settings, handling imbalanced datasets, and mitigating biases that can inflate AUC scores. This includes developing new algorithms and loss functions to directly optimize AUC, as well as exploring methods for decomposing AUC to understand model performance across different subgroups and features. Accurate and unbiased AUC calculation is crucial for reliable model evaluation across diverse applications, from medical diagnosis to recommendation systems.

Papers

August 29, 2024

Coalitions of AI-based Methods Predict 15-Year Risks of Breast Cancer Metastasis Using Real-World Clinical Data with AUC up to 0.9
Xia Jiang, Yijun Zhou, Alan Wells, Adam Brufsky
Patient Data Prognosis Prediction AI Based Coalition Formation AUC Metric Neoadjuvant Chemotherapy Breast Cancer Metastasis

April 19, 2024

Multiclass ROC
Liang Wang, Luis Carvalho
Multi Class Classification Multiclass Classification AUC Metric

April 10, 2024

On Fixing the Right Problems in Predictive Analytics: AUC Is Not the Problem
Ryan S. Baker, Nigel Bosch, Stephen Hutt, Andres F. Zambrano, Alex J. Bowers
Predictive Model Right Problem Area Under the ROC Curve Bias Metric AUC Metric

February 29, 2024

Lower-Left Partial AUC: An Effective and Efficient Optimization Metric for Recommendation
Wentao Shi, Chenxu Wang, Fuli Feng, Yang Zhang, Wenjie Wang, Junkang Wu, Xiangnan He
Effective Recommendation Recommendation System Recommendation Task AUC Metric Partial AUC

February 24, 2024

Teacher-Student Learning on Complexity in Intelligent Routing
Shu-Ting Pi, Michael Yang, Yuying Zhu, Qun Liu
Complexity Matter E Commerce Machine Learning Framework Teacher Student Customer Experience Intelligent Routing AUC Metric

September 20, 2023

Fairness Hub Technical Briefs: AUC Gap
Jinsook Lee, Chris Brooks, Renzhe Yu, Rene Kizilcec
Absolute Stance Bias Model Bias Fairness Assessment AUC Metric Fairness Toolkits Intersectional Group

May 9, 2023

Towards Understanding Generalization of Macro-AUC in Multi-label Learning
Guoqiang Wu, Chongxuan Li, Yilong Yin
Strong Generalization Multi Label Generalization Property AUC Metric Label Imbalance

April 17, 2023

Enhancing Personalized Ranking With Differentiable Group AUC Optimization
Xiao Sun, Bo Zhang, Chenrui Zhang, Han Ren, Mingchen Cai
Recommendation Model Interest Loss Personalized Ranking AUC Metric AUC Optimization

March 29, 2023

Problems and shortcuts in deep learning for screening mammography
Trevor Tsue, Brent Mombourquette, Ahmed Taha, Thomas Paul Matthews, Yen Nhi Truong Vu, Jason Su
Deep Learning Deep Learning Model Model Bias Related Problem AI Model Mammography Report Shortcut Feature AUC Metric

March 3, 2023

Attention-based Saliency Maps Improve Interpretability of Pneumothorax Classification
Alessandro Wollek, Robert Graf, Saša Čečatka, Nicola Fink, Theresa Willem, Bastian O. Sabel, Tobias Lasser
Inherent Interpretability Saliency Map Chest Radiograph Pneumothorax Segmentation AUC Metric

May 24, 2022

Attributing AUC-ROC to Analyze Binary Classifier Performance
Arya Tafvizi, Besim Avci, Mukund Sundararajan
Area Under the ROC Curve AUC Metric Concept Attribution

March 1, 2022

When AUC meets DRO: Optimizing Partial AUC for Deep Learning with Non-Convex Convergence Guarantee
Dixian Zhu, Gang Li, Bokun Wang, Xiaodong Wu, Tianbao Yang
Deep Learning Robust Optimization Distributionally Robust Optimization Value at Risk AUC Metric Partial AUC Efficient Gradient

November 24, 2021

Fairness for AUC via Feature Augmentation
Hortense Fong, Vineet Kumar, Anay Mehrotra, Nisheeth K. Vishnoi
Procedural Fairness Fairness Accuracy Feature Augmentation Area Under the ROC Curve AUC Metric