Calibrated Learning

Calibrated learning aims to improve the reliability and trustworthiness of machine learning models by ensuring their predicted probabilities accurately reflect the true likelihood of outcomes. Current research focuses on developing methods to achieve better calibration, particularly in challenging scenarios like federated learning, long-tailed data distributions, and multi-task/multi-modality settings, often employing techniques like Bayesian methods, ensemble learning, and novel loss functions. These advancements are crucial for deploying AI systems in high-stakes applications where accurate uncertainty quantification is paramount, improving decision-making in areas such as autonomous driving, healthcare, and industrial automation.

Papers

May 9, 2024

Compressed Bayesian Federated Learning for Reliable Passive Radio Sensing in Industrial IoT
Luca Barbieri, Stefano Savazzi, Monica Nicoli
Federated Learning Calibrated Learning Bayesian Federated Learning

April 17, 2024

Calibrating Bayesian Learning via Regularization, Confidence Minimization, and Selective Inference
Jiayi Huang, Sangwoo Park, Osvaldo Simeone
Bayesian Learning Selective Inference Calibrated Learning Calibrated Out of Distribution Confidence Minimization

September 26, 2023

MoCaE: Mixture of Calibrated Experts Significantly Improves Object Detection
Kemal Oksuz, Selim Kuzucu, Tom Joy, Puneet K. Dokania
Object Detector Mixture Component Open Vocabulary Object Detection Detection Confidence Calibrated Learning

July 31, 2023

FULLER: Unified Multi-modality Multi-task 3D Perception via Multi-level Gradient Calibration
Zhijian Huang, Sihao Lin, Guiyu Liu, Mukun Luo, Chaoqiang Ye, Hang Xu, Xiaojun Chang, Xiaodan Liang
Autonomous Driving Multi Task Learning Multi Task Multi Modality Fusion Calibrated Learning Multi Task 3D

December 7, 2022

A Flexible Nadaraya-Watson Head Can Offer Explainable and Calibrated Classification
Alan Q. Wang, Mert R. Sabuncu
Inherent Interpretability Neural Network Architecture Human Head Classification Head Nadaraya Watson Calibrated Learning Prediction Head

October 13, 2022

Smooth Calibration, Leaky Forecasts, Finite Recall, and Nash Dynamics
Dean P. Foster, Sergiu Hart
Nash Equilibrium Model Calibration Approximate Nash Equilibrium Online Linear Regression Calibrated Learning Deterministic Forecast Coarse Correlated Equilibrium Bounded Recall

July 25, 2022

Calibrated One-class Classification for Unsupervised Time Series Anomaly Detection
Hongzuo Xu, Yijie Wang, Songlei Jian, Qing Liao, Yongjun Wang, Guansong Pang
Anomaly Detection Unsupervised Time Series Anomaly Detection Calibrated Learning Normality Learning Calibration Network

June 10, 2022

Balanced Product of Calibrated Experts for Long-Tailed Recognition
Emanuel Sanchez Aimar, Arvi Jonnarth, Michael Felsberg, Marco Kuhlmann
Real World Long Tailed Long Tailed Recognition ImageNet LT Ensemble Diversity Balanced Approach Tail Label Calibrated Learning

April 18, 2022

Expert-Calibrated Learning for Online Optimization with Switching Costs
Pengfei Li, Jianyi Yang, Shaolei Ren
Hidden CoST Online Convex Optimization Online Algorithm Online Optimization Learning Augmented Learning Based Optimization Calibrated Learning

February 8, 2022

Calibrated Learning to Defer with One-vs-All Classifiers
Rajeev Verma, Eric Nalisnick
Calibrated Learning Multiclass Learning One v All Classifier

November 24, 2021

Adaptively Calibrated Critic Estimates for Deep Reinforcement Learning
Nicolai Dorka, Tim Welschehold, Joschka Boedecker, Wolfram Burgard
Deep Reinforcement Learning Adaptive Importance Policy Reinforcement Learning Multiple Critic Robotics Benchmark Quantile Temporal Difference Calibrated Learning Control Benchmark