Aware Calibration

Aware calibration focuses on improving the reliability of predictions from various machine learning models, ensuring that predicted probabilities accurately reflect the true likelihood of outcomes. Current research emphasizes methods that leverage contextual information, such as neighboring data points in image segmentation or graph neural networks, or utilize nearest-neighbor techniques to refine predictions, particularly in few-shot learning scenarios. These advancements are crucial for enhancing the trustworthiness of AI systems across diverse applications, from medical image analysis and natural language processing to safety-critical decision-making systems. The goal is to achieve well-calibrated models that maintain high accuracy while providing reliable confidence estimates.

Papers

February 21, 2024

Accuracy-Preserving Calibration via Statistical Modeling on Probability Simplex
Yasushi Esaki, Akihiro Nakamura, Keisuke Kawano, Ryoko Tokuhisa, Takuro Kutsuna
Deep Neural Network Calibration Performance Cross Entropy Loss DNN Model Statistical Model Probability Simplex Aware Calibration

January 25, 2024

Neighbor-Aware Calibration of Segmentation Networks with Penalty-Based Constraints
Balamurali Murugesan, Sukesh Adiga Vasudeva, Bingyuan Liu, Hervé Lombaert, Ismail Ben Ayed, Jose Dolz
Segmentation Network Pixel Level Label Smoothing Segmentation Benchmark Penalty Based Aware Calibration

December 5, 2022

Improving Few-Shot Performance of Language Models via Nearest Neighbor Calibration
Feng Nie, Meixi Chen, Zhirui Zhang, Xu Cheng
Language Model Context Learning Shot Learning System Performance Aware Calibration

July 11, 2022

Personalizing Federated Medical Image Segmentation via Local Calibration
Jiacheng Wang, Yueming Jin, Liansheng Wang
Medical Image Segmentation Task Personalized Model Consistent Prediction Federated Medical Image Segmentation Aware Calibration

June 3, 2022

On Calibration of Graph Neural Networks for Node Classification
Tong Liu, Yushan Liu, Marcel Hildebrandt, Mitchell Joblin, Hang Li, Volker Tresp
Graph Neural Network Graph Drawing Link Prediction Node Classification Calibration Performance Graph Structure Aware Calibration

Aware Calibration

Papers

Accuracy-Preserving Calibration via Statistical Modeling on Probability Simplex

Neighbor-Aware Calibration of Segmentation Networks with Penalty-Based Constraints

Improving Few-Shot Performance of Language Models via Nearest Neighbor Calibration

Personalizing Federated Medical Image Segmentation via Local Calibration

On Calibration of Graph Neural Networks for Node Classification