Conformal Prediction

Conformal prediction is a model-agnostic framework for generating prediction intervals or sets with guaranteed coverage probabilities, addressing the crucial need for reliable uncertainty quantification in machine learning. Current research focuses on improving the efficiency and robustness of conformal prediction methods, particularly for non-i.i.d. data (e.g., time series, graphs) and biased models, exploring techniques like adaptive conformal prediction, weighted conformal prediction, and score refinement to achieve this. These advancements are significant because they enhance the trustworthiness and applicability of machine learning models in high-stakes domains such as healthcare, autonomous systems, and financial modeling, where reliable uncertainty estimates are paramount for safe and informed decision-making.

Papers

December 15, 2023

Reliable Prediction Intervals with Regression Neural Networks
Harris Papadopoulos, Haris Haralambous
Conformal Prediction Prediction Interval Point Prediction Regression Network Conventional Neural Network

December 14, 2023

Verification of Neural Reachable Tubes via Scenario Optimization and Conformal Prediction
Albert Lin, Somil Bansal
High Dimensional Conformal Prediction Verification Task Reachability Analysis Safety Guarantee Scenario Selection Reachable Tube

December 12, 2023

December 11, 2023

Uncertainty quantification in automated valuation models with locally weighted conformal prediction
Anders Hjort, Gudmund Horn Hermansen, Johan Pensar, Jonathan P. Williams
Uncertainty Quantification Conformal Prediction Prediction Uncertainty Valuation Model Non Parametric Learning Confidence Set

December 8, 2023

Conformal Prediction in Multi-User Settings: An Evaluation
Enrique Garcia-Ceja, Luciano Garcia-Banuelos, Nicolas Jourdan
Global Evaluation Conformal Prediction Performance Metric Multi User Conformal Score

November 21, 2023

On the Out-of-Distribution Coverage of Combining Split Conformal Prediction and Bayesian Deep Learning
Paul Scemama, Ariel Kapusta
Conformal Prediction Distribution Data Bayesian Deep Learning Split Conformal Prediction

November 14, 2023

Identifying Light-curve Signals with a Deep Learning Based Object Detection Algorithm. II. A General Light Curve Classification Framework
Kaiming Cui, D. J. Armstrong, Fabo Feng
Time Series Conformal Prediction Deep Learning Based Object Light Curve Novel Deep Learning Framework

November 10, 2023

Early-Exit Neural Networks with Nested Prediction Sets
Metod Jazbec, Patrick Forré, Stephan Mandt, Dan Zhang, Eric Nalisnick
Conformal Prediction Prediction Set Sequential Prediction Early Exit Neural Network Confidence Sequence

November 2, 2023

Conformal Policy Learning for Sensorimotor Control Under Distribution Shifts
Huang Huang, Satvik Sharma, Antonio Loquercio, Anastasios Angelopoulos, Ken Goldberg, Jitendra Malik
Reinforcement Learning Distribution Shift Conformal Prediction Conformalized Quantile Regression Policy Prediction Sensorimotor Control

November 1, 2023

Conformalized Deep Splines for Optimal and Efficient Prediction Sets
Nathaniel Diamant, Ehsan Hajiramezanali, Tommaso Biancalani, Gabriele Scalia
Conformal Prediction Prediction Interval Prediction Set Conformal Regression Conformal Score

October 25, 2023

Agreeing to Stop: Reliable Latency-Adaptive Decision Making via Ensembles of Spiking Neural Networks
Jiechen Chen, Sangwoo Park, Osvaldo Simeone
Spiking Neural Network Conformal Prediction Diverse Ensemble Recurrent Network Ensemble Model Latency Accuracy

October 24, 2023

Guaranteed Coverage Prediction Intervals with Gaussian Process Regression
Harris Papadopoulos
Uncertainty Estimation Conformal Prediction Gaussian Process Regression Prediction Interval

October 17, 2023

On the Temperature of Bayesian Graph Neural Networks for Conformal Prediction
Seohyeon Cha, Honggu Kang, Joonhyuk Kang
Graph Neural Network Uncertainty Quantification Conformal Prediction Prediction Set Bayesian Learning Temperature Measurement

October 16, 2023

October 15, 2023

Estimating Uncertainty in Multimodal Foundation Models using Public Internet Data
Shiladitya Dutta, Hongbo Wei, Lars van der Laan, Ahmed M. Alaa
Zero Shot Uncertainty Estimation Conformal Prediction Zero Shot Classification Available Datasets Multimodal Foundation Model Conformal Prediction Set

October 10, 2023

Conformal Prediction for Deep Classifier via Label Ranking
Jianguo Huang, Huajun Xi, Linjun Zhang, Huaxiu Yao, Yue Qiu, Hongxin Wei
Conformal Prediction Prediction Set Deep Learning Classification Training Label Adaptive Prediction

October 9, 2023

Conformal Decision Theory: Safe Autonomous Decisions from Imperfect Predictions
Jordan Lekeufack, Anastasios N. Angelopoulos, Andrea Bajcsy, Michael I. Jordan, Jitendra Malik
Conformal Prediction Planning Algorithm Decision Theory Safe Autonomous Decision Imperfect Prediction Autonomous Manufacturing

October 5, 2023

Distribution-free risk assessment of regression-based machine learning algorithms
Sukrita Singh, Neeraj Sarna, Yuanyuan Li, Yang Li, Agni Orfanoudaki, Michael Berger
Conformal Prediction Risk Assessment Distribution Free Machine Learning Regression Failure Probability

Conformal Prediction

Papers

Reliable Prediction Intervals with Regression Neural Networks

Verification of Neural Reachable Tubes via Scenario Optimization and Conformal Prediction

Empirical Validation of Conformal Prediction for Trustworthy Skin Lesions Classification

Multi-Modal Conformal Prediction Regions with Simple Structures by Optimizing Convex Shape Templates

Uncertainty quantification in automated valuation models with locally weighted conformal prediction

Conformal Prediction in Multi-User Settings: An Evaluation

On the Out-of-Distribution Coverage of Combining Split Conformal Prediction and Bayesian Deep Learning

Identifying Light-curve Signals with a Deep Learning Based Object Detection Algorithm. II. A General Light Curve Classification Framework

Early-Exit Neural Networks with Nested Prediction Sets

Conformal Policy Learning for Sensorimotor Control Under Distribution Shifts

Conformalized Deep Splines for Optimal and Efficient Prediction Sets

Agreeing to Stop: Reliable Latency-Adaptive Decision Making via Ensembles of Spiking Neural Networks

Guaranteed Coverage Prediction Intervals with Gaussian Process Regression

On the Temperature of Bayesian Graph Neural Networks for Conformal Prediction

Forking Uncertainties: Reliable Prediction and Model Predictive Control with Sequence Models via Conformal Risk Control

Conformal Contextual Robust Optimization

Estimating Uncertainty in Multimodal Foundation Models using Public Internet Data

Conformal Prediction for Deep Classifier via Label Ranking

Conformal Decision Theory: Safe Autonomous Decisions from Imperfect Predictions

Distribution-free risk assessment of regression-based machine learning algorithms