Adaptive Conformal Inference

Overview

Adaptive conformal inference studies how to update prediction thresholds as outcomes arrive, helping prediction intervals or sets maintain target error rates when data drift or are selected online. Most methods use coverage feedback to adjust a quantile or significance level; variants add sliding windows, recency or density weighting, regime-specific calibration, and score correction. Researchers assess both coverage and efficiency, such as interval width, in settings including time-series forecasting, anomaly detection, and safety-critical control, alongside analyses using online learning and game theory. The central challenge is balancing narrow predictions and responsive adaptation against reliable guarantees: exchangeability supports finite-sample marginal validity for classical conformal methods, while distribution shift and sparse calibration data can cause undercoverage, and stronger conditional guarantees often require tradeoffs or extra assumptions.

Latest papers

32 papers4 papers in the last four weeks, level with the four weeks before. 0.0% of all new papers.

Oct 17, 2025

Adaptive Conformal Inference through the Lens of Blackwell Approachability

This article considers an online version of conformal inference, called adaptive conformal inference [ACI] and introduced by Gibbs and Candès (2021): prediction sets are issued sequentially, after observing features and before the outcomes are revealed. These sets are evaluated both in terms of validity (the fraction of rounds where the outcome was lying in the prediction set) and efficiency (the average lengths of the prediction sets). The two criteria point to different directions (validity favors larger sets). We also target a wide range of scenarios, with exchangeable data and arbitrary data (lack of any stochastic guarantees) as two extremes. A series of existing strategies for ACI typically guarantee that empirical coverage converges to the desired level for arbitrary sequences, but they generally lack simultaneous efficiency guarantees. To provide a unified study, we first formulate ACI as a repeated two-player game with finite action sets and vector-valued payoffs encoding validity and efficiency. Building on this reformulation, we introduce a strategy based on Blackwell approachability and on its opportunistic extension by Bernstein et al. (2014) that ensures validity while adapting the efficiency of the prediction intervals to the underlying degree of stochasticity of the opponent player. The resulting guarantee is "best of many worlds": it recovers the relevant efficiency guarantees in exchangeable and adversarial settings, and provides guarantees in intermediate settings that arise in typical applications such as the forecasting of time series.
May 27, 2025

Scalable and adaptive prediction bands with kernel sum-of-squares

Conformal Prediction (CP) is a popular framework for constructing prediction bands with valid coverage in finite samples, while being free of any distributional assumption. A well-known limitation of conformal prediction is the lack of adaptivity, although several works introduced practically efficient alternate procedures. In this work, we build upon recent ideas that rely on recasting the CP problem as a statistical learning problem, directly targeting coverage and adaptivity. This statistical learning problem is based on reproducible kernel Hilbert spaces (RKHS) and kernel sum-of-squares (SoS) methods. First, we extend previous results with a general representer theorem and exhibit the dual formulation of the learning problem. Crucially, such dual formulation can be solved efficiently by accelerated gradient methods with several hundreds or thousands of samples, unlike previous strategies based on off-the-shelf semidefinite programming algorithms. Second, we introduce a new hyperparameter tuning strategy tailored specifically to target adaptivity through bounds on test-conditional coverage. This strategy, based on the Hilbert-Schmidt Independence Criterion (HSIC), is introduced here to tune kernel lengthscales in our framework, but has broader applicability since it could be used in any CP algorithm where the score function is learned. Finally, extensive experiments are conducted to show how our method compares to related work. All figures can be reproduced with the accompanying code at gitlab.com/drti/ksos-bands.