cs.LGMay 27, 2025

Scalable and adaptive prediction bands with kernel sum-of-squares

Authors: Louis Allain, S{é}bastien da Veiga, Brian Staber

Organizations: ENSAI, CREST · ENSAI, CREST, RT-UQ · Safran Tech, Digital Sciences & Technologies, 78114 Magny-Les-Hameaux, France

Abstract

Conformal Prediction (CP) is a popular framework for constructing prediction bands with valid coverage in finite samples, while being free of any distributional assumption. A well-known limitation of conformal prediction is the lack of adaptivity, although several works introduced practically efficient alternate procedures. In this work, we build upon recent ideas that rely on recasting the CP problem as a statistical learning problem, directly targeting coverage and adaptivity. This statistical learning problem is based on reproducible kernel Hilbert spaces (RKHS) and kernel sum-of-squares (SoS) methods. First, we extend previous results with a general representer theorem and exhibit the dual formulation of the learning problem. Crucially, such dual formulation can be solved efficiently by accelerated gradient methods with several hundreds or thousands of samples, unlike previous strategies based on off-the-shelf semidefinite programming algorithms. Second, we introduce a new hyperparameter tuning strategy tailored specifically to target adaptivity through bounds on test-conditional coverage. This strategy, based on the Hilbert-Schmidt Independence Criterion (HSIC), is introduced here to tune kernel lengthscales in our framework, but has broader applicability since it could be used in any CP algorithm where the score function is learned. Finally, extensive experiments are conducted to show how our method compares to related work. All figures can be reproduced with the accompanying code at gitlab.com/drti/ksos-bands.

Explore similar work

Mar 2, 2026

Co-optimization for Adaptive Conformal Prediction

In regression, conformal prediction often suffers from inefficiency under heteroscedasticity and skewness due to fixed, non-adaptive interval centering. We propose CoCP, a new method that parameterizes prediction intervals by a center m(x)m(x) and a radius h(x)h(x), and alternates between learning h(x)h(x) via smooth quantile regression on folded residuals and refining m(x)m(x) using a smooth interval loss. This corrects mis-centering and drives the interval toward high-density regions. Beyond finite-sample marginal validity via split-conformal calibration, we prove that CoCP also attains asymptotic conditional coverage and optimal interval length if the base estimators are consistent. Experiments demonstrate that CoCP yields tighter intervals and achieves improved empirical conditional-coverage reliability. Code is available at https://github.com/jjoeysu/cocp.
May 1, 2026

Optimal Spatio-Temporal Decoupling for Bayesian Conformal Prediction

Online conformal prediction must balance fast adaptation to distribution shift against stable coverage: feedback-driven methods react quickly but become volatile, while strongly discounted Bayesian methods lag and inflate intervals at tight coverage. We introduce State-Adaptive Bayesian Conformal Prediction (SA-BCP), which forms the predictive quantile as a gated convex combination of long-term temporal inertia and local spatial evidence from a kernel density estimate, controlled by a single interpretable evidence threshold KK. We establish three results: (i) asymptotic marginal validity of the resulting intervals up to a gate-controlled bias that vanishes as spatial evidence accumulates (exact under recurrent states); (ii) a closed-form expression for the MSE-optimal threshold, KMSE∗=α(1−α)/MTK^*_{\mathrm{MSE}}=α(1-α)/M^{\mathcal{T}}, trading the coverage-indicator (Bernoulli) variance against the temporal structural bias MTM^{\mathcal{T}}; and (iii) a rolling-origin procedure for selecting KK online -- consistent under stationarity, with O(Tlog⁡N)O(\sqrt{T\log N}) regret against the best fixed KK and, for a segmented variant, a sublinear dynamic-regret bound under sublinearly many (BT=o(T)B_T=o(T)) threshold shifts. Across four financial-volatility and weather datasets, three target coverage levels, and eight baselines, SA-BCP attains at-or-above-nominal coverage in most settings while producing substantially sharper intervals -- up to roughly 3×3\times lower Winkler score than discounted Bayesian CP at the tightest coverage -- and a coverage-matched audit confirms these efficiency gains are not an artifact of under-coverage. We disclose our principal limitation: a volatility-specialized CF-GARCH competitor remains more efficient on its home volatility-base series, though it does not transfer across domains.
Mar 23, 2026

CRPS-Optimal Binning for Univariate Conformal Regression

We propose a method for non-parametric conditional distribution estimation based on partitioning covariate-sorted observations into contiguous bins and using the within-bin empirical CDF as the predictive distribution. Bin boundaries are chosen to minimise the total leave-one-out Continuous Ranked Probability Score (LOO-CRPS), which admits a closed-form cost function with O(n2log⁡n)O(n^2 \log n) precomputation and O(n2)O(n^2) storage; the globally optimal KK-partition is recovered by a dynamic programme in O(n2K)O(n^2 K) time. Minimisation of within-sample LOO-CRPS turns out to be inappropriate for selecting KK as it results in in-sample optimism. We instead select KK by KK-fold cross-validation of test CRPS, which yields a U-shaped criterion with a well-defined minimum. Having selected K∗K^* and fitted the full-data partition, we form two complementary predictive objects: the Venn prediction band and a conformal prediction set based on CRPS as the nonconformity score, which carries a finite-sample marginal coverage guarantee at any prescribed level ε\varepsilon. The conformal prediction is transductive and data-efficient, as all observations are used for both partitioning and p-value calculation, with no need to reserve a hold-out set. On real benchmarks against split-conformal competitors (Gaussian split conformal, CQR, CQR-QRF, and conformalized isotonic distributional regression), the method produces substantially narrower prediction intervals while maintaining near-nominal coverage.