Split Conformal Interval Length under Huber Contamination

Xiao Zhang; Bowen Zhang; Lie He

cs.LGOct 11, 2026

Split Conformal Interval Length under Huber Contamination

Authors: Xiao Zhang, Bowen Zhang, Lie He

Organizations: Shanghai University of Finance and Economics · MoE Key Laboratory of Interdisciplinary Research of Computation and Economics, Shanghai University of Finance and Economics

Abstract

Existing analyses of interval length in split conformal prediction largely focus on clean data. However, how contamination at the training and calibration stages alters interval length relative to the corresponding clean population oracle remains understudied. In this paper, we consider Huber contamination and derive a finite-sample upper bound that holds uniformly over contaminating distributions, with leading terms scaling as $L(δ_1+n_1^{-1/2}) +(n_2^{-1/2}+δ_2)/((1-δ_2)r_0)$ , in addition to a clean-sample estimation term. We further establish a minimax lower bound matching the upper-bound rate up to constant factors on a regular asymmetric location submodel. For calibration contamination, we obtain an exact worst-case characterization of the population interval-length deviation. We also extend the upper-bound results to conformalized quantile regression and adaptive residual scores. Experiments on synthetic and real-world datasets provide empirical support for our theoretical findings on training and calibration contamination.

Figures & tables

Figure 1: Clean and contaminated absolute-score distributions on Abalone .

Figure 2: Joint training and calibration contamination for three conformal prediction nonconformity scores: Absolute, Adaptive, and CQR. Both stages use

Q_{1}=Q_{2}=\operatorname{Dirac}((-1,-1,-1,-1,+1),-80)

Panel of Figure 3 — Figure 3: Training and calibration sensitivity. (a) Predictor perturbation along a fixed contamination path. (b) Local score-density minimum $\widehat{r}_{0}(\delta_{2})$ over the relevant quantile interval.

Panel of Figure 4 — Figure 4: Mean interval-length deviation under point-mass contamination: $Q_{1}=Q_{2}=\operatorname{Dirac}(x^{\star},-80)$ , with $x^{\star}=(-1,-1,-1,-1,+1)$ .

Figure 5: Calibration-only interval-length deviation on Abalone under four structured contamination families using the absolute-residual score. The dashed curve shows the finite high-score Dirac reference with score

200

for

\delta_{2}<\alpha

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Symbol	Description
Data, distributions, and contamination
$(\mathbf{X}_{i},Y_{i})$ , $\mathbf{x}\in\mathcal{X}\subseteq\mathbb{R}^{d}$	Observation pairs and a generic feature value in the $d$ -dimensional feature space.
$\mathcal{D}_{0},\mathcal{D}_{1},\mathcal{D}_{2}$ ; $n_{1},n_{2}$	Coupled clean training sample, contaminated training sample, calibration sample, and their sample sizes.
$P$	A generic distribution on the observation space; subscripts identify its role.
$P_{0}$	Clean distribution.
$Q$ , $\delta$ , $P_{\delta}$	Contamination distribution, contamination level, and Huber mixture $P_{\delta}=(1-\delta)P_{0}+\delta Q$ .

Appendix

Table 6

Family	$X_{Q}$	$Y_{Q}$	Parameter values
Mean shift	$X$	$Y+a$	$a\in\{3,5,10,15\}$
Variance inflation	$X$	$f(X)+s\sigma\varepsilon_{Q}$	$s\in\{3,5,10,15\}$
Heavy tail	$X$	$f(X)+c\sigma T_{3}/\sqrt{3}$	$c\in\{1,3,5,10\}$
Covariate shift + heteroscedastic noise	$X+b\mathbf{1}_{5}$	$f(X_{Q})+\sigma(0.5$ $+1.5\\|X_{Q}\\|_{\infty})\varepsilon_{Q}$	$b\in\{1.5,3,5,8\}$

Appendix

Table 2: Structured contamination for the synthetic experiments (

\sigma=1

). Here

\varepsilon_{Q}\sim N(0,1)

and

T_{3}

has a Student-

t

distribution with three degrees of freedom.

Figure 6: Joint training and calibration contamination for Absolute, Adaptive, and CQR. Both stages use $Q_{1}=Q_{2}=\operatorname{Dirac}((-1,-1,-1,-1,+1),-80)$ . The two horizontal axes are $\delta_{1}$ and $\delta_{2}$ ; height and color show the mean length deviation over 20 repetitions at $\alpha=0.1$ . Height uses a linear $0$ – $200$ scale, and color follows the labeled nonlinear scale.

Figure 7: Calibration-level correction under low-score contamination in the synthetic model with

\sigma=3

. Columns correspond to Absolute, Adaptive, and CQR. The top row shows clean-test coverage and the bottom row the signed length difference. Curves compare

\alpha

\alpha(1-\delta_{2})

, and

\alpha-\delta_{2}

, with target coverage

0.9

. Results are means over 20 repetitions. The additive rule has infinite length at

\delta_{2}=\alpha=0.1

and is undefined beyond this point.

Figure 8: Synthetic calibration contamination at

\alpha=0.1

. Columns show mean shift, variance inflation, heavy tails, and covariate shift plus heteroscedastic noise. Colors indicate severity and marker shapes distinguish scores. Points show mean length deviation over 20 repetitions. The score-

200

Dirac references show mean plus one standard deviation for

\delta_{2}<\alpha

. Broken vertical axes separate these large reference values from the structured results; the final column uses three segments. Tick ranges differ between panels.

Figure 9: Synthetic score distributions at

\delta_{2}=0.1

, using the fixed repetition zero. Rows correspond to Absolute, Adaptive, and CQR; columns use

a=5

s=5

c=10

, and

b=8

. The respective replacement counts are

537

463

452

, and

491

out of

5{,}000

, shared across scores. Stacked bars distinguish retained clean observations from replacements. Vertical lines show the clean and contaminated conformal thresholds at

\alpha=0.1

. Each panel reports their difference.

Dataset	$d$	Pool	Train	Calibration	Test
Abalone	8	3,341	1,670	1,670	836
California Housing	8	16,512	8,256	8,256	4,128
CPU Small	12	6,553	3,276	3,276	1,639
YearPredictionMSD	90	50,000	5,000	5,000	10,000

Appendix

Table 3: Real-data dimensions and sample sizes. The training pool is fixed before the repeated train–calibration splits.

Component	Setting
Architecture	$d$ –64–32–1, with ReLU activations
Optimizer	AdamW; learning rate $10^{-3}$ ; weight decay $10^{-4}$
Training	Batch size 256; at most 100 epochs; gradient norm clipped at 5
Early stopping	Internal validation fraction $0.1$ ; patience 10; minimum improvement $10^{-4}$ ; restore the best validation checkpoint

Appendix

Table 4: Neural-network configuration for all real-data experiments.

Dataset	Minimum–maximum mean coverage
Abalone	$0.916$ – $0.934$
California Housing	$0.888$ – $0.899$
CPU Small	$0.886$ – $0.908$
YearPredictionMSD	$0.887$ – $0.897$

Appendix

Table 5: Range of mean clean-test coverage with

\alpha^{\prime}=\alpha(1-\delta_{2})

. Each range spans all three scores and all 191 contamination levels; each mean uses 20 repetitions.

Figure 10: Calibration-level correction on Abalone (top two rows) and California Housing (bottom two rows). Columns correspond to Absolute, Adaptive, and CQR. Each dataset has a coverage row and a signed length difference row. Curves compare

\alpha

\alpha(1-\delta_{2})

, and

\alpha-\delta_{2}

under low-score contamination and average 20 repetitions. The dashed horizontal line marks coverage

0.9

, and the dotted vertical line marks

\delta_{2}=0.9

. The additive rule has infinite length at

\delta_{2}=0.1

. Length differences use the original response units.

Figure 11: Calibration-level correction on CPU Small (top two rows) and YearPredictionMSD (bottom two rows), using the same conventions as Figure 10 . Columns correspond to Absolute, Adaptive, and CQR. Results average 20 repetitions, and length differences are in the original response units. The additive rule has infinite length at

\delta_{2}=0.1

Figure 12: Real-data calibration contamination at

\alpha=0.1

. Rows correspond to Abalone, California Housing, CPU Small, and YearPredictionMSD. Columns show mean shift, variance inflation, heavy tails, and covariate shift plus heteroscedastic noise. Colored markers show severity levels, and marker shapes distinguish scores. Points show mean length deviation over 20 repetitions, in original response units. The score-

200

Dirac curves show mean plus one standard deviation only for

\delta_{2}<\alpha

. Each panel uses two independently labeled vertical-axis segments; both California Housing segments share the displayed

10^{6}

multiplier.

Dataset	$n_{\mathrm{cal}}$	Shift	Variance	Heavy tail	Composite
Abalone	1,670	183	161	144	161
California Housing	8,256	870	791	769	812
CPU Small	3,276	349	303	304	338
YearPredictionMSD	5,000	537	463	452	491

Appendix

Table 6: Realized replacement counts for the real-data score histograms at

\delta_{2}=0.1

in repetition zero. Columns use

a=5

s=5

c=10

, and

b=8

, respectively. “Composite” denotes covariate shift plus heteroscedastic noise.

Figure 13: Score mixtures on Abalone (top) and California Housing (bottom), using repetition zero and

\delta_{2}=0.1

. Within each dataset, rows show Absolute, Adaptive, and CQR. Columns use mean shift

a=5

, variance inflation

s=5

, heavy tails

c=10

, and covariate shift plus heteroscedastic noise

b=8

. Stacked bars show the source components; vertical lines are conformal thresholds at

\alpha=0.1

. Panel annotations report their differences.

Figure 14: Score mixtures on CPU Small (top) and YearPredictionMSD (bottom), with the same ordering and conventions as Figure 13 . Results use repetition zero at

\delta_{2}=0.1

Panel of Figure 15 — Figure 15: Training-only and calibration-only mean length deviation relative to the clean population oracle, averaged over 20 repetitions. Both panels use the setting of Figure 4 (a) and the same numerical reference for the population calibration term.

Conformal Prediction

Conformal prediction wraps predictive models with calibrated sets or intervals that provide statistical coverage guarantees.

See all 352 papers

Jun 30, 2026

On Optimal Data Splitting for Split Conformal Prediction

Conformal prediction and its variants, including the split conformal prediction, provide a distribution-free framework for uncertainty quantification by constructing prediction intervals or sets with finite-sample coverage guarantees. The statistical efficiency of these intervals depends critically on how the data are split into training and calibration samples. Despite its practical importance, a principled characterization of the training-calibration split that minimizes prediction interval length while maintaining coverage has remained largely unresolved. In this paper, we develop a theoretical framework for optimal data splitting in split conformal prediction. We first analyze the problem in a general setting and derive analytical characterizations of the length-optimal split ratio under both symmetric and asymmetric regimes. We then show how the general results specialize to several commonly used regression settings, including linear regression, nonparametric regression, and neural networks, thereby demonstrating the scope of the framework. We also describe a data-based method for selecting the optimal proportion. Our analysis clarifies how model-related features govern the optimal allocation of samples between training and calibration and provides principled guidance for constructing shorter prediction intervals. Experiments on both synthetic and real-world datasets demonstrate the applicability of the proposed methodology across a variety of practical scenarios.

Sayan Das, Bahram Yaghooti, Todd A. Kuffner +1

Department of Statistics and Data Science, Washington University in St. Louis, St. Louis, MO 63130, USA · Department of Electrical and Systems Engineering, Washington University in St. Louis, St. Louis, MO 63130, USA

Conformal Prediction Statistical Learning Theory Regression Prediction Intervals

Ask PDF

Aug 6, 2026

Beyond Marginal Validity: Finite-Sample Guarantees for Localized Conformal Prediction

Conformal prediction endows arbitrary black-box predictors with finite-sample, distribution-free marginal coverage, yet marginal validity can hide severe covariate-specific miscalibration, while exact distribution-free conditional coverage is finite-sample unattainable. Randomly localized conformal prediction (RLCP) mitigates this gap by calibrating near the test point while preserving marginal coverage. Existing theory, however, lacks finite-sample guarantees for the realized localized set that jointly control conditional validity and oracle efficiency. We provide such guarantees. For any fixed score, under Hölder regularity of the conditional score CDF and standard density and kernel assumptions, we prove high-probability bounds, uniform over a realized localization neighbourhood, for the conditional-coverage gap and the length error relative to the oracle. The bounds decompose into an

O(h^β)

localization bias and a calibration term decreasing with calibration size, clarifying the bandwidth bias-variance tradeoff and when RLCP tracks the oracle. We also analyze data-split learned scores: when the score targets a pivotal score, as in conformalized quantile regression, uniform local guarantees decompose into fixed-score calibration and uniform score-estimation errors, showing that improved learning sharpens localized guarantees.

Anton Conrad, Rustam Isaev, Denis Belomestny +2

Laboratoire de Recherche d’EPITA, 14–16 rue Voltaire, 94276 Le Kremlin-Bicêtre CEDEX, France · Faculty of Computer Science, HSE University, 11 Pokrovsky Boulevard, Moscow 109028, Russia · Faculty of Computational Mathematics and Cybernetics, Lomonosov Moscow State University, 1 Leninskie Gory, Building 52, Moscow 119991, Russia +2

Conformal Prediction Statistical Learning Theory

Ask PDF

Mar 2, 2026

Co-optimization for Adaptive Conformal Prediction

Conformal prediction (CP) provides finite-sample, distribution-free marginal coverage, but standard conformal regression intervals can be inefficient under heteroscedasticity and skewness. In particular, popular constructions such as conformalized quantile regression (CQR) often inherit a fixed notion of center and enforce equal-tailed errors, which can displace the interval away from high-density regions and produce unnecessarily wide sets. We propose Co-optimization for Adaptive Conformal Prediction (CoCP), a framework that learns prediction intervals by jointly optimizing a center

m(x)

and a radius

h(x)

.CoCP alternates between (i) learning

h(x)

via quantile regression on the folded absolute residual around the current center, and (ii) refining

m(x)

with a differentiable soft-coverage objective whose gradients concentrate near the current boundaries, effectively correcting mis-centering without estimating the full conditional density. Finite-sample marginal validity is guaranteed by split-conformal calibration with a normalized nonconformity score. Theory characterizes the population fixed point of the soft objective and shows that, under standard regularity conditions, CoCP asymptotically approaches the length-minimizing conditional interval at the target coverage level as the estimation error and smoothing vanish. Experiments on synthetic and real benchmarks demonstrate that CoCP yields consistently shorter intervals and achieves state-of-the-art conditional-coverage diagnostics.

Xiaoyi Su, Zhixin Zhou, Rui Luo

Department of Systems Engineering, City University of Hong Kong, Hong Kong SAR, China · Alpha Benito Research, Los Angeles, USA

Conformal Prediction Regression Prediction Intervals

Ask PDF

Abstract

Conformal Prediction

Explore similar work

On Optimal Data Splitting for Split Conformal Prediction

Beyond Marginal Validity: Finite-Sample Guarantees for Localized Conformal Prediction

Co-optimization for Adaptive Conformal Prediction