Selection-Safe Conformal Early Exit: Distribution-Free Coverage under Adaptive Depth

Adria Binte Habib; Ibne Farabi Shihab

cs.LGOct 14, 2026

Selection-Safe Conformal Early Exit: Distribution-Free Coverage under Adaptive Depth

Authors: Adria Binte Habib, Ibne Farabi Shihab

Organizations: Department of Computer Science, BRAC University · Department of Computer Science, Iowa State University

Abstract

Calibrating a conformal prediction set at each early exit does not preserve coverage once a data-dependent exit is selected. We construct one terminal path score from prefix maxima, calibrate a single split-conformal quantile, and return nested prefix sets that all contain the valid terminal set. The guarantee is selection-safe without an alpha/L correction and holds simultaneously for every post-calibration label-free stopping policy that preserves terminal-set containment. We also bound the efficiency cost of the path maximum by the empirical tail of the early-over-terminal score excess. Across eight encoder task-backbone cells, pathwise selection attains macro coverage 0.901 versus 0.814 for naive layerwise selection, with average set size 1.41 and latency fraction 0.71. On CIFAR-100 with a 24-exit ViT-L/16 (n_cal=500), first-eligible stopping at budget b=4 attains coverage 0.902, set size 2.76, and latency fraction 0.57, and the same calibration supports deployment-time budget changes without refitting. In a joint exit-count/calibration-size sweep at n=100, pathwise set size moves only from 3.02 to 3.11 as exits increase from 4 to 24, while Bonferroni degenerates to the full label space for 12 or more exits. Paired comparison against MinSE valid selection at (n,L)=(100,24) gives a set-size difference of 0.80 [0.47, 1.13]. Full-vocabulary decoder experiments remain a negative boundary, with sets of 7,162-9,620 tokens. Pathwise calibration is most useful when candidate exits are many relative to calibration size, when budgets change after calibration, and when shallow heads return actionable sets.

Figures & tables

Method	Controlled target	Selected validity	Avg. set	Latency/full	Exit/full
Naive layerwise CP	fixed-layer coverage	$0.814$	$1.22$	$0.51$	$0.50$
Bonferroni CP	simultaneous coverage	$0.954$	$1.71$	$0.79$	$0.79$
EENN-AVCS ( Jazbec et al., 2024a )	AVCS coverage	$0.911$	$1.54$	$0.76$	$0.75$
MinSE valid selection ( Hegazy et al., 2025 )	selected-set coverage	$0.903$	$1.40$	$0.70$	$0.69$
FyS supervised ( Jazbec et al., 2024b )	$0$ – $1$ risk	$0.900$	n/a	$0.67$	$0.66$
Pathwise conformal	selected-set coverage	$0.901$	$1.41$	$0.71$	$0.71$

Table 1: Matched five-seed encoder comparison at target coverage

0.90

. Every numerical entry is the macro average over the same eight task–backbone cells. Fast-yet-Safe controls supervised

0

–

1

risk rather than conformal set coverage; paired uncertainty is reported separately in Tables 11 and 12 .

Panel of Figure 1 — Figure 1: Per-cell selected-coverage distributions. Points are cell means. For SST-2, MNLI, and BoolQ, bars are the $10$ th– $90$ th percentiles across paired calibration–test roles. RTE uses one fixed calibration/test partition; its bars are the corresponding percentiles across the $5$ head-training seeds. Exact means, model accuracies, and familywise paired intervals appear in Appendix L .

Panel of Figure 2 — Figure 2: CIFAR-100 latency–set-size tradeoff for the explicit $(n,L)=(500,24)$ configuration used in the scaling grid. Bars are paired $95\%$ hierarchical-bootstrap intervals over head seeds and calibration subsamples. For each budget $b\in\{1,2,4,8,16\}$ , first-eligible is the earliest admissible stop; the design-chosen slack policy is a restriction and therefore lies weakly to its right in latency while returning smaller sets. The MinSE, EENN-AVCS, and Bonferroni markers are exactly the $(500,24)$ entries of Appendix N . Exact coverage, empty-terminal rate, and frontier values are reported in Appendix M .

Panel of Figure 3 — Figure 3: Scaling with the number of candidate exits on CIFAR-100. Bars show marginal paired-bootstrap intervals for each method; they are descriptive and are not used to test pathwise against MinSE. The left panel uses $n=100$ and a log $y$ -axis because Bonferroni has rank $n+1$ at $L=12$ and $L=24$ , forcing the returned set to all $100$ classes. The right panel uses $n=2000$ . Paired pathwise–MinSE difference intervals, which exploit the shared calibration subsample and test set, appear in Appendix N .

Appendix figures & tables21 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Protected object	Guarantee	Main requirement
Layerwise CP	set at fixed exit	marginal coverage per exit	exit fixed in advance
EENN-AVCS ( Jazbec et al., 2024a )	nested exit sequence	anytime-valid predictive set	posterior-stability construction
Fast-yet-Safe ( Jazbec et al., 2024b )	adaptive point or distribution	expected or high-probability risk	specified loss and calibrated policy
Valid selection ( Hegazy et al., 2025 )	chosen conformal set	post-selection coverage	stability correction
Pathwise CP	returned prefix set	exact marginal coverage	exchangeability and frozen path

Appendix

Table 2: Relationship to the closest approaches. “Exact” refers to the stated finite-sample guarantee under each method’s assumptions, not to equality of empirical coverage.

Task	Backbone	$t_{\star}$	$m_{t_{\star}}/n$	$B$ gap	$w_{t_{\star}}$	$\sum_{y}\widehat{M}_{y}$	RHS	observed inflation	RHS / observed
SST-2	BERT	$0.145$	$0.089$	$0.026$	$0.171$	$0.095$	$0.016$	$0.01$	$1.62$
SST-2	DistilBERT	$0.149$	$0.086$	$0.022$	$0.171$	$0.101$	$0.017$	$0.01$	$1.73$
MNLI	BERT	$0.082$	$0.088$	$0.029$	$0.111$	$0.620$	$0.069$	$0.06$	$1.15$
MNLI	DistilBERT	$0.086$	$0.087$	$0.034$	$0.120$	$0.720$	$0.086$	$0.08$	$1.08$
RTE	BERT	$0.052$	$0.081$	$0.021$	$0.073$	$0.780$	$0.057$	$0.05$	$1.14$
RTE	DistilBERT	$0.055$	$0.084$	$0.025$	$0.080$	$0.940$	$0.075$	$0.07$	$1.07$

Appendix

Table 3: Theorem 2 in measured units.

t_{\star}

is the smallest threshold satisfying

m_{t}\leq n-k

; “

B

gap” is

B_{(k+m_{t})}-B_{(k)}

; RHS is

w_{t}\sum_{y}\widehat{M}_{y}

from the independent design role; observed inflation is the paired test mean of

(|\mathcal{C}_{L}^{\rm path}|-|\mathcal{C}_{L}^{\rm last}|)_{+}

. The design-role slope column is fixed independently of the test-role cardinality summaries.

Stage	Operation
Train	Fit the backbone and exit heads on training data.
Design	Choose exits and increasing score maps without using final calibration or test labels; a provisional budget may guide this search.
Calibrate	Compute $S_{i}=\max_{\ell\leq L}r_{\ell}(X_{i},Y_{i})$ and the single quantile $\widehat{q}$ on $\mathcal{D}_{\mathrm{cal}}$ .
Execute	Update $A_{\ell}(x,y)$ candidatewise and construct $\mathcal{C}_{\ell}(x)$ after every executed exit.
Stop	Return an eligible prefix set; if none is eligible, return $\mathcal{C}_{L}(x)$ , or $\mathcal{Y}$ when $\mathcal{C}_{L}(x)$ is empty.

Appendix

Table 4: Execution order. Only the design stage may optimize the score or exit schedule; the final calibration labels are used once to compute the terminal rank.

Target	Guarantee	Required condition	Not implied
Selected set	marginal coverage $\geq 1-\alpha$	exchangeable terminal scores; frozen path	feature-conditional validity
Realized calibration	Beta or DKW lower floor	independent calibration draw; stated continuity or conservative ties	per-input coverage
Group shift	coverage within each fixed group	within-group exchangeability and valid $n_{g}$ quantile	arbitrary covariate shift
Hybrid output	error $\leq\alpha_{\rm pt}+\alpha_{\rm set}$	joint point-error control and valid fallback	error $\leq\alpha$ when both branches use $\alpha$

Appendix

Table 5: Guarantees and their load-bearing assumptions. Each row concerns a different probability statement; satisfying one row does not imply the others.

PAC size	$n=200$	$n=500$	$n=1500$	required minimum
Fraction above floor	$0.9985$	$0.9992$	$0.9992$	$0.90$ – $0.99$
Group method	easy	medium	hard	shifted marginal
Pooled	$1.0000$	$0.9913$	$0.6887$	$0.8112$
Mondrian	$0.9177$	$0.9054$	$0.9149$	$0.9129$

Appendix

Table 6: Controlled PAC and Mondrian verification at

\alpha=0.1

. PAC entries give the minimum fraction of

4000

calibration draws above the DKW floor over

\delta\in\{0.1,0.05,0.01\}

. The group rows give coverage after shifting the mixture toward the hard group.

$n$	coverage (mean $\pm$ SD)	Beta floor	DKW floor	set size	worst draw
$100$	$0.905\pm 0.028$	$0.848$	$0.778$	$1.47$	$0.841$
$200$	$0.900\pm 0.018$	$0.864$	$0.813$	$1.44$	$0.871$
$500$	$0.905\pm 0.011$	$0.877$	$0.845$	$1.46$	$0.884$
$1000$	$0.902\pm 0.009$	$0.884$	$0.861$	$1.44$	$0.885$
$1500$	$0.903\pm 0.006$	$0.887$	$0.868$	$1.44$	$0.888$

Appendix

Table 7: Calibration-size scaling on MNLI/BERT over repeated calibration draws at

\alpha=0.1

and

\delta=0.05

. Coverage is mean

\pm

SD; Beta and DKW columns are training-conditional lower floors.

Shift	calibration	hard $n_{\rm cal}$	hard cov.	shifted cov.	set	exit/full
MNLI label	pooled	$1719$	$0.894$	$0.900$	$1.44$	$0.73$
MNLI label	Mondrian	$1719$	$0.911$	$0.906$	$1.42$	$0.72$
BoolQ label	pooled	$618$	$0.811$	$0.860$	$1.56$	$0.86$
BoolQ label	Mondrian	$618$	$0.895$	$0.888$	$1.51$	$0.85$
SST-2 difficulty	pooled	$72$	$0.669$	$0.817$	$1.04$	$0.32$
SST-2 difficulty	Mondrian	$72$	$0.805$	$0.881$	$1.23$	$0.48$

Appendix

Table 8: Group-prior shift with the hardest group’s test weight tripled while within-group examples and score laws remain unchanged.

$\alpha_{\rm pt}/\alpha_{\rm set}$	commit $c$	cond. commit err. $e_{c}$	joint point err. $ce_{c}$	fallback share $1-c$	fallback cov. $q_{f}$	joint fallback miss	overall cov.	latency/full
$0.02/0.08$	$0.180$	$0.041$	$0.0074$	$0.820$	$0.923$	$0.0631$	$0.9295$	$0.690$
$0.05/0.05$	$0.340$	$0.061$	$0.0207$	$0.660$	$0.951$	$0.0323$	$0.9470$	$0.660$
$0.08/0.02$	$0.470$	$0.079$	$0.0371$	$0.530$	$0.983$	$0.0090$	$0.9539$	$0.640$

Appendix

Table 9: Risk-budgeted point-prediction hybrid. Conditional commit error uses only committed examples; fallback coverage uses only the

E^{c}

denominator. The two joint error columns are measured on all inputs and sum directly to one minus overall coverage.

Task	Backbone	returned path set	empty terminal	mean positive inflation
SST-2	BERT	$1.03$	$0.1\text{--}0.2\%$	$0.01$
SST-2	DistilBERT	$1.04$	$0.1\text{--}0.2\%$	$0.01$
MNLI	BERT	$1.44$	–	$0.06$
MNLI	DistilBERT	$1.62$	–	$0.08$
RTE	BERT	$1.49$	$0.1\text{--}0.2\%$	$0.05$
RTE	DistilBERT	$1.68$	$0.1\text{--}0.2\%$	$0.07$

Appendix

Table 10: Cost diagnostic for the terminal path maximum. Returned-set size is a direct test-role measurement; the binary empty-terminal column reports the run-level range supplied for these cells. Positive inflation is the paired statistic bounded by Theorem 2 ,

\mathbb{E}[(|\mathcal{C}_{L}^{\rm path}|-|\mathcal{C}_{L}^{\rm last}|)_{+}]

. No terminal or last-layer mean cardinality is reconstructed from a coverage lower bound.

Task	Backbone	$\Delta_{c}$	familywise $95\%$ interval
SST-2	BERT	$0.057$	$[0.036,\,0.078]$
SST-2	DistilBERT	$0.046$	$[0.022,\,0.070]$
MNLI	BERT	$0.080$	$[0.062,\,0.098]$
MNLI	DistilBERT	$0.060$	$[0.039,\,0.081]$
RTE	BERT	$0.105$	$[0.061,\,0.149]$
RTE	DistilBERT	$0.134$	$[0.083,\,0.185]$

Appendix

Table 11: Paired selection-gap inference. Intervals are familywise

95\%

hierarchical paired-bootstrap intervals obtained from the maximum absolute studentized statistic across the eight declared cells.

Comparator	$\Delta$ validity	$\Delta$ latency/full	$\Delta$ avg. set
MinSE valid selection	$-0.002\ [-0.011,\,0.007]$	$0.010\ [-0.012,\,0.032]$	$0.010\ [-0.036,\,0.056]$
EENN-AVCS	$-0.012\ [-0.022,\,-0.002]$	$-0.040\ [-0.064,\,-0.016]$	$-0.130\ [-0.188,\,-0.072]$
Bonferroni CP	$-0.055\ [-0.066,\,-0.044]$	$-0.070\ [-0.095,\,-0.045]$	$-0.300\ [-0.363,\,-0.237]$

Appendix

Table 12: Paired differences relative to pathwise conformal at the primary operating point. All point estimates subtract macro averages over the same eight cells in Table 1 ; positive latency and set differences mean pathwise is larger. Brackets give

95\%

hierarchical paired-bootstrap intervals.

Task	$\Pr(J\leq L/4)$	$\mathbb{E}[J/L]$	$\Pr(J=L)$	$\Pr(D>0)$	$Q_{.95}(D)$	false removal at $L/2$
SST-2	$0.62$	$0.29$	$0.10$	$0.64$	$0.18$	$0.88$
MNLI	$0.21$	$0.61$	$0.33$	$0.37$	$0.11$	$0.61$
RTE	$0.13$	$0.71$	$0.45$	$0.23$	$0.07$	$0.55$
BoolQ	$0.17$	$0.68$	$0.39$	$0.32$	$0.09$	$0.58$

Appendix

Table 13: Mechanism diagnostics pooled across backbones.

D=[\max_{j<L}r_{j}-r_{L}]_{+}

is the excess entering Theorem 2 ;

Q_{.95}(D)

is its

95

th percentile on true labels.

Score or schedule	coverage range	avg. set	exit/full	latency/full
$-\log p$	$0.890$ – $0.906$	$1.41$	$0.71$	$0.71$
Design affine	$0.887$ – $0.915$	$1.45$	$0.68$	$0.68$
Design ECDF	$0.890$ – $0.950$	$1.54$	$0.70$	$0.71$
APS	$0.885$ – $0.935$	$1.75$	$0.84$	$0.84$
RAPS	$0.887$ – $0.934$	$1.71$	$0.82$	$0.83$
Sparse schedule	$0.892$ – $0.929$	$1.39$	$0.65$	$0.66$

Appendix

Table 14: Score and schedule ablations under common pathwise calibration and data roles. Coverage ranges span the eight encoder task–backbone cells.

$n$	$\delta$	mean cov.	path floor	fraction above	12-exit floor
$200$	$0.10$	$0.9004$	$0.8241$	$0.9985$	$0.7906$
$200$	$0.05$	$0.9004$	$0.8135$	$1.0000$	$0.7829$
$200$	$0.01$	$0.9004$	$0.7927$	$1.0000$	$0.7669$
$500$	$0.10$	$0.9001$	$0.8520$	$0.9992$	$0.8308$
$500$	$0.05$	$0.9001$	$0.8453$	$1.0000$	$0.8260$
$500$	$0.01$	$0.9001$	$0.8321$	$1.0000$	$0.8158$

Appendix

Table 15: Controlled PAC verification. “Fraction above” is the fraction of

4000

calibration draws whose conditional coverage exceeded the DKW floor. The final column shows the corresponding floor after a twelve-exit simultaneous correction.

Item	Configuration
Accelerator	NVIDIA A $100$ SXM $4$ , $40$ GB, single GPU
CUDA / PyTorch	$12.1$ / $2.4.1$
Precision	FP $16$ inference
Batch size	$1$
Maximum encoded length	$128$ tokens
Warm-up	$200$ complete executions per method

Appendix

Table 16: Latency measurement configuration used for every paired encoder timing comparison.

Task	Backbone	path cov.	returned set	empty terminal	naive cov.	gap	latency/full
SST-2	BERT	$0.906$	$1.03$	$0.1\text{--}0.2\%$	$0.849$	$0.057$	$0.30$
SST-2	DistilBERT	$0.890$	$1.04$	$0.1\text{--}0.2\%$	$0.844$	$0.046$	$0.34$
MNLI	BERT	$0.904$	$1.44$	–	$0.824$	$0.080$	$0.75$
MNLI	DistilBERT	$0.902$	$1.62$	–	$0.842$	$0.060$	$0.83$
RTE	BERT	$0.901$	$1.49$	$0.1\text{--}0.2\%$	$0.796$	$0.105$	$0.88$
RTE	DistilBERT	$0.903$	$1.68$	$0.1\text{--}0.2\%$	$0.769$	$0.134$	$0.92$

Appendix

Table 17: Exact means underlying Figure 1 . Path coverage and returned-set size are direct test-role measurements. For the six binary cells, the empty-terminal column reports the supplied run-level range before the

\varnothing\to\mathcal{Y}

fallback; MNLI is left unreported here because it is not needed for the binary identity. RTE uses one fixed training-split calibration role with

n_{\rm cal}=498

and the fixed official development test set; its repeated estimates vary over head-training seeds rather than role assignments.

Task	Backbone	full-depth acc.	singleton rate	singleton acc.
SST-2	BERT	$0.932$	$0.970$	$0.903$
SST-2	DistilBERT	$0.918$	$0.960$	$0.885$
MNLI	BERT	$0.846$	$0.604$	$0.842$
MNLI	DistilBERT	$0.828$	$0.553$	$0.826$
RTE	BERT	$0.718$	$0.510$	$0.806$
RTE	DistilBERT	$0.697$	$0.320$	$0.697$

Appendix

Table 18: Encoder point-prediction context for interpreting set sizes. Full-depth accuracy is the deepest head’s top-1 accuracy. For the six binary rows, singleton rate and singleton accuracy are algebraic audit summaries implied by the directly measured returned-set size and coverage in the preceding table; the MNLI singleton columns are measured directly. RTE uses the fixed partition described in Section 4 .

Policy	budget $b$	coverage (95% CI)	avg. set (95% CI)	latency/full (95% CI)	empty-terminal
First eligible	$1$	$0.900\ [0.894,0.906]$	$1.94\ [1.87,2.01]$	$0.68\ [0.667,0.693]$	$0.06\%$
First eligible	$2$	$0.901\ [0.895,0.907]$	$2.21\ [2.13,2.29]$	$0.61\ [0.597,0.623]$	$0.06\%$
First eligible	$4$	$0.902\ [0.896,0.908]$	$2.76\ [2.65,2.87]$	$0.57\ [0.550,0.590]$	$0.06\%$
First eligible	$8$	$0.903\ [0.897,0.909]$	$3.90\ [3.76,4.04]$	$0.54\ [0.525,0.555]$	$0.06\%$
First eligible	$16$	$0.904\ [0.898,0.910]$	$5.48\ [5.30,5.66]$	$0.52\ [0.503,0.537]$	$0.06\%$
Design restriction	$1$	$0.900\ [0.894,0.906]$	$1.90\ [1.83,1.97]$	$0.70\ [0.687,0.713]$	$0.06\%$

Appendix

Table 19: Exact CIFAR-100 values underlying Figure 2 for

(n,L)=(500,24)

. Parentheses are paired

95\%

hierarchical-bootstrap intervals. All pathwise rows reuse one terminal calibration; empty-terminal is measured before the

\varnothing\to\mathcal{Y}

fallback. The three safe-baseline set-size/latency pairs are identical to the

(500,24)

scaling-grid row.

$n$	$L$	Pathwise	Bonferroni	EENN-AVCS	MinSE valid selection	paired MinSE $-$ Path
$50$	$4$	$3.24\pm 0.35/0.60\pm 0.04$	$3.71\pm 0.43/0.66\pm 0.05$	$3.49\pm 0.41/0.64\pm 0.05$	$3.34\pm 0.39/0.61\pm 0.05$	$0.10\ [-0.04,0.24]$
$50$	$12$	$3.30\pm 0.37/0.60\pm 0.04$	$100/1.00$	$4.18\pm 0.51/0.74\pm 0.06$	$3.81\pm 0.55/0.68\pm 0.06$	$0.51\ [0.21,0.81]$
$50$	$24$	$3.34\pm 0.39/0.61\pm 0.04$	$100/1.00$	$4.89\pm 0.63/0.84\pm 0.07$	$4.27\pm 0.92/0.76\pm 0.07$	$0.93\ [0.41,1.45]$
$100$	$4$	$3.02\pm 0.26/0.59\pm 0.03$	$3.36\pm 0.31/0.64\pm 0.04$	$3.21\pm 0.30/0.62\pm 0.04$	$3.10\pm 0.29/0.60\pm 0.04$	$0.08\ [-0.01,0.17]$
$100$	$8$	$3.05\pm 0.27/0.59\pm 0.03$	$3.91\pm 0.35/0.72\pm 0.04$	$3.55\pm 0.33/0.68\pm 0.04$	$3.29\pm 0.34/0.63\pm 0.04$	$0.24\ [0.11,0.37]$
$100$	$12$	$3.07\pm 0.28/0.59\pm 0.03$	$100/1.00$	$3.82\pm 0.36/0.73\pm 0.05$	$3.45\pm 0.39/0.66\pm 0.05$	$0.38\ [0.20,0.56]$

Appendix

Table 20: CIFAR-100 scaling grid at target coverage

0.90

. Method entries are average set size

\pm

marginal paired-bootstrap half-width / latency fraction

\pm

half-width. The final column is the directly paired MinSE-minus-pathwise set-size difference with its paired

95\%

interval, computed from the shared calibration subsample and test set; significance is assessed from this column, not from overlap of marginal intervals. Bonferroni entries marked

100/1.00

are exact rank-

n+1

degeneracies and therefore have zero sampling interval.

Dataset	independently trained path	$L$	$n_{\rm cal}$	full-depth acc.	Path	MinSE valid selection	Bonferroni
CIFAR-100	ViT-B/16	$12$	$500$	$0.927$	$2.84\pm 0.10/0.58\pm 0.02$	$3.08\pm 0.14/0.64\pm 0.02$	$3.73\pm 0.17/0.74\pm 0.02$
CIFAR-100	ViT-L/16	$24$	$500$	$0.936$	$2.76\pm 0.11/0.57\pm 0.02$	$3.18\pm 0.16/0.65\pm 0.02$	$4.05\pm 0.19/0.78\pm 0.03$
ImageNet-1k	ViT-B/16	$12$	$1000$	$0.798$	$8.31\pm 0.31/0.65\pm 0.02$	$9.02\pm 0.40/0.69\pm 0.02$	$11.57\pm 0.52/0.77\pm 0.03$
ImageNet-1k	ViT-L/16	$24$	$1000$	$0.823$	$7.86\pm 0.29/0.61\pm 0.02$	$9.41\pm 0.43/0.70\pm 0.02$	$13.88\pm 0.61/0.82\pm 0.03$

Appendix

Table 21: Replication beyond one frozen score path. “Path” and “MinSE” entries are average set size / latency fraction at target coverage

0.90

; Bonferroni is shown for the same trained path.

	DistilGPT2	Pythia-1.4B	Pythia-2.8B
Full-depth accuracy	$0.261\pm 0.002$	$0.391\pm 0.004$	$0.404\pm 0.005$
Selected coverage	$0.899\pm 0.004$	$0.903\pm 0.005$	$0.902\pm 0.003$
Average exit/full	$1.000$	$1.000$	$0.998$
Average set size	$9620$	$7162$	$8331$
Early-exit rate	$0.01\%$	$0.01\%$	$0.52\%$

Appendix

Table 22: Candidatewise next-token sets over the full vocabulary. Coverage and accuracy are mean

\pm

SD over five independent head-training seeds.

Conformal Prediction

Conformal prediction wraps predictive models with calibrated sets or intervals that provide statistical coverage guarantees.

See all 352 papers

May 18, 2026

Decoupled Conformal Optimisation: Efficient Prediction Sets via Independent Tuning and Calibration

Bayesian conformal optimisation methods often use the same held-out data both to search for efficient prediction sets and to certify coverage or risk. This coupling is natural for high-probability risk-control guarantees, but it is not necessary when the target is standard finite-sample marginal conformal coverage. We propose Decoupled Conformal Optimisation (DCO), a train-tune-calibrate design principle that uses an independent tuning split for efficiency-oriented structural selection and a fresh calibration split for the final conformal quantile. Conditional on the tuned structure, standard split-conformal exchangeability yields finite-sample marginal coverage for any candidate class, without a confidence parameter or multiple-testing correction. DCO therefore targets a different finite-sample guarantee from PAC-style methods: marginal conformal coverage rather than high-probability risk control. Under consistency assumptions on the coupled risk bound, the two approaches nevertheless converge to the same population threshold. Across classification and regression benchmarks, including ImageNet-A, CIFAR-100, Diabetes, California Housing, and Concrete, DCO tracks the nominal coverage level closely while often reducing average prediction-set size or interval width relative to PAC-style calibration. On ImageNet-A, for example, the average set size decreases from

26.52

25.26

and the 95th-percentile set size from

58.95

53.73

; on Diabetes, the average interval width decreases from

2.098

1.914

Fanyi Wu, Lihua Niu, Samuel Kaski +1

Department of Computer Science, University of Manchester, Manchester, UK · UKRI AI Centre for Doctoral Training in Decision Making for Complex Systems · Department of Computer Science, Aalto University, Espoo, Finland +1

Conformal Prediction Regression Prediction Intervals

Ask PDF

Aug 21, 2026

Reaching the Tail: Calibration Diversity Drives Conformal Coverage under Data Scarcity

Multi-horizon rare-event forecasting is hard under long macroeconomic series' data constraints: labeled events are scarce, and standard uncertainty quantification assumes an exchangeability that autocorrelation violates. A controlled ablation shows an apparent rare-event threshold for Adaptive Conformal Inference instead reflects calibration-set size. Across 200 random calibration sets, support width of the nonconformity-score distribution explains up to 85% of coverage variance versus 2% for rare-event count; the same, not the same magnitude, replicates across synthetic conditions and five countries (five-country Spearman

ρ

0.45-0.66 vs. 0.02-0.23). A diversity-maximizing selector built on this is the only strategy tested that improves long-horizon coverage (67.8% to 81.4% at six months); Mondrian, shift-robust, and extreme-value alternatives fail to close it. Mondrian even worsens coverage under oracle labels. A compact proposition explains why: coverage deficit reflects how closely the calibration set's upper quantile reaches the test distribution's. Diversity is necessary, not sufficient. Demonstrated on a two-stage U.S. recession-forecasting framework with RegressorChain, whether six-month coverage reaches 90% under honest scoring remains open, a question this paper quantifies rather than resolves.

Donald Aadithiyan

Department of Computer Science and Engineering, University of Moratuwa, Moratuwa, Sri Lanka

Rare-Event Prediction Conformal Prediction Adaptive Conformal Inference Data Selection Financial Forecasting

Ask PDF

May 21, 2025

Adaptive Cumulative Mass Calibration with Conformal Prediction

Reliable probability estimates by classifiers are essential in high-risk applications. In practice, however, predicted probabilities are often miscalibrated, and many existing post-hoc calibration methods typically lack guarantees that a specific notion of calibration is achieved after the correction procedure is applied. We introduce a set-based perspective on calibration through the notion of cumulative mass calibration and the corresponding error measures. We propose a new calibration procedure based on conformal prediction that forms cumulative probabilities with guaranteed marginal coverage. We introduce an adaptive temperature scaling algorithm, with the temperature tuned for each input to satisfy the conformal coverage constraint. As we show, this procedure can be efficiently implemented. Across image classification tasks, particularly in settings with many classes, our method improves newly introduced calibration error measures (CMCE and

α

-CMCE) and standard metrics (such as ECE, cw-ECE, MCE) over the existing baselines.

Daniil Kazantsev, Eric Moulines, Maxim Panov +2

Mohamed bin Zayed University of Artificial Intelligence (MBZUAI), Abu Dhabi, United Arab Emirates · École pour l’informatique et les techniques avancées (EPITA), Paris, France

Post-Hoc Calibration Conformal Prediction

Ask PDF

Abstract

Conformal Prediction

Explore similar work

Decoupled Conformal Optimisation: Efficient Prediction Sets via Independent Tuning and Calibration

Reaching the Tail: Calibration Diversity Drives Conformal Coverage under Data Scarcity

Adaptive Cumulative Mass Calibration with Conformal Prediction