Conformal Bayes combines Bayesian posterior predictive scores with conformal calibration, but under continuous label shift both the score and calibration weight depend on the unknown response-marginal density ratio. Existing methods typically estimate one shift parameter from pseudo-labels or predictive samples and plug it into calibration. We instead propose Joint Tilt-Sensitivity Conformal Bayes (JTS-CB), which performs sensitivity analysis over a prespecified set of plausible tilts; its split-conformal realization is JTS-SCB. Each tilt jointly determines the Bayesian conformal score and conformal importance weight. JTS-SCB forms a bounded sensitivity envelope over candidate tilts, but its calibration-only construction does not inherit the exact finite-sample weighted-conformal guarantee. We therefore study a separate candidate-weighted exact counterpart and show that its usefulness depends sharply on tail behavior. For scalar linear exponential tilts, any nonzero candidate tilt makes the exact set unbounded. More generally, tail-growing density ratios produce the same pathology, whereas quadratic tilts with a negative coefficient on y2 have vanishing tail weights and admit bounded exact inference on the original target. Ratio clipping provides a complementary bounded exact construction for a surrogate target when tails grow. Experiments show that strong plug-in predictive sampling can match the oracle when the shift is well identified, while sensitivity analysis is most useful for richer, weakly identified, or systematically biased shift models, at the cost of wider prediction sets.
Figures & tables
Figure 1: Illustration of exponential-family response-marginal tilting from a standard Gaussian source. (a) The source marginal is ps(y)=N(0,1) . (b) The linear tilt w(y)∝exp(βy) with β=1 shifts the mean while preserving the variance. (c) The quadratic tilt w(y)∝exp(β1y+β2y2) with (β1,β2)=(0.5,0.15) changes both location and spread. The linear model is nested at β2=0 .
Figure 2: Schematic view of practical JTS-SCB. A candidate tilt β jointly determines the Bayesian conformal score and the conformal importance weight. The inner step computes the weighted empirical quantile (equivalently, the smallest minimizer of a weighted quantile-regression problem) and returns the calibration threshold for that fixed tilt. The sensitivity aggregation then takes the largest threshold over the uncertainty set. JTS-SCB therefore provides a sensitivity-analysis alternative to estimating one target tilt from pseudo-labels or predictive samples.
Figure 3: Two complementary ways to handle continuous label shift. Pseudo-label SCB estimates a single target tilt from pseudo-labels and calibrates once; JTS-SCB instead computes one sensitivity threshold over an analyst-specified tilt uncertainty set and uses it to form the practical prediction set. The right panel separates this method from two theoretical counterparts used to analyze validity: the exact candidate-weighted construction and its clipped surrogate version.
Table 1: Exact candidate-weighted validity benchmarks, high-SNR regime, nominal 0.90 , averaged over 60 trials. Linear rows use truth β⋆=0.9 and candidate interval [0,B] ; their numerical width is measured on [−20,30] , while Proposition 4 gives the true infinite width for every B>0 . The final row uses quadratic truth β⋆=(0.30,−0.10) and an 11×11 grid over BQ−=[0,0.30]×[−0.15,−0.05] ; numerical width is measured on [−8,8] , with no accepted points at either grid boundary in any trial, and Corollary 3 certifies finite true width.
M
2
4
M⋆≈5.8
8
12
20
∞
width
2.36
2.26
2.30
2.28
2.33
2.29
38.5
cov (true)
.961
.941
.945
.951
.945
.951
.942
Table 2: Clipping the density ratio to [1/M,M] , β⋆=1.2 , high-SNR, β⋆ budget. Proposition 5 guarantees boundedness for M≤M⋆=αn/(1−α)≈5.8 . The larger tested values M=8,12,20 also show no tail divergence in the numerical evaluation, illustrating that the sufficient worst-case threshold is conservative in this experiment; the unclipped object ( M=∞ ) diverges (width grid-capped). “cov” is under the true exponential target. 25 trials.
Method
β⋆=0.3
0.6
0.9
1.2
1.5
Coverage
Source-CB
.899
.879
.864
.830
.798
Oracle-WT
.900
.895
.900
.886
.887
Robust-IW
.906
.894
.902
.894
.878
PL-point
.901
.894
.900
.888
.897
Exact counterpart ‡
.915
.924
.948
.957
.979
Table 3: Coverage and mean width vs. shift, high-SNR regime ( vs2=1.3 , R2=0.77 ), nominal 0.90 , one-sided 1× budget aligned with the positive target tilt. Standard errors ≤.01 . † uncertified diagnostic. ‡ the exact union is unbounded for every column (Prop. 4 ); the width block therefore reports its true Lebesgue width ∞ . Finite-window tail diagnostics are reported separately in Table 1 .
Table 5: Strong pseudo-labels (PL-tilt) vs. JTS, high-SNR, ntg=300 , nominal 0.90 , coverage/width. PL-tilt matches the oracle; JTS-SCB is wider at no coverage benefit.
Predictive model
β^PL
PL-tilt
JTS CJTSprac
Well-specified
1.05
.886 / 1.82
.944 / 2.24
Over-regularized
1.03
.898 / 1.85
.951 / 2.28
Mean-biased ( +0.5 )
1.57
.835 / 2.48
.972 / 2.86
Table 6: Model misspecification, β⋆=1.0 , ntg=300 , coverage/width. A biased predictive mean breaks PL-tilt; JTS-SCB holds by widening.
Condition
PL-point
PL-boot
JTS CJTSprac
Well-specified, ntg=300
.891 / 1.81
.909 / 1.92
.946 / 2.25
Well-specified, ntg=10
.880 / 1.78
.947 / 2.24
.951 / 2.33
Biased predictive +0.5
.832 / 2.47
.855 / 2.58
.972 / 2.86
Biased predictive −0.5
.841 / 2.41
.861 / 2.52
.921 / 2.91
Table 7: Uncertainty-propagated pseudo-labels (PL-boot) vs. point plug-in and JTS, coverage/width. PL-boot matches JTS-SCB at less width under variance unreliability (small ntg ) but, like any variance-hedge, fails under systematic bias, where JTS-SCB’s zero-anchored band still covers.
Multiplier
0.25×
0.50×
0.75×
1.0×
1.5×
2.0×
3.0×
CJTSprac coverage
.877
.908
.934
.957
.976
.988
.995
CJTSprac width
1.97
2.11
2.20
2.37
2.65
2.88
3.42
C∗ coverage
.854
.877
.881
.900
.896
.903
.861
C∗ width
1.88
1.92
1.92
1.99
2.09
2.13
2.30
Table 8: Budget sensitivity at β⋆=1.2 , high-SNR regime. Multiplier is the β -bound divided by β⋆ . The practical JTS-SCB sets are pathwise nested as the budget expands (Prop. 2 ), so their marginal coverage is nondecreasing; C∗ is an uncertified heuristic.
Table 13: One encompassing quadratic family when the true target tilt is randomly linear or quadratic. Entries are pooled coverage/mean width across the linear- and quadratic-truth trials, nominal 0.90 , from the primary 30 -trial mixed-truth run. The same JTS-Q box and the same universal TPS-Q procedure are used on every trial.
Method
Coverage
Width
Oracle-WT
.902
3.292
Beta-specific practical union
.967
4.061
Common-threshold JTS-Q envelope
.974
4.234
Table 14: Decomposition of JTS-Q conservatism in the mixed linear-or-quadratic experiment with ntg=30 . Entries are averages over an independent 40 -trial diagnostic run; Table 13 uses the primary 30 -trial run, so repeated Oracle-WT and JTS-Q summaries differ slightly by Monte Carlo variation. Both JTS rows are practical calibration-only constructions and carry no exact finite-sample certificate.
Conditional-noise scale c
1.00
1.25
1.50
2.00
CJTSprac coverage
.947
.857
.745
.516
Symmetric bimodal δ
0.00
0.50
1.00
1.50
CJTSprac coverage
.948
.938
.925
.898
Asymmetric bimodal δ
0.50
1.00
1.50
2.00
CJTSprac coverage
.948
.945
.932
.904
Table 15: Shift-model stress tests, high-SNR regime, nominal 0.90 . In the conditional-noise block, c=1 is the correctly specified linear-shift baseline with β⋆=1.0 , while c>1 breaks label shift. The bimodal blocks are separate response-marginal constructions; δ=0 in the symmetric block is a no-shift baseline. Reported entries are practical JTS-SCB coverage.
Conformal Bayes combines Bayesian posterior predictives with conformal calibration to produce prediction sets that are both statistically valid and geometrically efficient. We study conformal Bayes under label shift from a unified perspective, identifying two complementary approaches that restore nominal target-domain coverage through importance-weighted conformal calibration but operate through independent mechanisms. Post-hoc calibration tilts the posterior predictive toward the target domain and corrects the conformal threshold via an importance-weighted quantile, leaving the parameter posterior unchanged. In-training adaptation tilts the parameter posterior itself to the target domain, producing a corrected predictive whose highest predictive density region serves as the highest predictive density (HPD)-based prediction set under the fitted target predictive; efficiency is model-dependent and does not imply finite-sample conditional optimality. Two controlled experiments isolate the regime-dependence of each strategy: in the low-dimensional, well-estimated regime StrategyA produces the narrowest valid intervals, while in the high-dimensional, underdetermined regime StrategyB achieves up to 43% width reduction at unchanged coverage, under the stated source-sampling and label-shift assumptions.
Seungjin Choi
1CROID Research and aSSIST University, Seoul, Korea.
Conformal prediction provides distribution-free uncertainty quantification under exchangeability. However, this assumption is violated by label shift, where the marginal distribution of labels changes while the conditional distribution of inputs given labels remains stable. Under such shifts, standard conformal procedures no longer maintain their intended coverage behavior. Existing approaches address this via importance weighting. They pair the reweighting with residual-based nonconformity scores that ignore predictive uncertainty. The resulting intervals have uniform width. Bayesian conformal methods produce adaptive intervals by leveraging predictive distributions. They evaluate conformity under the source predictive, which is misaligned with the target domain under label shift. We propose the Label-Shift-Adjusted Bayesian Score (LSA score), a nonconformity score derived from a posterior predictive tilting identity. This identity shows that the target predictive is an importance-weighted transformation of the source predictive. We use it to derive a direct correction to the Bayesian score. We evaluate the method on molecular property prediction under controlled label shift. The LSA score consistently yields shorter intervals than residual-based and source-based Bayesian scores. Coverage in the target domain remains comparable. Under stronger shift, all methods incur some coverage loss due to pseudo-label-based density-ratio estimation. The LSA score is defined for any source predictive with a tractable log-density. We instantiate it with Bayesian Ridge Regression, where the correction admits a closed form.
Hyeonsu Lee, Juyeon Kim, Erkhembayar Jadamba +2
MOGAM Institute for Biomedical Research, Korea · CROID Research and aSSIST University, Korea
Machine learning predictors are rarely deployed on data that match the data they were calibrated on. Conformal prediction and its risk-control generalization promise distribution-free guarantees at deployment, but only under exchangeability or a covariate shift whose likelihood ratio is known exactly -- and the estimated-ratio case, the one that occurs in practice, is explicitly open, out of reach of the Bayesian construction available under exchangeability. We reach it from a different starting point: pushing a Dirichlet process prior on the calibration distribution through the likelihood ratio yields the exact Bayesian posterior over the deployed risk. That posterior is exact given the weight function, which at deployment is itself estimated, so we further prove finite-sample upper and lower bounds on the risk the selected threshold actually incurs, in terms of the weight-estimation error and the conditional shift. On synthetic and real covariate shifts, in regression and classification, the realized failure rate then stays close to target where shift-blind selection fails, and what deployment receives is a posterior over the risk of every threshold rather than a single number.
Xiayin Lou, Peng Luo
Technical University of Munich · Massachusetts Institute of Technology