Conformal prediction can lose coverage when the data distribution changes after deployment. We study adaptation using labeled source data and unlabeled target inputs, allowing both the input distribution and its relationship with outcomes to change. We use Exponential Tilt Reweighting Alignment (ExTRA), introduced for classification by Maity et al. (2023), to estimate structured distribution shifts. We compare using its estimated weights in conformal calibration with additionally tilting the source predictive distribution. Shared learned predictors, estimated weights, calibration samples, and test observations isolate the effect of tilting. Existing theory gives both procedures target coverage with true weights and a common coverage bound with estimated weights. Identification calculations and an analysis of how scoring interacts with weight estimation error help explain why their performance can nevertheless differ. In a synthetic regression setting where the assumed models match the data-generating process and target inputs are informative about the shift, tilting reduces mean set length by about 30% relative to weighting alone, with both methods attaining coverage near nominal. Tilting can instead cause substantial coverage losses in synthetic classification and in regression when target inputs provide little information about the response shift. Real-data experiments also show no consistent benefit. Good coverage from weighted calibration alone does not ensure that adding predictive tilting will preserve coverage. Deciding when to apply this additional adjustment using only source labels and target inputs remains an open problem.
Figures & tables
Figure 1: ExTRA-WCP and ExTRA-WCP-T share the fitted joint ratio and calibration weights. Predictive tilting changes only the score. The source conditional estimate pP also supplies the conditional moments used to fit ExTRA. All fitted functions are fixed before calibration.
Ratio
Method
Coverage
Set size
None
Standard CP
0.775 (0.003)
0.914 (0.004)
Fitted
ExTRA-WCP
0.924 (0.007)
1.445 (0.062)
Fitted
ExTRA-WCP-T
0.814 (0.020)
1.219 (0.017)
Oracle
ExTRA-WCP
0.903 (0.003)
1.246 (0.012)
Oracle
ExTRA-WCP-T
0.903 (0.003)
1.202 (0.010)
Table 1: Synthetic classification at nominal coverage 0.90 . Entries are means (SEs) over 30 replications. Oracle rows use the true joint ratio while retaining the same learned predictor, calibration sample, and test observations.
Figure 2: Regression at η=1 , showing target coverage (top) and mean total set length (bottom), with means ± one SE over 30 replications. Axes give true (a⋆,b⋆) . The left column varies the response shift alone, while the right adds a shift toward noisier inputs. The first points, (0,0) and (1,0) , are no-shift and covariate-shift controls. Both coefficients are estimated in all six cases. Dashed lines mark 0.90 coverage, and horizontal offsets separate methods.
Method
Coverage
Total length
Standard CP
0.755 (0.003)
1.488 (0.010)
Input-only WCP
0.815 (0.004)
1.728 (0.015)
ExTRA-WCP
0.904 (0.003)
2.190 (0.021)
ExTRA-WCP-T
0.899 (0.003)
1.525 (0.023)
Two-stage WCP
0.902 (0.003)
2.181 (0.024)
Two-stage WCP-T
0.902 (0.003)
1.535 (0.019)
Table 2: Target coverage and mean total prediction-set length under the combined shift (a⋆,b⋆)=(1,1.2) , at nominal coverage 0.90 . Entries are means (SEs) over 30 replications. Input-only WCP uses Mβ(x) as an unnormalized input-ratio estimate. The two-stage baseline estimates the covariate tilt and mode prior separately. Oracle methods use the true joint ratio. All methods share the learned source predictor and evaluation samples.
WCP
WCP-T
Ratio estimator
Coverage
Length
Coverage
Length
TV
ExTRA, full
0.900 (0.004)
1.760 (0.022)
0.899 (0.003)
1.155 (0.012)
0.041 (0.003)
ExTRA, separable
0.875 (0.003)
1.621 (0.014)
0.892 (0.002)
1.442 (0.010)
0.384 ( <0.001 )
Oracle, full
0.900 (0.004)
1.760 (0.022)
0.899 (0.002)
1.146 (0.010)
0
Table 3: Response interaction at (a⋆,b⋆,c⋆)=(1,0,2) . Entries are means (SEs) over 30 replications at nominal coverage 0.90 . The full family includes Usign(Y) , which separable fits omit. Oracle weights retain the learned source predictor. TV uses population normalization. The full and separable ExTRA rows compare the effect of including the interaction.
Figure 3: Sensitivity at fixed shift (a⋆,b⋆)=(1,1.2) . Reducing η weakens mode information in the inputs while preserving mode frequencies and component distributions. It also changes the source score’s alignment with target responses. ExTRA estimation deteriorates before nonidentification at η=0 . Two-stage results in the text show that some loss is estimator-specific. Points show means ± one SE over 30 replications. The η=1 results also appear in Figure 2 . The dashed line marks 0.90 coverage.
Setting
ExTRA TV
Two-stage TV
ExTRA bound
Difference
95% interval
Classification
0.187 (0.028)
—
0.713
−0.110
[−0.164,−0.057]
Main regression ( η=1 ), (a⋆,b⋆)
(0,0)
0.039 (0.005)
0.014 (0.001)
0.861
−0.001
[−0.003,0.000]
(0,0.6)
0.045 (0.005)
0.020 (0.002)
0.855
−0.001
[−0.006,0.003]
(0,1.2)
0.040 (0.004)
0.023 (0.003)
0.860
−0.003
[−0.007,0.002]
(1,0)
0.046 (0.006)
0.019 (0.002)
0.854
−0.003
[−0.005,−0.001]
Table 4: Joint-ratio error and paired coverage differences for synthetic classification and separable regression over 30 replications. TV entries are means (SEs) for each estimator. The bound and last two columns refer to ExTRA and report 0.90−d , WCP-T minus WCP coverage, and its paired 95% interval, respectively. The two-stage baseline is not used for classification.
Figure 4: Gas classification and Crime regression at nominal coverage 0.90 (dashed). Points show means ± one SE across eight source partitions of each fixed dataset. Horizontal offsets separate methods, and lines connect discrete target groups. Tilting gives smaller Gas sets but substantial undercoverage in batches 9 and 10. Crime shows no consistent length reduction, and all methods under-cover in South.