Beyond Data Splitting: Full-Data Conformal Prediction by Differential Privacy

Young Hyun Cho; Jordan Awan

stat.MLMar 8, 2026

Beyond Data Splitting: Full-Data Conformal Prediction by Differential Privacy

Authors: Young Hyun Cho, Jordan Awan

Organizations: Purdue University · University of Pittsburgh

Abstract

Privacy protection and uncertainty quantification are increasingly important in data-driven decision making. Conformal prediction provides finite-sample marginal coverage, but existing private approaches often rely on data splitting, reducing the effective sample size. We propose a full-data privacy-preserving conformal prediction framework that avoids splitting. Our framework leverages stability induced by differential privacy to control the gap between in-sample and out-of-sample conformal scores, and pairs this with a conservative private quantile routine designed to prevent under-coverage. We show that a generic differential privacy guarantee yields a universal coverage floor, yet cannot generally recover the nominal $1-α$ level. We then provide a refined, mechanism-specific stability analysis and yields asymptotic recovery of the nominal level. Experiments demonstrate sharper prediction sets than the split-based private baseline.

Figures & tables

Figure 1: Conceptual illustration of the distributional shift. The top row represents the ideal “exchangeable” world where

\theta_{n+1}

is trained on all data points including the test point. The bottom row represents the actual “non-exchangeable” world where

\theta_{n}

is used; the test score

S_{n+1}^{(n)}

(red) is an out-of-sample evaluation. DP acts as a stabiliser, bounding the distance between

\theta_{n+1}

and

\theta_{n}

, thereby ensuring that the red box remains distributionally close to the blue box.

Privacy Budget	Method	Coverage	Efficiency $\downarrow$	Informativeness $\uparrow$
$\varepsilon=0.5$	DP-SCP-F (Finite)	0.912 (0.005)	1.746 (0.065)	0.509 (0.021)
	DP-SCP-A (Asymp.)	0.898 (0.006)	1.632 (0.057)	0.549 (0.022)
	DP-Split	0.900 (0.007)	2.095 (0.083)	0.363 (0.021)
$\varepsilon=1.0$	DP-SCP-F (Finite)	0.906 (0.006)	1.576 (0.035)	0.574 (0.015)
	DP-SCP-A (Asymp.)	0.898 (0.006)	1.521 (0.033)	0.597 (0.015)
	DP-Split	0.901 (0.006)	2.028 (0.066)	0.379 (0.018)

Table 1: Performance on BloodMNIST Classification with balanced budget allocation (

p=0.5

). The target coverage is

1-\alpha=0.90

. We report the mean and standard deviation (in parentheses) over 30 independent trials.

Privacy Budget	Method	Coverage	Avg. Width $\downarrow$
$\varepsilon=0.5$	DP-SCP-F (Finite)	0.913 (0.004)	2.306 (0.049)
	DP-SCP-A (Asymp.)	0.898 (0.005)	2.119 (0.043)
	DP-Split	0.898 (0.007)	2.193 (0.106)
$\varepsilon=1.0$	DP-SCP-F (Finite)	0.907 (0.004)	2.209 (0.041)
	DP-SCP-A (Asymp.)	0.899 (0.005)	2.113 (0.035)
	DP-Split	0.898 (0.006)	2.183 (0.096)

Table 2: Performance on California Housing Regression with budget allocation

p=0.5

. The target coverage is

1-\alpha=0.90

. We report the mean and standard deviation (in parentheses) of the interval width on the original scale over 30 independent trials.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Method	$p$	Coverage	Efficiency (Set Size)	Informativeness	Train $\epsilon$	$\sigma_{q}$
DP-SCP-F	0.3	0.911 (0.005)	2.072 (0.121)	0.406 (0.030)	0.148	35.83
	0.5	0.912 (0.005)	1.746 (0.065)	0.509 (0.021)	0.245	39.98
	0.7	0.915 (0.005)	1.697 (0.058)	0.528 (0.020)	0.341	47.90
	0.9	0.926 (0.007)	1.768 (0.067)	0.506 (0.022)	0.448	79.07
DP-SCP-A	0.3	0.898 (0.005)	1.936 (0.106)	0.442 (0.029)	0.148	35.83
	0.5	0.898 (0.006)	1.632 (0.057)	0.549 (0.022)	0.245	39.98

Appendix

Table S1: BloodMNIST performance comparison for privacy budget

\varepsilon=0.5

(

\delta=10^{-5}

Method	$p$	Coverage	Efficiency (Set Size)	Informativeness	Train $\epsilon$	$\sigma_{q}$
DP-SCP-F	0.3	0.905 (0.006)	1.645 (0.045)	0.545 (0.017)	0.290	19.07
	0.5	0.906 (0.006)	1.576 (0.035)	0.574 (0.015)	0.490	21.08
	0.7	0.907 (0.006)	1.569 (0.030)	0.578 (0.013)	0.697	24.92
	0.9	0.911 (0.006)	1.590 (0.035)	0.569 (0.015)	0.894	37.60
DP-SCP-A	0.3	0.898 (0.006)	1.588 (0.044)	0.567 (0.018)	0.290	19.07
	0.5	0.898 (0.006)	1.521 (0.033)	0.597 (0.015)	0.490	21.08

Appendix

Table S2: BloodMNIST performance comparison for privacy budget

\varepsilon=1.0

(

\delta=10^{-5}

Method	$p$	Coverage	Efficiency (Set Size)	Informativeness	Train $\epsilon$	$\sigma_{q}$
DP-SCP-F	0.3	0.902 (0.006)	1.536 (0.030)	0.590 (0.014)	0.597	10.15
	0.5	0.903 (0.006)	1.521 (0.026)	0.598 (0.013)	0.991	10.92
	0.7	0.903 (0.006)	1.518 (0.025)	0.599 (0.012)	1.396	12.45
	0.9	0.905 (0.006)	1.530 (0.026)	0.594 (0.013)	1.795	19.27
DP-SCP-A	0.3	0.898 (0.006)	1.507 (0.027)	0.603 (0.014)	0.597	10.15
	0.5	0.898 (0.006)	1.492 (0.025)	0.611 (0.014)	0.991	10.92

Appendix

Table S3: BloodMNIST performance comparison for privacy budget

\varepsilon=2.0

(

\delta=10^{-5}

Sample Size	Method ( $p$ )	Coverage	Avg. width (orig. scale)
–	DP-SCP-F (0.3)	0.912 (0.004)	2.313 (0.051)
	DP-SCP-F (0.5)	0.913 (0.004)	2.306 (0.049)
	DP-SCP-F (0.7)	0.917 (0.005)	2.351 (0.063)
	DP-SCP-F (0.9)	0.927 (0.006)	2.521 (0.083)
	DP-SCP-A (0.3)	0.899 (0.004)	2.140 (0.041)
	DP-SCP-A (0.5)	0.898 (0.005)	2.119 (0.043)

Appendix

Table S4: California Housing regression results for privacy budget

\varepsilon=0.5

(

\delta=10^{-5}

Sample Size	Method ( $p$ )	Coverage	Avg. width (orig. scale)
–	DP-SCP-F (0.3)	0.906 (0.004)	2.203 (0.040)
	DP-SCP-F (0.5)	0.907 (0.004)	2.209 (0.041)
	DP-SCP-F (0.7)	0.908 (0.004)	2.231 (0.041)
	DP-SCP-F (0.9)	0.914 (0.005)	2.320 (0.056)
	DP-SCP-A (0.3)	0.899 (0.004)	2.116 (0.035)
	DP-SCP-A (0.5)	0.899 (0.005)	2.113 (0.035)

Appendix

Table S5: California Housing regression results for privacy budget

\varepsilon=1.0

(

\delta=10^{-5}

Sample Size	Method ( $p$ )	Coverage	Avg. width (orig. scale)
–	DP-SCP-F (0.3)	0.902 (0.004)	2.158 (0.036)
	DP-SCP-F (0.5)	0.902 (0.004)	2.160 (0.034)
	DP-SCP-F (0.7)	0.903 (0.004)	2.171 (0.036)
	DP-SCP-F (0.9)	0.907 (0.004)	2.217 (0.036)
	DP-SCP-A (0.3)	0.898 (0.005)	2.110 (0.033)
	DP-SCP-A (0.5)	0.898 (0.004)	2.109 (0.032)

Appendix

Table S6: California Housing regression results for privacy budget

\varepsilon=2.0

(

\delta=10^{-5}

Figure S1: Trajectory stability vs. estimation error under synchronized coupling. Each panel corresponds to $\epsilon\in\{0.5,1,2\}$ (with $\delta=10^{-5}$ ), reporting the mean over $R=30$ runs with a shaded uncertainty band.

Sample Size ( $n$ )	Method	Coverage	Efficiency (Set Size)	Informativeness
10000	DP-SCP-F	0.927 (0.009)	3.243 (0.126)	0.114 (0.019)
	DP-SCP-A	0.902 (0.007)	2.976 (0.123)	0.149 (0.021)
	DP-Split	0.904 (0.011)	3.339 (0.139)	0.060 (0.024)
15000	DP-SCP-F	0.920 (0.006)	2.824 (0.116)	0.186 (0.023)
	DP-SCP-A	0.905 (0.006)	2.662 (0.105)	0.214 (0.023)
	DP-Split	0.908 (0.007)	3.064 (0.124)	0.120 (0.020)

Appendix

Table S7: Performance comparison for privacy budget

\varepsilon=0.5

. We report the mean and standard deviation over

30

independent trials.

Sample Size ( $n$ )	Method	Coverage	Efficiency (Set Size)	Informativeness
10000	DP-SCP-F	0.917 (0.006)	2.958 (0.122)	0.148 (0.019)
	DP-SCP-A	0.905 (0.005)	2.828 (0.125)	0.168 (0.020)
	DP-Split	0.904 (0.007)	3.288 (0.107)	0.063 (0.021)
15000	DP-SCP-F	0.913 (0.006)	2.599 (0.104)	0.222 (0.023)
	DP-SCP-A	0.905 (0.005)	2.519 (0.103)	0.238 (0.024)
	DP-Split	0.908 (0.005)	3.016 (0.104)	0.123 (0.023)

Appendix

Table S8: Performance comparison for privacy budget

\varepsilon=1.0

. We report the mean and standard deviation over

30

independent trials.

Sample Size ( $n$ )	Method	Coverage	Efficiency (Set Size)	Informativeness
10000	DP-SCP-F	0.912 (0.006)	2.854 (0.111)	0.161 (0.023)
	DP-SCP-A	0.905 (0.006)	2.778 (0.104)	0.174 (0.022)
	DP-Split	0.904 (0.006)	3.266 (0.096)	0.066 (0.020)
15000	DP-SCP-F	0.909 (0.006)	2.517 (0.086)	0.235 (0.022)
	DP-SCP-A	0.904 (0.007)	2.469 (0.083)	0.246 (0.022)
	DP-Split	0.909 (0.005)	3.014 (0.096)	0.122 (0.022)

Appendix

Table S9: Performance comparison for privacy budget

\varepsilon=2.0

. We report the mean and standard deviation over

30

independent trials.

Explore similar work

Differentially Private Conformal Prediction
Apr 16, 2026Jiamei Wu, Ce Zhang, Zhipeng Cai +4Conformal Prediction Differential Privacy
School of Mathematics and Statistics Beijing Jiaotong University Beijing, Beijing 100044, China · Department of Mathematical and Statistical Sciences University of Alberta Edmonton, AB T6G 2G1, Canada
On Optimal Data Splitting for Split Conformal Prediction
Jun 30, 2026Sayan Das, Bahram Yaghooti, Todd A. Kuffner +1Conformal Prediction Statistical Learning Theory
Department of Statistics and Data Science, Washington University in St. Louis, St. Louis, MO 63130, USA · Department of Electrical and Systems Engineering, Washington University in St. Louis, St. Louis, MO 63130, USA
Multi-Variable Conformal Prediction: Optimizing Prediction Sets without Data Splitting
May 12, 2026Laura Lützow, Simone Garatti, Marco C. Campi +2Conformal Prediction
TU Munich & MCML, Germany · Politecnico di Milano, Italy · University of Brescia, Italy +1

Apr 16, 2026J/K move · Enter open · S save

Differentially Private Conformal Prediction

Jiamei Wu, Ce Zhang, Zhipeng Cai, Jingsen Kong+3

School of Mathematics and Statistics Beijing Jiaotong University Beijing, Beijing 100044, China · Department of Mathematical and Statistical Sciences University of Alberta Edmonton, AB T6G 2G1, Canada

Conformal Prediction Differential Privacy

Open paper AskPDF arXiv

Conformal prediction (CP) has attracted broad attention as a simple and flexible framework for uncertainty quantification through prediction sets. In this work, we study how to deploy CP under differential privacy (DP) in a statistically efficient manner. We first introduce differential CP, a non-splitting conformal procedure that avoids the efficiency loss caused by data splitting and serves as a bridge between oracle CP and private conformal inference. By exploiting the stability properties of DP mechanisms, differential CP establishes a direct connection to oracle CP and inherits corresponding validity behavior. Building on this idea, we develop Differentially Private Conformal Prediction (DPCP), a fully private procedure that combines DP model training with a private quantile mechanism for calibration. We establish the end-to-end privacy guarantee of DPCP and investigate its coverage properties under additional regularity conditions. We further study the efficiency of both differential CP and DPCP under empirical risk minimization and general regression models, showing that DPCP can produce tighter prediction sets than existing private split conformal approaches under the same privacy budget. Numerical experiments on synthetic and real datasets demonstrate the practical effectiveness of the proposed methods.

Abstract

Explore similar work

Differentially Private Conformal Prediction

On Optimal Data Splitting for Split Conformal Prediction

Multi-Variable Conformal Prediction: Optimizing Prediction Sets without Data Splitting