Differentially Private Covariate Balancing Causal Inference

Yuki Ohnishi; Jordan Awan

stat.MEOct 18, 2024

Differentially Private Covariate Balancing Causal Inference

Authors: Yuki Ohnishi, Jordan Awan

Organizations: Department of Biostatistics, Yale School of Public Health · Department of Statistics, University of Pittsburgh

Abstract

Differential privacy is the leading mathematical framework for privacy protection, providing a probabilistic guarantee that safeguards individuals' private information when publishing statistics from a dataset. This guarantee is achieved by applying a randomized algorithm to the original data, which introduces unique challenges in data analysis by distorting inherent patterns. In particular, causal inference using observational data while preserving privacy is challenging because it requires a good covariate balance between treatment groups, but checking the covariate balance is challenging when privacy is a primary concern because releasing any statistic (e.g., t-statistic) for the balance check compromises privacy. Additionally, the performance of the privatized estimator critically depends on the choice of differential privacy mechanisms, and it remains unexplored how privacy-protecting causal inference could be conducted with observational data. In this article, we present a differentially private two-stage covariate balancing weighting estimator to infer causal effects from observational data. Our algorithm produces both point and interval estimators with statistical guarantees, such as covariate balance, rate optimality, and oracle budget allocation, under a given privacy budget.

Figures & tables

$(\alpha,\beta)$	Target	$w_{1}(e)$	$w_{0}(e)$	$S(e,1)$	$S(e,0)$	$M$
$(-1,-1)$	ATE	$1/e$	$1/(1-e)$	$\log\frac{e}{1-e}-\frac{1}{e}$	$\log\frac{1-e}{e}-\frac{1}{1-e}$	$1/\eta$
$(-1,0)$	ATC	$(1-e)/e$	$1$	$-1/e$	$\log\frac{1-e}{e}$	$(1-\eta)/\eta$
$(0,-1)$	ATT	$1$	$e/(1-e)$	$\log\frac{e}{1-e}$	$-1/(1-e)$	$(1-\eta)/\eta$
$(0,0)$	ATO	$1-e$	$e$	$\log e$	$\log(1-e)$	$1-\eta$

Table 1: Targets, weights, and scoring rules. Scores are specified up to constants independent of

e

M

is a public bound on either arm’s weight defined in ( 7 ).

$n$	$\epsilon_{\rm total}$	Bias	MSE	Coverage	Mean width
10,000	1	0.003640	$3.02\times 10^{-4}$	1.000	1.3215
10,000	5	-0.000159	$8.72\times 10^{-5}$	0.978	0.0549
100,000	1	0.000061	$1.06\times 10^{-5}$	0.956	0.0146
100,000	5	0.000210	$8.53\times 10^{-6}$	0.952	0.0115

Table 2: Evaluation metrics of the proposed method under correctly specified propensity score model.

Linear outcome
Estimator	Bias	RMSE	Balance	Coverage	Mean length
CBSR, NP	-0.0001	0.0029	0.0000	0.955	0.0115
CBSR, DP (Ours)	-0.0001	0.0029	0.0012	0.957	0.0117
Logistic, NP	-0.0071	0.0077	0.0261	0.322	0.0116
KNG-logistic, DP	-0.0071	0.0077	0.0261	0.328	0.0117
Lee et al., DP	0.0015	0.0063	–	–	–

Table 3: Propensity misspecification with

n=10^{5}

, using 1,000 linear and 2,000 quadratic datasets paired across methods. NP is nonprivate. DP budgets are 1.6 for points and 0.4 for intervals. Lee is

(1.6,10^{-6})

-DP without intervals. Other complete releases are pure 2-DP. Intervals have nominal 95% coverage and are intersected with

[-1,1]

. Balance is the norm of arm-normalized feature-mean differences.

\dagger

denotes the Guha–Reiter interval adaptation in Appendix F.5 .

Setting	Equal allocation	Oracle allocation	Reduction
Asymptotic	9,106	5,376	41.0%
Empirical, $n=10^{4}$	8,838 (240)	5,355 (172)	39.4%
Empirical, $n=10^{6}$	8,726 (318)	5,367 (157)	38.5%
Empirical, $n=10^{8}$	8,728 (240)	5,503 (176)	36.9%

Table 4: Scaled privacy-perturbation cost under equal and oracle budget allocations, with

\epsilon_{\rm point}=0.8

. The asymptotic prediction is

C(\pi)

. Empirical costs estimate

n^{2}\epsilon_{\rm point}^{2}\mathrm{E}\{(\tilde{\tau}_{b}-\hat{\tau}_{b})^{2}\mid D\}

, averaged over 20 independent databases with 150 mechanism draws per database and allocation. Parentheses give Monte Carlo standard errors across databases. Reduction is relative to equal allocation.

$\epsilon_{\rm total}$	Estimate	95% confidence interval	Interval width	RMSE
Nonprivate	4.519	$[3.888,5.149]$	1.261	–
1	4.634	$[2.902,6.367]$	3.465	0.141
5	4.571	$[3.921,5.221]$	1.300	0.029
10	4.512	$[3.816,5.209]$	1.393	0.017

Table 5: Email campaign effects. Estimates, confidence limits, widths, and RMSE are in percentage points. At each budget, the estimate, 95% confidence interval, and its width use the same first scheduled draw. RMSE uses all 100 independent privatizations and compares with the nonprivate CBSR estimate. All 300 CBSR draws completed.

$\epsilon_{\rm total}$	First-draw imbalance	Mean imbalance	90th percentile
Nonprivate	$1.40\times 10^{-14}$	–	–
1	0.00936	0.02197	0.03650
5	0.00237	0.00407	0.00709
10	0.00080	0.00222	0.00361

Table 6: Covariate balance in the email campaign. Imbalance is

\|\mathcal{B}_{b}(\tilde{\theta})\|_{2}

on the transformed feature scale, with smaller values indicating better balance. The first-draw values use the same private propensity draws as Table 5 . Means and 90th percentiles use all 100 draws at each budget. The nonprivate row reports the single fitted value.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Estimator	Bias	RMSE	Balance	Coverage	Mean length
CBSR, NP	0.0000	0.0029	0.0000	0.946	0.0115
CBSR, DP	0.0000	0.0030	0.0060	0.937	0.0119
Logistic, NP	0.0000	0.0029	0.0004	0.948	0.0115
KNG-logistic, DP	0.0000	0.0029	0.0043	0.945	0.0124
Lee et al., DP	0.0161	0.0171	–	–	–
Guha–Reiter † , DP	-0.0004	0.0181	–	0.950	0.0756

Appendix

Table 7: Method comparison under the correctly specified multivariate propensity model with

n=10^{5}

, using 1,000 datasets paired across methods. NP is nonprivate. DP budgets are 1.6 for points and 0.4 for intervals. Lee is

(1.6,10^{-6})

-DP without intervals. Other complete releases are pure 2-DP. Intervals have nominal 95% coverage and are intersected with

[-1,1]

. Balance is the norm of arm-normalized feature-mean differences.

\dagger

denotes the Guha–Reiter interval adaptation in Appendix F.5 . All attempts completed.

$n$	Sequence	$\epsilon_{\rm total}$	Full	Omit KNG	Wald	Width
$10^{4}$	Fixed	1	0.959	0.900	0.815	0.0808
$10^{4}$	Critical	0.3	0.992	0.922	0.771	0.6786
$10^{6}$	Fixed	1	0.945	0.945	0.944	0.003517
$10^{6}$	Critical	0.03	0.938	0.793	0.374	0.01672
$10^{6}$	Low	0.009487	0.930	0.775	0.172	0.06937
$10^{8}$	Fixed	1	0.935	0.936	0.936	0.0003486

Appendix

Table 8: Superpopulation 95% coverage and mean interval width, using 1,000 databases per row. Ablations share the private point estimate and moment transcript. All 9,000 analyses completed without point flooring or arm projection.

$n$	Allocation	Cost (MCSE)	Mean bound	Nontrivial
$10^{4}$	Equal	$1.38\times 10^{-4}$ ( $3.74\times 10^{-6}$ )	$4.00$	0%
$10^{4}$	Oracle	$8.37\times 10^{-5}$ ( $2.69\times 10^{-6}$ )	$4.00$	0%
$10^{6}$	Equal	$1.36\times 10^{-8}$ ( $4.97\times 10^{-10}$ )	$2.17\times 10^{-3}$	100%
$10^{6}$	Oracle	$8.39\times 10^{-9}$ ( $2.46\times 10^{-10}$ )	$7.35\times 10^{-4}$	100%
$10^{8}$	Equal	$1.36\times 10^{-12}$ ( $3.75\times 10^{-14}$ )	$2.17\times 10^{-7}$	100%
$10^{8}$	Oracle	$8.60\times 10^{-13}$ ( $2.74\times 10^{-14}$ )	$7.35\times 10^{-8}$	100%

Appendix

Table 9: Unscaled conditional squared cost and explicit bound. Costs average draws within each of 20 databases, then across databases. Parentheses give Monte Carlo standard errors across databases. Nontrivial bounds are below 4.

Lee ridge $\lambda_{\rm Lee}$	0.25	0.50	0.75
0.001	0.0595/0.0584	0.0308/0.0281	0.0221/0.0210
0.01	0.0104/0.0124	0.0097/0.0114	0.0111/0.0134
0.1	0.0055/0.0544	0.0063/0.0544	0.0090/0.0552
Guha–Reiter $K_{\rm GR}$	50	100	200	500
RMSE	0.0355/0.0363	0.0189/0.0184	0.0116/0.0096	0.0086/0.0048
Coverage	0.958/0.944	0.955/0.942	0.917/0.945	0.654/0.940

Appendix

Table 10: Linear/quadratic results on the Table 3 datasets. Lee entries are RMSE. Columns give Lee training fractions or Guha–Reiter partition counts, fixed before evaluation. Guha–Reiter intervals use the stated adaptation.

$\epsilon_{\rm total}$	Estimate	Paired RMSE	Mean width	Width 90th pct.
Nonprivate	4.519	–	1.261	–
1	4.503	0.161	2.790	4.101
5	4.528	0.033	1.299	1.570
10	4.509	0.017	1.265	1.411

Appendix

Table 11: Logistic IPW for the email campaign with CBSR’s transforms, protected queries, and budget shares. Each private row completed 100 draws. Units and first-draw reporting follow Table 5 .

Causal Effect Estimation

Causal effect estimation measures how outcomes would change under interventions, using experiments or observational data and explicit assumptions.

See all 268 papers

Workload-Preserving Differentially Private Synthetic Data for Causal Inference via Maximum-Entropy Calibration
Jul 9, 2026Amir Asiaee, Kaveh AryanCausal Effect Estimation Synthetic Data Generation
Department of Biostatistics, Vanderbilt University Medical Center, Nashville, Tennessee, USA · Department of Informatics, King’s College London, London, UK
Differentially Private Average Treatment Effect Estimation by Propensity Score Blocking
Sep 8, 2026Duncan Stewardson, Grayson W. White, Adam GroceInverse Probability Weighting Causal Effect Estimation
Department of Computer Science Reed College Portland, OR, 97202 · Department of Mathematics and Statistics Reed College Portland, OR, 97202
Private Adaptive Covariance Estimation via Gaussian Graphical Models
May 22, 2026Cecilia Ferrando, Miguel Fuentes, Brett Mullins +2Differential Privacy Covariance Estimation
Manning College of Information and Computer Sciences University of Massachusetts Amherst

Jul 9, 2026J/K move · Enter open · S save

Workload-Preserving Differentially Private Synthetic Data for Causal Inference via Maximum-Entropy Calibration

Amir Asiaee, Kaveh Aryan

Department of Biostatistics, Vanderbilt University Medical Center, Nashville, Tennessee, USA · Department of Informatics, King’s College London, London, UK

Causal Effect Estimation Synthetic Data Generation Differential Privacy Causal Inference

Open paper AskPDF arXiv

Workload-based differentially private (DP) synthetic data methods privately measure aggregate queries and post-process the noisy answers into synthetic records. Generic workloads can achieve strong distributional fidelity, but causal estimands such as the average treatment effect (ATE) depend on treatment-arm balance and outcome moments that generic marginals need not preserve. We propose causal workloads: DP query sets designed around the orthogonal moments used by doubly robust causal estimators. The released workload can be used directly by stable moment-map estimators or reconstructed by maximum-entropy calibration into reusable synthetic data; our theory decomposes ATE error into sampling, privacy, workload-approximation, Monte Carlo, and calibration terms. We also introduce Causal-AIM, an adaptive workload selector, and a noise-aware multiple-imputation (NA+MI) procedure for confidence intervals from DP synthetic data. Because the workload is released once, the same DP synthetic table can support ATE, ATT, and subgroup analyses without additional privacy spending. Empirically, causal workloads are most useful at strict privacy budgets and for calibrated uncertainty, while generic workloads often retain an advantage for point RMSE as privacy relaxes. The broader lesson is a tradeoff: distributional fidelity can help point accuracy, but valid causal inference requires preserving causal moments and propagating DP noise rather than treating synthetic rows as real.

Abstract

Causal Effect Estimation

Explore similar work

Workload-Preserving Differentially Private Synthetic Data for Causal Inference via Maximum-Entropy Calibration

Differentially Private Average Treatment Effect Estimation by Propensity Score Blocking

Private Adaptive Covariance Estimation via Gaussian Graphical Models