Adaptive Diffusion Freezing: Privacy-preserving Diffusion Models Against Membership Inference Attacks

Jialu Guo; Xiao Han; Junjie Wu

cs.CRSep 8, 2026

Adaptive Diffusion Freezing: Privacy-preserving Diffusion Models Against Membership Inference Attacks

Authors: Jialu Guo, Xiao Han, Junjie Wu

Organizations: MIIT Key Laboratory of Data Intelligence and Management Beihang University Beijing, China

Abstract

Diffusion models have achieved remarkable success in generative tasks across various areas, however their training process raises significant privacy concerns, particularly under membership inference attacks (MIAs). Prior studies on privacy-preserving of diffusion models fail to balance privacy, utility, and efficiency. To address this gap, we propose a novel framework of privacy-preserving diffusion models, Adaptive Diffusion Freezing (ADF), which can defend against MIAs with better trade-off. By leveraging cross-timestep adaptive freezing training, ADF explicitly control the participation of different data subsets across diffusion timesteps via a mask matrix, which reduces the over-memorization and leads to more uniform model behaviors between member and nonmember samples. To construct a freezing mask matrix that effectively reduce membership leakage without unnecessarily harming generation quality, we introduce a pretraining-based risk-aware freezing policy to estimate MIA risk based on memorization tendency, and suppress the contribution of the subset-timestep pairs with higher risk. Evaluations on multiple datasets demonstrate that ADF provides effective defense performance as well as state-of-the-art privacy-utility-efficiency trade-off performance compared to various baselines.

Figures & tables

Figure 1. Adaptive Diffusion Freezing (ADF)

Panel of Figure 2 — Figure 2. Privacy-utility trade-off performance of ADF on DDPM compared with baselines. X-axis: FID of the target model; Y-axis: ASR or AUC of MIA attacks. Points closer to the lower-left corner indicate a better privacy-utility trade-off.

Panel of Figure 3 — Figure 3. Privacy-utility trade-off performance of ADF on LDM compared with baselines. X-axis: FID of the target model; Y-axis: ASR or AUC of MIA attacks. Points closer to the lower-left corner indicate a better privacy-utility trade-off.

		SecMI stat		SecMI nn		$GSA$
Model	FID	ASR	AUC	ASR	AUC	ASR	AUC
SD3	31.223	0.520	0.519	0.519	0.516	0.627	0.682
ADF	26.479	0.517	0.516	0.511	0.501	0.601	0.653

Table 1. Evaluation of ADF on Stable Diffusion 3.

Panel of Figure 4 — Figure 4. Effect of hyper-parameters on privacy-utility trade-off performance.

Figure 5. Ablation of risk-aware freezing policy on CIFAR-10 against

GSA

Figure 6. Evaluation of ADF against Adaptive Attacks.

Panel of Figure 7 — Figure 7. Comparison of the distribution of attack feature: t-error.

Items	Analysis	Value
Loss reduction	MIA-success members	0.9722
	MIA-missed members	0.9655
	Success vs. missed	0.0067 ( $t=32.806^{***}$ )
MIA Success	Point biserial correlation	$r_{\mathrm{pb}}=0.231^{***}$
Membership Confidence	Pearson correlation	$r=0.207^{***}$
Membership Confidence	Spearman correlation	$\rho=0.112^{***}$

Table 2. Validation of loss reduction as a membership-risk proxy.

Panel of Figure 8 — Figure 8. Comparison of the synthetic images: W/O (left), DP-SGD (middle), Ours (right).

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Defense	FID	$SecMI_{stat}$				$SecMI_{nn}$				$GSA$
Dataset	Defense	FID	ASR	AUC	@1%	@0.1%	ASR	AUC	@1%	@0.1%	ASR	AUC	@1%	@0.1%
CIFAR-10	W/O	18.227	0.678	0.737	0.033	0.004	0.720	0.789	0.082	0.013	0.772	0.858	0.239	0.084
	Cutout	29.108	0.625	0.665	0.021	0.003	0.629	0.675	0.035	0.005	0.599	0.635	0.021	0.003
	L2	27.201	0.518	0.521	0.009	0.001	0.505	0.504	0.011	0.001	0.543	0.559	0.014	0.001
	DP-SGD	22.121	0.548	0.563	0.013	0.002	0.528	0.538	0.012	0.001	0.680	0.745	0.091	0.029
	KD	55.203	0.523	0.529	0.010	0.001	0.507	0.505	0.011	0.001	0.533	0.548	0.013	0.001

Appendix

Table 3. Privacy-utility trade-off performance of ADF across MIA settings. @1% and @0.1% denote TPR@1%FPR and [email protected]%FPR, respectively.

	ASR		AUC		TPR@1%FPR		[email protected]%FPR
Attack	Mean $\pm$ Std	Range	Mean $\pm$ Std	Range	Mean $\pm$ Std	Range	Mean $\pm$ Std	Range
$SecMI_{stat}$	$0.585\pm 0.002$	$[0.582,\,0.587]$	$0.614\pm 0.002$	$[0.611,\,0.617]$	$0.0160\pm 0.0010$	$[0.0150,\,0.0172]$	$0.0017\pm 0.0003$	$[0.0012,\,0.0022]$
$SecMI_{nn}$	$0.572\pm 0.005$	$[0.566,\,0.578]$	$0.600\pm 0.006$	$[0.595,\,0.608]$	$0.0215\pm 0.0007$	$[0.0206,\,0.0225]$	$0.0027\pm 0.0005$	$[0.0021,\,0.0034]$
$GSA$	$0.587\pm 0.007$	$[0.579,\,0.596]$	$0.636\pm 0.010$	$[0.622,\,0.644]$	$0.0313\pm 0.0040$	$[0.0276,\,0.0374]$	$0.0055\pm 0.0011$	$[0.0042,\,0.0069]$
FID			Mean $\pm$ std: $17.169\pm 0.165$			Range: $[16.961,\,17.396]$

Appendix

Table 4. Sensitivity analysis of ADF across different partition seeds.

Method	Trainable	Extra	Time	Runtime
	Params	Params	(s)	vs. DDPM
W/O	35.75M	0	1684.36	1.00 $\times$
Cutout	35.75M	0	1695.37	1.01 $\times$
L2	35.75M	0	1707.31	1.01 $\times$
DP-SGD	35.75M	0	4278.79	2.54 $\times$
KD	71.49M	61.31M	7458.01	4.43 $\times$

Appendix

Table 5. Computational efficiency comparison of different defense methods.

Privacy-Preserving ML

Privacy-preserving machine learning develops methods to learn, share, and run models while limiting exposure of sensitive data.

See all 423 papers

DIME: Query-Efficient Framework for Membership Inference on Diffusion Models
Aug 24, 2026Tue Do, Daniel AlabiMembership Inference Attacks Diffusion Models
University of Illinois at Urbana-Champaign
Black-box Membership Inference Attacks on the Pre-training Data of Image-generation Models
May 26, 2026Tao Qi, Huili Wang, Yuanhong Huang +6Membership Inference Attacks Diffusion-Based Image Generation
Beijing University of Posts and Telecommunications · Tsinghua University
Membership Inference Attacks on Discrete Diffusion Language Models
May 15, 2026Shailesh KasivelrajanMasked Diffusion Models Membership Inference Attacks
Indian Institute of Technology Madras

Aug 24, 2026J/K move · Enter open · S save

DIME: Query-Efficient Framework for Membership Inference on Diffusion Models

Tue Do, Daniel Alabi

University of Illinois at Urbana-Champaign

Membership Inference Attacks Diffusion Models

Open paper AskPDF arXiv

Membership inference attacks expose whether individual records were used to train a model, yet existing attacks on diffusion models are largely heuristic and can require substantial query budgets. We introduce DIME (Denoiser Ideal Membership Error), a theoretically grounded and query-efficient framework for membership inference on diffusion models. Our starting point is an exact characterization of the optimal diffusion denoiser for a finite training set, which reveals that membership leakage is governed by the denoiser's implicit reconstruction error. This error decomposes into two complementary signals: a bias term, capturing reconstruction accuracy, and a previously unexplored local crowding term, capturing the geometry of nearby training examples. Both admit efficient estimators using only model queries, yielding a practical attack with as few as two queries. Across CIFAR-10/100, STL10-U, CelebA, and ImageNet, DIME consistently outperforms prior attacks at comparable or substantially lower query cost, improving TPR at 1% FPR by up to

3\times

; remarkably, its two-query variant can outperform existing 30-query baselines. Finally, we suggest, discuss, and evaluate specific defenses to counteract such powerful membership tests.

Abstract

Privacy-Preserving ML

Explore similar work

DIME: Query-Efficient Framework for Membership Inference on Diffusion Models

Black-box Membership Inference Attacks on the Pre-training Data of Image-generation Models

Membership Inference Attacks on Discrete Diffusion Language Models