Membership inference attacks expose whether individual records were used to train a model, yet existing attacks on diffusion models are largely heuristic and can require substantial query budgets. We introduce DIME (Denoiser Ideal Membership Error), a theoretically grounded and query-efficient framework for membership inference on diffusion models. Our starting point is an exact characterization of the optimal diffusion denoiser for a finite training set, which reveals that membership leakage is governed by the denoiser's implicit reconstruction error. This error decomposes into two complementary signals: a bias term, capturing reconstruction accuracy, and a previously unexplored local crowding term, capturing the geometry of nearby training examples. Both admit efficient estimators using only model queries, yielding a practical attack with as few as two queries. Across CIFAR-10/100, STL10-U, CelebA, and ImageNet, DIME consistently outperforms prior attacks at comparable or substantially lower query cost, improving TPR at 1% FPR by up to 3×; remarkably, its two-query variant can outperform existing 30-query baselines. Finally, we suggest, discuss, and evaluate specific defenses to counteract such powerful membership tests.
Figures & tables
Figure 1: Intuition for DIME’s two signals. For a candidate x∗ , mlocal is the responsibility-weighted mean of nearby training points. An isolated member’s mlocal coincides with x∗ (low bias, low crowding); an isolated non-member’s deviates substantially (high bias). A non-member inside a dense cluster can have mlocal coincidentally close to x∗ (low bias) while the contributing points remain widely spread (high crowding, shaded ellipse).
CIFAR-10 (%)
CIFAR-100 (%)
STL10-U (%)
CelebA (%)
Method
#Query ↓
ASR ↑
AUC ↑
TPR@1%FPR ↑
ASR ↑
AUC ↑
TPR@1%FPR ↑
ASR ↑
AUC ↑
TPR@1%FPR ↑
ASR ↑
AUC ↑
TPR@1%FPR ↑
Loss
1
77.30
84.38
6.59
75.31
82.51
9.44
93.85
97.95
52.14
67.32
73.50
5.67
PIA
2
81.67
88.58
14.20
79.14
86.34
15.26
95.28
98.77
70.54
72.63
80.06
9.75
SecMI stat
∼ 12
81.45
88.51
10.45
80.85
88.05
13.34
93.98
98.20
57.76
71.08
78.03
8.64
SimA ( ℓ4 )
1
83.60
90.43
14.07
82.40
89.40
13.64
95.42
98.61
62.68
71.09
78.21
8.91
SimA-MC ( ℓ4 , #mc=10)
10
84.26
91.34
16.95
83.22
90.55
19.03
97.25
99.31
80.92
74.33
82.05
9.91
Table 1: Performance of MIA attacks on DDPM across four benchmark datasets. DIME outperforms all other methods in all metrics at comparable query costs, with the biggest gains in TPR@1%FPR.
Method
#Query ↓
ASR ↑
AUC ↑
TPR@1%FPR ↑
Loss
1
74.62
80.64
1.60
PIA
2
67.56
69.88
1.07
SecMI stat
∼ 12
66.67
50.59
0.93
SimA ( ℓ4 )
1
83.84
86.35
7.20
SimA-MC ( ℓ4 , mc=10)
10
84.13
89.41
9.33
SimA-MC ( ℓ4 , mc=30)
30
82.58
87.45
1.87
Table 2: Performance on ImageNet-1k / Guided Diffusion (256x256, class-conditional). DIME outperforms all other methods in all metrics at comparable query costs, with the biggest gains in TPR@1%FPR.
Figure 2: DIME test statistic distributions for samples from member and held-out sets. The vertical dashed line denotes the selected threshold τ for each figure.
Figure 3: Robustness of the DIME test to timestep across five datasets. DDPM models remain stable across all time-steps, while the Guided Diffusion model is most vulnerable to membership inference at early timesteps.
Loss
PIA
SecMI stat
SimA ( ℓ4 )
DIME (Ours)
ASR ↑
AUC ↑
T@1%F ↑
ASR ↑
AUC ↑
T@1%F ↑
ASR ↑
AUC ↑
T@1%F ↑
ASR ↑
AUC ↑
T@1%F ↑
ASR ↑
AUC ↑
T@1%F ↑
ϵ=1
0.530
0.521
1.00
0.526
0.516
1.60
0.517
0.500
1.80
0.517
0.505
0.40
0.529
0.517
0.40
ϵ=4
0.539
0.518
1.40
0.506
0.481
0.60
0.522
0.488
1.20
0.515
0.502
0.20
0.525
0.515
1.20
ϵ=10
0.519
0.509
1.40
0.506
0.483
0.80
0.523
0.519
1.00
0.510
0.489
1.00
0.537
0.529
0.80
Vanilla
0.690
0.754
10.20
0.753
0.829
15.40
0.710
0.777
13.60
0.745
0.827
16.20
0.798
0.869
22.40
Table 3: Attack accuracy under DP-SGD defense (CelebA, n=1000 , identity-aware split, 500 epochs). All five attack methods’ accuracy collapses under DP-SGD, regardless of ϵ . ‘Vanilla’ means without DP-SGD. DIME ’s #Query is qref+1 (11 at qref=10 , the reported tier, roughly comparable to SimA-MC’s 10). The highest accuracy in each column is bolded.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: Membership signal of Bias and Variance term from the decomposition in Equation 5 from idealized denoiser over simulated data. The variance term is more effective at capturing membership signal of held-out data in a dense region of member data.
Figure 5: DIME vs. SimA-MC for the idealized denoiser on simulated data with q=10 , over various dataset sizes. The additional crowding signal captured in DIME improves membership signal decay with more diffusion noise.
Figure 6: DIME vs. SimA-MC for the idealized denoiser on simulated data with n=100 , over various query budgets. The additional crowding signal captured in DIME improves membership signal decay with more diffusion noise.
Figure 7: The idealized denoiser on synthetic 1D data. ϵ^∗(x,t) , computed exactly from the closed-form optimal denoiser, on synthetic training sets of size n=2,3,10 (vertical dashed lines mark training points), at four noise levels αt .
The rapid advancement of diffusion-based image generation models has raised serious concerns regarding potential copyright and privacy infringements involving human-created data. Membership inference attacks (MIAs) have emerged as a promising tool for identifying unauthorized data usage during model training. Existing methods typically assess the ability of model to denoise perturbed suspect images as an indicator of membership status. However, the discriminative power of such features is highly dependent on the degree of model memorization and deteriorates significantly when applied to less exposed data (e.g., pre-training data). Although several methods attempt to enhance detection by leveraging internal model features, these features are generally inaccessible in mainstream closed-source image generation platforms, limiting their practicality. In this paper, we demonstrate that analyzing how a black-box diffusion model denoises a target image and corresponding perturbed textual instructions can reveal more distinctive membership cues. Based on this insight, we propose a black-box membership inference attack framework (named SD-MIA) that leverages a cross-modal data perturbation mechanism to detect pre-training data in diffusion models. We conduct extensive experiments on both a public benchmark dataset and a newly constructed dataset, each comprising pre-training membership and non-membership samples with identical distributions. Experimental results demonstrate that SD-MIA achieves superior performance compared to existing baselines, including those with the unfair advantage of accessing internal model features.
Tao Qi, Huili Wang, Yuanhong Huang +6
Beijing University of Posts and Telecommunications · Tsinghua University
Diffusion models have achieved remarkable success in generative tasks across various areas, however their training process raises significant privacy concerns, particularly under membership inference attacks (MIAs). Prior studies on privacy-preserving of diffusion models fail to balance privacy, utility, and efficiency. To address this gap, we propose a novel framework of privacy-preserving diffusion models, Adaptive Diffusion Freezing (ADF), which can defend against MIAs with better trade-off. By leveraging cross-timestep adaptive freezing training, ADF explicitly control the participation of different data subsets across diffusion timesteps via a mask matrix, which reduces the over-memorization and leads to more uniform model behaviors between member and nonmember samples. To construct a freezing mask matrix that effectively reduce membership leakage without unnecessarily harming generation quality, we introduce a pretraining-based risk-aware freezing policy to estimate MIA risk based on memorization tendency, and suppress the contribution of the subset-timestep pairs with higher risk. Evaluations on multiple datasets demonstrate that ADF provides effective defense performance as well as state-of-the-art privacy-utility-efficiency trade-off performance compared to various baselines.
Jialu Guo, Xiao Han, Junjie Wu
MIIT Key Laboratory of Data Intelligence and Management Beihang University Beijing, China
Membership inference attacks (MIAs) test whether a candidate example was used to train a language model. Existing attacks on fine-tuned discrete diffusion language models (dLLMs) often aggregate reconstruction signals across many mask configurations, requiring repeated model evaluations. We propose Joint Uncertainty Guided Mask Probing (JUMP), an efficient MIA that exploits the ability of dLLMs to predict masked tokens in parallel. Using the pre-fine-tuning checkpoint as a reference, JUMP selects low-confidence positions, masks them jointly, and aggregates clipped target-reference reconstruction gaps. This focuses the attack on positions that reveal stronger membership signals from fine-tuning. After mask selection, all selected tokens are evaluated with one scoring query per model. Across six MIMIR domains, JUMP improves mean ROC-AUC over a prior multi-mask attack from 0.819 to 0.902 on LLaDA and from 0.851 to 0.942 on Dream. Including mask selection, it requires only three forward passes per example, compared with 32 for the baseline. We further extend JUMP to the target-only setting by replacing target-reference scoring with relative token preference, which compares the observed token with alternative predictions at the same masked position. Target-Only JUMP achieves mean ROC-AUCs of 0.609 and 0.638 on LLaDA and Dream.
Yeachan Jun, Albert No
Department of Artificial Intelligence Yonsei University, Seoul, Korea