JUMP: Efficient Membership Inference on Fine-Tuned Diffusion Language Models

Yeachan Jun; Albert No

cs.AIMay 9, 2026

JUMP: Efficient Membership Inference on Fine-Tuned Diffusion Language Models

Authors: Yeachan Jun, Albert No

Organizations: Department of Artificial Intelligence Yonsei University, Seoul, Korea

Abstract

Membership inference attacks (MIAs) test whether a candidate example was used to train a language model. Existing attacks on fine-tuned discrete diffusion language models (dLLMs) often aggregate reconstruction signals across many mask configurations, requiring repeated model evaluations. We propose Joint Uncertainty Guided Mask Probing (JUMP), an efficient MIA that exploits the ability of dLLMs to predict masked tokens in parallel. Using the pre-fine-tuning checkpoint as a reference, JUMP selects low-confidence positions, masks them jointly, and aggregates clipped target-reference reconstruction gaps. This focuses the attack on positions that reveal stronger membership signals from fine-tuning. After mask selection, all selected tokens are evaluated with one scoring query per model. Across six MIMIR domains, JUMP improves mean ROC-AUC over a prior multi-mask attack from 0.819 to 0.902 on LLaDA and from 0.851 to 0.942 on Dream. Including mask selection, it requires only three forward passes per example, compared with 32 for the baseline. We further extend JUMP to the target-only setting by replacing target-reference scoring with relative token preference, which compares the observed token with alternative predictions at the same masked position. Target-Only JUMP achieves mean ROC-AUCs of 0.609 and 0.638 on LLaDA and Dream.

Figures & tables

Figure 1: Overview of JUMP . The attack first selects informative positions under the reference model, then masks the selected positions jointly and scores them with the target and reference dLLMs. The final MIA statistic is computed from clipped token-level target/reference reconstruction gaps. In the visualization, general tokens are indicated by solid-line boxes, while confidence scores are denoted by dashed-line boxes.

LLaDA-8B-Base
	ArXiv					GitHub					HackerNews
Method	Attack quality ( $\uparrow$ )				Cost ( $\downarrow$ )	Attack quality ( $\uparrow$ )				Cost ( $\downarrow$ )	Attack quality ( $\uparrow$ )				Cost ( $\downarrow$ )
	AUC	T@10	T@1	T@0.1	NFE	AUC	T@10	T@1	T@0.1	NFE	AUC	T@10	T@1	T@0.1	NFE
Loss	0.52	0.12	0.01	0.00	16	0.59	0.20	0.04	0.01	16	0.51	0.11	0.01	0.01	16
ZLIB	0.52	0.13	0.01	0.00	16	0.61	0.23	0.07	0.02	16	0.51	0.10	0.01	0.01	16
SAMA	0.82	0.54	0.28	0.07	32	0.77	0.43	0.14	0.07	32	0.71	0.34	0.05	0.01	32

Table 1: Per-domain reference-assisted results on LLaDA-8B-Base and Dream-v0-Base-7B. We report ROC-AUC and TPR at fixed FPR (

\uparrow

) with total NFE (

\downarrow

). SAMA uses

T=16

(32 forwards) and JUMP uses 3; best quality values are in bold .

Family	Method	AUC	T@10	T@1	T@0.1
LLaDA	SAMA	0.8193	0.5393	0.2625	0.1220
	PRISM-Free	0.8677	0.6527	0.3238	0.1443
	JUMP	0.9024	0.7500	0.4663	0.2320
Dream	SAMA	0.8513	0.6122	0.2862	0.1660
	PRISM-Free	0.9404	0.8322	0.4997	0.2198
	JUMP	0.9415	0.8438	0.5505	0.2653

Table 2: PRISM-Free equal-weight six-domain means.

Appendix figures & tables26 assets

Supplementary material from the paper’s appendix.

Appendix

		SAMA				JUMP				Target-Only JUMP
Domain	Checkpoint	AUC	T@10	T@1	T@0.1	AUC	T@10	T@1	T@0.1	AUC	T@10	T@1	T@0.1
GitHub	Selected	0.77	0.43	0.15	0.07	0.86	0.66	0.32	0.18	0.60	0.21	0.03	0.01
GitHub	Epoch 4	0.83	0.58	0.25	0.08	0.91	0.77	0.50	0.32	0.63	0.24	0.04	0.01
HackerNews	Selected	0.71	0.34	0.05	0.01	0.76	0.39	0.10	0.02	0.57	0.15	0.02	0.00
HackerNews	Epoch 4	0.80	0.51	0.14	0.02	0.86	0.64	0.21	0.08	0.63	0.20	0.03	0.00

Appendix

Table 4: Checkpoint-selection robustness on GitHub and HackerNews. “Selected” denotes the checkpoint used in the main evaluation (

\sim

3 epochs), while “Epoch 4” evaluates checkpoint 80.

Domain	AUC	T@10	T@1	T@0.1
ArXiv	0.94 [.93,.95]	0.84 [.80,.86]	0.55 [.45,.60]	0.30 [.10,.46]
GitHub	0.86 [.84,.87]	0.66 [.62,.70]	0.32 [.24,.44]	0.18 [.00,.25]
HackerNews	0.76 [.74,.78]	0.39 [.35,.45]	0.10 [.07,.16]	0.02 [.00,.08]
PubMed Central	0.92 [.90,.93]	0.75 [.71,.80]	0.43 [.31,.52]	0.12 [.04,.33]
Wikipedia (en)	0.98 [.97,.99]	0.95 [.94,.97]	0.80 [.76,.85]	0.67 [.61,.78]
Pile-CC	0.96 [.95,.97]	0.90 [.88,.93]	0.60 [.52,.67]	0.10 [.01,.53]

Appendix

Table 5: Per-domain JUMP results on fine-tuned LLaDA-8B-Base, with 95% bootstrap CIs in brackets.

Domain	AUC	T@10	T@1	T@0.1
ArXiv	0.93 [.92,.94]	0.81 [.78,.84]	0.52 [.39,.58]	0.32 [.22,.40]
GitHub	0.91 [.89,.92]	0.75 [.69,.80]	0.36 [.28,.48]	0.01 [.00,.29]
HackerNews	0.93 [.92,.94]	0.81 [.77,.83]	0.49 [.28,.58]	0.23 [.15,.30]
PubMed Central	0.95 [.94,.96]	0.86 [.83,.89]	0.53 [.49,.66]	0.35 [.28,.51]
Wikipedia (en)	0.96 [.95,.97]	0.89 [.87,.92]	0.63 [.51,.73]	0.29 [.22,.52]
Pile-CC	0.97 [.97,.98]	0.94 [.92,.95]	0.77 [.66,.81]	0.38 [.13,.68]

Appendix

Table 6: Per-domain JUMP results on fine-tuned Dream-v0-Base-7B, with 95% bootstrap CIs in brackets.

Domain	AUC	T@10	T@1	T@0.1
ArXiv	0.62 [.59,.64]	0.21 [.17,.26]	0.03 [.01,.06]	0.01 [.00,.02]
GitHub	0.60 [.57,.62]	0.21 [.17,.24]	0.03 [.02,.05]	0.01 [.00,.02]
HackerNews	0.57 [.54,.59]	0.15 [.12,.19]	0.02 [.01,.03]	0.00 [.00,.02]
PubMed Central	0.62 [.60,.64]	0.21 [.17,.26]	0.03 [.02,.06]	0.01 [.00,.02]
Wikipedia (en)	0.64 [.62,.66]	0.22 [.18,.26]	0.03 [.02,.05]	0.01 [.00,.02]
Pile-CC	0.62 [.59,.64]	0.20 [.16,.24]	0.02 [.01,.04]	0.01 [.00,.01]

Appendix

Table 7: Per-domain Target-Only JUMP results on LLaDA-8B-Base, with 95% bootstrap CIs in brackets.

Domain	AUC	T@10	T@1	T@0.1
ArXiv	0.66 [.63,.68]	0.23 [.18,.28]	0.04 [.02,.08]	0.02 [.01,.03]
GitHub	0.58 [.56,.61]	0.13 [.10,.17]	0.01 [.00,.02]	0.00 [.00,.01]
HackerNews	0.66 [.63,.68]	0.22 [.19,.25]	0.04 [.01,.07]	0.00 [.00,.02]
PubMed Central	0.64 [.61,.66]	0.20 [.16,.25]	0.02 [.00,.04]	0.00 [.00,.01]
Wikipedia (en)	0.65 [.63,.68]	0.19 [.15,.25]	0.02 [.00,.04]	0.00 [.00,.01]
Pile-CC	0.64 [.62,.66]	0.23 [.18,.27]	0.04 [.02,.06]	0.01 [.00,.02]

Appendix

Table 8: Per-domain Target-Only JUMP results on Dream-v0-Base-7B, with 95% bootstrap CIs in brackets.

Exposure	Loss AUC	ZLIB AUC	SAMA AUC	JUMP AUC	JUMP TPR@1%
Epoch 1	0.5020	0.5055	0.5137	0.5263	0.0127
Epoch 2	0.5056	0.5091	0.6600	0.7316	0.0942
Selected $\leq 4$	0.5273	0.5311	0.8193	0.9024	0.4663

Appendix

Table 9: Attack performance as fine-tuning exposure increases. Values are six-domain means.

Exposure	Member loss	Non-member loss	Gap	Held-out gain
Epoch 1	2.4271	2.4432	0.0161	0.0020
Epoch 2	2.3878	2.4179	0.0301	0.0273
Selected $\leq 4$	2.2604	2.3453	0.0850	0.0998

Appendix

Table 10: Member/non-member reconstruction losses for the same exposure controls. “Held-out gain” is the reduction in non-member loss relative to the base model.

Method	NFE	AUC	TPR@10%	TPR@1%	TPR@0.1%
SAMA ( $T=2$ )	4	0.6283	0.2482	0.0515	0.0197
SAMA ( $T=16$ )	32	0.7116	0.3660	0.1265	0.0510
JUMP	3	0.7466	0.4192	0.1450	0.0572

Appendix

Table 11: Mixed six-domain target. The primary result is the equal-weight mean over six within-domain ROCs.

Learned selector	AUC	TPR@10%	TPR@1%	TPR@0.1%
LLaDA-8B-Base	0.9024	0.7500	0.4663	0.2320
Distilled 5.36B	0.8966	0.7245	0.4090	0.1885
LLaDA-1.5 (8B-class)	0.9035	0.7470	0.4548	0.3152
LLaDA-8B-Instruct	0.9040	0.7470	0.4655	0.2550

Appendix

Table 12: Selector-backbone transfer with the target and exact-base scoring pair fixed.

Scoring reference	AUC	TPR@10%	TPR@1%	TPR@0.1%
Exact pre-FT LLaDA-8B	0.9024	0.7500	0.4663	0.2320
Distilled 5.36B	0.6052	0.2137	0.0233	0.0067
LLaDA-1.5	0.6949	0.3422	0.0666	0.0182
LLaDA-8B-Instruct	0.6953	0.3448	0.0670	0.0190

Appendix

Table 13: Diagnostic results with non-exact scoring references.

Family	Method	ROC-AUC	TPR@10%	TPR@1%	TPR@0.1%
LLaDA	Random Joint	0.8173	0.5297	0.2073	0.0890
	Entropy Top	0.8867	0.7043	0.3650	0.1453
	RareToken (C4-freq)	0.8235	0.5489	0.2078	0.0394
	SAMA ( $T=16$ )	0.8193	0.5393	0.2625	0.1220
	JUMP	0.9024	0.7500	0.4663	0.2320
Dream	Random Joint	0.8700	0.6337	0.2927	0.1365

Appendix

Table 14: Selector and joint-probing controls across model families (six-domain mean). Joint variants use

K=64

and

\tau=\log 1.5

; RareToken uses C4 frequencies.

Selector corpus	Mean ROC-AUC
C4	0.9024
SlimPajama	0.8983
FineWeb	0.8972

Appendix

Table 15: Six-domain mean AUC for matched selector-training corpora.

Method	ROC-AUC	TPR@10%	TPR@1%	TPR@0.1%
RareToken (C4-freq)	0.8235	0.5489	0.2078	0.0394
SAMA ( $T=16$ )	0.8193	0.5393	0.2625	0.1220
JUMP	0.9024	0.7500	0.4663	0.2320

Appendix

Table 16: LLaDA selector diagnostics (six-domain mean); RareToken uses C4 frequencies.

C4 samples	ROC-AUC	TPR@10%	TPR@1%	TPR@0.1%	Train time
$20\mathrm{k}$	0.8960	0.7303	0.4000	0.1897	190s
$50\mathrm{k}$	0.9002	0.7437	0.4152	0.2437	421s
$100\mathrm{k}$	0.8959	0.7195	0.4068	0.2105	813s
$200\mathrm{k}$	0.9024	0.7500	0.4663	0.2320	1,613s
$400\mathrm{k}$	0.9052	0.7477	0.4498	0.2643	3,158s
$500\mathrm{k}$	0.9014	0.7432	0.4558	0.2783	3,922s

Appendix

Table 17: C4 selector-training sample-size ablation, averaged over six LLaDA domains.

$K$	ROC-AUC	TPR@10%	TPR@1%	TPR@0.1%
32	0.8880	0.7100	0.3995	0.1798
64	0.9024	0.7500	0.4663	0.2320
128	0.8650	0.6525	0.2430	0.0442

Appendix

Table 18: Selected-token budget sweep with

\tau=\log 1.5

, averaged over six LLaDA domains.

Method	ROC-AUC	TPR@10%	TPR@1%	TPR@0.1%
No clip mean	0.8166	0.5478	0.2323	0.1003
Clipped mean	0.9024	0.7500	0.4663	0.2320

Appendix

Table 19: Clipping ablation averaged over six MIMIR domains.

Method	Inference	Head	Total
SAMA	1.863	N/A	1.863
JUMP (C4)	0.184	0.134	0.318
JUMP (SlimPajama)	0.185	0.120	0.305
JUMP (FineWeb)	0.183	0.134	0.317

Appendix

Table 20: Runtime per sample in seconds. “Head” denotes amortized one-time selector training.

Model	Score	All	$p\leq.50$	$p\leq.90$	$p\leq.95$	$p\leq.99$
LLaDA	Masked Score	.5752	.5688	.5858	.5866	.5854
LLaDA	Target-Only JUMP	.6038	.5785	.6052	.6080	.6095
Dream	Masked Score	.6248	.6019	.6393	.6446	.6454
Dream	Target-Only JUMP	.6266	.6015	.6348	.6377	.6375

Appendix

Table 21: Mean ROC-AUC across six domains under different near-deterministic filtering thresholds. “All” uses every valid position.

Model	All	$p\leq.50$	$p\leq.90$	$p\leq.95$	$p\leq.99$
LLaDA	315.0	59.6	102.8	118.9	155.3
Dream	305.9	57.4	100.9	117.4	156.3

Appendix

Table 22: Mean number of retained positions per document under different probability thresholds, averaged equally across the six domains.

Model	Domain	Saturated gap	Non-saturated gap	Saturated fraction
LLaDA	ArXiv	0.00018	0.04816	44.8%
LLaDA	GitHub	-0.00090	0.04816	76.6%
LLaDA	HackerNews	-0.00002	0.02818	40.1%
LLaDA	Pile-CC	0.00003	0.05321	35.7%
LLaDA	PubMed Central	-0.00004	0.04752	46.0%
LLaDA	Wikipedia (en)	-0.00021	0.06306	53.9%

Appendix

Table 23: Token-level member–non-member difference in

q+H

inside and outside the saturated regime. Saturated positions satisfy

p_{\mathrm{true}}>0.99

	LLaDA		Dream
Domain	$q$	$q+H$	$q$	$q+H$
ArXiv	.6018	.6169	.6576	.6592
GitHub	.5769	.5955	.6372	.5810
HackerNews	.5409	.5662	.6528	.6574
PubMed Central	.5982	.6212	.6331	.6362
Wikipedia (en)	.6037	.6382	.6474	.6526

Appendix

Table 24: Per-domain ROC-AUC using the same

p_{\mathrm{true}}\leq 0.99

support. Masked Score averages

q

, while Target-Only JUMP averages

q+H

Model	Threshold	$q$	$q+H$	$(q+H)/\sigma$
LLaDA	$p\leq.50$	.5688	.5785	.5634
LLaDA	$p\leq.90$	.5858	.6052	.5923
LLaDA	$p\leq.95$	.5866	.6080	.5962
LLaDA	$p\leq.99$	.5854	.6095	.5994
Dream	$p\leq.50$	.6019	.6015	.5739
Dream	$p\leq.90$	.6393	.6348	.6215

Appendix

Table 25: Mean ROC-AUC across six domains with and without vocabulary-scale normalization.

Measurement	Contexts	AUC	T@10	T@1	T@0.1
F16 once	16	.6095	.1995	.0270	.0060
F16 twice	32	.6130	.1952	.0295	.0062
F32 once	32	.6056	.1915	.0255	.0068

Appendix

Table 26: Grouped reconstruction diagnostic on LLaDA. Values are six-domain means under the final

p_{\mathrm{true}}\leq 0.99

Target-Only JUMP score.

$r/\alpha$	Trainable	PPL	AUC		TPR@1%
			JUMP	Target-Only	JUMP	Target-Only
16/32	0.10%	5.22	0.512	0.498	0.007	0.012
64/128	0.42%	4.89	0.545	0.503	0.018	0.013
256/512	1.65%	4.71	0.588	0.512	0.026	0.013
1024/2048	6.28%	4.66	0.675	0.544	0.066	0.011
Full FT	100.00%	4.74	0.938	0.617	0.539	0.033

Appendix

Table 27: ArXiv LoRA capacity sweep. JUMP denotes the reference-assisted attack and Target-Only denotes Target-Only JUMP.

Method	Target $\varepsilon$	PPL	AUC		TPR@1%
			JUMP	Target-Only	JUMP	Target-Only
Non-DP LoRA	N/A	4.66	0.675	0.544	0.066	0.011
DP-LoRA	0.01	55.77	0.529	0.497	0.013	0.012
DP-LoRA	1	18.67	0.502	0.498	0.012	0.011
DP-LoRA	10	7.89	0.514	0.498	0.013	0.011
DP-LoRA	20	7.05	0.520	0.498	0.014	0.011

Appendix

Table 28: Matched non-DP LoRA and DP-LoRA on ArXiv across target privacy budgets.

Asset	Role in this paper	Citation / source	License / terms
LLaDA-8B-Base	Main base/reference dLLM and fine-tuning target	Nie et al. (2025)	MIT
Dream-v0-Base-7B	Second dLLM family for cross-architecture evaluation	Ye et al. (2025)	Apache-2.0
MIMIR	Member/non-member evaluation benchmark	Duan et al. (2024)	MIT; gated access terms on Hugging Face
C4	PRISM selector training corpus	Raffel et al. (2020)	ODC-BY; Common Crawl Terms of Use
SlimPajama	Selector-corpus ablation	Soboleva et al. (2023)	Apache-2.0 for the original Cerebras release; subset-specific terms if using a split or reupload
FineWeb	Selector-corpus ablation	Penedo et al. (2024)	ODC-By v1.0; Common Crawl Terms of Use

Appendix

Table 29: Existing assets used in this paper.

Explore similar work

Membership Inference Attacks on Discrete Diffusion Language Models
May 15, 2026Shailesh KasivelrajanMasked Diffusion Language Models Membership Inference Attacks
Indian Institute of Technology Madras
CheckMIABench: Firm Foundations For Membership Inference Attacks on Language Models
Jun 16, 2026Jeffrey G. Wang, Jason Wang, Marvin Li +1Membership Inference Attacks Attacker Large Language Model
Harvard University · Harvard Business School
MaskForge: Structure-Aware Adaptive Attacks for Jailbreaking Diffusion Large Language Models
Jun 1, 2026Yingzi Ma, Zhengyue Zhao, Xiaogeng Liu +3Large Language Model Jailbreaks Large Language Model Baselines
University of Wisconsin-Madison · Johns Hopkins University · Responsible AI Research (RAIR) Centre, The University of Adelaide +1

May 15, 2026, cs.LGJ/K move · Enter open · S save

Membership Inference Attacks on Discrete Diffusion Language Models

Shailesh Kasivelrajan

Indian Institute of Technology Madras

Masked Diffusion Language Models Membership Inference Attacks Diffusion Language Models Membership Inference Xgboost Model

Open paper AskPDF arXiv

Masked Diffusion Language Models MDLMs replace autoregressive generation with iterative demasking and their privacy properties are largely unstudied. We study membership inference attacks MIA on fine tuned MDLMs and show they are significantly more vulnerable than current grey box baselines suggest. We extract a 46 dimensional feature vector from the models reconstruction loss at four masking ratios and train XGBoost and MLP classifiers on top. On the MIMIR benchmark across six text domains XGBoost achieves mean AUC 0.878 peaking at 0.930 on Pile CC and beats the SAMA grey box baseline by 0.062 AUC on average. A leave one signal out ablation shows that the ELBO trajectory alone drives most of this with a mean drop of 0.130 when removed while attention features add almost nothing below 0.003. We also design a shadow model transfer attack where K equals 3 surrogate MDLMs trained on data from unrelated domains generate classifier labels with no access to the target domain. This achieves 0.858 mean AUC within 0.020 of the white box oracle and establishes shadow model transfer as a practical and near equally effective attack path.

Abstract

Explore similar work

Membership Inference Attacks on Discrete Diffusion Language Models

CheckMIABench: Firm Foundations For Membership Inference Attacks on Language Models

MaskForge: Structure-Aware Adaptive Attacks for Jailbreaking Diffusion Large Language Models