Detecting Non-Membership in LLM Training Data via Rank Correlations

Pranav Shetty; Mirazul Haque; Zhiqiang Ma; Xiaomo Liu

cs.CLMar 24, 2026

Detecting Non-Membership in LLM Training Data via Rank Correlations

Authors: Pranav Shetty, Mirazul Haque, Zhiqiang Ma, Xiaomo Liu

Organizations: JPMorgan AI Research

Abstract

As large language models (LLMs) are trained on increasingly vast and opaque text corpora, determining which data contributed to training has become essential for copyright enforcement, compliance auditing, and user trust. While prior work focuses on detecting whether a dataset was used in training (membership inference), the complementary problem -- verifying that a dataset was not used -- has received little attention. We address this gap by introducing PRISM, a test that detects dataset-level non-membership using only grey-box access to model logits. Our key insight is that two models that have not seen a dataset exhibit higher rank correlation in their normalized token log probabilities than when one model has been trained on that data. Using this observation, we construct a correlation-based test that detects non-membership. Empirically, PRISM reliably rules out membership in training data across all datasets tested while avoiding false positives, thus offering a framework for verifying that specific datasets were excluded from LLM training.

Figures & tables

Figure 1: Overview of PRISM . Logits of Reference LLM, Target LLM, and Distilled Reference LLMs are used to calculate Min-K%++ scores, and the corresponding ranking of the scores. Final statistical testing for non-membership detection is dependent on the rank correlation between the reference and target model, and the rank correlation between the distilled reference and target model.

Dataset	Pythia-1b		Pythia-2.8b		Pythia-6.9b
	410m	410m-CPT	410m	410m-CPT	410m	410m-CPT
ArXiv	0.82	0.45	0.75	0.42	0.72	0.37
HN	0.71	0.40	0.74	0.40	0.68	0.41
CC	0.80	0.50	0.77	0.46	0.70	0.42
Wikipedia	0.84	0.25	0.80	0.25	0.77	0.21
PubMed	0.84	0.33	0.79	0.29	0.77	0.30

Table 1: Rank correlation coefficients over different datasets between Pythia reference models of different sizes computed against Pythia-410m and Pythia-410m-CPT. Each of the datasets was held out from training the vanilla model as well as each reference model in the columns, but was used during continued pretraining of Pythia-410m-CPT with 6 billion additional tokens.

Figure 2: Overview of distilling the reference model. The distillation is dependent on two components: cross entropy of the reference model and KL divergence between the output probability of the target and reference model.

Target Model	PubMed	HN	ArXiv	CC	Ubuntu	Freelaw	Enron	Reddit
Pythia-410m-CPT	1.000	0.354	0.113	0.302	0.162	0.982	0.128	0.817
Pythia-410m	1.0e-4	1.0e-4	1.0e-4	1.0e-4	1.0e-4	1.0e-4	2.0e-4	1.0e-4
Pythia-1b	1.0e-4	1.0e-4	1.0e-4	1.0e-4	1.0e-4	1.0e-4	1.0e-4	1.0e-4
Pythia-2.8b	1.0e-4	1.0e-4	1.0e-4	1.0e-4	1.0e-4	1.0e-4	1.0e-4	1.0e-4
Pythia-6.9b	1.0e-4	1.0e-4	1.0e-4	1.0e-4	1.0e-4	1.0e-4	2.0e-4	1.0e-4

Table 2: p-values for non-member detection. None of the datasets below were used while training the vanilla Pythia models, but were used to train Pythia-410m-CPT. Pythia-410m is used as the reference model for all cases except Pythia-410m and Pythia-410m-CPT where Pythia-1b was the reference model.

	PubMed	HN	ArXiV	CC	Reddit	Ubuntu	Freelaw	Enron
p-value	0.84	0.01	1.0e-4	0.02	0.03	1.0e-4	1.0e-4	1.0e-4

Table 3: p-values computed using PRISM on Pythia-410m-CPT using a split of each of the below datasets that was held out during continued pre-training

Method	PubMed		HN		ArXiv		CommonCrawl		Reddit
	410m	410m-CPT	410m	410m-CPT	410m	410m-CPT	410m	410m-CPT	410m	410m-CPT
LLM-DI	0.604	0.775	0.395	0.022	0.781	0.083	0.642	0.045	0.437	0.064
PaCoST	3.0e-13	0.045	0.683	0.999	0.158	0.122	0.002	0.009	0.001	4.0e-9

Table 4: p-values for dataset level membership detection

Reference Model	PubMed		HN		ArXiv		CommonCrawl
	410m	410m-CPT	410m	410m-CPT	410m	410m-CPT	410m	410m-CPT
Pythia-70m	0.080	0.858	1.0e-4	0.002	2.0e-4	0.030	0.043	0.030
Pythia-160m	1.0e-4	0.461	2.0e-4	0.017	1.0e-4	0.012	1.0e-4	4.0e-4
Pythia-1b	1.0e-4	1.000	1.0e-4	0.354	1.0e-4	0.113	1.0e-4	0.302

Table 5: p-values for different reference models used for non-membership detection on Pythia-410m and Pythia-410m-CPT

Figure 3: p-value for non-member detection when different number of samples are used

Panel of Figure 4 — Figure 4: a) Spearman rank correlation difference of Pythia-410m and Pythia-410m-CPT against Pythia-1b reference model at different values of $K$ in Min-K%++ b) Change in rank of a document when Min-K%++ is computed using Pythia-410m-CPT relative to the rank of the Pythia-410m model.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Num. Documents	Tokens
PubMed	500	154,387
HN	500	160,637
ArXiv	500	162,267
CommonCrawl	500	129,029
Reddit	650	125,789
Ubuntu	688	294,341

Appendix

Table 6: Dataset statistics

	PubMed			HN			ArXiv			CommonCrawl			Wiki
Score	410m	CPT	$\Delta$	410m	CPT	$\Delta$	410m	CPT	$\Delta$	410m	CPT	$\Delta$	410m	CPT	$\Delta$
Loss	0.99	0.92	0.07	0.98	0.83	0.15	0.99	0.84	0.15	0.98	0.72	0.26	0.98	0.55	0.43
zlib	0.99	0.93	0.06	0.99	0.93	0.06	0.99	0.89	0.10	0.98	0.82	0.16	0.98	0.57	0.41
DC-PDD	0.98	0.92	0.06	0.95	0.62	0.33	0.97	0.85	0.12	0.90	0.50	0.40	0.98	0.47	0.51
Min-K%	0.96	0.79	0.17	0.90	0.64	0.26	0.93	0.59	0.34	0.94	0.75	0.19	0.94	0.42	0.52
Min-K%++	0.84	0.33	0.51	0.75	0.41	0.34	0.82	0.45	0.37	0.79	0.51	0.28	0.84	0.25	0.59

Appendix

Table 7: Spearman rank correlation of Pythia-410m and Pythia-410m-CPT models against Pythia-1b reference model computed using different MIA scores

Panel of Figure 5 — Figure 5: Hyperparameter study to determine the value of a) $\tau$ and b) $\lambda$ . Pythia-410m and Pythia-410m-CPT are the target models while Pythia-1b is the reference model.

Target Model	PubMed			HN			ArXiv			CommonCrawl
	$\rho_{RT}$	$\rho_{DT}$	$\Delta_{95\%CI}$	$\rho_{RT}$	$\rho_{DT}$	$\Delta_{95\%CI}$	$\rho_{RT}$	$\rho_{DT}$	$\Delta_{95\%CI}$	$\rho_{RT}$	$\rho_{DT}$	$\delta_{95\%CI}$
Pythia-410m-CPT	0.328	0.590	[-0.371, -0.151]	0.404	0.384	[-0.080, 0.121]	0.447	0.393	[-0.034, 0.142]	0.512	0.488	[-0.068, 0.116]
Pythia-410m	0.836	0.420	[0.322, 0.511]	0.752	0.467	[0.207, 0.366]	0.819	0.506	[0.232, 0.394]	0.791	0.446	[0.261, 0.434]
Pythia-1b	0.844	0.513	[0.249, 0.416]	0.722	0.512	[0.127, 0.291]	0.812	0.539	[0.200, 0.348]	0.840	0.482	[0.282, 0.439]
Pythia-2.8b	0.783	0.498	[0.203, 0.369]	0.661	0.307	[0.262, 0.446]	0.760	0.443	[0.239, 0.396]	0.807	0.361	[0.356, 0.536]
Pythia-6.9b	0.757	0.385	[0.283, 0.463]	0.651	0.321	[0.240, 0.420]	0.733	0.452	[0.204, 0.358]	0.769	0.302	[0.371, 0.563]

Appendix

Table 8: Spearman rank correlations using for each target model and dataset. This is the raw data corresponding to the first four datasets in Table 2 .

\rho_{RT}

is the Spearman rank correlation between the Min-K%++ scores from the reference and target model over the dataset,

\rho_{DT}

is the Spearman rank correlation between the Min-K%++ scores from the distilled reference and target model and

\Delta_{95\%CI}

is the 95 % confidence interval for

\rho_{RT}-\rho_{DT}

obtained by bootstrapping.

Target Model	Ubuntu			Freelaw			Enron			Reddit
	$\rho_{RT}$	$\rho_{DT}$	$\Delta_{95\%CI}$	$\rho_{RT}$	$\rho_{DT}$	$\Delta_{95\%CI}$	$\rho_{RT}$	$\rho_{DT}$	$\Delta_{95\%CI}$	$\rho_{RT}$	$\rho_{DT}$	$\delta_{95\%CI}$
Pythia-410m-CPT	0.312	0.259	[-0.054, 0.160]	0.411	0.490	[-0.151, -0.006]	0.432	0.355	[-0.055, 0.208]	0.449	0.488	[-0.123, 0.045]
Pythia-410m	0.784	0.266	[0.429, 0.605]	0.826	0.591	[0.184, 0.289]	0.703	0.440	[0.137, 0.392]	0.672	0.080	[0.503, 0.678]
Pythia-1b	0.767	0.395	[0.302, 0.441]	0.825	0.498	[0.277, 0.377]	0.646	0.287	[0.226, 0.496]	0.721	0.070	[0.560, 0.736]
Pythia-2.8b	0.718	0.308	[0.331, 0.489]	0.753	0.398	[0.298, 0.413]	0.530	0.188	[0.202, 0.479]	0.763	0.052	[0.626, 0.793]
Pythia-6.9b	0.675	0.298	[0.295, 0.459]	0.708	0.359	[0.289, 0.410]	0.443	0.178	[0.126, 0.405]	0.774	0.038	[0.650, 0.817]

Appendix

Table 9: Spearman rank correlations using for each target model and dataset. This is the raw data corresponding to the last four datasets in Table 2 .

\rho_{RT}

is the Spearman rank correlation between the Min-K%++ scores from the reference and target model over the dataset,

\rho_{DT}

is the Spearman rank correlation between the Min-K%++ scores from the distilled reference and target model and

\Delta_{95\%CI}

is the 95 % confidence interval for

\rho_{RT}-\rho_{DT}

obtained by bootstrapping.

Target Model	PubMed	HN	ArXiv	CommonCrawl	Ubuntu	Freelaw	Enron	Reddit
Pythia-410m-CPT	2.6e-5	1.7e-24	3.7e-45	8.0e-8	1.2e-16	9.9e-34	3.6e-8	2.5e-8
Pythia-410m	0.18	9.6e-22	9.5e-11	2.8e-21	0.54	0.03	5.7e-17	2.5e-6
Pythia-1b	1.9e-11	2.2e-3	2.6e-47	7.0e-10	1.2e-18	3.7e-35	0.84	3.3e-6
Pythia-2.8b	7.8e-15	0.63	2.3e-8	0.06	0.04	1.6e-31	0.36	2.4e-6
Pythia-6.9b	4.8e-5	1.0	2.8e-38	0.01	2.7e-4	0.03	0.92	2.0e-6

Appendix

Table 10: Non-Member detection baseline results using Min-K%++ values from target and distilled reference model.

	Pythia-70m	Pythia-160m	Pythia-410m	Pythia-1b
$p$ -value	1.0e-4	1.0e-4	1.0e-4	1.0e-4

Appendix

Table 11: p-values for different Pythia reference models with OLMo-1b as the target model for the Reddit dataset.

Correlation Method	PubMed	HN	ArXiv	CommonCrawl	Reddit
Spearman $\rho$	1.0e-4	1.0e-4	1.0e-4	1.0e-4	1.0e-4
Kendall $\tau$	1.0e-4	1.0e-4	1.0e-4	1.0e-4	1.0e-4

Appendix

Table 12: Non-Member detection p-values using PRISM where the rank correlation is computed using Spearman

\rho

and Kendall

\tau

Panel of Figure 6 — Figure 6: Spearman rank correlation difference of Pythia-410m and Pythia-410m-CPT against Pythia-1b reference model at different values of $K$ in Min-K%++ for different datasets a) CommonCrawl b) Reddit c) HackerNews d) PubMed

Panel of Figure 7 — Figure 7: Change in rank of a document when Min-K%++ is computed using Pythia-410m-CPT relative to the rank of the Pythia-410m model for different datasets a) CommonCrawl b) Reddit c) HackerNews d) PubMed

LLM Auditing

Large language model auditing tests models and deployed systems for bias, safety, privacy, reliability, and provenance using behavioral and internal evidence.

See all 639 papers

Detecting Data Contamination in Large Language Models
Apr 21, 2026Juliusz Janicki, Savvas Chamezopoulos, Evangelos Kanoulas +1Membership Inference Attacks LLM Auditing
University of Amsterdam · Elsevier
OLMo-Detect: A Multi-Stage, Confounder-Controlled Benchmark for Membership Inference on Large Language Models
Oct 2, 2026Tao Shi, Chaoyi Xiang, Qiongkai Xu +1Membership Inference Attacks Privacy Leakage in Language Models
The University of Melbourne · Macquarie University
Have You Ever Seen Them? Entity-level Membership Inference through Interrogating Large Language Models
Jun 22, 2026Yiran Zhu, Ziqi YangMembership Inference Attacks LLM Auditing
Zhejiang University

Apr 21, 2026J/K move · Enter open · S save

Detecting Data Contamination in Large Language Models

Juliusz Janicki, Savvas Chamezopoulos, Evangelos Kanoulas, Georgios Tsatsaronis

University of Amsterdam · Elsevier

Membership Inference Attacks LLM Auditing Privacy Leakage in Language Models

Open paper AskPDF arXiv

Large Language Models (LLMs) utilize large amounts of data for their training, some of which may come from copyrighted sources. Membership Inference Attacks (MIA) aim to detect those documents and whether they have been included in the training corpora of the LLMs. The black-box MIAs require a significant amount of data manipulation; therefore, their comparison is often challenging. We study state-of-the-art (SOTA) MIAs under the black-box assumptions and compare them to each other using a unified set of datasets to determine if any of them can reliably detect membership under SOTA LLMs. In addition, a new method, called the Familiarity Ranking, was developed to showcase a possible approach to black-box MIAs, thereby giving LLMs more freedom in their expression to understand their reasoning better. The results indicate that none of the methods are capable of reliably detecting membership in LLMs, as shown by an AUC-ROC of approximately 0.5 for all methods across several LLMs. The higher TPR and FPR for more advanced LLMs indicate higher reasoning and generalizing capabilities, showcasing the difficulty of detecting membership in LLMs using black-box MIAs.

Abstract

LLM Auditing

Explore similar work

Detecting Data Contamination in Large Language Models

OLMo-Detect: A Multi-Stage, Confounder-Controlled Benchmark for Membership Inference on Large Language Models

Have You Ever Seen Them? Entity-level Membership Inference through Interrogating Large Language Models