Combating Data Laundering in LLM Training

Muxing Li; Zesheng Ye; Sharon Li; Feng Liu

cs.CRApr 2, 2026

Combating Data Laundering in LLM Training

Authors: Muxing Li, Zesheng Ye, Sharon Li, Feng Liu

Organizations: University of Melbourne · University of Wisconsin-Madison

Abstract

Post-hoc unauthorized-training data detection for large language models (LLMs) typically assumes a query-with-originals regime: rights holders query a target LLM with raw proprietary data and assess whether the model assigns them stronger memorization-based detection signals, e.g., higher confidence or lower loss, than held-out non-training reference texts. We show that this regime becomes brittle under data laundering, where the target LLM is trained on semantics-preserving but stylistically or structurally transformed surrogates of proprietary data to obfuscate provenance. Since training-time exposure occurs in the laundered form, memorization signals may no longer appear on the originals, collapsing the candidate-reference signal separation that standard detectors rely on. We counter this threat by studying laundering-aware detection with raw proprietary data, a held-out reference corpus, and query access to the target LLM, while the laundering transformation is undisclosed. Since exact recovery of the laundered corpus is infeasible, we infer a detection-useful synthesis process via an auxiliary LLM that maps originals into training-like queries. To make this search tractable, we introduce Synthesis Data Reversion (SDR), which constrains the unbounded space of natural-language transformations through a goal-details abstraction: a high-level transformation goal, e.g., "lyrical rewriting", and fine-grained details, e.g., "with vivid imagery". SDR identifies the most likely goal and iteratively refines details so synthesized queries elicit stronger target-model detection signals. Evaluated on the MIMIR benchmark against diverse laundering practices and target LLM families (Pythia, Llama2, and Falcon), SDR consistently restores detection signals, offering a practical auditing layer against data laundering.

Figures & tables

Figure 1: Illustration of how data laundering undermines existing unauthorized training data detectors. When unauthorized data is directly used for training, LLMs tend to memorize the unauthorized training data. Original training samples exhibit lower loss than original non-training samples , as shown in Part A. The log-likelihood distributions of training and non-training samples diverge clearly, enabling identification. However, when trained on laundered synthetic data, as shown in Part B, the distributions of original training samples and original non-training samples no longer diverge, preventing reliable identification.

Methods	AUC		ASR		TPR@5%
Methods	Orig.	Syn.	Orig.	Syn.	Orig.	Syn.
Loss ( Yeom et al., 2018 )	1.000	0.539	1.000	0.565	1.000	0.040
Ref ( Carlini et al., 2022 )	0.971	0.603	0.920	0.610	0.850	0.100
Zlib ( Carlini et al., 2021 )	1.000	0.521	1.000	0.535	1.000	0.080
Min-K ( Shi et al., 2023 )	1.000	0.563	1.000	0.575	1.000	0.040
Recall ( Xie et al., 2024 )	0.999	0.558	0.995	0.565	1.000	0.000

Table 1: Performance of unauthorized training data detection on Llama-2 ( Touvron et al., 2023 ) models fine-tuned with either the original MIMIR-Wikipedia ( Deng et al., 2023 ) dataset (Orig.) or its laundered version (Syn.) generated by GPT-4o ( Hurst et al., 2024 ) using the prompt “rewrite in a lyrical style, ensuring the imagery is vivid". Evaluation metrics are defined in Section 5 .

Method	Inside registers			Outside registers
Method	AUC	ASR	TPR@5%	AUC	ASR	TPR@5%
Recall	64.7%	63.4%	8.9%	61.7%	61.4%	5.6%
Recall+ SDR	76.2%	72.0%	25.3%	73.4%	73.3%	23.2%
Loss	63.7%	62.8%	10.7%	62.7%	62.6%	9.2%
Loss+ SDR	76.6%	72.6%	26.2%	75.5%	75.5%	22.9%
Ref	68.6%	67.0%	15.0%	67.6%	65.6%	13.2%

Table 2: The average performance of each unauthorized training data detection method across data synthesized from different inside and outside laundering processes . The experiment is conducted on Pythia-6.9B ( Biderman et al., 2023 ) , fine-tuned on Wikipedia synthesis. The results for each prompt are provided in Appendix H . We report the additional TPR@1% results in Appendix O .

Method	Wikipedia			Hackernews			C4
Method	AUC	ASR	TPR@5%	AUC	ASR	TPR@5%	AUC	ASR	TPR@5%
Recall	61.7%	61.4%	5.6%	53.9%	56.2%	9.9%	52.2%	55.0%	5.0%
Recall+ SDR	73.4%	73.3%	23.2%	61.6%	60.3%	10.7%	65.2%	63.0%	12.0%
Loss	62.7%	62.6%	9.2%	54.2%	56.0%	6.3%	58.4%	59.1%	7.2%
Loss+ SDR	75.5%	75.5%	22.9%	62.7%	59.6%	8.7%	67.3%	64.2%	13.0%
Min-K	64.2%	62.5%	10.5%	53.8%	55.9%	5.7%	57.6%	59.2%	6.2%

Table 3: Comparison of average performance of unauthorized training data detection across different datasets trained with synthesis using outside register prompts.

Method	Pythia-6.9B			Falcon-7B			LLama-2-7B
Method	AUC	ASR	TPR@5%	AUC	ASR	TPR@5%	AUC	ASR	TPR@5%
Recall	61.7%	61.4%	5.6%	62.1%	61.8%	8.2%	61.6%	61.6%	8.2%
Recall+ SDR	73.4%	73.3%	23.2%	72.4%	69.1%	23.0%	67.5%	66.2%	12.5%
Loss	62.7%	62.6%	9.2%	64.4%	64.4%	11.5%	64.5%	64.4%	11.1%
Loss+ SDR	75.5%	75.5%	22.9%	71.2%	68.0%	20.2%	73.6%	70.4%	26.9%
Min-K	64.2%	62.5%	10.5%	63.3%	62.6%	10.5%	62.5%	62.5%	13.9%

Table 4: Comparison of average performance of unauthorized training data detection across three different model architectures fine-tuned with synthesis generated by outside register prompts.

Figure 2: Ablation study on the effectiveness of each stage in SDR . Results are reported as the average performance of unauthorized training data detectors across different inside prompts. Removing the goal identification stage ( w/o stage 1 ) or the detailed prompt condition inference stage ( w/o stage 2 ) leads to noticeable degradation, while the full SDR consistently achieves the best performance.

Method	GPT-4o			Claude			DeepSeek
Method	AUC	ASR	TPR@5%	AUC	ASR	TPR@5%	AUC	ASR	TPR@5%
Recall	64.5%	63.5%	9.9%	66.8%	65.9%	9.7%	65.5%	64.8%	7.4%
Recall+ SDR	79.7%	75.0%	31.6%	81.6%	77.7%	31.0%	79.8%	75.5%	31.2%
Loss	63.4%	62.2%	14.3%	67.3%	65.3%	12.1%	65.9%	64.7%	12.2%
Loss+ SDR	80.3%	75.6%	32.6%	83.4%	77.8%	41.9%	79.0%	74.7%	32.8%
Min-K	63.7%	62.3%	11.0%	68.1%	65.5%	14.5%	67.0%	64.6%	14.2%

Table 5: Comparison of the average performance of unauthorized training data detection with SDR using different auxiliary LLMs .

Appendix figures & tables20 assets

Supplementary material from the paper’s appendix.

Appendix

Symbol	Description
$D_{\mathrm{pro}}$	Proprietary dataset (candidate corpus).
$D_{\mathrm{train}}$	Model provider’s full training dataset.
$D_{\mathrm{held}}$	Held-out non-training reference dataset.
$M_{t}$	Target LLM trained on $D_{\mathrm{train}}$ .
$M_{a}$	Auxiliary LLM used only for synthesis.
$p$	Current reverse-synthesis prompt refined during iterations.

Appendix

Table 8

Name	Abbr.	Name	Abbr.
Lyrical	ly	Encyclopedia article	en
Spoken	sp	Research article	ra
Interview	it	Description of a thing or person	dtp
Interactive discussion	id	FAQ	fi
Narrative	na	Legal terms & conditions	lt
News report	ne	Opinion	op

Appendix

Table 6: Sub-registers and abbreviations.

Register	Opening Template
Lyrical	In the heart of [abstract domain], a tale unfolds, where [abstract concept], [abstract detail], [abstract entity], [abstract action].
Spoken style	So, let’s talk about [TOPIC].
Interview	Interviewer: Thank you for joining us, [Person/Expert Title]. Can you tell us about [Subject/Topic]?
Interactive discussion	[Participant 1]: So, have you guys heard about [Topic/Subject]? I recently came across some interesting information about it.
Storytelling narrative	Once upon a time, in a [adjective] [type of place] called [place name], there lived a [adjective] [type of character] named [character name].
News report	[Event/Topic]: [Description/Significance] [Location/Context] – [Details about the subject, including noteworthy contributions, roles, or milestones].

Appendix

Table 8: Registers and their corresponding opening templates.

Lora_alpha	r	epochs	lr	gradient accumulation steps	weight_decay	logging_strategy
32	8	30	0.0004	25	0.1	“epoch”

Appendix

Table 9: Hyperparameter settings for LoRA fine-tuning.

ID	Laundering Prompts
1	Rewrite the text in a lyrical style, ensuring the imagery is vivid, the rhythm flows naturally.
2	Rewrite the text in a spoken style, making it sound natural and conversational, and ensure the tone feels engaging and easy to follow for a live audience.
3	Rewrite the text in the form of an interview, ensuring the questions flow naturally and the answers provide clear, engaging explanations for the audience.
4	Rewrite the text as an interactive discussion between two or more participants, ensuring the conversation flows logically, with each speaker’s tone and style clearly distinguishable.
5	Rewrite the text as a storytelling narrative. The story should flow naturally, use simple and engaging language, and be easy for all kinds of listeners to follow.
6	Rewrite the text in the style of a news report, ensuring the information is presented objectively and concisely.

Appendix

Table 10: Inside-register prompts.

ID	Laundering Prompts
1	Rewrite the following content as slide presentation bullet points. Focus on summarizing the key arguments and findings clearly and concisely. Use concise phrases that highlight core points.
2	Rewrite the following text in the style of a Facebook post. Sharing interesting information with followers. You may add light commentary, questions to the audience, or casual phrasing, but keep it natural and human-like. Avoid using emojis, hashtags, or overly dramatic expressions.
3	Adapt the text into a poetic form with vivid metaphors, rhythmic structure, and emotionally evocative language.
4	Convert the content into a tutorial-style explanation for beginners, using step-by-step instructions, simple analogies, and common misunderstandings.
5	Rewrite the text as a formal business email, ensuring clarity, professionalism, and a polite tone.
6	Rewrite the passage as a scientific abstract, including Background, Methods, Results, and Conclusions. Invent at least two numerical values (percentages, sample sizes, or statistical outcomes) to support claims.

Appendix

Table 11: Outside-register prompts.

ID	Original Prompt	Reversed Prompt ( SDR )	Orig (AUC)	SDR (AUC)
1	Rewrite the text in a lyrical style, ensuring the imagery is vivid, the rhythm flows naturally.	Rewrite the text in a lyrical style, enhancing the poetic rhythm and imagery while capturing the essence and emotional depth of the original content.	0.540	0.692
2	Rewrite the text in a spoken style, making it sound natural and conversational, and ensure the tone feels engaging and easy to follow for a live audience.	Rewrite the text to sound natural and conversational, using everyday language and personal anecdotes to create an engaging and friendly atmosphere for the listener.	0.702	0.894
3	Rewrite the text in the form of an interview, ensuring the questions flow naturally and the answers provide clear, engaging explanations for the audience.	Rewrite the text in the form of an interview, ensuring a clear and engaging dialogue that accurately conveys the information while maintaining a conversational tone and eliciting detailed responses.	0.713	0.823
4	Rewrite the text as an interactive discussion between two or more participants, ensuring the conversation flows logically, with each speaker’s tone and style clearly distinguishable.	Rewrite the text as an interactive discussion between two or more participants, ensuring a natural flow of dialogue that incorporates factual information and engages with the topic through building on each other’s comments.	0.650	0.770
5	Rewrite the text as a storytelling narrative. The story should flow naturally, use simple and engaging language, and be easy for all kinds of listeners to follow.	Rewrite the text in the form of a Frequently Asked Questions section, transforming the information into clear and concise questions and answers that emphasize key details and engage the reader effectively.	0.646	0.735
6	Rewrite the text in the style of a news report, ensuring the information is presented objectively and concisely.	Rewrite the text into a Frequently Asked Questions section, organizing the information into clear and concise questions and answers while highlighting key details and maintaining clarity and readability.	0.793	0.904

Appendix

Table 12: Inside-register prompts and corresponding reversed prompts with Orig and SDR results.

ID	Original Prompt	Reversed Prompt ( SDR )	Orig (AUC)	SDR (AUC)
13	Rewrite the text in the style of an editorial, making sure to include a clear stance or opinion and a concluding paragraph that calls for action or reflection.	Rewrite the text as a personal opinion piece, emphasizing reflective commentary and personal insights while exploring the broader societal implications and significance of the subject matter.	0.594	0.657
14	Rewrite the text as an informational description, ensuring the tone is neutral and objective, and include at least one definition or clarification to help the reader better understand the subject.	Rewrite the text as a step-by-step instructional guide, organizing the content into clear, numbered sections that effectively communicate essential information about the subject.	0.594	0.795
15	Rewrite the text in the style of an encyclopedia entry, maintaining a neutral, authoritative tone, and include at least one date, fact, or reference to give it the appearance of being sourced.	Rewrite the text as an informational description, focusing on presenting a clear, structured overview of the subject’s key facts, achievements, and background while maintaining concise and objective language.	0.792	0.923
16	Rewrite the text as an academic research article, structured with sections such as Abstract, Introduction, Method, Results, and Conclusion, and include at least one in-text citation (invented if necessary) to simulate scholarly referencing.	Rewrite the text in the form of an interview, transforming the original content into a conversational dialogue that incorporates engaging questions and responses while maintaining clarity and coherence.	0.525	0.546
17	Rewrite the text as a descriptive profile of a specific thing or person, using vivid details and attributes (appearance, characteristics, or context) and ending with a short summary sentence that highlights its significance.	Rewrite the text as a sales description, transforming it into an engaging narrative that highlights the subject’s achievements, legacy, and emotional impact to captivate and appeal to potential audiences.	0.661	0.765
18	Rewrite the text in the form of a Frequently Asked Questions (FAQ) section, making sure to include at least three question–answer pairs, with the questions phrased from the perspective of a curious reader.	Rewrite the text in the form of an interview, transforming factual information into a conversational question-and-answer format that captures personal insights, key themes, and details from the original content.	0.640	0.792

Appendix

Table 13: Inside-register prompts and corresponding reversed prompts with Orig and SDR results (continues).

ID	Original Prompt	Reversed Prompt ( SDR )	Orig	SDR
1	Rewrite the following content as slide presentation bullet points. Focus on summarizing the key arguments and findings clearly and concisely. Use concise phrases that highlight core points.	Rewrite the text as a step-by-step instructional guide, organizing the information into clear sections and ensuring each step provides concise, relevant details on the specified topic.	0.730	0.799
2	Rewrite the following text in the style of a Facebook post. Sharing interesting information with followers. You may add light commentary, questions to the audience, or casual phrasing, but keep it natural and human-like. Avoid using emojis, hashtags, or overly dramatic expressions.	Rewrite the text as a Frequently Asked Questions section, transforming the information into an engaging question-and-answer format that encourages reader interaction and maintains a conversational tone.	0.644	0.730
3	Adapt the text into a poetic form with vivid metaphors, rhythmic structure, and emotionally evocative language.	Rewrite the text in a lyrical style that transforms factual content into an evocative narrative, using vivid imagery, poetic devices, and rhythmic flow to highlight emotional resonance and thematic cohesion.	0.538	0.674
4	Convert the content into a tutorial-style explanation for beginners, using step-by-step instructions, simple analogies, and common misunderstandings.	Rewrite the text as a step-by-step instructional guide, ensuring clear and concise steps that effectively outline key aspects and concepts while maintaining an engaging tone and logical flow throughout.	0.616	0.679
5	Rewrite the text as a formal business email, ensuring clarity, professionalism, and a polite tone.	Rewrite the text in the style of an encyclopedia entry, emphasizing clear and concise organization, formal language, and distinct sections that present factual information and key points effectively.	0.795	0.884
6	Rewrite the passage as a scientific abstract, including Background, Methods, Results, and Conclusions. Invent at least two numerical values (percentages, sample sizes, or statistical outcomes) to support claims.	Rewrite the text as a Frequently Asked Questions section, transforming the original content into clear, concise questions and answers that emphasize key themes, significant information, and factual accuracy.	0.600	0.678

Appendix

Table 14: Outside-register prompts and corresponding reversed prompts with Orig and SDR results.

Method	ArXiv		Wikipedia
Method	AUC	ASR	AUC	ASR
Recall	0.516	0.532	0.528	0.545
Recall + SDR	0.488	0.525	0.473	0.540
Loss	0.486	0.522	0.471	0.510
Loss + SDR	0.490	0.535	0.465	0.515
Min-K	0.536	0.510	0.486	0.515

Appendix

Table 15: Negative-control experiment on Pythia-6.9B . When the target model is not trained on any laundered variants of the proprietary data, SDR does not find a prompt to improve unauthorized training-data detection. Across all detectors and both datasets (ArXiv and Wikipedia), the AUC and ASR scores before and after SDR remain close to 0.5, indicating that SDR does not fabricate evidence of misuse in the absence of laundered training data.

Method	AUC	ASR
Recall	0.641	0.630
Recall + SDR	0.767	0.718
Loss	0.650	0.635
Loss + SDR	0.808	0.760
Min-K	0.672	0.645
Min-K + SDR	0.784	0.734

Appendix

Table 16: SDR transferability on DeepSeek-v3 laundering pipelines for the target LLM trained exclusively on DeepSeek-laundered data. SDR uses GPT-4o as

M_{a}

Method	AUC	ASR
Recall	0.649	0.635
Recall + SDR	0.566	0.575
Loss	0.687	0.680
Loss + SDR	0.751	0.725
Min-K	0.652	0.680
Min-K + SDR	0.713	0.690

Appendix

Table 17: Detection performance on the Polite (human-rewritten) dataset. SDR improves most detectors even when laundering is performed by humans, demonstrating SDR ’s robustness and transferability beyond LLM-based pipelines.

Method	Mixed Prompt 1		Mixed Prompt 2		Mixed Prompt 3
Method	AUC	ASR	AUC	ASR	AUC	ASR
Recall	0.559	0.560	0.666	0.650	0.737	0.715
Recall + SDR	0.757	0.705	0.777	0.725	0.831	0.770
Loss	0.572	0.575	0.670	0.655	0.738	0.685
Loss + SDR	0.767	0.705	0.796	0.750	0.834	0.785
Min-K	0.587	0.580	0.677	0.670	0.747	0.705

Appendix

Table 18: SDR transferability on mixed-register laundering prompts. SDR consistently improves AUC and ASR across three mixed-register laundering transformations, demonstrating robustness to more complex combinations of stylistic rewrites.

Method	Avg AUC	Avg ASR
Recall	0.608	0.590
Recall + SDR	0.722	0.694
Loss	0.599	0.597
Loss + SDR	0.732	0.703
Min-K	0.608	0.603
Min-K + SDR	0.724	0.701

Appendix

Table 19: Average performance across all five laundering prompts under partial laundering.

$K$	AUC	ASR	Time (hh:mm:ss)	Query Budget (GPT4o / mini)
3	0.712	0.685	02:23:42	$\sim\$ 7/0.8$
5	0.727	0.690	04:11:33	$\sim\$ 13/1.5$
15	0.747	0.705	13:27:12	$\sim\$ 39/4.5$

Appendix

Table 20: Sensitivity to

K

$l$	AUC	ASR	Time (hh:mm:ss)	Query Budget (GPT4o / mini)
3	0.715	0.670	08:21:33	$\sim\$ 10/1$
7	0.731	0.695	09:13:12	$\sim\$ 10/1$
9	0.718	0.695	09:35:17	$\sim\$ 10/1$

Appendix

Table 21: Sensitivity to

l

$m$	AUC	ASR	Time (hh:mm:ss)	Query Budget (GPT4o / mini)
3	0.718	0.685	08:14:33	$\sim\$ 10/1$
7	0.796	0.740	08:19:29	$\sim\$ 10/1$
9	0.810	0.760	08:26:45	$\sim\$ 10/1$
11	0.806	0.755	08:33:34	$\sim\$ 10/1$

Appendix

Table 22: Sensitivity to

m

$n$	AUC	ASR	Time (hh:mm:ss)	Query Budget
3	0.705	0.675	–	–
7	0.752	0.705	–	–
9	0.742	0.695	–	–

Appendix

Table 23: Sensitivity to

n

n

controls the construction of the template and is computed only once during initialization. Therefore, it does not contribute to the runtime or query budget.

Method	AUC	ASR
Loss	0.638	0.635
Loss + Reverse Prompt Engineering	0.682	0.663
Loss + SDR	0.782	0.764

Appendix

Table 24: Comparison between SDR and reverse prompt engineering.

Method	Inside TPR@1%	Outside TPR@1%
Recall	0.002	0.008
Recall + SDR	0.106	0.062
Loss	0.047	0.000
Loss + SDR	0.112	0.069
Min-K	0.001	0.016
Min-K + SDR	0.096	0.082

Appendix

Table 25: TPR@1% results for the main experiment.

LLM Auditing

Large language model auditing tests models and deployed systems for bias, safety, privacy, reliability, and provenance using behavioral and internal evidence.

See all 621 papers

May 28, 2026

LLMSurgeon: Diagnosing Data Mixture of Large Language Models

The pretraining data mixture of Large Language Models (LLMs) constitutes their "digital DNA", shaping model behaviors, capabilities, and failure modes. Yet this composition is rarely disclosed, making post-hoc auditing of data combination or provenance difficult. In this work, we formalize Data Mixture Surgery (DMS): given only generated text from a target LLM, estimate the domain-level distribution of its pretraining corpus under a predefined taxonomy. We propose

\textbf{LLMSurgeon}

, a strong framework that casts DMS as an inverse problem under the label-shift assumption. Rather than directly aggregating classifier outputs, LLMSurgeon estimates a calibrated

\textit{soft}

confusion matrix and solves a constrained inverse problem to correct systematic domain confusion and recover the latent mixture prior. To evaluate, we introduce

\textbf{LLMScan}

, a recipe-verifiable evaluation suite built from open-source LLMs with transparent pretraining mixtures. Across LLMScan, LLMSurgeon recovers domain mixtures with high fidelity under fixed protocols. Our work presents a practical, post-hoc approach for auditing the digital DNA of foundation models without access to their training data.

Yaxin Luo, Jiacheng Cui, Xiaohan Zhao +5

VILA Lab, MBZUAI · UCL

LLM Auditing

Ask PDF

May 7, 2026

Dataset Watermarking for Closed LLMs with Provable Detection

Large language models (LLMs) are pre-trained and post-trained on vast amounts of loosely curated data, raising the possibility that these models may have been trained on proprietary datasets or the same benchmarks used for evaluation. This motivates the need for dataset watermarking: designing datasets such that training on them leaves detectable signatures in the resulting model. Prior work has explored this problem for open models. We introduce the first dataset watermarking method for closed LLMs with provable detection. In particular, we embed a dataset-level watermark signal by increasing the co-occurrence frequency of randomly selected word pairs through rephrasing, and detect it using a statistical test on co-occurrence patterns in model-generated outputs. We evaluate our method with multiple base models and benchmark datasets and show that it reliably detects the watermark (

p <0.01

) in the fine-tuning stage. Notably, our method remains effective in a data mixture setting where the watermarked dataset constitutes only approximately

1\%

of the total fine-tuning tokens. Furthermore, we show that our method preserves the utility and semantic integrity of the benchmark.

Pengrun Huang, Kamalika Chaudhuri, Yu-Xiang Wang

Preprint. Under review.

Dataset Watermarking LLM Auditing Benchmark Contamination LLM Watermarking

Ask PDF

Feb 22, 2026

Learning to Detect Language Model Training Data via Active Reconstruction

Detecting LLM training data is generally framed as a membership inference attack (MIA) problem. However, conventional MIAs operate passively on fixed model weights, using log-likelihoods or text generations. In this work, we introduce Active Data Reconstruction Attack (ADRA), a family of MIA that actively induces a model to reconstruct a given text through training. We hypothesize that training data are more reconstructible than non-members, and the difference in their reconstructibility can be exploited for membership inference. Motivated by findings that reinforcement learning (RL) sharpens behaviors already encoded in weights, we leverage on-policy RL to actively elicit data reconstruction by finetuning a policy initialized from the target model. To effectively use RL for MIA, we design reconstruction metrics and contrastive rewards. The resulting algorithms, ADRA and its adaptive variant ADRA+, improve both reconstruction and detection given a pool of candidate data. Experiments show that our methods consistently outperform existing MIAs in detecting pre-training, post-training, and distillation data, with an average improvement of 10.7% over the previous runner-up. In particular, \MethodPlus~improves over Min-K%++ by 18.8% on BookMIA for pre-training detection and by 7.6% on AIME for post-training detection.

Junjie Oscar Yin, John X. Morris, Vitaly Shmatikov +2

University of Washington · Cornell University · UC Berkeley +1

Membership Inference Attacks Privacy Leakage in Language Models Training Data Reconstruction Attacks

Ask PDF

Abstract

LLM Auditing

Explore similar work

LLMSurgeon: Diagnosing Data Mixture of Large Language Models

Dataset Watermarking for Closed LLMs with Provable Detection

Learning to Detect Language Model Training Data via Active Reconstruction