cs.CRApr 2, 2026
SaveCombating Data Laundering in LLM Training
Organizations: University of Melbourne · University of Wisconsin-Madison
Abstract
Post-hoc unauthorized-training data detection for large language models (LLMs) typically assumes a query-with-originals regime: rights holders query a target LLM with raw proprietary data and assess whether the model assigns them stronger memorization-based detection signals, e.g., higher confidence or lower loss, than held-out non-training reference texts. We show that this regime becomes brittle under data laundering, where the target LLM is trained on semantics-preserving but stylistically or structurally transformed surrogates of proprietary data to obfuscate provenance. Since training-time exposure occurs in the laundered form, memorization signals may no longer appear on the originals, collapsing the candidate-reference signal separation that standard detectors rely on. We counter this threat by studying laundering-aware detection with raw proprietary data, a held-out reference corpus, and query access to the target LLM, while the laundering transformation is undisclosed. Since exact recovery of the laundered corpus is infeasible, we infer a detection-useful synthesis process via an auxiliary LLM that maps originals into training-like queries. To make this search tractable, we introduce Synthesis Data Reversion (SDR), which constrains the unbounded space of natural-language transformations through a goal-details abstraction: a high-level transformation goal, e.g., "lyrical rewriting", and fine-grained details, e.g., "with vivid imagery". SDR identifies the most likely goal and iteratively refines details so synthesized queries elicit stronger target-model detection signals. Evaluated on the MIMIR benchmark against diverse laundering practices and target LLM families (Pythia, Llama2, and Falcon), SDR consistently restores detection signals, offering a practical auditing layer against data laundering.
Figures & tables
Figure 1: Illustration of how data laundering undermines existing unauthorized training data detectors. When unauthorized data is directly used for training, LLMs tend to memorize the unauthorized training data. Original training samples exhibit lower loss than original non-training samples , as shown in Part A. The log-likelihood distributions of training and non-training samples diverge clearly, enabling identification. However, when trained on laundered synthetic data, as shown in Part B, the distributions of original training samples and original non-training samples no longer diverge, preventing reliable identification.
| Methods | AUC | ASR | TPR@5% | |||
| Orig. | Syn. | Orig. | Syn. | Orig. | Syn. | |
| Loss ( Yeom et al., 2018 ) | 1.000 | 0.539 | 1.000 | 0.565 | 1.000 | 0.040 |
| Ref ( Carlini et al., 2022 ) | 0.971 | 0.603 | 0.920 | 0.610 | 0.850 | 0.100 |
| Zlib ( Carlini et al., 2021 ) | 1.000 | 0.521 | 1.000 | 0.535 | 1.000 | 0.080 |
| Min-K ( Shi et al., 2023 ) | 1.000 | 0.563 | 1.000 | 0.575 | 1.000 | 0.040 |
| Recall ( Xie et al., 2024 ) | 0.999 | 0.558 | 0.995 | 0.565 | 1.000 | 0.000 |
Table 1: Performance of unauthorized training data detection on Llama-2 ( Touvron et al., 2023 ) models fine-tuned with either the original MIMIR-Wikipedia ( Deng et al., 2023 ) dataset (Orig.) or its laundered version (Syn.) generated by GPT-4o ( Hurst et al., 2024 ) using the prompt “rewrite in a lyrical style, ensuring the imagery is vivid". Evaluation metrics are defined in Section 5 .
| Method | Inside registers | Outside registers | ||||
| AUC | ASR | TPR@5% | AUC | ASR | TPR@5% | |
| Recall | 64.7% | 63.4% | 8.9% | 61.7% | 61.4% | 5.6% |
| Recall+ SDR | 76.2% | 72.0% | 25.3% | 73.4% | 73.3% | 23.2% |
| Loss | 63.7% | 62.8% | 10.7% | 62.7% | 62.6% | 9.2% |
| Loss+ SDR | 76.6% | 72.6% | 26.2% | 75.5% | 75.5% | 22.9% |
| Ref | 68.6% | 67.0% | 15.0% | 67.6% | 65.6% | 13.2% |
Table 2: The average performance of each unauthorized training data detection method across data synthesized from different inside and outside laundering processes . The experiment is conducted on Pythia-6.9B ( Biderman et al., 2023 ) , fine-tuned on Wikipedia synthesis. The results for each prompt are provided in Appendix H . We report the additional TPR@1% results in Appendix O .
| Method | Wikipedia | Hackernews | C4 | ||||||
| AUC | ASR | TPR@5% | AUC | ASR | TPR@5% | AUC | ASR | TPR@5% | |
| Recall | 61.7% | 61.4% | 5.6% | 53.9% | 56.2% | 9.9% | 52.2% | 55.0% | 5.0% |
| Recall+ SDR | 73.4% | 73.3% | 23.2% | 61.6% | 60.3% | 10.7% | 65.2% | 63.0% | 12.0% |
| Loss | 62.7% | 62.6% | 9.2% | 54.2% | 56.0% | 6.3% | 58.4% | 59.1% | 7.2% |
| Loss+ SDR | 75.5% | 75.5% | 22.9% | 62.7% | 59.6% | 8.7% | 67.3% | 64.2% | 13.0% |
| Min-K | 64.2% | 62.5% | 10.5% | 53.8% | 55.9% | 5.7% | 57.6% | 59.2% | 6.2% |
Table 3: Comparison of average performance of unauthorized training data detection across different datasets trained with synthesis using outside register prompts.
| Method | Pythia-6.9B | Falcon-7B | LLama-2-7B | ||||||
| AUC | ASR | TPR@5% | AUC | ASR | TPR@5% | AUC | ASR | TPR@5% | |
| Recall | 61.7% | 61.4% | 5.6% | 62.1% | 61.8% | 8.2% | 61.6% | 61.6% | 8.2% |
| Recall+ SDR | 73.4% | 73.3% | 23.2% | 72.4% | 69.1% | 23.0% | 67.5% | 66.2% | 12.5% |
| Loss | 62.7% | 62.6% | 9.2% | 64.4% | 64.4% | 11.5% | 64.5% | 64.4% | 11.1% |
| Loss+ SDR | 75.5% | 75.5% | 22.9% | 71.2% | 68.0% | 20.2% | 73.6% | 70.4% | 26.9% |
| Min-K | 64.2% | 62.5% | 10.5% | 63.3% | 62.6% | 10.5% | 62.5% | 62.5% | 13.9% |
Table 4: Comparison of average performance of unauthorized training data detection across three different model architectures fine-tuned with synthesis generated by outside register prompts.
Figure 2: Ablation study on the effectiveness of each stage in SDR . Results are reported as the average performance of unauthorized training data detectors across different inside prompts. Removing the goal identification stage ( w/o stage 1 ) or the detailed prompt condition inference stage ( w/o stage 2 ) leads to noticeable degradation, while the full SDR consistently achieves the best performance.
| Method | GPT-4o | Claude | DeepSeek | ||||||
| AUC | ASR | TPR@5% | AUC | ASR | TPR@5% | AUC | ASR | TPR@5% | |
| Recall | 64.5% | 63.5% | 9.9% | 66.8% | 65.9% | 9.7% | 65.5% | 64.8% | 7.4% |
| Recall+ SDR | 79.7% | 75.0% | 31.6% | 81.6% | 77.7% | 31.0% | 79.8% | 75.5% | 31.2% |
| Loss | 63.4% | 62.2% | 14.3% | 67.3% | 65.3% | 12.1% | 65.9% | 64.7% | 12.2% |
| Loss+ SDR | 80.3% | 75.6% | 32.6% | 83.4% | 77.8% | 41.9% | 79.0% | 74.7% | 32.8% |
| Min-K | 63.7% | 62.3% | 11.0% | 68.1% | 65.5% | 14.5% | 67.0% | 64.6% | 14.2% |
Table 5: Comparison of the average performance of unauthorized training data detection with SDR using different auxiliary LLMs .
Appendix figures & tables20 assets
Supplementary material from the paper’s appendix.
Appendix
| Symbol | Description |
| Proprietary dataset (candidate corpus). | |
| Model provider’s full training dataset. | |
| Held-out non-training reference dataset. | |
| Target LLM trained on . | |
| Auxiliary LLM used only for synthesis. | |
| Current reverse-synthesis prompt refined during iterations. |
| Name | Abbr. | Name | Abbr. | |
| Lyrical | ly | Encyclopedia article | en | |
| Spoken | sp | Research article | ra | |
| Interview | it | Description of a thing or person | dtp | |
| Interactive discussion | id | FAQ | fi | |
| Narrative | na | Legal terms & conditions | lt | |
| News report | ne | Opinion | op |
Table 6: Sub-registers and abbreviations.
| Register | Opening Template |
| Lyrical | In the heart of [abstract domain], a tale unfolds, where [abstract concept], [abstract detail], [abstract entity], [abstract action]. |
| Spoken style | So, let’s talk about [TOPIC]. |
| Interview | Interviewer: Thank you for joining us, [Person/Expert Title]. Can you tell us about [Subject/Topic]? |
| Interactive discussion | [Participant 1]: So, have you guys heard about [Topic/Subject]? I recently came across some interesting information about it. |
| Storytelling narrative | Once upon a time, in a [adjective] [type of place] called [place name], there lived a [adjective] [type of character] named [character name]. |
| News report | [Event/Topic]: [Description/Significance] [Location/Context] – [Details about the subject, including noteworthy contributions, roles, or milestones]. |
Table 8: Registers and their corresponding opening templates.
| Lora_alpha | r | epochs | lr | gradient accumulation steps | weight_decay | logging_strategy |
| 32 | 8 | 30 | 0.0004 | 25 | 0.1 | “epoch” |
Table 9: Hyperparameter settings for LoRA fine-tuning.
| ID | Laundering Prompts |
| 1 | Rewrite the text in a lyrical style, ensuring the imagery is vivid, the rhythm flows naturally. |
| 2 | Rewrite the text in a spoken style, making it sound natural and conversational, and ensure the tone feels engaging and easy to follow for a live audience. |
| 3 | Rewrite the text in the form of an interview, ensuring the questions flow naturally and the answers provide clear, engaging explanations for the audience. |
| 4 | Rewrite the text as an interactive discussion between two or more participants, ensuring the conversation flows logically, with each speaker’s tone and style clearly distinguishable. |
| 5 | Rewrite the text as a storytelling narrative. The story should flow naturally, use simple and engaging language, and be easy for all kinds of listeners to follow. |
| 6 | Rewrite the text in the style of a news report, ensuring the information is presented objectively and concisely. |
Table 10: Inside-register prompts.
| ID | Laundering Prompts |
| 1 | Rewrite the following content as slide presentation bullet points. Focus on summarizing the key arguments and findings clearly and concisely. Use concise phrases that highlight core points. |
| 2 | Rewrite the following text in the style of a Facebook post. Sharing interesting information with followers. You may add light commentary, questions to the audience, or casual phrasing, but keep it natural and human-like. Avoid using emojis, hashtags, or overly dramatic expressions. |
| 3 | Adapt the text into a poetic form with vivid metaphors, rhythmic structure, and emotionally evocative language. |
| 4 | Convert the content into a tutorial-style explanation for beginners, using step-by-step instructions, simple analogies, and common misunderstandings. |
| 5 | Rewrite the text as a formal business email, ensuring clarity, professionalism, and a polite tone. |
| 6 | Rewrite the passage as a scientific abstract, including Background, Methods, Results, and Conclusions. Invent at least two numerical values (percentages, sample sizes, or statistical outcomes) to support claims. |
Table 11: Outside-register prompts.
| ID | Original Prompt | Reversed Prompt ( SDR ) | Orig (AUC) | SDR (AUC) |
| 1 | Rewrite the text in a lyrical style, ensuring the imagery is vivid, the rhythm flows naturally. | Rewrite the text in a lyrical style, enhancing the poetic rhythm and imagery while capturing the essence and emotional depth of the original content. | 0.540 | 0.692 |
| 2 | Rewrite the text in a spoken style, making it sound natural and conversational, and ensure the tone feels engaging and easy to follow for a live audience. | Rewrite the text to sound natural and conversational, using everyday language and personal anecdotes to create an engaging and friendly atmosphere for the listener. | 0.702 | 0.894 |
| 3 | Rewrite the text in the form of an interview, ensuring the questions flow naturally and the answers provide clear, engaging explanations for the audience. | Rewrite the text in the form of an interview, ensuring a clear and engaging dialogue that accurately conveys the information while maintaining a conversational tone and eliciting detailed responses. | 0.713 | 0.823 |
| 4 | Rewrite the text as an interactive discussion between two or more participants, ensuring the conversation flows logically, with each speaker’s tone and style clearly distinguishable. | Rewrite the text as an interactive discussion between two or more participants, ensuring a natural flow of dialogue that incorporates factual information and engages with the topic through building on each other’s comments. | 0.650 | 0.770 |
| 5 | Rewrite the text as a storytelling narrative. The story should flow naturally, use simple and engaging language, and be easy for all kinds of listeners to follow. | Rewrite the text in the form of a Frequently Asked Questions section, transforming the information into clear and concise questions and answers that emphasize key details and engage the reader effectively. | 0.646 | 0.735 |
| 6 | Rewrite the text in the style of a news report, ensuring the information is presented objectively and concisely. | Rewrite the text into a Frequently Asked Questions section, organizing the information into clear and concise questions and answers while highlighting key details and maintaining clarity and readability. | 0.793 | 0.904 |
Table 12: Inside-register prompts and corresponding reversed prompts with Orig and SDR results.
| ID | Original Prompt | Reversed Prompt ( SDR ) | Orig (AUC) | SDR (AUC) |
| 13 | Rewrite the text in the style of an editorial, making sure to include a clear stance or opinion and a concluding paragraph that calls for action or reflection. | Rewrite the text as a personal opinion piece, emphasizing reflective commentary and personal insights while exploring the broader societal implications and significance of the subject matter. | 0.594 | 0.657 |
| 14 | Rewrite the text as an informational description, ensuring the tone is neutral and objective, and include at least one definition or clarification to help the reader better understand the subject. | Rewrite the text as a step-by-step instructional guide, organizing the content into clear, numbered sections that effectively communicate essential information about the subject. | 0.594 | 0.795 |
| 15 | Rewrite the text in the style of an encyclopedia entry, maintaining a neutral, authoritative tone, and include at least one date, fact, or reference to give it the appearance of being sourced. | Rewrite the text as an informational description, focusing on presenting a clear, structured overview of the subject’s key facts, achievements, and background while maintaining concise and objective language. | 0.792 | 0.923 |
| 16 | Rewrite the text as an academic research article, structured with sections such as Abstract, Introduction, Method, Results, and Conclusion, and include at least one in-text citation (invented if necessary) to simulate scholarly referencing. | Rewrite the text in the form of an interview, transforming the original content into a conversational dialogue that incorporates engaging questions and responses while maintaining clarity and coherence. | 0.525 | 0.546 |
| 17 | Rewrite the text as a descriptive profile of a specific thing or person, using vivid details and attributes (appearance, characteristics, or context) and ending with a short summary sentence that highlights its significance. | Rewrite the text as a sales description, transforming it into an engaging narrative that highlights the subject’s achievements, legacy, and emotional impact to captivate and appeal to potential audiences. | 0.661 | 0.765 |
| 18 | Rewrite the text in the form of a Frequently Asked Questions (FAQ) section, making sure to include at least three question–answer pairs, with the questions phrased from the perspective of a curious reader. | Rewrite the text in the form of an interview, transforming factual information into a conversational question-and-answer format that captures personal insights, key themes, and details from the original content. | 0.640 | 0.792 |
Table 13: Inside-register prompts and corresponding reversed prompts with Orig and SDR results (continues).
| ID | Original Prompt | Reversed Prompt ( SDR ) | Orig | SDR |
| 1 | Rewrite the following content as slide presentation bullet points. Focus on summarizing the key arguments and findings clearly and concisely. Use concise phrases that highlight core points. | Rewrite the text as a step-by-step instructional guide, organizing the information into clear sections and ensuring each step provides concise, relevant details on the specified topic. | 0.730 | 0.799 |
| 2 | Rewrite the following text in the style of a Facebook post. Sharing interesting information with followers. You may add light commentary, questions to the audience, or casual phrasing, but keep it natural and human-like. Avoid using emojis, hashtags, or overly dramatic expressions. | Rewrite the text as a Frequently Asked Questions section, transforming the information into an engaging question-and-answer format that encourages reader interaction and maintains a conversational tone. | 0.644 | 0.730 |
| 3 | Adapt the text into a poetic form with vivid metaphors, rhythmic structure, and emotionally evocative language. | Rewrite the text in a lyrical style that transforms factual content into an evocative narrative, using vivid imagery, poetic devices, and rhythmic flow to highlight emotional resonance and thematic cohesion. | 0.538 | 0.674 |
| 4 | Convert the content into a tutorial-style explanation for beginners, using step-by-step instructions, simple analogies, and common misunderstandings. | Rewrite the text as a step-by-step instructional guide, ensuring clear and concise steps that effectively outline key aspects and concepts while maintaining an engaging tone and logical flow throughout. | 0.616 | 0.679 |
| 5 | Rewrite the text as a formal business email, ensuring clarity, professionalism, and a polite tone. | Rewrite the text in the style of an encyclopedia entry, emphasizing clear and concise organization, formal language, and distinct sections that present factual information and key points effectively. | 0.795 | 0.884 |
| 6 | Rewrite the passage as a scientific abstract, including Background, Methods, Results, and Conclusions. Invent at least two numerical values (percentages, sample sizes, or statistical outcomes) to support claims. | Rewrite the text as a Frequently Asked Questions section, transforming the original content into clear, concise questions and answers that emphasize key themes, significant information, and factual accuracy. | 0.600 | 0.678 |
Table 14: Outside-register prompts and corresponding reversed prompts with Orig and SDR results.
| Method | ArXiv | Wikipedia | ||
| AUC | ASR | AUC | ASR | |
| Recall | 0.516 | 0.532 | 0.528 | 0.545 |
| Recall + SDR | 0.488 | 0.525 | 0.473 | 0.540 |
| Loss | 0.486 | 0.522 | 0.471 | 0.510 |
| Loss + SDR | 0.490 | 0.535 | 0.465 | 0.515 |
| Min-K | 0.536 | 0.510 | 0.486 | 0.515 |
Table 15: Negative-control experiment on Pythia-6.9B . When the target model is not trained on any laundered variants of the proprietary data, SDR does not find a prompt to improve unauthorized training-data detection. Across all detectors and both datasets (ArXiv and Wikipedia), the AUC and ASR scores before and after SDR remain close to 0.5, indicating that SDR does not fabricate evidence of misuse in the absence of laundered training data.
| Method | AUC | ASR |
| Recall | 0.641 | 0.630 |
| Recall + SDR | 0.767 | 0.718 |
| Loss | 0.650 | 0.635 |
| Loss + SDR | 0.808 | 0.760 |
| Min-K | 0.672 | 0.645 |
| Min-K + SDR | 0.784 | 0.734 |
Table 16: SDR transferability on DeepSeek-v3 laundering pipelines for the target LLM trained exclusively on DeepSeek-laundered data. SDR uses GPT-4o as .
| Method | AUC | ASR |
| Recall | 0.649 | 0.635 |
| Recall + SDR | 0.566 | 0.575 |
| Loss | 0.687 | 0.680 |
| Loss + SDR | 0.751 | 0.725 |
| Min-K | 0.652 | 0.680 |
| Min-K + SDR | 0.713 | 0.690 |
Table 17: Detection performance on the Polite (human-rewritten) dataset. SDR improves most detectors even when laundering is performed by humans, demonstrating SDR ’s robustness and transferability beyond LLM-based pipelines.
| Method | Mixed Prompt 1 | Mixed Prompt 2 | Mixed Prompt 3 | |||
| AUC | ASR | AUC | ASR | AUC | ASR | |
| Recall | 0.559 | 0.560 | 0.666 | 0.650 | 0.737 | 0.715 |
| Recall + SDR | 0.757 | 0.705 | 0.777 | 0.725 | 0.831 | 0.770 |
| Loss | 0.572 | 0.575 | 0.670 | 0.655 | 0.738 | 0.685 |
| Loss + SDR | 0.767 | 0.705 | 0.796 | 0.750 | 0.834 | 0.785 |
| Min-K | 0.587 | 0.580 | 0.677 | 0.670 | 0.747 | 0.705 |
Table 18: SDR transferability on mixed-register laundering prompts. SDR consistently improves AUC and ASR across three mixed-register laundering transformations, demonstrating robustness to more complex combinations of stylistic rewrites.
| Method | Avg AUC | Avg ASR |
| Recall | 0.608 | 0.590 |
| Recall + SDR | 0.722 | 0.694 |
| Loss | 0.599 | 0.597 |
| Loss + SDR | 0.732 | 0.703 |
| Min-K | 0.608 | 0.603 |
| Min-K + SDR | 0.724 | 0.701 |
Table 19: Average performance across all five laundering prompts under partial laundering.
| AUC | ASR | Time (hh:mm:ss) | Query Budget (GPT4o / mini) | |
| 3 | 0.712 | 0.685 | 02:23:42 | \sim\7/0.8$ |
| 5 | 0.727 | 0.690 | 04:11:33 | \sim\13/1.5$ |
| 15 | 0.747 | 0.705 | 13:27:12 | \sim\39/4.5$ |
Table 20: Sensitivity to .
| AUC | ASR | Time (hh:mm:ss) | Query Budget (GPT4o / mini) | |
| 3 | 0.715 | 0.670 | 08:21:33 | \sim\10/1$ |
| 7 | 0.731 | 0.695 | 09:13:12 | \sim\10/1$ |
| 9 | 0.718 | 0.695 | 09:35:17 | \sim\10/1$ |
Table 21: Sensitivity to .
| AUC | ASR | Time (hh:mm:ss) | Query Budget (GPT4o / mini) | |
| 3 | 0.718 | 0.685 | 08:14:33 | \sim\10/1$ |
| 7 | 0.796 | 0.740 | 08:19:29 | \sim\10/1$ |
| 9 | 0.810 | 0.760 | 08:26:45 | \sim\10/1$ |
| 11 | 0.806 | 0.755 | 08:33:34 | \sim\10/1$ |
Table 22: Sensitivity to .
| AUC | ASR | Time (hh:mm:ss) | Query Budget | |
| 3 | 0.705 | 0.675 | – | – |
| 7 | 0.752 | 0.705 | – | – |
| 9 | 0.742 | 0.695 | – | – |
Table 23: Sensitivity to . controls the construction of the template and is computed only once during initialization. Therefore, it does not contribute to the runtime or query budget.
| Method | AUC | ASR |
| Loss | 0.638 | 0.635 |
| Loss + Reverse Prompt Engineering | 0.682 | 0.663 |
| Loss + SDR | 0.782 | 0.764 |
Table 24: Comparison between SDR and reverse prompt engineering.
| Method | Inside TPR@1% | Outside TPR@1% |
| Recall | 0.002 | 0.008 |
| Recall + SDR | 0.106 | 0.062 |
| Loss | 0.047 | 0.000 |
| Loss + SDR | 0.112 | 0.069 |
| Min-K | 0.001 | 0.016 |
| Min-K + SDR | 0.096 | 0.082 |
Table 25: TPR@1% results for the main experiment.
More in this topic
LLM Auditing
Large language model auditing tests models and deployed systems for bias, safety, privacy, reliability, and provenance using behavioral and internal evidence.
See all 621 papers