Logit Laundering: Evading Data-Use Auditing in Large Language Models
Authors: Ruihan Hu, Wei Luo, Yu-Ming Shang, Jiakai Wang, Chuxuan Zhang, Mei Li, Xi Zhang, Meikang Qiu
Organizations: Beijing University of Posts and Telecommunications, Beijing, China · Zhongguancun Laboratory, Beijing, China · Augusta University, Augusta, Georgia, USA
Data-use auditing in Large Language Models (LLMs) aims to determine whether the given data has been used for training, thereby supporting data ownership safeguards. A dominant solution is data watermarking, which embeds traceable signals into training data and detects them in model outputs. Recent studies have proposed many watermarking-based auditing methods in LLMs. However, systematic investigation from an adversarial perspective remains limited: non-compliant LLM providers would use unauthorized copyrighted datasets for training their own models to enhance model capabilities, while attempting to evade detection by auditors, thereby avoiding legal liability. To fill this research gap, we propose a novel evasion framework, termed Logit Laundering, where the non-compliant LLM provider manipulates model outputs at inference time to undermine detectable watermark signals. Specifically, we design a logit reshaping module to suppress watermark-induced statistical signals by integrating a statistical bias estimator as a plug-and-play component. This design keeps the model parameters unchanged and performs manipulation only in the logit space, thereby enabling auditing evasion while minimizing the impact on model's learned knowledge. To further balance evasion effectiveness and model utility, we introduce a confidence-aware scaling module to adaptively calibrate the suppression strength based on next-token prediction confidence. Extensive evaluations show that Logit Laundering can successfully evade existing various state-of-the-art auditing methods, significantly attenuating detectable signals, while preserving model utility on downstream tasks. Our work highlights the need for exploring more robust auditing methods to guard against evasion behaviors.
Figures & tables
Figure 1. (a) Watermark-based auditing paradigm . Data Owner embeds watermarks into datasets prior to distribution, enabling the detection of unauthorized use via model outputs. (b) Evasion behaviors : (1) Data-level : disrupting text watermarks but compromising data integrity for training; (2) Model-level : manipulating model output behaviors to circumvent detection (our Logit Laundering).
Figure 2. Threat Model Scenario.
Own Model Knowledge
Watermark Knowledge
Model Params.
Training Data
WM Algorithm
Detection Algorithm
✓
✓
✗
✗
Table 1. Knowledge of the Bob company and the information that the Auditor is permitted to access via Bob’s API.
Canary-filtering
Unicode-filtering
Original
100.00%
100.00%
After removal
0.00%
0.00%
Table 2. Canaries and Unicode characters hit rates before and after filtering, indicating that injection-based watermarks can be easily removed through data preprocessing.
Text Transformation
P-SP ↑
PPL ↓
z-score ↓
p-value ↑
Char (homoglyph)
0.625
102.918
4.007
3.749×10−3
Char (insertion)
0.596
59.723
4.414
1.429×10−3
Char (typo)
0.592
292.106
4.612
9.162×10−4
Paraphrase
0.730
36.794
5.929
7.152×10−5
Synonym
0.714
95.515
6.503
9.983×10−5
Translation
0.570
47.960
3.697
6.324×10−3
Table 3. P-SP, PPL scores, and watermark detection performance on text after disrupting watermarks.
Figure 3. Rank-Swap is insufficient to erase watermark signals. Under radioactive detection, the aggregate green-token bias remains above the null expectation γ . Under STAMP, the public/private PPL gap remains statistically significant.
Figure 4. Logit Laundering. Stage 1: Continual pretraining, where the base model is trained on collected corpora containing unauthorized data. Stage 2: Logit reshaping, where a statistical bias estimator is incorporated at inference time to suppress detectable signals in the output distribution. Confidence-aware scaling adaptively calibrates the suppression strength.
Figure 5. Illustration of logit reshaping.
arXiv papers
Pile-CC
Hacker News
Books
Source
Public
Public
Public
Copyrighted
Quantity
500
1,000
646
4,959
Table 4. Statistics of collected corpora.
STAMP Detection
LLAMA-3.2-1B-Instruct
LLAMA-3.2-3B-Instruct
Qwen-2.5-1.5B-Instruct
Gemma-2-2B-IT
Stable-LM-2-1.6B-Chat
Metrics
pt -value ↑
t-statistic
pt -value ↑
t-statistic
pt -value ↑
t-statistic
pt -value ↑
t-statistic
pt -value ↑
t-statistic
Non Evasion
1.778×10−3
-5.371
4.076×10−3
-4.213
1.250×10−5
-10.420
2.624×10−3
-4.364
1.990×10−4
-5.564
Data-level Evasion
Char (homoglyph)
1.985×10−3
-5.217
4.134×10−2
-2.840
1.465×10−6
-10.704
5.131×10−1
0.024
5.077×10−2
-2.472
Char (insertion)
1.670×10−3
-5.701
3.550×10−2
-2.952
1.644×10−6
-10.658
4.492×10−1
-0.234
5.186×10−2
-2.435
Char (typo)
1.893×10−3
-5.328
3.893×10−2
-2.890
1.388×10−6
-10.760
4.578×10−1
-0.204
4.602×10−2
-2.522
Table 5. Comparison of auditing performance under evasion behaviors. Light green cells indicate p>0.05 , suggesting successful evasion: the statistical evidence is insufficient to determine infringement. Darker green values highlight the largest p-value, indicating the strongest signal weakening. We prioritize the pt -value and pz -value, as they provide more principled measures of statistical significance.
STAMP variants
PPL
Min-K
Min-K ++
Lowercase
Zlib
Non Evasion
1.778×10−3
6.411×10−3
5.230×10−3
1.434×10−1
1.499×10−2
Data-level Evasion
Char (homoglyph)
1.985×10−3
5.383×10−3
1.738×10−3
1.740×10−1
1.661×10−2
Char (insertion)
1.670×10−3
2.860×10−3
6.139×10−4
1.405×10−1
1.293×10−2
Char (typo)
1.893×10−3
6.369×10−3
3.700×10−3
1.537×10−1
1.578×10−2
Token Swap
1.507×10−3
3.935×10−4
1.780×10−3
1.010×10−1
9.385×10−3
Table 6. STAMP variants evasion performance.
Radioactive variants
Greedy (top-1)
Top-k sampling
Nucleus sampling
Non Evasion
1.995×10−4
8.123×10−4
9.911×10−4
Data-level Evasion
Char (homoglyph)
2.131×10−3
6.913×10−3
7.244×10−3
Char (insertion)
5.667×10−4
1.739×10−3
1.871×10−3
Char (typo)
6.921×10−4
1.298×10−3
1.753×10−3
Token Swap
9.819×10−4
1.774×10−4
1.412×10−4
Table 7. Radioactive variants under decoding strategies.
Figure 6. Illustrations before and after logit reshaping under STAMP and radioactive watermark detection.
Copyright Knowledge
LA-1B
LA-3B
Qw-1.5B
Ge-2B
SLM-1.6B
Foundation Model
31.8%
64.5%
71.8%
56.2%
35.2%
Non Evasion
73.4%
82.9%
79.7%
83.6%
59.2%
Data-level Evasion
Char (homoglyph)
51.5%
79.1%
75.9%
63.3%
45.0%
Char (insertion)
50.6%
79.8%
76.9%
63.2%
46.3%
Char (typo)
51.6%
79.3%
77.3%
59.1%
44.1%
Table 8. Knowledge Retention via MCQs.Values in green indicate the knowledge gain over the foundation model.
Method
Fluency-PPL ↓
MAUVE ↑
LLM Judge ↑
Non Evasion
6.222
0.178
83.7
Rank-Swap (2S)
11.250
0.200
78.0
Rank-Swap (SS)
36.480
0.055
43.9
Rank-Swap (SK)
13.904
0.193
73.4
Logit Laundering
5.752
0.255
79.9
Table 9. General utility: measured by generated text quality.
Model
Approach
ARC-C
MMLU
TriviaQA
GSM8K
acc
acc
exact-match
acc
Llama-3.2-1B
Non Evasion
0.310
0.455
0.290
0.260
Rank-Swap (2S)
0.290
0.385
0.110
0.040
Rank-Swap (SS)
0.300
0.383
0.160
0.010
Rank-Swap (SK)
0.310
0.364
0.170
0.030
Logit Laundering
0.310
0.453
0.270
0.230
Table 10. General utility: performance on general task.
Figure 7. Performance under different watermark strengths controlled by induced bias δ , reported as p -values.
Figure 8. Detectable signals under different estimator sizes
Figure 9. Hyperparameter analysis of segment ratio α and scaling smoothing parameter S .
Figure 10. Evasion performance under different JS budget range settings, [τmin,τmax] .
Model
Variant
λ sched.
MAUVE ↑
p -value ↑
Llama-3.2-1B
adaptive
[0.170,0.560]
0.107
0.748
w/o scaling
λ=0.170
0.018 ↓↓
0.044
w/o scaling
λ=0.365
0.016 ↓↓
0.752
w/o scaling
λ=0.560
0.018 ↓↓
0.982
Qwen2.5-1.5B
adaptive
[0.140,0.450]
0.151
0.412
w/o scaling
λ=0.140
0.019 ↓↓
0.069
Table 11. Ablation of confidence scaling module
Model
True-key ratio
Random-key mean
Shift
Clean
0.2496
0.2476
+0.0019
Non-evasion
0.2781
0.2503
+0.0278
Laundered
0.2400
0.2513
-0.0112 FN flag
Table 12. True-key placebo check. A negative shift indicates signal suppression and may flag a suspicious false negative.
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 11. Text transformations ( 84 , 40 ) would distort the original content, impairing their value as training data.
Decoding
Margin Group
Candidate Overlap
Dist. Shift
Cont. Drift
Greedy
Large
1.000
0.136
0.000
Greedy
Small
0.917
0.748
0.066
Top- k
Large
0.952
0.000
0.056
Top- k
Small
0.928
0.006
0.130
Top- p
Large
1.000
0.000
0.059
Top- p
Small
0.895
0.005
0.070
Appendix
Table 13. Sensitivity to first-step logit perturbations across margin groups.
Target LLM
Release time
LLAMA-3.2-1B-Instruct
2024-09
LLAMA-3.2-3B-Instruct
2024-09
Qwen-2.5-1.5B-Instruct
2024-09
Gemma-2-2B-IT
2024-07
Stable-LM-2-1.6B-Chat
2024-02
Our copyrighted-book corpus
2025
Appendix
Table 14. Release time of evaluated foundation models and publication time of our collected copyrighted book corpus.
Source
n
Mean (tokens)
Median (tokens)
Out-of-training sources
Wikipedia (en)
84
217.6
220
Books3
83
219.9
222
StackExchange
83
217.0
219
In-training sources
Copyrighted Books
63
235.1
229
Appendix
Table 15. Statistics of the 500 generation-quality evaluation samples, grouped by whether the source belongs to the continual pretraining corpus D .
τmin & τmax
LA-1B
Qw-1.5B
LA-3B
Ge-2B
SLM-1.6B
0.01
0.17
0.14
0.15
0.14
0.12
0.02
0.24
0.19
0.21
0.20
0.18
0.03
0.29
0.24
0.26
0.25
0.22
0.04
0.34
0.28
0.30
0.28
0.25
0.05
0.38
0.31
0.34
0.32
0.28
0.06
0.42
0.34
0.37
0.35
0.31
Appendix
Table 16. Recommended λ under each τ value for all five foundation models.
Copyright Knowledge
LA-1B
LA-3B
Qw-1.5B
Ge-2B
SLM-1.6B
Foundation Model
33.8%
80.7%
87.1%
76.2%
29.4%
Non Evasion
86.6%
93.8%
94.4%
94.1%
65.7%
Data-level Evasion
Char (homoglyph)
44.6%
87.7%
87.2%
58.2%
39.3%
Char (insertion)
44.3%
88.6%
88.2%
60.1%
40.8%
Char (typo)
44.3%
88.3%
88.0%
59.6%
38.8%
Appendix
Table 17. Knowledge Retention Performance on the Claude Sonnet 4.6-generated MCQ set.
Copyright Knowledge
LA-1B
LA-3B
Qw-1.5B
Ge-2B
SLM-1.6B
Foundation Model
29.8%
48.3%
56.4%
36.1%
40.9%
Non Evasion
60.1%
71.9%
65.0%
73.1%
52.6%
Data-level Evasion
Char (homoglyph)
58.3%
70.4%
64.6%
68.4%
50.7%
Char (insertion)
57.0%
70.9%
65.6%
66.4%
51.8%
Char (typo)
58.9%
70.3%
66.7%
58.6%
49.4%
Appendix
Table 18. Knowledge Retention Performance on the GPT-5.4-generated MCQ set.
Model
Approach
Avg. PPL ↓
Avg. MAUVE ↑
GPT-5.4 as Judge ↑
sonnet4.6 as Judge ↑
Llama-3.2-1B
Non Evasion
5.13
0.118
87.6
82
Rank-Swap (2S)
9.30
0.207
78.3
80
Rank-Swap (SS)
33.22
0.083
43.9
43
Rank-Swap (SK)
11.49
0.238
72.5
76
Logit Laundering
4.29
0.151
81.8
79
Qwen2.5-1.5B
Non Evasion
4.83
0.156
86.5
75
Appendix
Table 19. Generation quality under different evasion attacks. Lower Avg. PPL is better; higher Avg. MAUVE and judge scores are better.
Model
Variant
λ sched.
MAUVE ↑
p -value ↑
Llama-3.2-1B
adaptive
[0.17,0.56]
0.107
0.748
w/o scaling
λ=0.17
0.018
0.044
w/o scaling
λ=0.365
0.016
0.752
w/o scaling
λ=0.56
0.017
0.982
Qwen2.5-1.5B
adaptive
[0.14,0.45]
0.151
0.412
w/o scaling
λ=0.14
0.019
0.069
Appendix
Table 20. More ablation studies of confidence scaling module.
STAMP-Min-K
LLAMA-3.2-1B-Instruct
LLAMA-3.2-3B-Instruct
Qwen-2.5-1.5B-Instruct
Gemma-2-2B-IT
Stable-LM-2-1.6B-Instruct
Metrics
pt -value ↑
t-statistic
pt -value ↑
t-statistic
pt -value ↑
t-statistic
pt -value ↑
t-statistic
pt -value ↑
t-statistic
Non Evasion
6.410×10−3
-3.758
2.550×10−2
-2.849
9.744×10−1
10.276
1.175×10−2
-3.516
1.927×10−3
-4.053
Data-level Evasion
Char (homoglyph)
5.383×10−3
-3.826
1.497×10−1
-1.611
9.760×10−1
10.024
6.150×10−1
0.452
8.439×10−2
-2.049
Char (insertion)
2.860×10−3
-4.239
1.303×10−1
-1.754
9.774×10−1
10.046
5.705×10−1
0.268
8.421×10−2
-2.039
Char (typo)
6.369×10−3
-3.794
1.388×10−1
-1.678
9.717×10−1
9.640
5.966×10−1
0.361
7.812×10−2
-2.090
Appendix
Table 21. STAMP-like auditing performance under evasion behaviors (Part I: Min-K and Min-K++).
STAMP-Lowercase
LLAMA-3.2-1B-Instruct
LLAMA-3.2-3B-Instruct
Qwen-2.5-1.5B-Instruct
Gemma-2-2B-IT
Stable-LM-2-1.6B-Instruct
Metrics
pt -value ↑
t-statistic
pt -value ↑
t-statistic
pt -value ↑
t-statistic
pt -value ↑
t-statistic
pt -value ↑
t-statistic
Non Evasion
1.434×10−1
-2.843
2.377×10−1
-1.521
3.487×10−2
-4.628
2.956×10−2
-4.559
8.572×10−3
-3.894
Data-level Evasion
Char (homoglyph)
1.740×10−1
-2.371
4.776×10−1
-0.110
3.850×10−2
-4.456
3.505×10−1
-0.588
6.259×10−1
0.526
Char (insertion)
1.405×10−1
-2.858
4.766×10−1
-0.120
3.585×10−2
-4.676
3.098×10−1
-0.703
5.895×10−1
0.329
Char (typo)
1.537×10−1
-2.519
5.153×10−1
0.096
3.529×10−2
-5.368
2.557×10−1
-0.975
6.265×10−1
0.520
Appendix
Table 22. Comparison of STAMP-like auditing performance under evasion behaviors (Part II: Lowercase and Zlib).
Large language model auditing tests models and deployed systems for bias, safety, privacy, reliability, and provenance using behavioral and internal evidence.
Post-hoc unauthorized-training data detection for large language models (LLMs) typically assumes a query-with-originals regime: rights holders query a target LLM with raw proprietary data and assess whether the model assigns them stronger memorization-based detection signals, e.g., higher confidence or lower loss, than held-out non-training reference texts. We show that this regime becomes brittle under data laundering, where the target LLM is trained on semantics-preserving but stylistically or structurally transformed surrogates of proprietary data to obfuscate provenance. Since training-time exposure occurs in the laundered form, memorization signals may no longer appear on the originals, collapsing the candidate-reference signal separation that standard detectors rely on. We counter this threat by studying laundering-aware detection with raw proprietary data, a held-out reference corpus, and query access to the target LLM, while the laundering transformation is undisclosed. Since exact recovery of the laundered corpus is infeasible, we infer a detection-useful synthesis process via an auxiliary LLM that maps originals into training-like queries. To make this search tractable, we introduce Synthesis Data Reversion (SDR), which constrains the unbounded space of natural-language transformations through a goal-details abstraction: a high-level transformation goal, e.g., "lyrical rewriting", and fine-grained details, e.g., "with vivid imagery". SDR identifies the most likely goal and iteratively refines details so synthesized queries elicit stronger target-model detection signals. Evaluated on the MIMIR benchmark against diverse laundering practices and target LLM families (Pythia, Llama2, and Falcon), SDR consistently restores detection signals, offering a practical auditing layer against data laundering.
Muxing Li, Zesheng Ye, Sharon Li +1
University of Melbourne · University of Wisconsin-Madison
The proliferation of customized Large Language Models (LLMs) poses critical risks of Data Intellectual Property (Data IP) infringement via unauthorized fine-tuning on proprietary data. Existing audit techniques are limited, as they require intervention during data preparation or training and remain fragile under malicious obfuscations such as data paraphrasing and knowledge distillation. We propose Distribution Provenance Audit (DPA), a post-hoc framework for auditing data IP infringement in LLM fine-tuning under black-box and malicious settings. DPA is grounded in a critical insight: regardless of fine-tuning tactics to evade provenance, the practical necessity of maintaining utility constrains the model to preserve the fundamental intersection of semantic substance and lexical form. Accordingly, DPA captures this persistent lexical-semantic intersection as intrinsic distributional fingerprints. The framework formulates the audit as a statistical hypothesis test, effectively quantifying these fingerprints via unbiased output sampling to reliably reject the null hypothesis of non-usage. Extensive experiments on medical and legal fine-tuning tasks show that DPA consistently outperforms existing baselines, remaining robust against adversarial trainers employing paraphrasing and knowledge distillation. We further highlight a fundamental dual-use tension: the same high-fidelity distributional fingerprints enabling reliable auditing may also facilitate privacy attacks.
Zirui Huang, Yunlong Mao, Wei Tong +3
State Key Laboratory of Novel Software Technology, Nanjing University Nanjing, Jiangsu, China
This work investigates the application of the Membership Inference Test (MINT) to large language models (LLMs) fine-tuned with LoRA, aiming to determine whether specific samples were used during training and address concerns about data privacy, leakage, and transparency. Building on our previous LoRA-MINT framework, which relies exclusively on perplexity under a strict black-box setting, we propose an approach based on intermediate model representations, including activations and parameter-dependent signals. These representations enable a MINT classifier to distinguish training from non-training samples without requiring full access to model parameters. We evaluated the method on four LLMs (Qwen, Phi-4, DeepSeek-V3, and LLaMA 3.2-3B) fine-tuned on three datasets covering medical, mathematical, and academic reasoning tasks. Results show consistent performance across architectures and domains, achieving scores between 95% and 98%, with robustness to variations in model configurations and data distributions. The proposed approach enables efficient auditing of sensitive training data and supports data provenance and compliance with emerging privacy and AI governance regulations. Although our experiments focus on LoRA-based fine-tuning, the methodology can be extended to other LLM adaptation techniques and, more broadly, to other AI models.
BiometricsAI, Universidad Autonoma de Madrid, Spain · Mathematics Dept., Universidad de Las Palmas de Gran Canaria, Spain · Computer Engineering Dept., Universidad Autonoma de Madrid, Spain