Logit Laundering: Evading Data-Use Auditing in Large Language Models
Organizations: Beijing University of Posts and Telecommunications, Beijing, China · Zhongguancun Laboratory, Beijing, China · Augusta University, Augusta, Georgia, USA
Abstract
Data-use auditing in Large Language Models (LLMs) aims to determine whether the given data has been used for training, thereby supporting data ownership safeguards. A dominant solution is data watermarking, which embeds traceable signals into training data and detects them in model outputs. Recent studies have proposed many watermarking-based auditing methods in LLMs. However, systematic investigation from an adversarial perspective remains limited: non-compliant LLM providers would use unauthorized copyrighted datasets for training their own models to enhance model capabilities, while attempting to evade detection by auditors, thereby avoiding legal liability. To fill this research gap, we propose a novel evasion framework, termed Logit Laundering, where the non-compliant LLM provider manipulates model outputs at inference time to undermine detectable watermark signals. Specifically, we design a logit reshaping module to suppress watermark-induced statistical signals by integrating a statistical bias estimator as a plug-and-play component. This design keeps the model parameters unchanged and performs manipulation only in the logit space, thereby enabling auditing evasion while minimizing the impact on model's learned knowledge. To further balance evasion effectiveness and model utility, we introduce a confidence-aware scaling module to adaptively calibrate the suppression strength based on next-token prediction confidence. Extensive evaluations show that Logit Laundering can successfully evade existing various state-of-the-art auditing methods, significantly attenuating detectable signals, while preserving model utility on downstream tasks. Our work highlights the need for exploring more robust auditing methods to guard against evasion behaviors.
Figures & tables
| Own Model Knowledge | Watermark Knowledge | ||
|---|---|---|---|
| Model Params. | Training Data | WM Algorithm | Detection Algorithm |
| ✓ | ✓ | ✗ | ✗ |
| Canary-filtering | Unicode-filtering | |
|---|---|---|
| Original | 100.00% | 100.00% |
| After removal | 0.00% | 0.00% |
| Text Transformation | P-SP | PPL | z-score | p-value |
|---|---|---|---|---|
| Char (homoglyph) | 0.625 | 102.918 | 4.007 | |
| Char (insertion) | 0.596 | 59.723 | 4.414 | |
| Char (typo) | 0.592 | 292.106 | 4.612 | |
| Paraphrase | 0.730 | 36.794 | 5.929 | |
| Synonym | 0.714 | 95.515 | 6.503 | |
| Translation | 0.570 | 47.960 | 3.697 |
| arXiv papers | Pile-CC | Hacker News | Books | |
|---|---|---|---|---|
| Source | Public | Public | Public | Copyrighted |
| Quantity | 500 | 1,000 | 646 | 4,959 |
| STAMP Detection | LLAMA-3.2-1B-Instruct | LLAMA-3.2-3B-Instruct | Qwen-2.5-1.5B-Instruct | Gemma-2-2B-IT | Stable-LM-2-1.6B-Chat | |||||
|---|---|---|---|---|---|---|---|---|---|---|
| Metrics | -value | t-statistic | -value | t-statistic | -value | t-statistic | -value | t-statistic | -value | t-statistic |
| Non Evasion | -5.371 | -4.213 | -10.420 | -4.364 | -5.564 | |||||
| Data-level Evasion | ||||||||||
| Char (homoglyph) | -5.217 | -2.840 | -10.704 | 0.024 | -2.472 | |||||
| Char (insertion) | -5.701 | -2.952 | -10.658 | -0.234 | -2.435 | |||||
| Char (typo) | -5.328 | -2.890 | -10.760 | -0.204 | -2.522 | |||||
| STAMP variants | PPL | Min-K | Min-K ++ | Lowercase | Zlib |
|---|---|---|---|---|---|
| Non Evasion | |||||
| Data-level Evasion | |||||
| Char (homoglyph) | |||||
| Char (insertion) | |||||
| Char (typo) | |||||
| Token Swap | |||||
| Radioactive variants | Greedy (top-1) | Top-k sampling | Nucleus sampling |
|---|---|---|---|
| Non Evasion | |||
| Data-level Evasion | |||
| Char (homoglyph) | |||
| Char (insertion) | |||
| Char (typo) | |||
| Token Swap | |||
| Copyright Knowledge | LA-1B | LA-3B | Qw-1.5B | Ge-2B | SLM-1.6B |
|---|---|---|---|---|---|
| Foundation Model | 31.8% | 64.5% | 71.8% | 56.2% | 35.2% |
| Non Evasion | 73.4% | 82.9% | 79.7% | 83.6% | 59.2% |
| Data-level Evasion | |||||
| Char (homoglyph) | 51.5% | 79.1% | 75.9% | 63.3% | 45.0% |
| Char (insertion) | 50.6% | 79.8% | 76.9% | 63.2% | 46.3% |
| Char (typo) | 51.6% | 79.3% | 77.3% | 59.1% | 44.1% |
| Method | Fluency-PPL | MAUVE | LLM Judge |
|---|---|---|---|
| Non Evasion | 6.222 | 0.178 | 83.7 |
| Rank-Swap (2S) | 11.250 | 0.200 | 78.0 |
| Rank-Swap (SS) | 36.480 | 0.055 | 43.9 |
| Rank-Swap (SK) | 13.904 | 0.193 | 73.4 |
| Logit Laundering | 5.752 | 0.255 | 79.9 |
| Model | Approach | ARC-C | MMLU | TriviaQA | GSM8K |
|---|---|---|---|---|---|
| acc | acc | exact-match | acc | ||
| Llama-3.2-1B | Non Evasion | 0.310 | 0.455 | 0.290 | 0.260 |
| Rank-Swap (2S) | 0.290 | 0.385 | 0.110 | 0.040 | |
| Rank-Swap (SS) | 0.300 | 0.383 | 0.160 | 0.010 | |
| Rank-Swap (SK) | 0.310 | 0.364 | 0.170 | 0.030 | |
| Logit Laundering | 0.310 | 0.453 | 0.270 | 0.230 |
| Model | Variant | sched. | MAUVE | -value |
|---|---|---|---|---|
| Llama-3.2-1B | adaptive | 0.107 | 0.748 | |
| w/o scaling | 0.018 | 0.044 | ||
| w/o scaling | 0.016 | 0.752 | ||
| w/o scaling | 0.018 | 0.982 | ||
| Qwen2.5-1.5B | adaptive | 0.151 | 0.412 | |
| w/o scaling | 0.019 | 0.069 |
| Model | True-key ratio | Random-key mean | Shift |
|---|---|---|---|
| Clean | 0.2496 | 0.2476 | +0.0019 |
| Non-evasion | 0.2781 | 0.2503 | +0.0278 |
| Laundered | 0.2400 | 0.2513 | -0.0112 FN flag |
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
| Decoding | Margin Group | Candidate Overlap | Dist. Shift | Cont. Drift |
|---|---|---|---|---|
| Greedy | Large | 1.000 | 0.136 | 0.000 |
| Greedy | Small | 0.917 | 0.748 | 0.066 |
| Top- | Large | 0.952 | 0.000 | 0.056 |
| Top- | Small | 0.928 | 0.006 | 0.130 |
| Top- | Large | 1.000 | 0.000 | 0.059 |
| Top- | Small | 0.895 | 0.005 | 0.070 |
| Target LLM | Release time |
|---|---|
| LLAMA-3.2-1B-Instruct | 2024-09 |
| LLAMA-3.2-3B-Instruct | 2024-09 |
| Qwen-2.5-1.5B-Instruct | 2024-09 |
| Gemma-2-2B-IT | 2024-07 |
| Stable-LM-2-1.6B-Chat | 2024-02 |
| Our copyrighted-book corpus | 2025 |
| Source | Mean (tokens) | Median (tokens) | |
|---|---|---|---|
| Out-of-training sources | |||
| Wikipedia (en) | 84 | 217.6 | 220 |
| Books3 | 83 | 219.9 | 222 |
| StackExchange | 83 | 217.0 | 219 |
| In-training sources | |||
| Copyrighted Books | 63 | 235.1 | 229 |
| & | LA-1B | Qw-1.5B | LA-3B | Ge-2B | SLM-1.6B |
|---|---|---|---|---|---|
| 0.01 | 0.17 | 0.14 | 0.15 | 0.14 | 0.12 |
| 0.02 | 0.24 | 0.19 | 0.21 | 0.20 | 0.18 |
| 0.03 | 0.29 | 0.24 | 0.26 | 0.25 | 0.22 |
| 0.04 | 0.34 | 0.28 | 0.30 | 0.28 | 0.25 |
| 0.05 | 0.38 | 0.31 | 0.34 | 0.32 | 0.28 |
| 0.06 | 0.42 | 0.34 | 0.37 | 0.35 | 0.31 |
| Copyright Knowledge | LA-1B | LA-3B | Qw-1.5B | Ge-2B | SLM-1.6B |
|---|---|---|---|---|---|
| Foundation Model | 33.8% | 80.7% | 87.1% | 76.2% | 29.4% |
| Non Evasion | 86.6% | 93.8% | 94.4% | 94.1% | 65.7% |
| Data-level Evasion | |||||
| Char (homoglyph) | 44.6% | 87.7% | 87.2% | 58.2% | 39.3% |
| Char (insertion) | 44.3% | 88.6% | 88.2% | 60.1% | 40.8% |
| Char (typo) | 44.3% | 88.3% | 88.0% | 59.6% | 38.8% |
| Copyright Knowledge | LA-1B | LA-3B | Qw-1.5B | Ge-2B | SLM-1.6B |
|---|---|---|---|---|---|
| Foundation Model | 29.8% | 48.3% | 56.4% | 36.1% | 40.9% |
| Non Evasion | 60.1% | 71.9% | 65.0% | 73.1% | 52.6% |
| Data-level Evasion | |||||
| Char (homoglyph) | 58.3% | 70.4% | 64.6% | 68.4% | 50.7% |
| Char (insertion) | 57.0% | 70.9% | 65.6% | 66.4% | 51.8% |
| Char (typo) | 58.9% | 70.3% | 66.7% | 58.6% | 49.4% |
| Model | Approach | Avg. PPL | Avg. MAUVE | GPT-5.4 as Judge | sonnet4.6 as Judge |
|---|---|---|---|---|---|
| Llama-3.2-1B | Non Evasion | 5.13 | 0.118 | 87.6 | 82 |
| Rank-Swap (2S) | 9.30 | 0.207 | 78.3 | 80 | |
| Rank-Swap (SS) | 33.22 | 0.083 | 43.9 | 43 | |
| Rank-Swap (SK) | 11.49 | 0.238 | 72.5 | 76 | |
| Logit Laundering | 4.29 | 0.151 | 81.8 | 79 | |
| Qwen2.5-1.5B | Non Evasion | 4.83 | 0.156 | 86.5 | 75 |
| Model | Variant | sched. | MAUVE | -value |
|---|---|---|---|---|
| Llama-3.2-1B | adaptive | 0.107 | 0.748 | |
| w/o scaling | 0.018 | 0.044 | ||
| w/o scaling | 0.016 | 0.752 | ||
| w/o scaling | 0.017 | 0.982 | ||
| Qwen2.5-1.5B | adaptive | 0.151 | 0.412 | |
| w/o scaling | 0.019 | 0.069 |
| STAMP-Min-K | LLAMA-3.2-1B-Instruct | LLAMA-3.2-3B-Instruct | Qwen-2.5-1.5B-Instruct | Gemma-2-2B-IT | Stable-LM-2-1.6B-Instruct | |||||
|---|---|---|---|---|---|---|---|---|---|---|
| Metrics | -value | t-statistic | -value | t-statistic | -value | t-statistic | -value | t-statistic | -value | t-statistic |
| Non Evasion | -3.758 | -2.849 | 10.276 | -3.516 | -4.053 | |||||
| Data-level Evasion | ||||||||||
| Char (homoglyph) | -3.826 | -1.611 | 10.024 | 0.452 | -2.049 | |||||
| Char (insertion) | -4.239 | -1.754 | 10.046 | 0.268 | -2.039 | |||||
| Char (typo) | -3.794 | -1.678 | 9.640 | 0.361 | -2.090 | |||||
| STAMP-Lowercase | LLAMA-3.2-1B-Instruct | LLAMA-3.2-3B-Instruct | Qwen-2.5-1.5B-Instruct | Gemma-2-2B-IT | Stable-LM-2-1.6B-Instruct | |||||
|---|---|---|---|---|---|---|---|---|---|---|
| Metrics | -value | t-statistic | -value | t-statistic | -value | t-statistic | -value | t-statistic | -value | t-statistic |
| Non Evasion | -2.843 | -1.521 | -4.628 | -4.559 | -3.894 | |||||
| Data-level Evasion | ||||||||||
| Char (homoglyph) | -2.371 | -0.110 | -4.456 | -0.588 | 0.526 | |||||
| Char (insertion) | -2.858 | -0.120 | -4.676 | -0.703 | 0.329 | |||||
| Char (typo) | -2.519 | 0.096 | -5.368 | -0.975 | 0.520 | |||||
LLM Auditing
Large language model auditing tests models and deployed systems for bias, safety, privacy, reliability, and provenance using behavioral and internal evidence.
See all 723 papers