SPADE: Soft Prompts for Ai-text DEtection
Organizations: ChapsVision, France · Univ. Grenoble Alpes, CNRS, Grenoble INP*, LIG, 38000 Grenoble, France
Abstract
The easy access to Large Language Models (LLMs) has made AI-generated texts on the web increasingly ubiquitous, making AI-text detection a crucial task. Prompts greatly influence the generation process, but existing detectors focus solely on the generated texts --- supervised methods learn from labeled texts only, and zero-shot methods rely on the text's probability distributions --- ignoring the underlying generation process, which limits their robustness in open-world scenarios. We introduce SPADE: Soft Prompts for Ai-text DEtection, a novel framework for detecting AI-generated text that transfers better across datasets. Specifically, we measure the prompt's influence and use it as a discriminative signal. We characterize the prompt's influence by introducing a Pointwise Mutual Information (PMI) term that captures how much a prompt explains a text. Based on this quantity, SPADE trains one soft prompt per class to amplify the separation, and predicts the class whose prompt best explains the candidate text. Extensive experiments on several benchmarks against various detectors demonstrate the robustness and effectiveness of our method, achieving state-of-the-art (SOTA) performance on out-of-distribution (OOD) datasets.
Figures & tables
| harmful | peerread | pubmed | writing | Avg. | ||||||
| Detector | AUC | TPR | AUC | TPR | AUC | TPR | AUC | TPR | AUC | TPR |
| Zero-Shot Detectors | ||||||||||
| Likelihood | ||||||||||
| Entropy | ||||||||||
| DNA-GPT ( Yang et al., 2024 ) | ||||||||||
| Fast-DetectGPT ( Bao et al., 2024 ) | ||||||||||
| Abstracts | Books | News | Poetry | Recipes | Reviews | Wiki | Avg. | |||||||||||
| Detector | AUC | TPR | AUC | TPR | AUC | TPR | AUC | TPR | AUC | TPR | AUC | TPR | AUC | TPR | AUC | TPR | AUC | TPR |
| Zero-Shot Detectors | ||||||||||||||||||
| Likelihood | ||||||||||||||||||
| Entropy | ||||||||||||||||||
| DNA-GPT ( Yang et al., 2024 ) | ||||||||||||||||||
| Fast-DetectGPT ( Bao et al., 2024 ) | ||||||||||||||||||
| EvoBench | RAID | Avg. | ||||
| Detector | AUC | TPR | AUC | TPR | AUC | TPR |
| LLaMA-3.1-8B | ||||||
| Likelihood | ||||||
| Entropy | ||||||
| DNA-GPT | ||||||
| Fast-DetectGPT | ||||||
| Hinge | Score | EvoBench | RAID | Avg. | |||
|---|---|---|---|---|---|---|---|
| AUC | TPR | AUC | TPR | AUC | TPR | ||
| ✓ | LLR (ours) | 97.7 | 92.9 | 95.7 | 84.0 | 96.7 | 88.5 |
| ✓ | 90.7 | 81.5 | 90.5 | 79.8 | 90.6 | 80.7 | |
| ✓ | 86.3 | 57.9 | 90.6 | 35.4 | 88.5 | 46.7 | |
| ✗ | LLR | 95.7 | 85.2 | 79.7 | 46.6 | 87.7 | 65.9 |
| ✗ | 78.4 | 45.8 | 74.2 | 33.9 | 76.3 | 39.9 | |
| Detector | s / doc | s / 100 words | vs. SPADE | Rank |
|---|---|---|---|---|
| RoBERTa | 2 | |||
| Likelihood | 5 | |||
| Entropy | 8 | |||
| NTS | 11 | |||
| ImBD | 4 | |||
| Fast-DetectGPT | 9 |
Appendix figures & tables22 assets
Supplementary material from the paper’s appendix.
Appendix
| Dataset | License |
|---|---|
| MIRAGE ( Fu et al., 2025 ) | Pi-Lab License 1.0 |
| EvoBench ( Yu et al., 2025 ) | MIT |
| RAID ( Dugan et al., 2024 ) | MIT |
| DetectRL-X ( Wu et al., 2026 ) | Apache 2.0 |
| Family | Instruct | Base | |
|---|---|---|---|
| Cohere | |||
| Mistral | |||
| MPT | |||
| Mean |
| Detector | Base | Instruct |
| Likelihood | ✓ | ✗ |
| Entropy | ✓ | ✗ |
| DNA-GPT | ✓ | ✗ |
| Fast-DetectGPT | ✓ | ✓ |
| Binoculars | ✓ | ✓ |
| NTS | ✓ | ✗ |
| harmful | peerread | pubmed | writing | Avg. | |||||||||||
| Detector | AUC | TPR | F1 | AUC | TPR | F1 | AUC | TPR | F1 | AUC | TPR | F1 | AUC | TPR | F1 |
| Zero-Shot Detectors | |||||||||||||||
| Likelihood | |||||||||||||||
| Entropy | |||||||||||||||
| DNA-GPT ( 2024 ) | |||||||||||||||
| Fast-DetectGPT ( 2024 ) | |||||||||||||||
| Abstracts | Books | News | Poetry | Recipes | Reviews | Wiki | Avg. | ||||||||||||||||||||
| Detector | AUC | TPR | F1 | AUC | TPR | F1 | AUC | TPR | F1 | AUC | TPR | F1 | AUC | TPR | F1 | AUC | TPR | F1 | AUC | TPR | F1 | AUC | TPR | F1 | AUC | TPR | F1 |
| Zero-Shot Detectors | |||||||||||||||||||||||||||
| Likelihood | |||||||||||||||||||||||||||
| Entropy | |||||||||||||||||||||||||||
| DNA-GPT ( 2024 ) | |||||||||||||||||||||||||||
| Fast-DetectGPT ( 2024 ) | |||||||||||||||||||||||||||
| harmful | peerread | pubmed | writing | Avg. | |||||||||||
| Detector | AUC | TPR | F1 | AUC | TPR | F1 | AUC | TPR | F1 | AUC | TPR | F1 | AUC | TPR | F1 |
| Zero-Shot Detectors | |||||||||||||||
| Likelihood | |||||||||||||||
| Entropy | |||||||||||||||
| DNA-GPT ( 2024 ) | |||||||||||||||
| Fast-DetectGPT ( 2024 ) | |||||||||||||||
| Abstracts | Books | News | Poetry | Recipes | Reviews | Wiki | Avg. | ||||||||||||||||||||
| Detector | AUC | TPR | F1 | AUC | TPR | F1 | AUC | TPR | F1 | AUC | TPR | F1 | AUC | TPR | F1 | AUC | TPR | F1 | AUC | TPR | F1 | AUC | TPR | F1 | AUC | TPR | F1 |
| Zero-Shot Detectors | |||||||||||||||||||||||||||
| Likelihood | |||||||||||||||||||||||||||
| Entropy | |||||||||||||||||||||||||||
| DNA-GPT ( 2024 ) | |||||||||||||||||||||||||||
| Fast-DetectGPT ( 2024 ) | |||||||||||||||||||||||||||
| harmful | peerread | pubmed | writing | Avg. | |||||||||||
| Detector | AUC | TPR | F1 | AUC | TPR | F1 | AUC | TPR | F1 | AUC | TPR | F1 | AUC | TPR | F1 |
| Zero-Shot Detectors | |||||||||||||||
| Likelihood | |||||||||||||||
| Entropy | |||||||||||||||
| DNA-GPT ( 2024 ) | |||||||||||||||
| Fast-DetectGPT ( 2024 ) | |||||||||||||||
| Abstracts | Books | News | Poetry | Recipes | Reviews | Wiki | Avg. | ||||||||||||||||||||
| Detector | AUC | TPR | F1 | AUC | TPR | F1 | AUC | TPR | F1 | AUC | TPR | F1 | AUC | TPR | F1 | AUC | TPR | F1 | AUC | TPR | F1 | AUC | TPR | F1 | AUC | TPR | F1 |
| Zero-Shot Detectors | |||||||||||||||||||||||||||
| Likelihood | |||||||||||||||||||||||||||
| Entropy | |||||||||||||||||||||||||||
| DNA-GPT ( 2024 ) | |||||||||||||||||||||||||||
| Fast-DetectGPT ( 2024 ) | |||||||||||||||||||||||||||
| Train Method | RoBERTa | RAIDAR | DPIC | ImBD | SPADE (ours) | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| EvoBench | AUC | TPR | F1 | AUC | TPR | F1 | AUC | TPR | F1 | AUC | TPR | F1 | AUC | TPR | F1 |
| IN — training family present in EvoBench | |||||||||||||||
| Qwen | |||||||||||||||
| Claude | |||||||||||||||
| Gemini | |||||||||||||||
| GPT | |||||||||||||||
| Train Method | RoBERTa | RAIDAR | DPIC | ImBD | SPADE (ours) | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| EvoBench | AUC | TPR | F1 | AUC | TPR | F1 | AUC | TPR | F1 | AUC | TPR | F1 | AUC | TPR | F1 |
| IN — training family present in EvoBench | |||||||||||||||
| GPT | |||||||||||||||
| Claude | |||||||||||||||
| Qwen | |||||||||||||||
| Gemini | |||||||||||||||
| Train Method | RoBERTa | RAIDAR | DPIC | ImBD | SPADE (ours) | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| EvoBench | AUC | TPR | F1 | AUC | TPR | F1 | AUC | TPR | F1 | AUC | TPR | F1 | AUC | TPR | F1 |
| IN — training family present in EvoBench | |||||||||||||||
| Claude | |||||||||||||||
| GPT | |||||||||||||||
| Qwen | |||||||||||||||
| Gemini | |||||||||||||||
| Attack Method | Likelihood | Entropy | DNA-GPT | Fast-DetectGPT | Binoculars | SPADE | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| AUC | TPR | F1 | AUC | TPR | F1 | AUC | TPR | F1 | AUC | TPR | F1 | AUC | TPR | F1 | AUC | TPR | F1 | |
| No attack | ||||||||||||||||||
| Alt. spelling | ||||||||||||||||||
| Article deletion | ||||||||||||||||||
| Homoglyph | ||||||||||||||||||
| Insert paragraphs | ||||||||||||||||||
| Attack Method | Likelihood | Entropy | DNA-GPT | Fast-DetectGPT | Binoculars | SPADE | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| AUC | TPR | F1 | AUC | TPR | F1 | AUC | TPR | F1 | AUC | TPR | F1 | AUC | TPR | F1 | AUC | TPR | F1 | |
| No attack | ||||||||||||||||||
| Alt. spelling | ||||||||||||||||||
| Article deletion | ||||||||||||||||||
| Homoglyph | ||||||||||||||||||
| Insert paragraphs | ||||||||||||||||||
| Attack Method | Likelihood | Entropy | DNA-GPT | Fast-DetectGPT | Binoculars | SPADE | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| AUC | TPR | F1 | AUC | TPR | F1 | AUC | TPR | F1 | AUC | TPR | F1 | AUC | TPR | F1 | AUC | TPR | F1 | |
| No attack | ||||||||||||||||||
| Alt. spelling | ||||||||||||||||||
| Article deletion | ||||||||||||||||||
| Homoglyph | ||||||||||||||||||
| Insert paragraphs | ||||||||||||||||||
| Train Test | English | Chinese | Spanish | Arabic | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| F1B | F1F | F1@0 | F1B | F1F | F1@0 | F1B | F1F | F1@0 | F1B | F1F | F1@0 | |
| English | ||||||||||||
| Chinese | ||||||||||||
| Spanish | ||||||||||||
| Arabic | ||||||||||||
| French | ||||||||||||
| Backbone | per epoch | total (3 epochs) | peak VRAM |
|---|---|---|---|
| Qwen3-8B-Base | 2.1 h | 6.2 h | 33.6 GB |
| LLaMA-3.1-8B | 1.7 h | 5.2 h | 30.7 GB |
| Falcon-7B | 1.4 h | 4.2 h | 24.0 GB |
AI-Generated Text Detection
AI-generated text detection develops methods to distinguish human-written, machine-generated, and AI-edited text.
See all 185 papers