Robust LLM-Generated Text Detection through Matched Conditional-Response Probing
Organizations: School of Cyber Science and Engineering, Sichuan University · Cyber Science Research Institute, Sichuan University · Intelligent Policing Key Laboratory of Sichuan Province, Sichuan Police College · Key Laboratory of Data Protection and Intelligent Management, Sichuan University
Abstract
Reliable detection of large language model (LLM)-generated text must remain effective under meaning-preserving text revisions, such as rewriting, polishing, and expansion. In practice, LLM outputs are often rewritten, polished, expanded, translated, or lightly edited before publication, weakening the lexical, syntactic, and local discourse cues used by many existing detectors. We introduce Matched Conditional-Response Probing (MCRP), a frozen-LLM probing framework that measures how a candidate text responds when a semantically matched condition is introduced, removed, or replaced. For each candidate, the frozen probe LLM first generates a short semantic condition and then compares token-level losses and prompt--target attention measurements under matched, null, and control conditions. It aggregates bidirectional conditioned alignment, counterfactual sensitivity, and internal attention coupling into a probe representation, on which it trains a lightweight supervised random-forest classifier while keeping the probe LLM frozen. Across standard detection, robustness to meaning-preserving text revisions, multilingual, and multidomain evaluations, MCRP achieves consistently strong performance. Under rewriting, polishing, and expansion, it reaches 93.97% F1 and 96.24% ROC-AUC, outperforming the strongest evaluated baseline by 19.50 and 25.77 percentage points, respectively. Component ablations show that the response feature groups make complementary contributions to predictive performance. In the condition-control study, matched conditions consistently outperform random, mismatched, and corrupted alternatives, demonstrating a measurable incremental benefit from semantic matching.
Figures & tables
| Method | arXiv | Code | Essay | Yelp | Creative |
|---|---|---|---|---|---|
| Fast-DetectGPT | 89.72/95.85 | 67.04/66.28 | 97.99/99.41 | 71.94/71.60 | 91.59/95.64 |
| AdaDetectGPT | 82.14/85.92 | 68.61/63.19 | 77.18/80.70 | 67.73/59.47 | 82.38/81.44 |
| Binoculars | 90.42/95.48 | 62.09/69.96 | 99.85 / 99.99 | 75.62/78.30 | 98.96/99.79 |
| RADAR | 81.22/85.24 | 66.86/52.48 | 96.25/98.52 | 75.00/67.66 | 94.39/95.82 |
| ImBD | 94.16/96.86 | 84.10 / 87.45 | 99.40/99.71 | 82.35 / 85.55 | 97.48/98.66 |
| DNA-DetectLLM | 94.39 / 97.92 | 71.28/63.94 | 99.84/99.92 | 79.13/80.46 | 99.11 / 99.91 |
| Method | Rewrite | Polish | Expand | Avg. |
|---|---|---|---|---|
| Fast-DetectGPT | 68.98/63.10 | 68.72/62.20 | 71.85/68.59 | 69.85/64.64 |
| AdaDetectGPT | 69.88/65.55 | 71.65/69.79 | 70.62/69.01 | 70.71/68.12 |
| Binoculars | 68.47/60.75 | 72.82/67.70 | 77.06/76.17 | 72.79/68.21 |
| RADAR | 69.94/62.37 | 74.27 / 71.07 | 79.20 /77.96 | 74.47 / 70.47 |
| ImBD | 70.63 / 66.00 | 72.21/68.33 | 73.16/68.81 | 72.00/67.72 |
| DNA-DetectLLM | 67.75/58.71 | 74.26/69.40 | 77.96/ 79.71 | 73.32/69.27 |
| Method | Zh-Law | Russian | Arabic | Urdu | Bulgarian | Avg. |
|---|---|---|---|---|---|---|
| Fast-DetectGPT | 82.54/90.48 | 69.45/69.44 | 88.25/88.25 | 74.21/80.06 | 71.86/74.38 | 77.26/80.52 |
| AdaDetectGPT | 66.89/54.75 | 67.27/60.27 | 72.72/74.82 | 97.01/99.57 | 78.84/85.04 | 76.55/74.89 |
| Binoculars | 88.24 / 93.38 | 66.86/67.44 | 67.61/68.13 | 72.72/76.73 | 70.67/73.63 | 73.22/75.86 |
| RADAR | 87.97/85.35 | 66.76/59.12 | 66.67/62.21 | 66.67/54.00 | 68.98/67.53 | 71.41/65.64 |
| ImBD | 70.46/50.47 | 81.77 / 87.33 | 90.80 / 96.82 | 99.50/99.96 | 88.72 / 95.16 | 86.25 / 85.95 |
| DNA-DetectLLM | 86.74/92.43 | 66.89/63.62 | 66.69/56.04 | 77.87/84.42 | 69.30/70.92 | 73.50/73.49 |
| Method | Rewrite | Polish | Expand | Avg. |
|---|---|---|---|---|
| Panel A: Component ablation | ||||
| MCRP | 90.64 | 94.83 | 96.43 | 93.97 |
| w/o Sensitivity | 86.30 | 91.68 | 94.82 | 90.94 |
| w/o Coupling | 86.90 | 92.44 | 94.57 | 91.30 |
| Panel B: Null-condition variants | ||||
| Only Bidirectional Alignment | 85.11 | 91.41 | 94.30 | 90.27 |
| Probe Model | Rewrite | Polish | Expand | Avg |
|---|---|---|---|---|
| Llama-2-7B | 84.08/84.48 | 89.73/92.44 | 92.26/94.91 | 88.69/90.61 |
| Gemma-3-9B | 84.58/88.06 | 91.52/94.14 | 94.63 / 97.01 | 90.24/93.07 |
| Qwen3-8B | 86.49/88.24 | 88.69/91.76 | 91.11/93.63 | 88.76/91.21 |
| Llama-3-8B | 80.09/79.85 | 84.84/87.10 | 86.22/89.68 | 83.72/85.54 |
| Falcon-7B | 88.70 / 92.12 | 91.83 / 94.55 | 92.01/95.62 | 90.85 / 94.10 |
| Phi-3.5-mini-3.8B | 90.64/93.78 | 94.83/96.74 | 96.43/98.22 | 93.97/96.24 |
AI-Generated Text Detection
AI-generated text detection develops methods to distinguish human-written, machine-generated, and AI-edited text.
See all 171 papers