StealthRL: Reinforcement Learning Paraphrase Attacks for Multi-Detector Evasion of AI-Text Detectors

Suraj Ranganath; Atharv Ramesh

cs.LGFeb 9, 2026

StealthRL: Reinforcement Learning Paraphrase Attacks for Multi-Detector Evasion of AI-Text Detectors

Authors: Suraj Ranganath, Atharv Ramesh

Organizations: University of California, San Diego

Abstract

AI-text detectors are increasingly used in high-stakes settings, but their robustness to meaning-preserving adversarial rewriting remains unclear. We introduce StealthRL, a reinforcement learning framework for stress-testing detectors with adaptive paraphrase attacks. StealthRL trains a paraphrase policy against a detector ensemble while enforcing semantic fidelity, then evaluates transfer to held-out detector families on the full filtered MAGE test pool. Across RoBERTa, Fast-DetectGPT, Binoculars, and MAGE, the resulting attack sharply degrades detection performance, driving mean TPR@1%FPR down to 0.024 while retaining nontrivial semantic overlap and readable prose. The attack also transfers to detectors not used during training, indicating a broader robustness failure rather than overfitting to a single target model. We complement the main results with detector-score analysis, repeated-generation stability checks, and automatic and judge-based quality evaluation. Our findings suggest that current AI-text detectors remain brittle under adaptive paraphrasing pressure and motivate robustness evaluation at deployment-relevant operating points. Our code and the trained model checkpoint are publicly available.

Figures & tables

Panel of Figure 1 — Figure 1: StealthRL training and evaluation pipeline. A paraphrase policy (Qwen3-4B with LoRA) is trained via GRPO against a two-detector ensemble (RoBERTa + Fast-DetectGPT) with semantic similarity reward. The trained policy is then evaluated against four detectors, including the held-out Binoculars and MAGE detectors, at the 1% FPR operating point.

Figure 2: Detection evasion results for methods M0–M5. (a) AUROC by detector. (b) Mean AUROC with standard-deviation bars across detectors. (c) TPR at 1% FPR. (d) Mean attack success rate. StealthRL (M2, teal) achieves extremely low TPR on Binoculars, Fast-DetectGPT, and RoBERTa, with MAGE remaining the most robust detector in the panel.

Figure 3: TPR@1%FPR heatmap. Lower is better for the attacker; M2 collapses TPR on three detectors and degrades held-out MAGE.

Appendix figures & tables23 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Binoc. AUC	Binoc. TPR	Binoc. ASR
M0	0.705	0.367	0.633
M1	0.593	0.158	0.842
M2 (Ours)	0.055	0.002	0.998
M3	0.511	0.051	0.949
M4	0.559	0.036	0.964
M5	0.593	0.000	1.000

Appendix

Table 1: Main results on the full filtered MAGE test pool at the 1% FPR operating point for Binoculars. Lower TPR@1%FPR/AUROC is better for the attacker; higher ASR is better. Bold indicates the best value per metric.

Method	FastDGPT AUC	FastDGPT TPR	FastDGPT ASR
M0	0.661	0.388	0.612
M1	0.588	0.148	0.852
M2 (Ours)	0.089	0.002	0.998
M3	0.536	0.092	0.908
M4	0.518	0.067	0.933
M5	0.080	0.000	1.000

Appendix

Table 2: Main results on the full filtered MAGE test pool at the 1% FPR operating point for Fast-DetectGPT. Lower TPR@1%FPR/AUROC is better for the attacker; higher ASR is better. Bold indicates the best value per metric.

Method	MAGE AUC	MAGE TPR	MAGE ASR
M0	0.983	0.843	0.157
M1	0.973	0.720	0.280
M2 (Ours)	0.891	0.089	0.911
M3	0.967	0.666	0.334
M4	0.969	0.651	0.349
M5	0.772	0.136	0.864

Appendix

Table 3: Main results on the full filtered MAGE test pool at the 1% FPR operating point for MAGE. Lower TPR@1%FPR/AUROC is better for the attacker; higher ASR is better. Bold indicates the best value per metric.

Method	RoBERTa AUC	RoBERTa TPR	RoBERTa ASR	Mean TPR	Mean ASR
M0	0.806	0.203	0.797	0.450	0.550
M1	0.778	0.111	0.889	0.284	0.716
M2 (Ours)	0.691	0.002	0.998	0.024	0.976
M3	0.651	0.058	0.942	0.217	0.783
M4	0.627	0.056	0.944	0.203	0.797
M5	0.630	0.001	0.999	0.034	0.966

Appendix

Table 4: Main results on the full filtered MAGE test pool at the 1% FPR operating point, part II. Lower TPR@1%FPR/AUROC is better for the attacker; higher ASR is better. This panel reports RoBERTa and the mean TPR@1%FPR / mean ASR aggregates across the four-detector panel. Bold indicates the best value per metric.

Method	Binoc. TPR@5%	FastDGPT TPR@5%	MAGE TPR@5%	RoBERTa TPR@5%	Mean TPR@5%
M0	0.481	0.451	0.934	0.397	0.566
M1	0.269	0.252	0.876	0.294	0.423
M2 (Ours)	0.005	0.003	0.272	0.022	0.076
M3	0.135	0.180	0.846	0.159	0.330
M4	0.126	0.130	0.839	0.158	0.313
M5	0.004	0.000	0.238	0.027	0.067

Appendix

Table 5: Supplementary TPR@5%FPR results on the full filtered MAGE test pool. Lower is better for the attacker. Thresholds are recalibrated on the 15,310 human evaluation samples at the 5% false-positive operating point. Bold indicates the lowest TPR@5%FPR per column.

Run	Mean AUROC	Mean TPR	Mean ASR	Binoc.	Fast-DGPT	MAGE	RoBERTa
Original M2	0.432	0.024	0.976	0.002	0.002	0.089	0.002
Seed 4201	0.432	0.025	0.975	0.002	0.002	0.093	0.002
Seed 4202	0.433	0.024	0.976	0.002	0.002	0.091	0.002
Mean $\pm$ SD	0.432 $\pm$ 0.001	0.024 $\pm$ 0.001	0.976 $\pm$ 0.001	0.002 $\pm$ 0.000	0.002 $\pm$ 0.000	0.091 $\pm$ 0.002	0.002 $\pm$ 0.000

Appendix

Table 6: Full-pool stochastic repeat analysis for StealthRL (M2) at temperature 1.0 and top-p 0.9. All runs use one generation per input and no reranking; thresholds are the same 1% FPR thresholds calibrated on human evaluation samples. Detector columns report TPR@1%FPR.

Method	E5 Sim.	BERTScore	NLI	PPL	Edit Rate	Valid	Quality	Similarity
M0	1.000	1.000	0.991	26.662	0.000	1.000	—	—
M1	0.974	0.926	0.967	24.596	0.779	1.000	3.780	4.058
M2 (Ours)	0.901	0.868	0.316	148.686	0.835	1.000	2.514	2.644
M3	0.973	0.934	0.965	28.338	0.782	1.000	3.770	4.024
M4	0.958	0.902	0.829	18.909	0.861	0.998	3.152	3.704
M5	0.923	0.773	0.847	146.430	0.634	1.000	1.972	2.920

Appendix

Table 7: Quality and similarity metrics. E5 Sim., BERTScore, and NLI are computed on the full filtered evaluation pool. Quality and Similarity are mean Likert scores (1–5) from a gpt-5-nano judge on 500 matched samples for methods M1–M5; M0 reports automatic metrics only. Higher is better for E5, BERTScore, NLI, Valid, Quality, and Similarity. Lower PPL indicates lower GPT-2 language-model surprise, while Edit Rate is a magnitude-of-change diagnostic rather than a quality-positive score.

Original (M0)	StealthRL (M2)
During cardio the heart increases its workload and all the body’s other systems adjust to help support that endeavor. The blood vessels dilate, the muscles do their best to help pump blood back to the heart, and the lungs work harder to take in oxygen and remove waste gases like carbon dioxide.	In cardio, the heart ramps up its workload, prompting the body’s other systems to adapt and assist. Blood vessels widen, muscles strive to push blood back to the heart, and lungs intensify their efforts to absorb oxygen and expel waste gases such as carbon dioxide.
The engine has to endure the torque of powering two axels and a drive shaft generally the transfer casing connects to the drive shaft with a ujoint and same for power distribution. The electricity goes into the battery first then is sent to the alternator where it generates voltage that powers all other electrical components like the lights, radio.	The engine must handle the torque from two axles and a drive shaft, typically linked via a universal joint in the transfer casing for power distribution. Electricity first charges the battery, then flows to the alternator, which converts it into voltage to power essential electrical systems such as lights and the radio.

Appendix

Table 8: Representative paraphrases from the filtered MAGE test split (health and automotive domains).

Original (M0)	StealthRL (M2)
Machine learning algorithms have become increasingly sophisticated in recent years, with deep neural networks achieving state-of-the-art performance across a wide range of tasks including image classification, natural language processing, and reinforcement learning.	Recent advances in machine learning have led to highly sophisticated algorithms. Deep neural networks now achieve top performance in diverse areas, from classifying images and processing natural language to learning through reinforcement-based approaches.
The process of photosynthesis converts light energy into chemical energy that can be stored in glucose molecules. This process occurs in the chloroplasts of plant cells and involves two main stages: the light-dependent reactions and the Calvin cycle.	Photosynthesis transforms light energy into chemical energy, storing it within glucose molecules. Taking place in plant cell chloroplasts, this process unfolds in two key stages: light-dependent reactions followed by the Calvin cycle.

Appendix

Table 9: Representative paraphrases (science and technology domains).

Original (M0)	StealthRL (M2)
The global economy faces several interconnected challenges, including rising inflation, supply chain disruptions, and geopolitical tensions. Central banks worldwide have responded by tightening monetary policy, raising interest rates to combat price pressures.	Several intertwined challenges confront the global economy: climbing inflation, supply chain breakdowns, and geopolitical friction. In response, central banks around the world have tightened monetary policy and hiked interest rates to curb rising prices.
Renewable energy sources such as solar and wind power have seen dramatic cost reductions over the past decade, making them competitive with traditional fossil fuels in many markets. Government incentives and technological improvements continue to drive adoption rates higher.	Solar and wind power costs have fallen dramatically over the last decade, allowing renewables to compete with fossil fuels across numerous markets. Ongoing government incentives and technological progress continue pushing adoption rates upward.

Appendix

Table 10: Representative paraphrases (economics and energy domains).

Method	Cand.	Generator / transform	Decoding / seed policy
M0	1	Original AI text	Deterministic
M1	1	Qwen/Qwen3-4B-Instruct-2507 via vLLM	Temp. 0.9, top-p 0.95; vLLM default engine seed
M2 (ours)	1	StealthRL LoRA sampler on Qwen/Qwen3-4B-Instruct-2507	Temp. 1.0, top-p 0.9; original run plus repeat seeds 4201/4202
M3	4	Qwen/Qwen3-4B-Instruct-2507 via vLLM	Temp. 1.0, top-p 0.95; vLLM default engine seed
M4	1	authormist/authormist-originality via vLLM	Temp. 0.7, top-p 0.9; vLLM default engine seed
M5	1	SilverSpeak homoglyph transform	Substitution rate 0.10; seed 42

Appendix

Table 11: Per-method generation protocol. Table F reports the M2 training/evaluation configuration.

Method	Selection and attack-time query budget
M0	No generation and no detector queries.
M1	No reranking; 0 detector queries; 1 generation call.
M2 (ours)	No reranking; 0 evaluation-time detector queries; 1 policy generation call.
M3	Select lowest RoBERTa score among candidates with E5 $\geq 0.90$ ; fallback to lowest RoBERTa score; 4 RoBERTa queries and 4 E5 scores per sample.
M4	No reranking; 0 detector queries; 1 generation call.
M5	Deterministic character substitution; 0 detector queries and 0 LLM calls.

Appendix

Table 12: Per-method selection and query budget. The budget counts detector calls used to construct an attack output, not the post-hoc evaluation pass in which every final output is scored once by each evaluation detector.

Model & LoRA
Parameter	Value
Base model	Qwen/Qwen3-4B-Instruct-2507
LoRA rank	32
LoRA alpha	32
LoRA dropout	0.05
Training

Appendix

Table 13: Complete hyperparameters and configuration for reproducibility.

Reward setting	Detector prob.	Evasion rate	E5 Sim.	Total reward
$\alpha{=}1.0,\beta{=}0.0$	0.331	0.669	0.978	-0.136
$\alpha{=}1.0,\beta{=}0.1$ (selected)	0.226	0.774	0.970	0.127
$\alpha{=}1.0,\beta{=}0.3$	0.327	0.673	0.968	0.200
$\alpha{=}0.0,\beta{=}1.0$	0.359	0.641	0.964	0.970

Appendix

Table 14: Training-time reward-weight ablation on the small MAGE development setup. Detector probability is the mean training-ensemble probability of AI generation, so lower detector probability and higher evasion rate are better for the attacker. This diagnostic sweep selected the nonzero but small semantic weight

\beta{=}0.1

Selection run	Mean AUROC	Mean TPR@1%FPR	Mean ASR	E5 Sim.	PPL	Judge quality
No semantic reward / low-fidelity checkpoint	0.460	0.089	0.911	0.909	91.5	1.63
$\alpha{=}1.0,\beta{=}0.1$ (selected)	0.546	0.050	0.950	0.973	31.7	4.10
$\alpha{=}1.0,\beta{=}0.1$ (replicate)	0.554	0.063	0.937	0.974	30.5	–

Appendix

Table 15: Checkpoint-selection ablations on a fixed 1,000-human / 1,000-AI MAGE selection set. Metrics are averaged across RoBERTa, Fast-DetectGPT, and Binoculars; lower AUROC and TPR are better for the attacker, while higher ASR and quality metrics are better. The full paper results are recomputed separately on the full filtered MAGE test pool.

Method	AUROC [95% CI]	TPR@1%FPR [95% CI]	ASR [95% CI]
M0 No Attack	0.806 [0.801, 0.810]	0.203 [0.196, 0.209]	0.797 [0.791, 0.804]
M1 Simple Para.	0.778 [0.773, 0.783]	0.111 [0.106, 0.116]	0.889 [0.884, 0.894]
M2 StealthRL (Ours)	0.691 [0.685, 0.697]	0.002 [0.002, 0.003]	0.998 [0.997, 0.998]
M3 Adv. Para.	0.651 [0.645, 0.657]	0.058 [0.055, 0.062]	0.942 [0.938, 0.945]
M4 AuthorMist	0.627 [0.620, 0.633]	0.056 [0.052, 0.059]	0.944 [0.941, 0.948]
M5 Homoglyph	0.630 [0.624, 0.636]	0.001 [0.000, 0.001]	0.999 [0.999, 1.000]

Appendix

Table 16: RoBERTa results with 95% bootstrap confidence intervals. Bold indicates the best attacker value per metric.

Method	AUROC [95% CI]	TPR@1%FPR [95% CI]	ASR [95% CI]
M0 No Attack	0.661 [0.655, 0.668]	0.388 [0.380, 0.396]	0.612 [0.604, 0.620]
M1 Simple Para.	0.588 [0.582, 0.595]	0.148 [0.143, 0.154]	0.852 [0.846, 0.857]
M2 StealthRL (Ours)	0.089 [0.086, 0.092]	0.002 [0.001, 0.002]	0.998 [0.998, 0.999]
M3 Adv. Para.	0.536 [0.530, 0.543]	0.092 [0.087, 0.097]	0.908 [0.903, 0.913]
M4 AuthorMist	0.518 [0.511, 0.524]	0.067 [0.064, 0.072]	0.933 [0.928, 0.936]
M5 Homoglyph	0.080 [0.078, 0.083]	0.000 [0.000, 0.000]	1.000 [1.000, 1.000]

Appendix

Table 17: Fast-DetectGPT results with 95% bootstrap confidence intervals. Bold indicates the best attacker value per metric.

Method	AUROC [95% CI]	TPR@1%FPR [95% CI]	ASR [95% CI]
M0 No Attack	0.705 [0.699, 0.712]	0.367 [0.360, 0.375]	0.633 [0.625, 0.640]
M1 Simple Para.	0.593 [0.587, 0.599]	0.158 [0.152, 0.163]	0.842 [0.837, 0.848]
M2 StealthRL (Ours)	0.055 [0.052, 0.058]	0.002 [0.001, 0.003]	0.998 [0.997, 0.999]
M3 Adv. Para.	0.511 [0.504, 0.517]	0.051 [0.048, 0.055]	0.949 [0.945, 0.952]
M4 AuthorMist	0.559 [0.553, 0.566]	0.036 [0.033, 0.038]	0.964 [0.962, 0.967]
M5 Homoglyph	0.593 [0.587, 0.599]	0.000 [0.000, 0.000]	1.000 [1.000, 1.000]

Appendix

Table 18: Binoculars results with 95% bootstrap confidence intervals. Bold indicates the best attacker value per metric.

Method	AUROC [95% CI]	TPR@1%FPR [95% CI]	ASR [95% CI]
M0 No Attack	0.983 [0.981, 0.984]	0.843 [0.837, 0.849]	0.157 [0.151, 0.163]
M1 Simple Para.	0.973 [0.972, 0.975]	0.720 [0.713, 0.727]	0.280 [0.273, 0.287]
M2 StealthRL (Ours)	0.891 [0.888, 0.895]	0.089 [0.084, 0.093]	0.911 [0.907, 0.916]
M3 Adv. Para.	0.967 [0.965, 0.969]	0.666 [0.658, 0.674]	0.334 [0.326, 0.342]
M4 AuthorMist	0.969 [0.967, 0.971]	0.651 [0.644, 0.658]	0.349 [0.342, 0.356]
M5 Homoglyph	0.772 [0.767, 0.777]	0.136 [0.130, 0.141]	0.864 [0.859, 0.870]

Appendix

Table 19: MAGE detector results with 95% bootstrap confidence intervals. Bold indicates the best attacker value per metric.

Token range	100–500 tokens
Stage	Purpose	Human Samples	AI Samples
Train	RL fine-tuning (GRPO)	0	10,000
Test	Detection evaluation	15,310	14,656
Likert eval	Quality scoring (per method)	0	500
Source	MAGE benchmark Li et al. (2024)
Domains	Multiple (essays, Q&A, creative writing)

Appendix

Table 20: Dataset statistics for training and evaluation. The Likert evaluation uses 500 AI samples per attack method (M1–M5) with matched sample IDs.

Figure 4: Detector score distributions for AI samples across methods (one panel per detector). StealthRL (M2) and Homoglyph (M5) push scores below the detection threshold, explaining their near-zero TPR@1%FPR.

Figure 5: Evasion–quality tradeoff at the 1% FPR operating point. Each point represents a method, plotted by E5 semantic similarity (x-axis) against mean ASR@1%FPR (y-axis). Top-right is ideal (high similarity, high attack success). The dashed lines mark the 0.90 similarity floor and the random-detector ASR baseline at 1% FPR, and the solid line shows the Pareto frontier.

Figure 6: LLM-based quality evaluation on 500 matched samples. Panel (a) shows linguistic quality and panel (b) semantic similarity; higher is better. The dashed line marks the neutral midpoint (3.0).

Robustness of Machine-Generated Text Detection

Robustness of machine-generated text detection studies whether AI-text detectors remain reliable across shifts, edits, languages, and adversarial attacks.

See all 90 papers

Detector-Evasive LLM Paraphrasing via Constrained Policy Optimization
May 29, 2026Mingyi Wang, Zhuoer Shen, Yuheng Bu +1Constrained RL Text Generation
School of ECEE, Arizona State University · Department of Computer Science, University of California, Santa Barbara
Paraphrasing Attack Resilience of Various AI-Generated Text Detection Methods
May 14, 2026Andrii Shportko, Inessa VerbitskyAI-Generated Text Detection Robustness of Machine-Generated Text Detection
Northwestern University · Evanston, IL, USA
Attacking Only Half the Data: Two-Sided Evaluation of AI-Text Detector Robustness
Oct 13, 2026Apu Chandraw Shill, Rabindra Nath NandiAI-Generated Text Detection Robustness of Machine-Generated Text Detection
Independent Researcher

May 29, 2026J/K move · Enter open · S save

Detector-Evasive LLM Paraphrasing via Constrained Policy Optimization

Mingyi Wang, Zhuoer Shen, Yuheng Bu, Shaofeng Zou

School of ECEE, Arizona State University · Department of Computer Science, University of California, Santa Barbara

Constrained RL Text Generation Adversarial Text Attacks

Open paper AskPDF arXiv

AI-text detectors are vulnerable to paraphrasing and detector-guided paraphrasing attacks, but existing detector-evasion methods often lack precise control over semantic preservation. In particular, optimizing directly for detector evasion can degrade fine-grained semantics, whereas scalarized reward designs provide only indirect, weight-sensitive control over the evasion-semantics trade-off. We address this limitation by formulating detector-evasive LLM paraphrasing as a Constrained Markov Decision Process, where detector evasion is the primary objective and semantic preservation is enforced as an explicit constraint. We propose Detector Evasion Policy Optimization (DEPO), a Lagrangian primal-dual reinforcement learning algorithm with a novel GRPO-style group-based policy update. DEPO adaptively balances semantic preservation and detector evasion during training, enabling the policy to improve attack success within a prescribed semantic-preservation region. Experiments on MAGE, M4, RAID, and peer-review datasets, evaluated against MAGE, RoBERTa, RADAR, Binoculars, and Fast-DetectGPT detectors, show that DEPO achieves strong detector evasion while precisely satisfying the semantic preservation constraint. DEPO also exhibits cross-domain, cross-detector, and prompt-level robustness.

Abstract

Robustness of Machine-Generated Text Detection

Explore similar work

Detector-Evasive LLM Paraphrasing via Constrained Policy Optimization

Paraphrasing Attack Resilience of Various AI-Generated Text Detection Methods

Attacking Only Half the Data: Two-Sided Evaluation of AI-Text Detector Robustness