Agents Can Use Base Models to Evade AI Detection

Bhuwan Dhingra; Danish Pruthi

cs.CLSep 25, 2026

Agents Can Use Base Models to Evade AI Detection

Authors: Bhuwan Dhingra, Danish Pruthi

Organizations: Duke University · Indian Institute of Science

Abstract

We show that coding agents equipped with a base language model can successfully assemble responses from its samples to evade detection. Base models have been shown to evade commercial detectors, however, prior "humanization" techniques rely on using these models to paraphrase AI outputs over several iterations, which invariably results in semantic drift. In contrast, equipping coding agents to directly orchestrate the writing process by stitching text samples from a base model allows it to produce outputs that are coherent, task-specific and generally high quality. We find that Claude Opus 5 operating in a Claude Code harness effectively orchestrates a local 32B parameter OLMo-2 base LM and sacrifices little task accuracy across benchmarks spanning creative writing, factual grounding, health QA and instruction following, while using up to 90% base LM tokens. Responses constructed in this manner reduce the effectiveness of both post-hoc detectors (Pangram v4 detection rate drops from 77% to 24%) and soft watermarking applied a priori to the agent's generations (down to a simulated 10% detection at low FPR). While effective, this evasion requires a significantly larger number of input and output tokens from the agent, increasing the dollar cost per query up to 30x at API-pricing. Overall, this work demonstrates the effectiveness of a new class of adversarial attacks against AI text detection, and urges post-hoc detection providers to include outputs of base models in their training.

Figures & tables

Figure 1: Task accuracy and detection rate trade-off. HALO evades Pangram with minimal drop in quality.

Figure 2: System Overview. ( Left ) Given a writing task (IFEval), an LLM follows instructions precisely (exclude the words “reschedule” or “free”) but is detected as AI by Pangram. Humanization via paraphrasing ( Xu et al., 2026 ) evades detection, but its output fails to follow task instructions. ( Right ) Our proposed HALO framework: a coding agent (Claude Opus 5) is given access to a base LM (OLMo-2-32B) as a tool which it can prompt to generate continuations. The agent assembles its response by generating pointers (as <copy> tags) to the stored base LM text, which are spliced into the final output, evading detection while following task instructions.

	Mythos			FACTS			IFEval			HealthBench			Average
System	Qual. $\uparrow$	Pan. $\downarrow$	Len.	Qual. $\uparrow$	Pan. $\downarrow$	Len.	Qual. $\uparrow$	Pan. $\downarrow$	Len.	Qual. $\uparrow$	Pan. $\downarrow$	Len.	Qual. $\uparrow$	Pan. $\downarrow$
Baselines
Agent	1.000	1.000	534	0.823	0.130	517	0.960	0.965	277	0.703	1.000	644	0.871	0.774
+prompted	1.000	1.000	532	0.672	0.308	474	0.970	0.970	332	0.638	1.000	507	0.820	0.820
+detector †	1.000	0.960	548	0.683	0.166	428	1.000	0.866	406	0.584	1.000	650	0.817	0.748
Non-Agent	1.000	1.000	530	0.720	0.223	348	0.890	0.963	206	0.670	1.000	410	0.820	0.796

Table 1: Quality-Detection Trade-off Against Pangram. Task-specific quality (Qual.), Pangram-4 fraction_AI scores (Pan.) and median response length (Len.) across all four benchmarks, evaluated on

n{=}100

random examples each (

\dagger

n{=}30

). All methods use Claude-Opus-5 to generate the responses, except Frankentext which uses Gemini-2.5-Pro. HALO and the Agent baselines operate within the Claude Agent SDK harness, whereas Non-Agent directly prompts the model via the API. The paraphrasing attacks operate on top of the Agent outputs. On FACTS and IFEval, HALO (

p=0.90

) evades Pangram effectively while maintaining quality. On Mythos and HealthBench, there is a trade-off: evasion works better as

p

increases, but also leads to a drop in quality. Paraphrasing (HIP) provides a modest drop in detection, but leads to catastrophic decrease in quality for FACTS and IFEval. Prompting the Agent to generate human-like text (‘+prompted’) or giving it access to Pangram as a tool (‘+detector’) doesn’t lead to meaningful evasion, and leads to a drop in quality for FACTS and HealthBench.

Figure 3: Watermarking Evasion. ROC curves for the baseline Agent and HALO variants for watermark detection using LeftHash (Top) and WinMax (Bottom) schemes from Kirchenbauer et al. (2024) . The detection rates are computed/simulated using the next-token probabilities from Qwen-3.5-27B. The vertical lines and labeled TPRs correspond to commonly used FPRs in the literature.

h

is the number of context tokens used for computing the green-list hash. At

p=0.90

, the detection rate drops to nearly

0

at a low FPR for all benchmarks except HealthBench, where it drops to

0.40

and

0.28

for the two schemes.

Figure 4: Source pool of the copied tokens by benchmark and copy target

p

	Mythos				FACTS				IFEval				HealthBench
System	In	Out	$	S	In	Out	$	S	In	Out	$	S	In	Out	$	S
Baselines
Agent	3.4k	1.7k	0.062	0	15.8k	2.2k	0.110	0	4.3k	2.0k	0.075	0	4.6k	4.1k	0.134	0
Non-Agent	170	1.2k	0.032	0	3.7k	1.0k	0.044	0	303	813	0.022	0	516	1.8k	0.047	0
HALO (w/ OLMo-2-32B base LM)
HALO, $p{=}0.60$	787k	21.2k	1.377	34	323k	16.6k	0.872	5	398k	13.8k	0.829	26	1,932k	42.9k	2.916	85

Table 2: Cost of Orchestration. Mean input tokens (In), output tokens (Out), US dollar cost ($) and base-LM candidate generations drawn (S) per generated response. Dollar estimates only account for Opus-5 inference based on list pricing as of 2026-09-04 . HALO’s detector evasion comes at a significant cost increase, both for running the orchestrator agent and for drawing samples from the base LM.

	$p{=}0.60$			$p{=}0.90$
Mechanism	Qual. $\uparrow$	Pan. $\downarrow$	$	Qual. $\uparrow$	Pan. $\downarrow$	$
Re-generation	0.847	0.501	0.96	0.827	0.226	1.37
Pointer	0.845	0.506	1.50	0.831	0.245	1.82

Table 3: Pointing vs re-generating copied text. Scores averaged across all four benchmarks. While significantly cheaper, re-generation comes without any guarantees of watermark evasion.

	Pangram 3.3.2			Pangram 4				EditLens		Binoculars
System	score $\downarrow$	AI $\downarrow$	Mix $\downarrow$	score $\downarrow$	AI $\downarrow$	Mix $\downarrow$	is-hm $\downarrow$	score $\downarrow$	AI $\downarrow$	score $\uparrow$	AI $\downarrow$	Len.
Agent	0.793	78%	6%	0.769	76%	7%	0%	0.418	55%	0.997	0%	495
Base LM (OLMo-2, raw)	0.059	4%	7%	0.113	7%	9%	3%	0.132	7%	0.953	2%	628
Frankentext (released)	0.141	9%	17%	0.180	10%	23%	12%	0.134	0%	0.983	0%	556
HALO, Re-generation ( $p{=}0.90$ )	0.036	3%	3%	0.226	15%	19%	12%	0.106	3%	0.980	2%	624
HALO, Pointer ( $p{=}0.90$ )	0.052	3%	8%	0.242	19%	17%	17%	0.112	6%	0.983	1%	509

Table 4: Alternative detectors. Average AI scores and detection rates (AI/Mix) from different detectors over

n{=}400

examples across all benchmarks (lower scores indicate more “human”, except Binoculars where lower is more “AI”). For responses flagged as AI, Pangram 4 also reports an is_humanized binary prediction for every window of text, we report the average here (is-hm).

	Mythos		HealthBench
System	Qual. $\uparrow$	Pan. $\downarrow$	Qual. $\uparrow$	Pan. $\downarrow$
Agent	1.000	1.000	0.668	1.000
HALO ( $p=0.90$ )
OLMo-2-7B	0.733	0.222	0.569	0.898
Qwen3-8B	0.700	0.437	0.555	0.820
Gemma-3-27B	0.833	0.331	0.624	0.649

Table 5: Base-LM comparison. Quality and Pangram-4 AI scores (

n{=}30

) as HALO orchestrates various base LMs. Larger base LMs generally provide better quality and evasion.

Figure 5: Prompts issued by the orchestrator to the base LM. For each benchmark we show hand-picked examples of agent strategies when prompting the base LM to extract text suitable for inclusion in the final response. Instead of few-shot prompting, the agent generally tries to create a fake document which fits the style and content needed for the response, and samples a continuation (light gray text) from the base LM for that document.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Base LM	Checkpoint	Len.	Pan. $\downarrow$	AI $\downarrow$	Mixed $\downarrow$
Llama-3-8B	meta-llama/Meta-Llama-3-8B	254	0.282	7	3
Qwen3-8B	Qwen/Qwen3-8B-Base	246	0.417	11	3
OLMo-2-7B	allenai/OLMo-2-1124-7B	250	0.267	8	0
top- $p{=}1.0$	allenai/OLMo-2-1124-7B	248	0.380	11	1
Gemma-3-12B	google/gemma-3-12b-pt	249	0.167	5	0
Gemma-3-27B	google/gemma-3-27b-pt	230	0.085	2	1

Appendix

Table 6: Pangram-4 scores of raw base LM continuations. Mean fraction_AI (Pan.), number of generations labeled AI or Mixed out of

n{=}30

, and median length in words (Len.). The prompts are sampled from those written by the HALO orchestrator agent and span all four benchmarks. Larger base LMs are detected less often.

Setting	Value
Orchestrator	claude-opus-5 , Claude Agent SDK v0.2.128, effort xhigh , adaptive thinking
Base LM	allenai/OLMo-2-0325-32B , served with vLLM on $2{\times}$ A6000 (bf16, context length $4096$ )
Sampling budget $B$	$30$ sample calls per example
Samples per call	chosen by the agent, $1$ – $8$ (default $1$ )
Temperature	chosen by the agent (default $1.1$ ), top- $p{=}0.95$
Max sample length	chosen by the agent, default $150$ words, up to $400$

Appendix

Table 7: HALO settings. All HALO runs in the main paper use these values, varying only the copy target

p

Figure 6: Pangram-4 scores by query for +detector. Mean Pangram-4 fraction_AI (solid) of the draft scored across turns in the agent’s trajectories and of the best draft seen so far (dashed), over the

30

examples per benchmark. Most drafts produced by the agent score close to

1.0

and hence the agent receives little signal to optimize against.

	Sonnet-4.5			GPT-4.1
System	Coh. $\uparrow$	Rel. $\uparrow$	Likert $\uparrow$	Coh. $\uparrow$	Rel. $\uparrow$	Likert $\uparrow$	Pan. $\downarrow$	Len.
Baselines
Agent	1.000 $\pm$ 0.00	0.990	5.60	1.000	1.000	6.99	1.000 $\pm$ 0.00	534
+prompted	1.000 $\pm$ 0.00	1.000	5.80	1.000	1.000	6.99	1.000 $\pm$ 0.00	532
+detector	1.000 $\pm$ 0.00	1.000	6.17	1.000	1.000	7.00	0.960 $\pm$ 0.06	548
Non-Agent	1.000 $\pm$ 0.00	1.000	5.08	1.000	1.000	6.97	1.000 $\pm$ 0.00	530

Appendix

Table 8: Mythos all metrics. Fraction of stories judged coherent (Coh.) and relevant to the writing prompt (Rel.), and the overall 1–7 Likert quality score, using the prompts of Pham et al. [2026] , with Sonnet-4.5 and GPT-4.1 as judges. The Sonnet-4.5 Likert score is averaged over

5

samples at

T{=}1

per story, and the GPT-4.1 judgments are a single sample at

T{=}0

. Pan. is the Pangram-4 fraction_AI , and Len. the median length in words.

		Grounded
System	Elig. $\uparrow$	GPT-5 $\uparrow$	Gemini $\uparrow$	Unadj. $\uparrow$	Adj. $\uparrow$	Pan. $\downarrow$	Len.
Baselines
Agent	0.990	0.697	0.960	0.828	0.823 $\pm$ 0.05	0.130 $\pm$ 0.05	517
+prompted	0.990	0.505	0.848	0.677	0.672 $\pm$ 0.07	0.308 $\pm$ 0.07	474
+detector	1.000	0.500	0.867	0.683	0.683 $\pm$ 0.12	0.166 $\pm$ 0.10	428
Non-Agent	0.990	0.590	0.860	0.725	0.720 $\pm$ 0.07	0.223 $\pm$ 0.07	348

Appendix

Table 9: FACTS grounding scores. Fraction of responses judged eligible (i.e., addressing the user request; Elig.), fraction judged fully grounded in the context document by each of the two judges (GPT-5 and Gemini-3.6-Flash), and the grounding score averaged over the two judges before (Unadj.) and after (Adj.) setting ineligible responses to

0

[ Cheng et al., 2025 ] . Adj. is the main quality metric. Pan. and Len. as in Table 8 .

	Prompt-level		Instruction-level
System	Strict $\uparrow$	Loose $\uparrow$	Strict $\uparrow$	Loose $\uparrow$	Pan. $\downarrow$	Len.
Baselines
Agent	0.960 $\pm$ 0.03	1.000	0.974	1.000	0.965 $\pm$ 0.04	277
+prompted	0.970 $\pm$ 0.03	0.980	0.981	0.987	0.970 $\pm$ 0.03	332
+detector	1.000 $\pm$ 0.00	1.000	1.000	1.000	0.866 $\pm$ 0.12	406
Non-Agent	0.890 $\pm$ 0.06	0.930	0.916	0.955	0.963 $\pm$ 0.04	206

Appendix

Table 10: IFEval all metrics. Fraction of prompts for which all instructions are followed (prompt-level) and fraction of individual instructions followed (instruction-level), under the strict and loose verifiers of Zhou et al. [2023] . Prompt-level strict accuracy is the main quality metric. Pan. and Len. as in Table 8 .

		Axis
System	Score $\uparrow$	Acc. $\uparrow$	Compl. $\uparrow$	Comm. $\uparrow$	Context $\uparrow$	Instr. $\uparrow$	Pan. $\downarrow$	Len.
Baselines
Agent	0.703 $\pm$ 0.04	0.782	0.586	0.623	0.620	0.709	1.000 $\pm$ 0.00	644
+prompted	0.638 $\pm$ 0.05	0.705	0.531	0.638	0.619	0.662	1.000 $\pm$ 0.00	507
+detector	0.584 $\pm$ 0.10	0.758	0.372	0.618	0.485	0.726	1.000 $\pm$ 0.00	650
Non-Agent	0.670 $\pm$ 0.05	0.736	0.571	0.632	0.664	0.710	1.000 $\pm$ 0.00	410

Appendix

Table 11: HealthBench breakdown. Overall rubric score (Score, the main quality metric), and the scores restricted to the rubric criteria for each axis: accuracy, completeness, communication quality, context awareness and instruction following [ Arora et al., 2025 ] . All scores are clipped means over examples of the per-example weighted rubric score; axis scores are averaged over the examples which have criteria on that axis. Rubrics are graded by GPT-4.1. Pan. and Len. as in Table 8 .

Figure 7: Watermark detection as a function of

\bar{\gamma}_{w}

. TPR at a fixed FPR (

10^{-3}

and

10^{-5}

, rows) as a function of the green-list probability

\bar{\gamma}_{w}

at agent generated positions, for the Agent baseline and HALO, with

\gamma=0.25

and

h=1

. Markers show the TPR at the estimate

\hat{\gamma}

from Qwen-3.5-27B, and the dotted line the upper bound

\gamma^{\ast}

for

\delta=2

Adversarial Text Attacks

Adversarial text attacks test how small or meaning-preserving text changes can fool language models, classifiers, and safety systems.

See all 78 papers

Base Models Look Human To AI Detectors
May 19, 2026Yixuan Even Xu, Ziqian Zhong, Aditi Raghunathan +2AI-Generated Text Detection Language Model Generation Evaluation
Carnegie Mellon University
EASE: Entropy-Adaptive Distribution Shaping for Evading AI-generated Text Detectors
Oct 7, 2026Jicheng Zhou, Kahim Wong, Jialong Wang +1Adversarial Text Attacks Robustness of Machine-Generated Text Detection
Department of Artificial Intelligence, Faculty of Information Science and Computing State Key Laboratory of Internet of Things for Smart City University of Macau, Macau, China
READER: Reasoning-Enhanced AI-Generated Text Detection
May 24, 2026Pingfan Su, Kai Ye, Shijin Gong +4AI-Generated Text Detection Large Language Model-Based Text Classification
Department of Statistics, London School of Economics and Political Science · School of Management, University of Science and Technology of China · School of Mathematics, University of Birmingham

May 19, 2026J/K move · Enter open · S save

Base Models Look Human To AI Detectors

Yixuan Even Xu, Ziqian Zhong, Aditi Raghunathan, Fei Fang+1

Carnegie Mellon University

AI-Generated Text Detection Language Model Generation Evaluation Instruction Tuning Robustness of Machine-Generated Text Detection

Open paper AskPDF arXiv

As AI-generated text enters the real-world at scale, institutions increasingly use commercial AI-text detectors, especially in education and academic-integrity workflows. We report a surprising empirical finding about such systems: when evaluated by GPTZero and Pangram, generated text from base models is often judged overwhelmingly human, whereas text generated by their instruction-tuned counterparts is not. Building on this observation, we propose Humanization by Iterative Paraphrasing (HIP), a detector-agnostic pipeline that minimally fine-tunes a base model into a paraphraser and applies it iteratively. Compared with the baselines we test, HIP yields a stronger trade-off between semantic preservation and detector evasion on commercial detectors. Across Llama-3 and Qwen-3 families, spanning model sizes from 0.6B to 70B, HIP consistently improves detector human-likeness. Our findings suggest that current detectors are tracking artifacts of instruction tuning and local context more than any invariant notion of machine-generated text. This, in turn, calls for detector designs that model these factors more explicitly.

Abstract

Adversarial Text Attacks

Explore similar work

Base Models Look Human To AI Detectors

EASE: Entropy-Adaptive Distribution Shaping for Evading AI-generated Text Detectors

READER: Reasoning-Enhanced AI-Generated Text Detection