Robust Detection of LLM-Generated Text under Contamination

Jiaxun Li; Saptarshi Chakraborty; Ambuj Tewari

stat.MLSep 24, 2026

Robust Detection of LLM-Generated Text under Contamination

Authors: Jiaxun Li, Saptarshi Chakraborty, Ambuj Tewari

Organizations: Department of Statistics, University of Michigan

Abstract

We study the detection of LLM-generated text under editing and contamination. Modeling human and machine text as finite-order Markov processes with Huber contamination, we characterize an exact boundary for reliable detection under our assumptions. Detection is impossible when contamination is sufficiently large relative to clean-source separation. Below this boundary, a collection of clipped likelihood-ratio tests achieves vanishing worst-case errors. This construction motivates clipping as a simple modification of existing statistical detectors. For a broad class of additive scores, we identify conditions under which the clipped test is consistent while the raw test's worst-case power tends to zero. We evaluate seven detectors across three datasets and three generation models, and on the RAID benchmark. Clipping improves robustness in both studies, with gains varying across detectors and contamination settings. For example, at a target false-positive rate of 5%, clipping improves the log-likelihood--log-rank ratio (LRR) detector's true-positive rate by a median of 8.3 percentage points in the controlled study and 2.1 and 4.3 points in rate- and attack-specific RAID evaluations, respectively.

Figures & tables

Figure 1: Possibility and impossibility of robust detection.

Figure 2: Lower clipping of token scores. Scores below the floor are raised to it; all other scores remain unchanged. Larger scores favor machine-generated text.

Detector	Raw statistic	Orientation and clipping
Log likelihood	$\overline{\log p_{i}}$	Keep sign; clip token scores from below
Rank	$\overline{r_{i}}$	Reverse sign; clip oriented scores from below
Log rank	$\overline{\log r_{i}}$	Reverse sign; clip oriented scores from below
LRR	$\overline{-\log p_{i}}/\overline{\log r_{i}}$	Keep ratio’s sign; clip both contributions from above
Entropy	$\overline{h_{i}}$	Orient using tuning data; clip oriented scores from below
Entropy gap	$\overline{-\log p_{i}-h_{i}}$	Reverse sign; clip oriented scores from below

Table 1: Detector statistics and clipping operations. All bounds are applied to token-level quantities before averaging.

	0% replacement		20% replacement
	Clean		Random		Tail
Detector	Raw	Gain	Raw	Gain	Raw	Gain
Log likelihood	62.6	-2.9	9.2	+7.6	21.0	+7.7
Rank	70.7	-5.3	0.1	+21.4	0.3	+34.0
Log rank	67.0	-3.1	12.1	+9.5	24.8	+9.1
LRR	71.2	-3.5	22.3	+26.2	30.0	+21.4

Table 2: Controlled-study TPR at target 5% FPR, averaged equally across nine settings. Raw: TPR (%); gain: clipped minus raw (percentage points). Replacement rates are requested.

Figure 3: Binoculars TPR across nine settings versus requested replacement rate. Raw and clipped thresholds are separately calibrated at target 5% FPR.

	$0<\rho\leq 0.05$		$0.05<\rho\leq 0.1$		$0.1<\rho\leq 0.2$
Detector	Raw	Gain	Raw	Gain	Raw	Gain
Log likelihood	49.17	+0.32	55.62	+0.00	30.47	+0.00
Rank	29.48	+25.23	35.05	+26.40	24.87	+15.96
Log rank	53.24	+0.74	59.38	+0.68	35.79	+0.28
LRR	59.51	+2.28	64.48	+4.21	47.60	+0.73
Entropy	26.11	+0.09	35.07	+0.43	18.50	+0.01

Table 3: Rate-specific RAID TPR at target 5% FPR. Raw: TPR (%); gain: clipped minus raw (percentage points). Test counts: 19,894 / 4,682 / 7,137.

	Case change		Paraphrase		Alternative spelling
Detector	Raw	Gain	Raw	Gain	Raw	Gain
Log likelihood	44.96	+1.15	27.91	+0.00	59.22	+0.00
Rank	22.02	+31.37	27.89	+7.79	39.50	+23.83
Log rank	50.15	+2.21	31.87	-0.10	62.11	+0.19
LRR	59.16	+3.52	38.80	+9.30	65.04	+4.87
Entropy	24.20	+0.43	11.87	+0.10	33.52	+0.54

Table 4: Attack-specific RAID TPR at target 5% FPR, with 5,149 texts per attack. Raw: TPR (%); gain: clipped minus raw (percentage points).

Appendix figures & tables56 assets

Supplementary material from the paper’s appendix.

Appendix

Profile	Simpler sufficient inequality	Nonempty certified interval
Mild	$4/4$	$4/4$
Larger heterogeneity	$0/4$	$4/4$
Tail-heavy human control	$2/4$	$2/4$

Appendix

Table 5: Number of detectors satisfying each sufficient check, out of four, for every

K\in\{10,100\}

and

M\in\{500,1000\}

Figure 4: Estimated detection power as sequence length increases, for the mild profile with

M=1000

under the adaptive minimum-score attack. Blue solid curves show raw tests; orange dashed curves show clipped tests. Each detector uses its selected contamination level, and each test is calibrated to target a

5\%

false-positive rate. The top and bottom rows use

K=10

and

K=100

, respectively.

Attack	Description
Alternative spelling	Changes American spellings to British spellings.
Article deletion	Removes selected occurrences of a, an, or the.
Homoglyph	Replaces characters with visually similar Unicode characters.
Insert paragraphs	Adds paragraph breaks between selected sentences.
Number	Replaces digits in selected numbers with sampled digits.
Paraphrase	Rewrites the text using DIPPER-11B.

Appendix

Table 6: RAID attacks and the changes they make.

Cell	Detector	Clean TPR		Actual FPR		Area change
		Raw	Clip	Raw	Clip	Random	Tail
XSum / Granite 8B	Log likelihood	79.1	66.5	4.7	4.2	+1.3	+1.8
	Rank	99.8	80.9	3.8	3.8	+8.7	+15.3
	Log rank	96.0	83.5	4.7	4.0	+1.6	+2.1
	LRR	99.4	90.0	5.2	4.6	+22.3	+21.9
	Entropy	8.0	8.2	4.3	4.6	+0.5	+0.5

Appendix

Table 11

Cell	Detector	Clean TPR		Actual FPR		Area change
		Raw	Clip	Raw	Clip	Random	Tail
XSum / Granite 8B	Log likelihood	44.5	36.3	0.9	1.1	+0.5	+0.8
	Rank	99.6	51.3	0.5	0.9	+0.7	+2.9
	Log rank	74.9	51.9	1.1	0.9	-0.5	-0.6
	LRR	94.1	74.1	0.8	0.7	+12.5	+14.9
	Entropy	1.4	1.4	0.7	0.7	+0.3	+0.3

Appendix

Table 12

Detector	Raw FPR (95% CI)	Clip FPR (95% CI)	Clean $\Delta$ TPR (95% CI)
XSum / Granite 8B
Log likelihood	4.7 (3.8, 5.6)	4.2 (3.4, 5.1)	-12.7 (-14.2, -11.2)
Rank	3.8 (2.9, 4.7)	3.8 (2.9, 4.6)	-18.9 (-20.7, -17.2)
Log rank	4.7 (3.8, 5.5)	4.0 (3.2, 4.9)	-12.5 (-14.0, -11.1)
LRR	5.2 (4.2, 6.2)	4.6 (3.7, 5.5)	-9.4 (-10.6, -8.1)
Entropy	4.3 (3.5, 5.2)	4.6 (3.7, 5.5)	+0.2 (+0.1, +0.4)

Appendix

Table 13

Dataset	Model	Detector	$d$	Selected bound(s)
XSum	Granite 8B	Log likelihood	1	$L=-6.183$
XSum	Mistral 24B	Log likelihood	1	$L=-6.466$
XSum	Qwen 32B	Log likelihood	1	$L=-6.739$
SQuAD	Granite 8B	Log likelihood	1	$L=-7.076$
SQuAD	Mistral 24B	Log likelihood	1	$L=-6.036$
SQuAD	Qwen 32B	Log likelihood	1	$L=-0.523$

Appendix

Table 14

Figure 5: XSum: all seven methods at target 5% FPR.

Figure 6: SQuAD: all seven methods at target 5% FPR. Low values are retained; no test-selected direction reversal is applied.

Figure 7: WritingPrompts: all seven methods at target 5% FPR.

Figure 8: XSum: secondary target 1% FPR.

Figure 9: SQuAD: secondary target 1% FPR.

Figure 10: WritingPrompts: secondary target 1% FPR.

Condition	RAID paper	Our raw	95% CI	$\Delta$ (pp)
None	79.6	80.00	[78.91, 81.12]	+0.40
Paraphrase	80.3	80.83	[79.76, 81.92]	+0.53
Synonym	43.5	43.58	[42.26, 44.86]	+0.08
Misspelling	78.0	78.40	[77.28, 79.49]	+0.40
Homoglyph	37.7	37.56	[36.30, 38.82]	-0.14
Whitespace	70.1	70.09	[68.83, 71.26]	-0.01

Appendix

Table 11: Raw Binoculars TPR (%) compared with the published RAID results, at target 5% FPR.

Detector	Raw TPR	Clipped TPR	Change [95% CI]
Log likelihood	49.17	49.49	+0.32 [0.21, 0.44]
Rank	29.48	54.71	+25.23 [24.09, 26.41]
Log rank	53.24	53.98	+0.74 [0.54, 0.94]
LRR	59.51	61.79	+2.28 [1.72, 2.83]
Entropy	26.11	26.20	+0.09 [-0.20, 0.36]
Entropy gap	66.55	68.15	+1.60 [1.37, 1.84]

Appendix

Table 12: Rate-specific detection for

\rho\in(0,0.05]

, with 19894 attacked test texts.

Detector	Raw TPR	Clipped TPR	Change [95% CI]
Log likelihood	55.62	55.62	+0.00 [0.00, 0.00]
Rank	35.05	61.45	+26.40 [24.83, 27.98]
Log rank	59.38	60.06	+0.68 [0.32, 1.08]
LRR	64.48	68.69	+4.21 [3.36, 5.01]
Entropy	35.07	35.50	+0.43 [0.02, 0.85]
Entropy gap	71.64	72.77	+1.13 [0.75, 1.52]

Appendix

Table 13: Rate-specific detection for

\rho\in(0.05,0.1]

, with 4682 attacked test texts.

Detector	Raw TPR	Clipped TPR	Change [95% CI]
Log likelihood	30.47	30.47	+0.00 [0.00, 0.00]
Rank	24.87	40.83	+15.96 [15.00, 16.98]
Log rank	35.79	36.07	+0.28 [0.10, 0.48]
LRR	47.60	48.33	+0.73 [0.10, 1.33]
Entropy	18.50	18.51	+0.01 [-0.08, 0.11]
Entropy gap	46.77	47.79	+1.02 [0.56, 1.52]

Appendix

Table 14: Rate-specific detection for

\rho\in(0.1,0.2]

, with 7137 attacked test texts.

Detector	Raw TPR	Clipped TPR	Change [95% CI]
Log likelihood	27.59	27.59	+0.00 [0.00, 0.00]
Rank	25.62	33.27	+7.65 [6.32, 9.05]
Log rank	31.49	31.40	-0.09 [-0.34, 0.17]
LRR	37.36	39.28	+1.92 [1.07, 2.82]
Entropy	16.28	16.48	+0.20 [-0.06, 0.46]
Entropy gap	44.58	45.53	+0.95 [0.29, 1.59]

Appendix

Table 15: Rate-specific detection for

\rho\in(0.2,0.5]

, with 3490 attacked test texts.

Attack	Raw TPR	Clipped TPR	Change [95% CI]
Alternative spelling	59.22	59.22	+0.00 [0.00, 0.00]
Article deletion	49.99	49.99	+0.00 [0.00, 0.00]
Homoglyph	91.42	91.42	+0.00 [0.00, 0.00]
Paragraph insertion	56.69	56.69	+0.00 [0.00, 0.00]
Number change	52.34	53.06	+0.72 [0.49, 0.97]
Paraphrase	27.91	27.91	+0.00 [0.00, 0.00]

Appendix

Table 16: Log likelihood: attack-specific detection, with 5,149 test texts per attack.

Attack	Raw TPR	Clipped TPR	Change [95% CI]
Alternative spelling	39.50	63.33	+23.83 [22.57, 25.15]
Article deletion	36.63	56.09	+19.46 [18.14, 20.78]
Homoglyph	67.18	87.84	+20.66 [19.48, 21.81]
Paragraph insertion	38.98	60.61	+21.64 [20.31, 22.90]
Number change	37.19	56.79	+19.60 [18.31, 20.88]
Paraphrase	27.89	35.68	+7.79 [6.62, 8.93]

Appendix

Table 17: Rank: attack-specific detection, with 5,149 test texts per attack.

Attack	Raw TPR	Clipped TPR	Change [95% CI]
Alternative spelling	62.11	62.30	+0.19 [-0.02, 0.43]
Article deletion	54.44	54.67	+0.23 [0.00, 0.47]
Homoglyph	90.15	90.29	+0.14 [0.04, 0.25]
Paragraph insertion	59.68	60.01	+0.33 [0.14, 0.52]
Number change	55.64	56.24	+0.60 [0.21, 0.95]
Paraphrase	31.87	31.77	-0.10 [-0.25, 0.04]

Appendix

Table 18: Log rank: attack-specific detection, with 5,149 test texts per attack.

Attack	Raw TPR	Clipped TPR	Change [95% CI]
Alternative spelling	65.04	69.92	+4.87 [4.21, 5.57]
Article deletion	61.70	65.00	+3.30 [2.68, 3.94]
Homoglyph	76.42	81.98	+5.55 [4.89, 6.25]
Paragraph insertion	63.80	67.94	+4.14 [3.46, 4.82]
Number change	60.87	62.92	+2.06 [1.34, 2.84]
Paraphrase	38.80	48.11	+9.30 [8.47, 10.10]

Appendix

Table 19: LRR: attack-specific detection, with 5,149 test texts per attack.

Attack	Raw TPR	Clipped TPR	Change [95% CI]
Alternative spelling	33.52	34.06	+0.54 [0.12, 1.01]
Article deletion	24.92	24.68	-0.23 [-0.66, 0.17]
Homoglyph	88.35	88.60	+0.25 [-0.02, 0.52]
Paragraph insertion	33.89	34.45	+0.56 [0.12, 0.97]
Number change	30.67	31.07	+0.41 [-0.04, 0.85]
Paraphrase	11.87	11.96	+0.10 [-0.21, 0.41]

Appendix

Table 20: Entropy: attack-specific detection, with 5,149 test texts per attack.

Attack	Raw TPR	Clipped TPR	Change [95% CI]
Alternative spelling	72.54	73.39	+0.85 [0.54, 1.17]
Article deletion	68.89	68.89	+0.00 [0.00, 0.00]
Homoglyph	40.63	40.63	+0.00 [0.00, 0.00]
Paragraph insertion	70.40	70.40	+0.00 [0.00, 0.00]
Number change	67.92	70.36	+2.45 [1.96, 2.99]
Paraphrase	75.78	75.78	+0.00 [0.00, 0.00]

Appendix

Table 21: Entropy gap: attack-specific detection, with 5,149 test texts per attack.

Attack	Raw TPR	Clipped TPR	Change [95% CI]
Alternative spelling	78.71	79.03	+0.31 [0.12, 0.50]
Article deletion	74.66	74.62	-0.04 [-0.27, 0.21]
Homoglyph	37.56	37.56	+0.00 [0.00, 0.00]
Paragraph insertion	79.30	79.49	+0.19 [0.00, 0.39]
Number change	77.35	78.02	+0.66 [0.41, 0.93]
Paraphrase	80.83	80.83	+0.00 [0.00, 0.00]

Appendix

Table 22: Binoculars: attack-specific detection, with 5,149 test texts per attack.

Fitting group	Clean TPR	Change [95% CI]	Human FPR	Change [95% CI]
$(0,0.05]$	61.29/61.45	+0.16 [0.02, 0.29]	5.85/5.90	+0.06 [-0.06, 0.17]
$(0.05,0.1]$	61.29/61.29	+0.00 [0.00, 0.00]	5.85/5.85	+0.00 [0.00, 0.00]
$(0.1,0.2]$	61.29/61.29	+0.00 [0.00, 0.00]	5.85/5.85	+0.00 [0.00, 0.00]
$(0.2,0.5]$	61.29/61.29	+0.00 [0.00, 0.00]	5.85/5.85	+0.00 [0.00, 0.00]
Alternative spelling	61.29/61.29	+0.00 [0.00, 0.00]	5.85/5.85	+0.00 [0.00, 0.00]
Article deletion	61.29/61.29	+0.00 [0.00, 0.00]	5.85/5.85	+0.00 [0.00, 0.00]

Appendix

Table 23: Log likelihood: clean detection and false positives for each specific fit. Each entry Raw/Clip gives both operating points.

Fitting group	Clean TPR	Change [95% CI]	Human FPR	Change [95% CI]
$(0,0.05]$	47.91/64.71	+16.80 [15.42, 18.18]	5.54/5.92	+0.39 [-0.29, 1.13]
$(0.05,0.1]$	47.91/64.71	+16.80 [15.38, 18.20]	5.54/5.92	+0.39 [-0.33, 1.13]
$(0.1,0.2]$	47.91/65.35	+17.44 [16.14, 18.80]	5.54/5.57	+0.04 [-0.58, 0.66]
$(0.2,0.5]$	47.91/65.35	+17.44 [16.20, 18.70]	5.54/5.57	+0.04 [-0.58, 0.64]
Alternative spelling	47.91/65.35	+17.44 [16.12, 18.74]	5.54/5.57	+0.04 [-0.60, 0.68]
Article deletion	47.91/65.35	+17.44 [16.12, 18.70]	5.54/5.57	+0.04 [-0.62, 0.68]

Appendix

Table 24: Rank: clean detection and false positives for each specific fit. Each entry Raw/Clip gives both operating points.

Fitting group	Clean TPR	Change [95% CI]	Human FPR	Change [95% CI]
$(0,0.05]$	63.76/63.68	-0.08 [-0.37, 0.19]	5.98/5.85	-0.14 [-0.37, 0.08]
$(0.05,0.1]$	63.76/63.68	-0.08 [-0.37, 0.23]	5.98/5.85	-0.14 [-0.37, 0.10]
$(0.1,0.2]$	63.76/63.82	+0.06 [-0.12, 0.23]	5.98/5.83	-0.16 [-0.33, 0.02]
$(0.2,0.5]$	63.76/63.82	+0.06 [-0.10, 0.21]	5.98/5.83	-0.16 [-0.33, 0.02]
Alternative spelling	63.76/63.82	+0.06 [-0.12, 0.23]	5.98/5.83	-0.16 [-0.33, 0.02]
Article deletion	63.76/63.82	+0.06 [-0.10, 0.23]	5.98/5.83	-0.16 [-0.33, 0.00]

Appendix

Table 25: Log rank: clean detection and false positives for each specific fit. Each entry Raw/Clip gives both operating points.

Fitting group	Clean TPR	Change [95% CI]	Human FPR	Change [95% CI]
$(0,0.05]$	66.05/69.86	+3.81 [3.11, 4.58]	5.55/5.54	-0.02 [-0.49, 0.49]
$(0.05,0.1]$	66.05/69.86	+3.81 [3.13, 4.55]	5.55/5.54	-0.02 [-0.50, 0.49]
$(0.1,0.2]$	66.05/70.91	+4.86 [4.19, 5.50]	5.55/5.63	+0.08 [-0.39, 0.52]
$(0.2,0.5]$	66.05/69.86	+3.81 [3.09, 4.49]	5.55/5.54	-0.02 [-0.52, 0.50]
Alternative spelling	66.05/70.91	+4.86 [4.19, 5.52]	5.55/5.63	+0.08 [-0.39, 0.52]
Article deletion	66.05/70.91	+4.86 [4.21, 5.54]	5.55/5.63	+0.08 [-0.37, 0.51]

Appendix

Table 26: LRR: clean detection and false positives for each specific fit. Each entry Raw/Clip gives both operating points.

Fitting group	Clean TPR	Change [95% CI]	Human FPR	Change [95% CI]
$(0,0.05]$	34.65/35.42	+0.78 [0.35, 1.26]	5.75/5.79	+0.04 [-0.25, 0.33]
$(0.05,0.1]$	34.65/35.42	+0.78 [0.33, 1.22]	5.75/5.79	+0.04 [-0.25, 0.31]
$(0.1,0.2]$	34.65/34.90	+0.25 [0.10, 0.43]	5.75/5.77	+0.02 [-0.10, 0.14]
$(0.2,0.5]$	34.65/35.60	+0.95 [0.66, 1.26]	5.75/5.96	+0.21 [0.00, 0.45]
Alternative spelling	34.65/35.42	+0.78 [0.33, 1.24]	5.75/5.79	+0.04 [-0.23, 0.31]
Article deletion	34.65/35.42	+0.78 [0.33, 1.22]	5.75/5.79	+0.04 [-0.23, 0.33]

Appendix

Table 27: Entropy: clean detection and false positives for each specific fit. Each entry Raw/Clip gives both operating points.

Fitting group	Clean TPR	Change [95% CI]	Human FPR	Change [95% CI]
$(0,0.05]$	75.06/74.93	-0.14 [-0.45, 0.17]	5.07/5.09	+0.02 [-0.23, 0.29]
$(0.05,0.1]$	75.06/75.30	+0.23 [-0.02, 0.47]	5.07/4.97	-0.10 [-0.29, 0.10]
$(0.1,0.2]$	75.06/73.90	-1.17 [-1.63, -0.74]	5.07/5.36	+0.29 [-0.08, 0.66]
$(0.2,0.5]$	75.06/73.90	-1.17 [-1.59, -0.74]	5.07/5.36	+0.29 [-0.08, 0.66]
Alternative spelling	75.06/75.30	+0.23 [0.00, 0.47]	5.07/4.97	-0.10 [-0.29, 0.10]
Article deletion	75.06/75.06	+0.00 [0.00, 0.00]	5.07/5.07	+0.00 [0.00, 0.00]

Appendix

Table 28: Entropy gap: clean detection and false positives for each specific fit. Each entry Raw/Clip gives both operating points.

Fitting group	Clean TPR	Change [95% CI]	Human FPR	Change [95% CI]
$(0,0.05]$	80.00/79.92	-0.08 [-0.29, 0.14]	4.62/4.53	-0.10 [-0.33, 0.12]
$(0.05,0.1]$	80.00/79.72	-0.27 [-0.58, 0.02]	4.62/4.66	+0.04 [-0.25, 0.33]
$(0.1,0.2]$	80.00/79.72	-0.27 [-0.58, 0.02]	4.62/4.66	+0.04 [-0.27, 0.33]
$(0.2,0.5]$	80.00/80.00	+0.00 [0.00, 0.00]	4.62/4.62	+0.00 [0.00, 0.00]
Alternative spelling	80.00/80.09	+0.10 [-0.10, 0.29]	4.62/4.64	+0.02 [-0.16, 0.19]
Article deletion	80.00/79.92	-0.08 [-0.29, 0.14]	4.62/4.53	-0.10 [-0.31, 0.12]

Appendix

Table 29: Binoculars: clean detection and false positives for each specific fit. Each entry Raw/Clip gives both operating points.

Test population	Raw	Full	Change [95% CI]	Eligible	Change [95% CI]
All attacks	53.82	53.82	+0.00 [0.00, 0.00]	53.99	+0.17 [0.11, 0.22]
$0<\rho\leq 0.5$	44.10	44.10	+0.00 [0.00, 0.00]	44.34	+0.24 [0.16, 0.33]
$(0,0.05]$	49.17	49.17	+0.00 [0.00, 0.00]	49.49	+0.32 [0.21, 0.43]
$(0.05,0.1]$	55.62	55.62	+0.00 [0.00, 0.00]	55.83	+0.21 [0.02, 0.40]
$(0.1,0.2]$	30.47	30.47	+0.00 [0.00, 0.00]	30.73	+0.25 [0.08, 0.42]
$(0.2,0.5]$	27.59	27.59	+0.00 [0.00, 0.00]	27.42	-0.17 [-0.40, 0.06]

Appendix

Table 30: Log likelihood: secondary universal rules evaluated on matched test populations.

Fit	Clean TPR	Change [95% CI]	Human FPR	Change [95% CI]
Full	61.29/61.29	+0.00 [0.00, 0.00]	5.85/5.85	+0.00 [0.00, 0.00]
Eligible	61.29/61.45	+0.16 [0.02, 0.31]	5.85/5.90	+0.06 [-0.06, 0.19]

Appendix

Table 31: Log likelihood: clean TPR and human FPR for the two universal fits (Raw/Clip).

Test population	Raw	Full	Change [95% CI]	Eligible	Change [95% CI]
All attacks	37.90	57.64	+19.74 [19.00, 20.43]	57.75	+19.84 [19.08, 20.60]
$0<\rho\leq 0.5$	28.90	49.58	+20.68 [19.90, 21.62]	50.26	+21.36 [20.43, 22.28]
$(0,0.05]$	29.48	53.37	+23.89 [22.82, 24.97]	54.71	+25.23 [24.10, 26.38]
$(0.05,0.1]$	35.05	59.01	+23.96 [22.48, 25.43]	61.45	+26.40 [24.84, 28.01]
$(0.1,0.2]$	24.87	40.83	+15.96 [14.88, 17.00]	39.13	+14.26 [13.18, 15.40]
$(0.2,0.5]$	25.62	33.27	+7.65 [6.29, 8.99]	32.61	+6.99 [5.55, 8.32]

Appendix

Table 32: Rank: secondary universal rules evaluated on matched test populations.

Fit	Clean TPR	Change [95% CI]	Human FPR	Change [95% CI]
Full	47.91/65.35	+17.44 [16.12, 18.78]	5.54/5.57	+0.04 [-0.56, 0.70]
Eligible	47.91/64.71	+16.80 [15.46, 18.14]	5.54/5.92	+0.39 [-0.31, 1.07]

Appendix

Table 33: Rank: clean TPR and human FPR for the two universal fits (Raw/Clip).

Test population	Raw	Full	Change [95% CI]	Eligible	Change [95% CI]
All attacks	56.89	57.10	+0.21 [0.15, 0.29]	57.17	+0.28 [0.17, 0.40]
$0<\rho\leq 0.5$	48.36	48.69	+0.34 [0.24, 0.43]	48.87	+0.51 [0.37, 0.66]
$(0,0.05]$	53.24	53.63	+0.40 [0.27, 0.53]	53.98	+0.74 [0.55, 0.94]
$(0.05,0.1]$	59.38	59.85	+0.47 [0.23, 0.72]	60.06	+0.68 [0.32, 1.08]
$(0.1,0.2]$	35.79	36.07	+0.28 [0.11, 0.46]	35.88	+0.10 [-0.15, 0.35]
$(0.2,0.5]$	31.49	31.40	-0.09 [-0.32, 0.14]	31.35	-0.14 [-0.46, 0.17]

Appendix

Table 34: Log rank: secondary universal rules evaluated on matched test populations.

Fit	Clean TPR	Change [95% CI]	Human FPR	Change [95% CI]
Full	63.76/63.82	+0.06 [-0.10, 0.23]	5.98/5.83	-0.16 [-0.33, 0.02]
Eligible	63.76/63.68	-0.08 [-0.37, 0.21]	5.98/5.85	-0.14 [-0.35, 0.10]

Appendix

Table 35: Log rank: clean TPR and human FPR for the two universal fits (Raw/Clip).

Test population	Raw	Full	Change [95% CI]	Eligible	Change [95% CI]
All attacks	59.29	62.84	+3.55 [3.25, 3.87]	62.09	+2.79 [2.43, 3.16]
$0<\rho\leq 0.5$	55.56	57.68	+2.12 [1.76, 2.49]	57.65	+2.09 [1.66, 2.53]
$(0,0.05]$	59.51	61.85	+2.35 [1.83, 2.83]	61.79	+2.28 [1.72, 2.85]
$(0.05,0.1]$	64.48	68.11	+3.63 [2.87, 4.39]	68.69	+4.21 [3.36, 4.98]
$(0.1,0.2]$	47.60	48.33	+0.73 [0.14, 1.34]	47.86	+0.27 [-0.36, 0.90]
$(0.2,0.5]$	37.36	39.03	+1.66 [0.80, 2.52]	39.28	+1.92 [1.04, 2.79]

Appendix

Table 36: LRR: secondary universal rules evaluated on matched test populations.

Fit	Clean TPR	Change [95% CI]	Human FPR	Change [95% CI]
Full	66.05/70.91	+4.86 [4.19, 5.48]	5.55/5.63	+0.08 [-0.37, 0.52]
Eligible	66.05/69.86	+3.81 [3.13, 4.54]	5.55/5.54	-0.02 [-0.54, 0.47]

Appendix

Table 37: LRR: clean TPR and human FPR for the two universal fits (Raw/Clip).

Test population	Raw	Full	Change [95% CI]	Eligible	Change [95% CI]
All attacks	37.52	37.68	+0.16 [0.00, 0.32]	37.91	+0.39 [0.29, 0.50]
$0<\rho\leq 0.5$	24.78	24.75	-0.04 [-0.25, 0.16]	25.17	+0.38 [0.26, 0.50]
$(0,0.05]$	26.11	26.20	+0.09 [-0.19, 0.35]	26.65	+0.53 [0.38, 0.69]
$(0.05,0.1]$	35.07	35.50	+0.43 [0.00, 0.87]	35.73	+0.66 [0.37, 0.97]
$(0.1,0.2]$	18.50	17.82	-0.67 [-1.01, -0.35]	18.37	-0.13 [-0.33, 0.10]
$(0.2,0.5]$	16.28	16.19	-0.09 [-0.44, 0.26]	16.48	+0.20 [-0.06, 0.48]

Appendix

Table 38: Entropy: secondary universal rules evaluated on matched test populations.

Fit	Clean TPR	Change [95% CI]	Human FPR	Change [95% CI]
Full	34.65/35.42	+0.78 [0.33, 1.22]	5.75/5.79	+0.04 [-0.23, 0.31]
Eligible	34.65/35.60	+0.95 [0.66, 1.26]	5.75/5.96	+0.21 [0.00, 0.43]

Appendix

Table 39: Entropy: clean TPR and human FPR for the two universal fits (Raw/Clip).

Test population	Raw	Full	Change [95% CI]	Eligible	Change [95% CI]
All attacks	60.31	60.31	+0.00 [0.00, 0.00]	59.79	-0.52 [-0.68, -0.35]
$0<\rho\leq 0.5$	61.04	61.04	+0.00 [0.00, 0.00]	62.23	+1.19 [1.01, 1.37]
$(0,0.05]$	66.55	66.55	+0.00 [0.00, 0.00]	68.15	+1.60 [1.37, 1.86]
$(0.05,0.1]$	71.64	71.64	+0.00 [0.00, 0.00]	72.30	+0.66 [0.19, 1.13]
$(0.1,0.2]$	46.77	46.77	+0.00 [0.00, 0.00]	47.44	+0.67 [0.31, 1.03]
$(0.2,0.5]$	44.58	44.58	+0.00 [0.00, 0.00]	45.24	+0.66 [0.20, 1.14]

Appendix

Table 40: Entropy gap: secondary universal rules evaluated on matched test populations.

Fit	Clean TPR	Change [95% CI]	Human FPR	Change [95% CI]
Full	75.06/75.06	+0.00 [0.00, 0.00]	5.07/5.07	+0.00 [0.00, 0.00]
Eligible	75.06/74.93	-0.14 [-0.47, 0.17]	5.07/5.09	+0.02 [-0.23, 0.27]

Appendix

Table 41: Entropy gap: clean TPR and human FPR for the two universal fits (Raw/Clip).

Test population	Raw	Full	Change [95% CI]	Eligible	Change [95% CI]
All attacks	72.15	72.32	+0.17 [0.09, 0.25]	71.49	-0.66 [-0.82, -0.49]
$0<\rho\leq 0.5$	71.06	71.38	+0.32 [0.22, 0.41]	71.29	+0.22 [0.04, 0.42]
$(0,0.05]$	74.27	74.58	+0.31 [0.19, 0.43]	74.35	+0.08 [-0.15, 0.30]
$(0.05,0.1]$	82.79	83.04	+0.26 [0.04, 0.47]	83.06	+0.28 [-0.13, 0.66]
$(0.1,0.2]$	60.88	61.30	+0.42 [0.20, 0.65]	61.75	+0.87 [0.49, 1.26]
$(0.2,0.5]$	57.88	58.14	+0.26 [-0.03, 0.56]	57.56	-0.32 [-0.83, 0.18]

Appendix

Table 42: Binoculars: secondary universal rules evaluated on matched test populations.

Fit	Clean TPR	Change [95% CI]	Human FPR	Change [95% CI]
Full	80.00/80.09	+0.10 [-0.10, 0.29]	4.62/4.64	+0.02 [-0.16, 0.19]
Eligible	80.00/79.72	-0.27 [-0.62, 0.04]	4.62/4.66	+0.04 [-0.25, 0.33]

Appendix

Table 43: Binoculars: clean TPR and human FPR for the two universal fits (Raw/Clip).

Fitting group	Tuning texts	Floor	NLL cap	Log-rank cap
$(0,0.05]$	19989	-11.1402	–	–
$(0.05,0.1]$	4672	–	–	–
$(0.1,0.2]$	7169	–	–	–
$(0.2,0.5]$	3406	–	–	–
Alternative spelling	5148	–	–	–
Article deletion	5148	–	–	–

Appendix

Table 44: Log likelihood: selected bounds; direction

d=+1

Fitting group	Tuning texts	Floor	NLL cap	Log-rank cap
$(0,0.05]$	19989	-21	–	–
$(0.05,0.1]$	4672	-21	–	–
$(0.1,0.2]$	7169	-66	–	–
$(0.2,0.5]$	3406	-66	–	–
Alternative spelling	5148	-66	–	–
Article deletion	5148	-66	–	–

Appendix

Table 45: Rank: selected bounds; direction

d=-1

Fitting group	Tuning texts	Floor	NLL cap	Log-rank cap
$(0,0.05]$	19989	-5.2575	–	–
$(0.05,0.1]$	4672	-5.2575	–	–
$(0.1,0.2]$	7169	-6.49979	–	–
$(0.2,0.5]$	3406	-6.49979	–	–
Alternative spelling	5148	-6.49979	–	–
Article deletion	5148	-6.49979	–	–

Appendix

Table 46: Log rank: selected bounds; direction

d=-1

Fitting group	Tuning texts	Floor	NLL cap	Log-rank cap
$(0,0.05]$	19989	–	3.59968	3.04452
$(0.05,0.1]$	4672	–	3.59968	3.04452
$(0.1,0.2]$	7169	–	3.59968	4.18965
$(0.2,0.5]$	3406	–	3.59968	3.04452
Alternative spelling	5148	–	3.59968	4.18965
Article deletion	5148	–	3.59968	4.18965

Appendix

Table 47: LRR: selected bounds; direction

d=+1

Fitting group	Tuning texts	Floor	NLL cap	Log-rank cap
$(0,0.05]$	19989	-3.59632	–	–
$(0.05,0.1]$	4672	-3.59632	–	–
$(0.1,0.2]$	7169	-5.63195	–	–
$(0.2,0.5]$	3406	-4.41172	–	–
Alternative spelling	5148	-3.59632	–	–
Article deletion	5148	-3.59632	–	–

Appendix

Table 48: Entropy: selected bounds; direction

d=-1

Fitting group	Tuning texts	Floor	NLL cap	Log-rank cap
$(0,0.05]$	19989	-6.7337	–	–
$(0.05,0.1]$	4672	-8.05884	–	–
$(0.1,0.2]$	7169	-4.91388	–	–
$(0.2,0.5]$	3406	-4.91388	–	–
Alternative spelling	5148	-8.05884	–	–
Article deletion	5148	–	–	–

Appendix

Table 49: Entropy gap: selected bounds; direction

d=-1

Fitting group	Tuning texts	Floor	NLL cap	Log-rank cap
$(0,0.05]$	19989	–	10.25	–
$(0.05,0.1]$	4672	–	8.625	–
$(0.1,0.2]$	7169	–	8.625	–
$(0.2,0.5]$	3406	–	–	–
Alternative spelling	5148	–	11.3125	–
Article deletion	5148	–	10.25	–

Appendix

Table 50: Binoculars: selected bounds; direction

d=-1

Condition	Median	Q1–Q3	Zero (%)	Eligible (%)	Dense (%)
None	0.000	0.000–0.000	100.0	0.0	0.0
Alternative spelling	0.006	0.000–0.015	36.1	63.9	0.0
Article deletion	0.027	0.015–0.037	10.4	89.6	0.0
Homoglyph	0.963	0.933–0.977	0.0	0.0	100.0
Paragraph insertion	0.051	0.034–0.068	6.9	93.1	0.0
Number	0.000	0.000–0.013	54.0	46.0	0.0

Appendix

Table 51: Modification rates by attack, measured on 5,149 test texts per condition.

Observed generator identifier	Tuning	Calibration	Test	Total
chatgpt	337	150	320	807
cohere	315	146	292	753
cohere-chat	284	147	285	716
gpt2	610	268	593	1,471
gpt3	276	162	317	755
gpt4	294	169	329	792

Appendix

Table 52: Observed generator identifier: one selected machine family per human-source group.

Source domain	Tuning	Calibration	Test	Total
abstracts	681	341	681	1,703
books	687	344	687	1,718
news	687	343	687	1,717
poetry	683	342	683	1,708
recipes	684	342	684	1,710
reddit	687	343	687	1,717

Appendix

Table 53: Source domain: one selected machine family per human-source group.

Explore similar work

May 29, 2026stat.ME

A Distribution-Free Framework for Rewrite-Based Human-text Detection via Knockoff Filtering

We propose a distribution-free statistical framework that converts arbitrary rewrite-based detectors into detectors with finite-sample FDR guarantees without retraining. Our key observation is that rewrite-based detection implicitly constructs knockoff samples, enabling LLM-generated text detection to be formulated as a multiple hypothesis testing problem with knockoff structure. This perspective separates the design of detection statistics from the control of false discoveries, allowing existing rewrite detectors to inherit finite-sample false discovery rate (FDR) guarantees through a simple calibration procedure. We demonstrate reliable FDR control with meaningful detection power across three detection models, 19 domains, and four LLMs.

Yi Liu

Machine-Generated Text Detection False Discovery Rate

Ask PDF

Aug 11, 2026cs.CL

EVIL-Detect for NLPCC 2026 Shared Task 6: LLM-Generated Text Detection

The rapid development of large language models (LLMs) has increased the need for reliable detection of LLM-generated text, especially in realistic Chinese scenarios involving human-written text (HWT), LLM-generated text (LGT), and LLM-refined text (HLT). This paper presents EVIL-Detect, a multi-signal ensemble framework with conflict-aware fusion for NLPCC 2026 Shared Task 6. The system integrates edit-extent regression, zero-shot likelihood-contrast signals, lexical statistics, and conservative text rules. With calibrated decision boundaries and conflict-aware integration, our system improves robustness under strong out-of-distribution shifts, achieving a macro-F1 score of 0.8888 and ranking first in the official evaluation. Our code is available at https://github.com/bbbbhrrrr/evildetect.

Hongrui Bao, Hangyu Rong, Zhuoshang Wang +2

Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China

Machine-Generated Text Detection Large Language Model Evaluation Natural Language Processing Ensemble

Ask PDF

May 2, 2026cs.CL

LLM Output Detectability and Task Performance Can be Jointly Optimized

Detecting machine-generated text is essential for transparency and accountability when deploying large language models (LLMs). Among detection approaches, watermarking is a statistically reliable method by design -- it embeds detectable signals into LLM outputs by biasing their token distributions. However, it has been reported that watermarked LLMs often perform worse on downstream tasks. We propose PUPPET, a framework that fine-tunes an LLM via reinforcement learning to generate text that is both more detectable and better performing on downstream tasks. We use two reward functions: a detector that outputs a machine-class likelihood and an evaluator that measures a task-specific metric. Experiments on long-form QA, summarization, and essay writing show that LLMs trained with PUPPET achieve high detectability competitive with watermarking methods while outperforming them on downstream tasks. The analysis shows that this optimization can be performed efficiently with only a few thousand samples in 1--2 GPU hours. Moreover, these gains are consistent across out-of-domain tasks, different LLM families, and model sizes, and are even robust to paraphrasing attacks.

Koshiro Saito, Ryuto Koike, Masahiro Kaneko +1

Institute of Science Tokyo · MBZUAI · National Institute of Advanced Industrial Science and Technology +1

Large Language Model Watermarks Machine-Generated Text Detection Watermarking Attacker Large Language Model

Ask PDF