C-HAT-Bench: Benchmarking Chinese AI-Text Detection Beyond Fully Generated Text

Qing Yang; Zixiang Luo; Zhenyu Mao; Zezheng Wu; Xinghe Cheng; Haibo Chen; Qinggang Zhang; Jiapu Wang

cs.CLSep 26, 2026

C-HAT-Bench: Benchmarking Chinese AI-Text Detection Beyond Fully Generated Text

Authors: Qing Yang, Zixiang Luo, Zhenyu Mao, Zezheng Wu, Xinghe Cheng, Haibo Chen, Qinggang Zhang, Jiapu Wang,

Organizations: Guilin University of Electronic Technology, China · Jinan Unaiversity, China · Nanjing University of Science and Technology, China · Jilin Unaiversity, China

Abstract

Large Language Models (LLMs) increasingly participate in writing by modifying or extending human drafts, causing machine involvement to vary in both form and extent. Yet most Machine-Generated Text (MGT) detectors are evaluated only on fully human-written versus fully AI-generated text. Because human--AI collaboration can weaken or redistribute cues associated with machine generation, strong performance under this binary setting may overstate detector reliability. This mismatch remains underexplored in Chinese: detection cues are shaped by tokenization and language-specific text distributions, yet controlled resources spanning production settings, domains, and generators remain limited. To fill this gap, we present a Chinese Human-AI Collaborative Text Detection Benchmark (C-HAT-Bench), a unified benchmark that links $5,000$ human-written source texts from five domains to more than $240,000$ variants produced using six generative models under Prefix-Conditioned Continuation as a reference setting and three collaborative production modes. We evaluate $21$ detectors through four protocols spanning zero-shot and pretrained supervised document-level detection, boundary localization, and cross-condition generalization. Relative to Prefix-Conditioned Continuation, mean AUROC across document-level detectors is $12.0\%$ lower on the collaborative production modes, with the largest detector-specific relative decrease reaching $44.4\%$ . Transfer across collaborative production modes is also asymmetric, indicating that performance in a given production setting is not a reliable predictor of performance in other production settings.

Figures & tables

Figure 1: Overview of the C-HAT-Bench framework.

Figure 2: Composition of C-HAT-240K across production settings.

Detector	MGT	Content Filling				Machine Rewriting		Prompt-based Generation
Detector		LC-C	LC-D	HC-C	HC-D	Light	Deep	L1	L2	L3	L3+Ref.
Zero-shot detectors
Log-Likelihood	0.743	0.651	0.631	0.715	0.695	0.634	0.629	0.758	0.730	0.694	0.697
LogRank	0.746	0.644	0.624	0.709	0.689	0.623	0.616	0.762	0.727	0.688	0.691
Entropy	0.317	0.374	0.386	0.333	0.334	0.393	0.396	0.303	0.324	0.341	0.338
DetectLLM-LRR	0.712	0.578	0.560	0.635	0.610	0.529	0.516	0.723	0.661	0.606	0.608

Table 1: Detector performance under human–AI collaboration, measured by AUROC. The best two results are highlighted by 1st and 2nd . Here, MGT denotes the prefix-conditioned continuation condition; LC/HC indicate low/high coverage (15–35%/50–75%), and C/D indicate single-suffix/separated-span filling; Light/Deep denote expression polishing/structural rewriting; and L1–L3 indicate increasing prompt detail, with L3+Ref additionally using a same-domain reference.

Figure 3: Detector performance relative to MGT across human–AI collaboration conditions. Top and bottom rows show zero-shot and supervised detectors, respectively.

Figure 4: AUROC distributions across five text domains for zero-shot (top) and supervised (bottom) detectors on MGT and human–AI collaborative text. Box colors indicate median AUROC.

Figure 5: Boundary-localization performance of WCP and DeBERTa–BiGRU–CRF across Content Filling conditions.

Figure 6: Cross-domain accuracies of AIGC-detector-zhv2 across five text domains.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Stage	Criterion	Rule
Human-text selection	Length-based selection	Starting from 1250 characters, progressively expand the selection range toward shorter and longer texts until 1,000 samples are collected per domain.
Human-text selection	Chinese-character ratio	For news, law, Q&A, and encyclopedia, start at 90% and relax the threshold when insufficient samples are available, with a lower bound of 60%. The denominator excludes whitespace.
Format cleaning	Whitespace and markup	Normalize line endings, remove redundant whitespace and invisible characters, and strip HTML/Markdown markup, code fences, and recognizable task-description prefixes. Text lengths are computed after cleaning.

Appendix

Table 2: HWT cleaning and validation rules for C-HAT-240K.

Figure 7: Character composition of the selected human corpus. Percentages are averaged over 1,000 texts per domain, excluding whitespace.

Figure 8: stride-sensitivity analysis for representative zero-shot and supervised detectors. The left panel shows zero-shot detectors, and the right panel shows supervised detectors. The highlighted x-axis value indicates the selected stride for the corresponding experiment.

Detector	MGT		Content Filling								Machine Rewriting				Prompt-based Generation
			LC-C		LC-D		HC-C		HC-D		Light		Subst.		L1		L2		L3		L3+Ref.
	ACC	F1	ACC	F1	ACC	F1	ACC	F1	ACC	F1	ACC	F1	ACC	F1	ACC	F1	ACC	F1	ACC	F1	ACC	F1
1% FPR Zero-shot detectors
Log-Likelihood	58.5	22.5	52.6	10.4	52.3	9.6	53.8	14.1	54.8	15.7	52.5	10.0	52.6	10.3	56.7	18.9	53.4	12.2	53.6	12.7	53.9	13.5
LogRank	58.4	21.5	52.4	9.6	52.3	9.1	53.4	12.8	54.5	14.8	52.2	9.0	52.4	9.4	56.9	18.5	53.2	11.4	53.4	11.8	53.7	12.4
Entropy	50.1	2.3	49.9	1.7	49.9	1.6	49.9	1.4	50.0	1.7	49.9	1.4	50.0	1.7	50.0	1.7	49.8	1.1	50.0	1.7	50.0	1.6

Appendix

Table 6: Detector performance under human–AI collaboration, measured by ACC and F1 at 1%, 5%, and 10% FPR. The best two results are highlighted by 1st and 2nd . Values are reported as percentages and macro-averaged over 30 domain–generation-model combinations.

Detector	Law	Literature	News	Encyclopedia	Q&A
Zero-shot detectors
Log-Likelihood	0.891	0.575	0.618	0.676	0.698
LogRank	0.873	0.583	0.614	0.669	0.698
Entropy	0.182	0.479	0.399	0.346	0.337
DetectLLM-LRR	0.663	0.602	0.572	0.602	0.661
Fast-DetectGPT	0.783	0.602	0.552	0.604	0.655

Appendix

Table 7: Domain-wise AUROC of zero-shot and off-the-shelf supervised detectors. Each value is macro-averaged over six generation models and the fine-grained conditions within each of the four text-production settings, followed by an equal-weight average over the four settings. Within each detector family and domain, the best and second-best results are shown in boldface and underlined, respectively.

Condition	Detector	$n$	Sentence $\kappa$ $\uparrow$	WindowDiff $\downarrow$	Boundary F1@1 $\uparrow$
LC-C	WCP	1962	0.194 $\pm$ 0.343	0.428 $\pm$ 0.206	0.168 $\pm$ 0.226
LC-C	DeBERTa–BiGRU–CRF	1962	-0.054 $\pm$ 0.101	0.386 $\pm$ 0.184	0.004 $\pm$ 0.051
LC-D	WCP	1886	0.019 $\pm$ 0.163	0.381 $\pm$ 0.101	0.380 $\pm$ 0.232
LC-D	DeBERTa–BiGRU–CRF	1886	0.258 $\pm$ 0.250	0.309 $\pm$ 0.102	0.373 $\pm$ 0.271
HC-C	WCP	1877	0.166 $\pm$ 0.373	0.451 $\pm$ 0.185	0.171 $\pm$ 0.231
HC-C	DeBERTa–BiGRU–CRF	1877	0.005 $\pm$ 0.152	0.336 $\pm$ 0.089	0.341 $\pm$ 0.327

Appendix

Table 8: Detailed boundary-localization performance of the two fine-tuned detectors on the common content-filling test set. Here,

n

denotes the number of evaluated documents, and values are document-level means

\pm

population standard deviations. Boundary F1@1 uses one-to-one matching between predicted and gold boundaries within a tolerance of one sentence; arrows indicate the preferred direction.

Condition	Detector	$n$	No predicted boundary (%)	Mean predicted	Mean gold	Mean count error
LC-C	WCP	1962	4.18	4.497	1.000	-3.497
LC-C	DeBERTa–BiGRU–CRF	1962	70.90	0.629	1.000	0.371
LC-D	WCP	1886	5.51	4.073	4.914	0.841
LC-D	DeBERTa–BiGRU–CRF	1886	24.23	1.694	4.914	3.221
HC-C	WCP	1877	3.94	4.384	1.000	-3.384
HC-C	DeBERTa–BiGRU–CRF	1877	31.43	1.477	1.000	-0.477

Appendix

Table 9: Boundary-prediction diagnostics for the two fine-tuned detectors on the common content-filling test set. Here,

n

denotes the number of evaluated documents. No predicted boundary is the percentage of documents with zero predicted boundaries; Mean predicted and Mean gold are the document-level average numbers of predicted and gold boundaries, respectively. Mean count error is defined as gold minus predicted boundaries, so negative and positive values indicate over-segmentation and missed boundaries, respectively.

Figure 9: Cross-domain accuracy (%) of AIGC-detector-zhv2 and IMBD with cellwise Youden-optimal thresholds. Rows and columns denote training and test domains, respectively.

Figure 10: Accuracy (%) across collaboration types for AIGC-detector-zhv2 and IMBD with cellwise Youden-optimal thresholds. Rows and columns denote training and test collaboration types, respectively.

Figure 11: Cross-generator accuracy (%) of AIGC-detector-zhv2 and IMBD with cellwise Youden-optimal thresholds. Rows and columns denote training and test generators, respectively.

AI-Generated Text Detection

AI-generated text detection develops methods to distinguish human-written, machine-generated, and AI-edited text.

See all 170 papers

Detecting the Machine: A Comprehensive Benchmark of AI-Generated Text Detectors Across Architectures, Domains, and Adversarial Conditions
Mar 18, 2026Madhav S. Baidya, S. S. Baidya, Chirag ChawlaAI-Generated Text Detection Robustness of Machine-Generated Text Detection
Indian Institute of Technology (BHU), Varanasi, India · Indian Institute of Technology Guwahati, India
C-ReD: A Comprehensive Chinese Benchmark for AI-Generated Text Detection Derived from Real-World Prompts
Apr 13, 2026Chenxi Qing, Junxi Wu, Zheng Liu +5AI-Generated Text Detection Robustness of Machine-Generated Text Detection
Tsinghua University · Nankai University · Harbin Institute of Technology, Shenzhen +2
'Your AI Text is not Mine': Redefining and Evaluating AI-generated Text Detection under Realistic Assumptions
Jun 3, 2026Nils Dycke, Marina Sakharova, Nico Daheim +1AI-Generated Text Detection Generative AI Evaluation
Ubiquitous Knowledge Processing Lab (UKP Lab), Department of Computer Science, Technical University of Darmstadt · National Research Center for Applied Cybersecurity ATHENE, Germany · Zuse School ELIZA

Mar 18, 2026J/K move · Enter open · S save

Detecting the Machine: A Comprehensive Benchmark of AI-Generated Text Detectors Across Architectures, Domains, and Adversarial Conditions

Madhav S. Baidya, S. S. Baidya, Chirag Chawla

Indian Institute of Technology (BHU), Varanasi, India · Indian Institute of Technology Guwahati, India

AI-Generated Text Detection Robustness of Machine-Generated Text Detection

Open paper AskPDF arXiv

The rapid proliferation of large language models (LLMs) has created an urgent need for robust and generalizable detectors of machine-generated text. Existing benchmarks typically evaluate a single detector on a single dataset under ideal conditions, leaving open questions about cross-domain transfer, cross-LLM generalization, and adversarial robustness. We present a comprehensive benchmark evaluating diverse detection approaches across two corpora: HC3 (23,363 human-ChatGPT pairs) and ELI5 (15,000 human-Mistral-7B pairs). Methods include classical classifiers, fine-tuned transformer encoders (BERT, RoBERTa, ELECTRA, DistilBERT, DeBERTa-v3), a CNN, an XGBoost stylometric model, perplexity-based detectors, and LLM-as-detector prompting. Results show that transformer models achieve near-perfect in-distribution performance but degrade under domain shift. The XGBoost stylometric model matches performance while remaining interpretable. LLM-based detectors underperform and are affected by generator-detector identity bias. Perplexity-based methods exhibit polarity inversion, with modern LLM outputs showing lower perplexity than human text, but remain effective when corrected. No method generalizes robustly across domains and LLM sources.

Abstract

AI-Generated Text Detection

Explore similar work

Detecting the Machine: A Comprehensive Benchmark of AI-Generated Text Detectors Across Architectures, Domains, and Adversarial Conditions

C-ReD: A Comprehensive Chinese Benchmark for AI-Generated Text Detection Derived from Real-World Prompts

'Your AI Text is not Mine': Redefining and Evaluating AI-generated Text Detection under Realistic Assumptions