Why AI-Generated Text Detection Fails: Evidence from Explainable AI Beyond Benchmark Accuracy

Shushanta Pudasaini; Luis Miralles-Pechuán; David Lillis; Marisa Llorens Salvador

cs.CLMar 24, 2026

Why AI-Generated Text Detection Fails: Evidence from Explainable AI Beyond Benchmark Accuracy

Authors: Shushanta Pudasaini, Luis Miralles-Pechuán, David Lillis, Marisa Llorens Salvador

Organizations: Technological University Dublin · University College Dublin

Abstract

The widespread adoption of Large Language Models (LLMs) has made the detection of AI-Generated text a pressing and complex challenge. Although many detection systems report high benchmark accuracy, their reliability in real-world settings remains uncertain, and their interpretability is often unexplored. In this work, we investigate whether contemporary detectors genuinely identify machine authorship or merely exploit dataset-specific artefacts. We propose an interpretable detection framework that integrates linguistic feature engineering, machine learning, and explainable AI techniques. When evaluated on two prominent benchmark corpora, namely PAN CLEF 2025 and COLING 2025, our model trained on 30 linguistic features achieves leaderboard-competitive performance, attaining an F1 score of 0.9734. However, systematic cross-domain and cross-generator evaluation reveals substantial generalisation failure: classifiers that excel in-domain degrade significantly under distribution shift. Using SHAP- based explanations, we show that the most influential features differ markedly between datasets, indicating that detectors often rely on dataset-specific stylistic cues rather than stable signals of machine authorship. Further investigation with in-depth error analysis exposes a fundamental tension in linguistic-feature-based AI text detection: the features that are most discriminative on in-domain data are also the features most susceptible to domain shift, formatting variation, and text-length effects. We believe that this knowledge helps build AI detectors that are robust across different settings. To support replication and practical use, we release an open-source Python package that returns both predictions and instance-level explanations for individual texts.

Figures & tables

Feature	Description	How It Is Calculated
Surface and Length Features
Character Count	Total characters in the document.	Count all characters, including spaces.
Word Count	Total word tokens.	Split on whitespace and count tokens.
Sentence Count	Number of Sentences.	Sentence tokenisation count.
Paragraph Count	Number of paragraphs.	Count non-empty segments split by newlines.
Punctuation Count	Total punctuation symbols.	Count standard punctuation characters.

Table 1: Linguistic feature set with operational definitions and computational specifications. Sentence tokenisation uses NLTK Punkt [ 25 ] ; POS tagging, lemmatisation, and dependency parsing use spaCy en_core_web_sm [ 26 ] ; grammar checking uses LanguageTool v5.7 [ 27 ] ; sentiment analysis uses TextBlob [ 28 ] ; TF-IDF vectorisation uses scikit-learn [ 29 ] .

Attribute	PAN CLEF Dataset	COLING Dataset	Ghostbuster Dataset
Task Name	Voight–Kampff Generative AI Detection 2025 (Subtask 1)	GenAI Content Detection Task 1 (English & Multilingual)	Research benchmark (NAACL 2024)
Domains Considered	Essays, news, fiction	Peer reviews, student essays, scientific papers, news, and 8+ additional domains	Student essays, creative writing, news
Training Data Size	23,707 samples (9,101 human / 14,606 AI)	610,767 English samples (228,922 human / 381,845 machine)	Not used for training purposes
Dev / Validation Size	3,589 samples (1,277 human / 2,312 AI)	261,758 English samples (98,328 human / 163,430 machine)	Same-prompt splits; 6,000 text(2000 human-written,2000 generated with GPT and 2000 generated with Claude); used only for evaluation
Generators	GPT-4o and participant models with obfuscation	GPT-4/4o, Mistral, Llama 3.1, Qwen-2, Claude, others	GPT-3.5-turbo (training); Claude (generalisation)
Evaluation Metrics	ROC–AUC, Brier, C@1, F1, F0.5u, mean score, FPR/FNR	Macro-F1 (official), Accuracy, Micro-F1	F1 score, Accuracy

Table 2: Comparison of the benchmark datasets used for the experiment with their data size and diversity details.

Feature	PAN CLEF		COLING		Ghostbuster
	Human	AI	Human	AI	Human	AI
Character Count	$3941.826$	$3580.483$	$1527.844$	$1449.026$	$3535.748$	$3080.883$
Word Count	$695.456$	$574.973$	$259.964$	$237.821$	$586.937$	$475.910$
Paragraph Count	$26.409$	$17.884$	$7.116$	$3.786$	$12.587$	$9.107$
Stopword Count	$326.233$	$239.586$	$110.491$	$102.020$	$256.590$	$191.907$
Unique Word Count	$381.030$	$332.886$	$109.093$	$104.482$	$215.613$	$192.170$

Table 3: Average values of selected linguistic features for human-written and AI-generated texts across the PAN CLEF, COLING, and Ghostbuster benchmark datasets.

Figure 1: t-SNE projections of linguistic feature representations for the COLING dataset (left) and PAN CLEF dataset (right). Human-written samples are shown in purple and AI-generated samples in yellow.

Figure 2: Overview of the experimental pipeline, illustrating dataset usage, linguistic feature extraction, model training and evaluation, ensemble construction, and post-hoc interpretability using SHAP.

Algorithm	F1-score
mdok of KInIT [ 37 ]	98.90
Optimised Feature Set + SVC (Ours)	97.34
Optimised Feature Set + XGBoost (Ours)	96.94
Optimised Feature Set + Random Forest (Ours)	95.82
Optimised Feature Set + Logistic Regression (Ours)	95.48
Baseline TF-IDF SVM [ 5 ]	90.40

Table 4: Comparison of F-score performance on the Voight–Kampff Generative AI Detection 2025 leaderboard (F-score scaled to 0–100).

Model	Accuracy	Precision	Recall	F1 Score	FPR	FNR
XGBoost	96.04	96.45	97.45	96.94	6.50	2.55
Random Forest	94.51	94.12	97.58	95.82	11.04	2.42
Logistic Regression	94.09	94.19	96.80	95.48	10.08	3.20
SVM	96.55	96.71	97.97	97.34	6.03	2.03

Table 5: Comparison of machine learning model performance on the AI-generated text detection dataset using cross-validation. Reported metrics include Accuracy, Precision, Recall, F1-score, False Positive Rate (FPR), and False Negative Rate (FNR). The best-performing model across most evaluation metrics is highlighted in bold. All values are scaled from 0 to 100.

Figure 3: Confusion matrices for the four classifiers evaluated on the PAN CLEF in-domain test set. Rows indicate true labels and columns predicted labels. False positives (Human $\rightarrow$ AI) represent potential false accusations, while false negatives (AI $\rightarrow$ Human) represent missed detections. Despite similar F1 scores, models differ in their error distribution, illustrating why benchmark accuracy alone does not fully capture deployment risk.

Model	Accuracy	Precision	Recall	F1 Score	ROC-AUC
SVM	0.9697 $\pm$ 0.0045	0.9731 $\pm$ 0.0051	0.9778 $\pm$ 0.0036	0.9754 $\pm$ 0.0036	0.9942 $\pm$ 0.0015
XGBoost	0.9678 $\pm$ 0.0033	0.9717 $\pm$ 0.0028	0.9762 $\pm$ 0.0034	0.9739 $\pm$ 0.0027	0.9943 $\pm$ 0.0009
Random Forest	0.9505 $\pm$ 0.0044	0.9500 $\pm$ 0.0037	0.9707 $\pm$ 0.0044	0.9602 $\pm$ 0.0035	0.9878 $\pm$ 0.0019
Logistic Regression	0.9431 $\pm$ 0.0043	0.9490 $\pm$ 0.0048	0.9591 $\pm$ 0.0050	0.9540 $\pm$ 0.0034	0.9826 $\pm$ 0.0015

Table 6: Comparison of machine learning model performance using cross-validation. Reported metrics include Accuracy, Precision, Recall, F1 Score, and ROC-AUC. The best-performing model across most evaluation metrics is highlighted in bold. Values are reported as mean

\pm

standard deviation.

Comparison	$\Delta$ F1	t-test p-value	Significant?
SVM vs. XGBoost	+0.0015	0.0984	No
SVM vs. Random Forest	+0.0152	$<0.0001$	Yes
SVM vs. Logistic Regression	+0.0214	$<0.0001$	Yes
XGBoost vs. Random Forest	+0.0137	$<0.0001$	Yes
XGBoost vs. Logistic Regression	+0.0199	$<0.0001$	Yes
Random Forest vs. Logistic Regression	+0.0062	$<0.0001$	Yes

Table 7: Statistical significance testing of F1-score differences between machine learning models using paired t-test.

\Delta

F1 indicates the difference in mean F1-score between the compared models. Statistical significance is determined at

\alpha=0.05

Train Set	Algorithm	PAN CLEF	COLING
PAN CLEF	SVC	97.34	76.59
	XGBoost	96.94	67.23
	Random Forest	95.82	74.86
	Logistic Regression	95.48	76.53
COLING	SVC	78.05	76.79
	XGBoost	78.56	78.53

Table 8: Cross-dataset generalisation results on PAN CLEF and COLING test sets (F1 scaled to 1–100).

LLM	Best-PAN	Best-COL
GPT-5.2	23.25	3.42
DeepSeek V3.2	0.76	3.51
Gemini 3 Pro	0.43	18.40

Table 9: Cross-generator generalisation performance on unseen LLMs, reported as False Negative Rate (FNR, %). Best-PAN refers to the SVM model achieving the highest F1-score when trained and evaluated on the PAN CLEF dataset, while Best-COL denotes the Random Forest model achieving the highest F1-score when trained and evaluated on the COLING dataset.

Subset	Best-PAN	Best-COL	Ens-PAN	Ens-COL
WP	60.15	59.36	92.87	77.77
Reuters	55.80	85.59	96.88	90.98
Essay	76.07	90.83	94.09	92.64
Average	64.0	78.59	94.61	87.13

Table 10: Ensemble performance on the Ghostbuster dataset (F1-score). Ens-PAN and Ens-COL denote ensemble models combining SVM, Random Forest, XGBoost, and Logistic Regression, trained on the PAN CLEF and COLING datasets, respectively.

Figure 4: Feature importance obtained while training an XGBoost model on different datasets.

Panel of Figure 5 — Figure 5: SHAP summary plots showing feature importance distributions for different datasets.

Type	Count	Words (Mean)	Words (Median)	Paragraph (Mean)	Paragraph (Median)	GZIP (Mean)	GZIP (Median)
FN	139	177.48	183	13.24	3	0.724	0.557
FP	2753	221.48	171	4.13	1	0.574	0.548
TN	967	421.03	212	24.69	17	0.628	0.518
TP	6141	242.89	198	5.36	1	0.535	0.510

Table 11: Descriptive statistics for text classification categories, including mean and median values for word count, paragraph count, and GZIP compression ratio. This table highlights structural differences between true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN).

Panel of Figure 6 — Figure 6: SHAP waterfall plots illustrating feature contributions for human-written and AI-generated samples.

Panel of Figure 7 — Figure 7: SHAP bar plots showing average feature impact for human-written and AI-generated samples.

AI-Generated Text Detection

AI-generated text detection develops methods to distinguish human-written, machine-generated, and AI-edited text.

See all 167 papers

Mar 18, 2026

Detecting the Machine: A Comprehensive Benchmark of AI-Generated Text Detectors Across Architectures, Domains, and Adversarial Conditions

The rapid proliferation of large language models (LLMs) has created an urgent need for robust and generalizable detectors of machine-generated text. Existing benchmarks typically evaluate a single detector on a single dataset under ideal conditions, leaving open questions about cross-domain transfer, cross-LLM generalization, and adversarial robustness. We present a comprehensive benchmark evaluating diverse detection approaches across two corpora: HC3 (23,363 human-ChatGPT pairs) and ELI5 (15,000 human-Mistral-7B pairs). Methods include classical classifiers, fine-tuned transformer encoders (BERT, RoBERTa, ELECTRA, DistilBERT, DeBERTa-v3), a CNN, an XGBoost stylometric model, perplexity-based detectors, and LLM-as-detector prompting. Results show that transformer models achieve near-perfect in-distribution performance but degrade under domain shift. The XGBoost stylometric model matches performance while remaining interpretable. LLM-based detectors underperform and are affected by generator-detector identity bias. Perplexity-based methods exhibit polarity inversion, with modern LLM outputs showing lower perplexity than human text, but remain effective when corrected. No method generalizes robustly across domains and LLM sources.

Madhav S. Baidya, S. S. Baidya, Chirag Chawla

Indian Institute of Technology (BHU), Varanasi, India · Indian Institute of Technology Guwahati, India

AI-Generated Text Detection Robustness of Machine-Generated Text Detection

Ask PDF

Jun 2, 2026

A Systematic Analysis of Linguistic Features in AI-Generated Text Detection Across Domains and Models

Interpretable linguistic features offer a promising approach for explaining why a given text appears machine-generated, particularly for non-expert users. However, existing findings on which features reliably indicate LLM-generated text remain fragmented across feature sets, models, and text domains. To address this gap, we conduct a large-scale empirical study assessing the robustness of linguistic signals for characterizing AI-generated text. Our analysis covers 284 interpretable linguistic features across outputs from 27 LLMs and ten text domains under cross-model and cross-domain generalization settings. We show that classifiers based solely on linguistic features can reliably distinguish AI-generated from human-written text. However, many previously proposed indicators prove strongly context-dependent, with the exception of measures of lexical richness, which remain robust signals across model families and text domains. These results demonstrate which linguistic signals generalize across contexts and provide a foundation for more reliable, interpretable analyses of AI-generated language.

Yassir El Attar, Esra Dönmez, Maximilian Maurer +1

Institute for Natural Language Processing, University of Stuttgart · Interchange Forum for Reflecting on Intelligent Systems, University of Stuttgart · GESIS Leibniz Institute for the Social Sciences +1

AI-Generated Text Detection Stylometry Language Model Robustness Robustness of Machine-Generated Text Detection Generative AI Evaluation

Ask PDF

Mar 19, 2026

Automatic detection of Gen-AI texts: A comparative framework of neural models

The rapid proliferation of Large Language Models has significantly increased the difficulty of distinguishing between human-written and AI generated texts, raising critical issues across academic, editorial, and social domains. This paper investigates the problem of AI generated text detection through the design, implementation, and comparative evaluation of multiple machine learning based detectors. Four neural architectures are developed and analyzed: a Multilayer Perceptron, a one-dimensional Convolutional Neural Network, a MobileNet-based CNN, and a Transformer model. The proposed models are benchmarked against widely used online detectors, including ZeroGPT, GPTZero, QuillBot, Originality.AI, Sapling, IsGen, Rephrase, and Writer. Experiments are conducted on the COLING Multilingual Dataset, considering both English and Italian configurations, as well as on an original thematic dataset focused on Art and Mental Health. Results show that supervised detectors achieve more stable and robust performance than commercial tools across different languages and domains, highlighting key strengths and limitations of current detection strategies.

Cristian Buttaro, Irene Amerini

Sapienza University in Rome Latina

AI-Generated Text Detection Robustness of Machine-Generated Text Detection Text Classification

Ask PDF

Abstract

AI-Generated Text Detection

Explore similar work

Detecting the Machine: A Comprehensive Benchmark of AI-Generated Text Detectors Across Architectures, Domains, and Adversarial Conditions

A Systematic Analysis of Linguistic Features in AI-Generated Text Detection Across Domains and Models

Automatic detection of Gen-AI texts: A comparative framework of neural models