We present NotAI.AI, an explainable AI-generated text detection system. Instead of returning only a binary label or confidence score, the system shows which signals influenced the prediction and lets users inspect an attribution-based sensitivity estimate obtained by subtracting selected local contributions. NotAI.AI combines sentence-level conditional probability curvature, a neural detector score, and interpretable stylometric and readability features in an XGBoost meta-classifier. It explains predictions with TreeSHAP feature contributions and can turn the resulting evidence into a concise natural-language explanation. We evaluate the system on a category-balanced subset of RAID containing human-written, clean AI-generated, and attacked AI-generated texts. The full model outperforms variants based on individual feature families, reaching 0.9685 F1 on the held-out within-subset test split. In an automatic evaluation, two model judges rate 94.5-98.6% of generated explanations as faithful to the supplied detector evidence. The web interface (https://notai-ai.vercel.app), source code (https://github.com/Oleksandr-MB/EMNLP2026DEMO_NotAI.AI), and demonstration video(https://youtu.be/l8Nk8kdBTHQ) are publicly available.
Figures & tables
Figure 1: NotAI.AI classifier architecture.
Figure 2: NotAI.AI user interface. Sample text generated using ChatGPT OpenAI (2023) .
Model
Acc.
Prec.
Rec.
ROC AUC
F1
All-AI majority
0.6650
0.6650
1.0000
0.5000
0.7988
Stylometric only
0.8895
0.9117
0.9233
0.9511
0.9174
ModernBERT only
0.8700
0.8749
0.9387
0.9122
0.9057
Sentence CPC only
0.8562
0.9306
0.8470
0.9246
0.8868
Full ensemble
0.9583
0.9731
0.9639
0.9924
0.9685
Table 1: Held-out test performance on the category-balanced RAID subset. Precision, recall, and F1 are computed for the AI class.
Criterion
Gemma (%)
Qwen (%)
Agree (%)
κ
Polarity correct
98.8
95.4
94.9
0.130
Values faithful
98.6
94.5
93.8
0.103
Summary consistent
98.8
98.9
98.5
0.356
No unsupported claims
98.8
94.5
94.1
0.110
No authorship claim
98.8
99.1
98.7
0.394
Overall faithful
98.6
94.5
93.8
0.103
Table 2: Automatic explanation-faithfulness results. The Gemma judge produced 500 valid judgments, and Qwen produced 454; agreement and Cohen’s κ are computed over the 454 paired judgments. Overall faithfulness requires all five component criteria to pass.
Figure 3: Distribution of absolute feature contributions for the full ensemble. Features are ordered by their average absolute contribution.
Figure 4: ModernBERT score dependence in the full ensemble. The axes show the raw neural score and its TreeSHAP contribution to the AI logit.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Figure 5: F1-Score heatmap per topic-generator model pair for the full ensemble classifier.
Research on AI-generated text detection has presented a number of approaches to discern human from AI prose, some of which achieving high in-distribution performance. However, real-world applicability has stalled because their outputs are misaligned with the needs of users, such as professors, who are presented with a numeric score that has no attached explanation. We tackle this issue with a novel architecture, TELL, that bakes explainability from the ground-up. While our system still offers a numerical score like other detectors for comparability, TELL takes a fundamentally different approach where we aim to show the user the "tells" by which the model believes a text is AI or human-written, to empower the user to decide who wrote a text using their own judgment and understanding of the context of the writing and its alleged author. We train TELL on a custom SFT dataset of domain-specific authorship annotations, and further refine the system using GRPO with curriculum learning to improve performance. We achieve competitive performance with state-of-the-art detectors (AUROC 0.927) while natively providing annotations that explain the basis for the detector's decision. We further evaluate the quality of our explanations using a dataset of human annotations and report a high (mean 72.3%) win-rate on annotation concreteness, falsifiability, coherence, plausibility and grounding, allowing users to critically think and decide for themselves. Our work thus reframes the problem of AI-generated text detection in a human-centric perspective and paves the way for a new family of detectors that focus on native explainability.
Aldan Creo, Suraj Ranganath
School of Computing, Information and Data Sciences University of California, San Diego United States of America
Although it is generally agreed that AI-generated text poses a broad societal risk, there is no common understanding in the AI-generated text detection literature on what constitutes harmful use. Rather, existing datasets and approaches often define their own criteria and make their own assumptions, sometimes implicitly, and often only loosely related to real-world needs and applications. To address this gap, we here systematically define various notions of AI-generated text and their characteristics. To study these, we collect AITDNA - a new benchmark of human-machine co-constructed texts that is annotated with detailed genesis information, such as the entire edit and AI-interaction history. We benchmark various machine-generated text detectors and find that they often only perform well for specific notions but not as broad detectors. We release code and data publicly.
Nils Dycke, Marina Sakharova, Nico Daheim +1
Ubiquitous Knowledge Processing Lab (UKP Lab), Department of Computer Science, Technical University of Darmstadt · National Research Center for Applied Cybersecurity ATHENE, Germany · Zuse School ELIZA
Recent advances in large language models (LLMs) have made it increasingly difficult to distinguish human-written text from AI-generated content. Many existing detectors train supervised neural classifiers that achieve strong in-distribution performance but are often opaque and can degrade substantially under distribution shift. We present READER, a reasoning-enhanced AI text detector that outputs both a human/AI label and a structured rationale describing the evidence for its decision. A key component of our approach is READ, a curated supervision set of rationales and verdicts. We fine-tune an LLM on READ to build READER, which reasons before detecting at inference time. Despite having only 1.5B parameters, READER consistently outperforms existing detectors as well as prompted, high-capacity LLM baselines (GPT-5.2, Gemini-3-Pro, and DeepSeek-V3.2), which are 100 to 1000 times larger in scale.
Pingfan Su, Kai Ye, Shijin Gong +4
Department of Statistics, London School of Economics and Political Science · School of Management, University of Science and Technology of China · School of Mathematics, University of Birmingham