NOTAI.AI: Explainable Detection of Machine-Generated Text via Curvature and Feature Attribution
Organizations: Department of Computer Science, Faculty of Science, Technology and Medicine, University of Luxembourg
Abstract
We present NotAI.AI, an explainable AI-generated text detection system. Instead of returning only a binary label or confidence score, the system shows which signals influenced the prediction and lets users inspect an attribution-based sensitivity estimate obtained by subtracting selected local contributions. NotAI.AI combines sentence-level conditional probability curvature, a neural detector score, and interpretable stylometric and readability features in an XGBoost meta-classifier. It explains predictions with TreeSHAP feature contributions and can turn the resulting evidence into a concise natural-language explanation. We evaluate the system on a category-balanced subset of RAID containing human-written, clean AI-generated, and attacked AI-generated texts. The full model outperforms variants based on individual feature families, reaching 0.9685 F1 on the held-out within-subset test split. In an automatic evaluation, two model judges rate 94.5-98.6% of generated explanations as faithful to the supplied detector evidence. The web interface (https://notai-ai.vercel.app), source code (https://github.com/Oleksandr-MB/EMNLP2026DEMO_NotAI.AI), and demonstration video(https://youtu.be/l8Nk8kdBTHQ) are publicly available.
Figures & tables
| Model | Acc. | Prec. | Rec. | ROC AUC | F1 |
|---|---|---|---|---|---|
| All-AI majority | 0.6650 | 0.6650 | 1.0000 | 0.5000 | 0.7988 |
| Stylometric only | 0.8895 | 0.9117 | 0.9233 | 0.9511 | 0.9174 |
| ModernBERT only | 0.8700 | 0.8749 | 0.9387 | 0.9122 | 0.9057 |
| Sentence CPC only | 0.8562 | 0.9306 | 0.8470 | 0.9246 | 0.8868 |
| Full ensemble | 0.9583 | 0.9731 | 0.9639 | 0.9924 | 0.9685 |
| Criterion | Gemma (%) | Qwen (%) | Agree (%) | |
|---|---|---|---|---|
| Polarity correct | 98.8 | 95.4 | 94.9 | 0.130 |
| Values faithful | 98.6 | 94.5 | 93.8 | 0.103 |
| Summary consistent | 98.8 | 98.9 | 98.5 | 0.356 |
| No unsupported claims | 98.8 | 94.5 | 94.1 | 0.110 |
| No authorship claim | 98.8 | 99.1 | 98.7 | 0.394 |
| Overall faithful | 98.6 | 94.5 | 93.8 | 0.103 |
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
AI-Generated Text Detection
AI-generated text detection develops methods to distinguish human-written, machine-generated, and AI-edited text.
See all 166 papers