Authors: George Alexandru Adam, Alexander Cui, Edwin Thomas, Emily Napier, Nazar Shmatko, Yue Gao, Zikun Chen, Jacob Schnell, Jacob Junqi Tian, Alekhya Dronavalli, Edward Tian, Dongwon Lee
Organizations: GPTZero · University of Waterloo · Vector Institute · Mila · Pennsylvania State University
While historical considerations surrounding text authenticity revolved primarily around plagiarism, the advent of large language models (LLMs) has introduced a new challenge: distinguishing human-authored from AI-generated text. This shift raises significant concerns, including the undermining of skill evaluations, the mass-production of low-quality content, and the proliferation of misinformation. Addressing these issues, we introduce GPTZero 5.0, a state-of-the-art industrial AI detection solution, offering reliable discernment between human and LLM-generated text. Our key contributions include: introducing a hierarchical, multi-task architecture enabling a flexible taxonomy of human and AI texts, setting a new state-of-the-art for AI detection accuracy on 11 independent third-party benchmarks and 18 writing domains, and achieving superior robustness to adversarial attacks and paraphrasing via multi-tiered automated red teaming. GPTZero offers accurate and explainable detection, and educates users on its responsible use, ensuring fair and transparent assessment of text.
Figures & tables
Figure 1: The process of developing and improving our deep learning model for detecting LLM-generated texts.
Figure 2: Distribution of GPTZero 5.0b’s human-written training texts by writing domain. Essay types (expository, argumentative, descriptive, persuasive and admissions essays) are grouped as Essays. The 16 domains that each account for less than 3% of texts are grouped as Smaller domains and broken down below, all percentages are shares of all human-written training texts.
Figure 3: Hierarchical Classifier Heads
Figure 4: GPTZero’s multi-tiered red teaming approach covers a variety of adversarial threats, providing unprecedented robustness.
Figure 5: Drops in AI probability after removing top-k% of high AI impact sentences predicted by our Advanced Scan Feature (Blue) and Detector Sentence Head (Orange)
Figure 6: Levensthein ratio for polished sample misclassifications.
Figure 7: Accuracy of mixed class ablation models on classifying documents evaluated on internal dataset of 40k examples (details in Appendix).
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Domain
Human texts
Share (%)
Essays
1.48M
26.8
Expository essay
1.08M
19.5
Argumentative essay
176K
3.2
Descriptive essay
125K
2.3
Persuasive essay
69.9K
1.3
Admissions essay
31.2K
0.6
Appendix
Table 10: Human-written training texts of GPTZero 5.0b by writing domain, the data behind Figure 2 .
Example
Label
Dataset
many lepidopteran insects are agricultural pests that affect stored grains , food and fiber crops .
Human
Scientific Papers
The sine-Gordon field theory and its associated massive Thirring model are quantum field theories that have been extensively studied by researchers.
AI
Scientific Papers
Suddenly , Google could find itself in a position where it has to explain that they can not perform miracles to a fully committed audience invested in the idea that they can . All of these and many , many more ways to fall off exist . There ’s no rest for the weary , as they say . :-)
Human
HC3 Plus
These include the financial performance and prospects of the company, the strength of its management and leadership, the stability and growth potential of its industry, and the overall state of the economy.Additionally, the value of a share may be influenced by the demand for the stock among investors, as well as the supply of the stock that is available for purchase.
AI
HC3 Plus
Appendix
Table 11: Example of formatting biases. The human document in Scientific Papers is all lowercase with extraneous spacing around punctuation, while the AI document has correct capitalization and spacing. The human document in HC3 Plus has an emoji and extraneous spacing, while the AI document is more formal, and is missing space before "Additionally".
Figure 8: Main user interface for document scans. An overall Classification is presented along with a user-friendly confidence category. Per-class likelihoods are shown in the Probability Breakdown section.
Figure 9: Our user feedback collection interface for improving predictions.
Figure 10: Example of the GPTZero Advanced Scan feature showing the most important sentences contributing to a prediction.
Class
Precision (%)
Recall (%)
Accuracy (%)
AI
90.0
91.0
91.0
Human
91.0
90.0
90.0
Appendix
Table 12: Classification metrics (in %) for AI and Human classes using mined features.
Figure 11: Confusion Matrix representing AI vs Pure AI classes.
Figure 12: Remapping outputs to reduce overconfident predictions limits the negative effects of false positives.
Prompt
Polished Text
Original Human Document
The author gave a loving mood in the memoir. I know this becacus he did not talk badily about anything and he talked about how his parents helped him and other people as a kit. in paragraph two he was talking about how his house was nice and comsortabur. I know this because he sais. the love of cooking and the passioniate Cuban music.
Polish this text:
The author creates a warm, loving tone in the memoir. I know this because he doesn’t speak negatively about anything; instead, he focuses on how his parents helped him and others when he was a child. In paragraph two, he describes his home as nice and comfortable. I can tell because he mentions “the love of cooking” and “the passionate Cuban music,” which help paint a cozy, caring atmosphere.
Make this text sound more fluid:
The author creates a warm, loving tone throughout the memoir. I can tell because he doesn’t speak negatively about anything—instead, he focuses on how his parents supported him and cared for others when he was a child. In paragraph two, he describes his home as nice and comfortable, shown through details like the love of cooking and the passionate Cuban music.
Improve the quality of this text while maintaining a human voice:
The author creates a warm, loving tone throughout the memoir. I can tell because he doesn’t speak negatively about anything. Instead, he focuses on how his parents supported him and how other people helped him when he was a kid. In paragraph two, he describes his home as nice and comfortable. You can see this in the details he shares, like the love of cooking and the passionate Cuban music.
Improve the spelling and grammar in this text:
The author created a loving mood in the memoir. I know this because he did not speak badly about anything. He talked about how his parents helped him and other people when he was a kid. In paragraph two, he talked about how his house was nice and comfortable. I know this because he says he loved cooking and passionate Cuban music.
Appendix
Table 13: A comparison of some polishing prompts used for generated texts trained under the Polished class.
Service
Bypassing ability on naive detectors
GPTinf
Low
Grubby AI
Medium
HIX
Low
HumanizeAIText
Medium
Phrasely
High
QuillBot
Low
Appendix
Table 14: Commercial humanizer and paraphrasing services used to create the bypassed AI texts in the AI bypasser evaluation (Section 5.5), with their bypassing ability on naive AI detection methods.
The rapid proliferation of large language models (LLMs) has created an urgent need for robust and generalizable detectors of machine-generated text. Existing benchmarks typically evaluate a single detector on a single dataset under ideal conditions, leaving open questions about cross-domain transfer, cross-LLM generalization, and adversarial robustness. We present a comprehensive benchmark evaluating diverse detection approaches across two corpora: HC3 (23,363 human-ChatGPT pairs) and ELI5 (15,000 human-Mistral-7B pairs). Methods include classical classifiers, fine-tuned transformer encoders (BERT, RoBERTa, ELECTRA, DistilBERT, DeBERTa-v3), a CNN, an XGBoost stylometric model, perplexity-based detectors, and LLM-as-detector prompting. Results show that transformer models achieve near-perfect in-distribution performance but degrade under domain shift. The XGBoost stylometric model matches performance while remaining interpretable. LLM-based detectors underperform and are affected by generator-detector identity bias. Perplexity-based methods exhibit polarity inversion, with modern LLM outputs showing lower perplexity than human text, but remain effective when corrected. No method generalizes robustly across domains and LLM sources.
Madhav S. Baidya, S. S. Baidya, Chirag Chawla
Indian Institute of Technology (BHU), Varanasi, India · Indian Institute of Technology Guwahati, India
An ideal detection system for machine generated content is supposed to work well on any generator as many more advanced LLMs come into existence day by day. Existing systems often struggle with accurately identifying AI-generated content over shorter texts. Further, not all texts might be entirely authored by a human or LLM, hence we focused more over partial cases i.e human-LLM co-authored texts. Our paper introduces a set of models built for the task of token classification which are trained on an extensive collection of human-machine co-authored texts, which performed well over texts of unseen domains, unseen generators, texts by non-native speakers and those with adversarial inputs. We also introduce a new dataset of over 2.4M such texts mostly co-authored by several popular proprietary LLMs over 23 languages. We also present findings of our models' performance over each texts of each domain and generator. Additional findings include comparison of performance against each adversarial method, length of input texts and characteristics of generated texts compared to the original human authored texts.
As large language models (LLMs) generate text that increasingly resembles human writing, the subtle cues that distinguish AI-generated content from human-written content become increasingly challenging to capture. Reliance on generator-specific artifacts is inherently unstable, since new models emerge rapidly and reduce the robustness of such shortcuts. This generalizes unseen generators as a central and challenging problem for AI-text detection. To tackle this challenge, we propose a progressively structured framework that disentangles AI-detection semantics from generator-aware artifacts. This is achieved through a compact latent encoding that encourages semantic minimality, followed by perturbation-based regularization to reduce residual entanglement, and finally a discriminative adaptation stage that aligns representations with task objectives. Experiments on MAGE benchmark, covering 20 representative LLMs across 7 categories, demonstrate consistent improvements over state-of-the-art methods, achieving up to 24.2% accuracy gain and 26.2% F1 improvement. Notably, performance continues to improve as the diversity of training generators increases, confirming strong scalability and generalization in open-set scenarios. Our source code will be publicly available at https://github.com/PuXiao06/DRGD.
Xiao Pu, Zepeng Cheng, Lin Yuan +2
Chongqing University of Posts and Telecommunications, China