The widespread adoption of large language models (LLMs) is reshaping how textual content is produced, complicating text provenance, and raising concerns for misinformation, plagiarism, and intellectual property. While prior work on AI-generated text detection has progressed, one-dimensional human-AI co-authorship scores lack signals directly relevant to ``AI involvement''. It remains unclear how adversarial paraphrasing distorts detection signals. In this work, we formulate a discrete-class text provenance detection framework. This framework allows an abstraction of realistic co-authorship scenarios into distinct degrees of AI involvement, differentiating the AI polishing on the surface level from the AI updating the discourse structures. This framework reveals the mechanisms of how adversarial paraphrasing blurs the decision boundaries for co-authorship detectors. To improve robustness of the detectors, we augment training data with adversarial paraphrases generated by the state-of-the-art paraphrasing attack model, exposing detectors to challenging meaning-preserving rewrites. Extensive experiments across multiple datasets and generation models demonstrate that adversarial paraphrasing substantially degrades detectors trained only on benign data, while adversarial data augmentation leads to consistent robustness gains and good cross-generator transferability. Overall, our findings highlight the value of discrete-class formulation and principled data augmentation for building reliable detectors of human-AI co-authorship.
Figures & tables
Figure 1: A ROCStories example showing that light GPT editing is correctly identified, while adversarial paraphrasing (DIPPER) preserves meaning but induces a false negative in an existing detector.
Generator
Data
Threshold
Low AI (%)
High AI (%)
GPT
XSum
0.233
95.00
97.67
arXiv
0.247
94.61
97.06
ROCStory
0.605
94.99
97.83
Claude
XSum
0.298
90.00
92.50
arXiv
0.366
94.61
91.18
ROCStory
0.678
94.99
98.83
Table 1: Evaluation of surface-level separation on AI-paraphrased data using normalized Levenshtein distance. The percentages indicate the proportion of samples classified as low/high-AI involvement under each threshold.
Figure 2: PCA visualization of document representations under increasing adversarial paraphrasing, with stronger paraphrases inducing shifts and overlap among AI-involved classes in representation space.
Data
GPT-4o
Claude Sonnet 4
Method
Acc
Macro-F1
AUC
Method
Acc
Macro-F1
AUC
XSum
Linguistic
0.8694
0.8682
0.9694
Linguistic
0.9472
0.9471
0.9904
Perplexity
0.5250
0.5054
0.7836
Perplexity
0.5722
0.5561
0.8253
BERT
0.9387
0.9388
0.9942
BERT
0.9526
0.9525
0.9946
RoBERTa
0.9716
0.9716
0.9995
RoBERTa
0.9638
0.9638
0.9974
Ours
0.9805
0.9805
0.9998
Ours
0.9749
0.9729
0.9979
Table 2: Detection performance of baseline methods and ours.
Settings
Benign Data Generator: GPT-4o
Settings
Benign Data Generator: Claude
Train
Test
Acc
F1
AUC
Train
Test
Acc
F1
AUC
(a) Same generator and same attack.
Benign
Benign
0.9805
0.9805
0.9998
Benign
Benign
0.9749
0.9729
0.9979
Level-1
0.9861
0.9861
0.9990
Level-1
0.9694
0.9693
0.9948
Level-2
0.9638
0.9636
0.9973
Level-2
0.9610
0.9609
0.9960
Benign
Level-1
0.7695
0.7560
0.9386
Benign
Level-1
0.6789
0.6796
0.8999
Table 3: Results on the XSum dataset under different training and testing conditions. Rows indicate increasing adversarial training levels based on DIPPER-based paraphrasing attack. Columns compare detectors trained on benign data from different generators. GPT-5.1 serves as a held-out adversarial paraphrasing attack method.
Augmentation (%)
Accuracy
Macro-F1
AUC
0%
0.7695
0.7560
0.9386
25%
0.9078
0.9127
0.9887
50%
0.9348
0.9378
0.9893
75%
0.9396
0.9427
0.9919
100%
0.9475
0.9514
0.9947
Table 4: Ablation study on the amounts of augmented data. All models are evaluated on the Level-1 test set.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 3: PCA visualization of document representations from a Longformer detector fine-tuned under a 10-class formulation. Each class corresponds to a benign or adversarial data variant treated as a separate label. The separation patterns illustrate that adversarial paraphrases occupy distinct directions in representation space rather than extending the benign AI-involvement continuum.
Dataset
Gen.
Human words
Low AI/Human
High AI/Human
Fully AI/Human
XSum
GPT
629.6 ± 285.0
0.967 ± 0.034
0.838 ± 0.151
1.012 ± 0.235
XSum
Claude
629.6 ± 285.0
0.984 ± 0.028
0.934 ± 0.128
1.206 ± 0.221
arXiv
GPT
758.3 ± 344.3
0.964 ± 0.026
0.884 ± 0.070
1.100 ± 0.244
arXiv
Claude
758.3 ± 344.3
0.944 ± 0.094
0.809 ± 0.098
0.955 ± 0.169
ROCStory
GPT
61.9 ± 4.2
1.079 ± 0.097
1.910 ± 0.249
1.659 ± 0.206
ROCStory
Claude
61.9 ± 4.2
1.180 ± 0.115
2.717 ± 0.388
1.760 ± 0.218
Appendix
Table 5: Length statistics across datasets and generators. Human words reports the word count of the source texts, while Low AI/Human, High AI/Human, and Fully AI/Human report generated-to-source word-count ratios. Values are mean ± standard deviation over aligned source-generation pairs.
Figure 4: Quantitative similarity analysis using lexical (Jaccard), n-gram (BLEU), and semantic (embedding cosine) metrics. (a) Similarity between benign AI-involved texts and their corresponding human-written sources decreases monotonically with increasing AI involvement. (b) Adversarial paraphrases introduce substantial additional surface-level divergence from their benign counterparts while largely preserving semantic similarity.
Data
Setting
Original (%)
Recalibrated (%)
XSum-GPT-4o
Benign
47.8
95.8
XSum-GPT-4o
Level-1
51.9
84.1
XSum-Claude
Benign
48.6
92.8
XSum-Claude
Level-1
51.9
78.9
Appendix
Table 6: Alignment between EditLens similarity-based categories and our process-defined AI-involvement labels using the original and recalibrated thresholds.
Dataset: XSum
Settings
F1 per Class
Overall Metrics
Train
Test
Human
Low AI
High AI
Fully AI
Acc
Macro F1
AUC
(a) Same generator and same attack.
Part I: Benign AI-involved Data generated by GPT-4o
Benign
Benign
0.9775
0.9721
0.9888
0.9836
0.9805
0.9805
0.9998
Level-1
0.6824
0.7151
0.6567
0.9697
0.7695
0.7560
0.9386
Appendix
Table 8: Results on the XSum dataset. Upward arrows ( ↑ ) indicate improved AUC relative to the immediately preceding training level under the same testing condition.
Dataset: ROCStory
Settings
F1 per Class
Overall Metrics
Train
Test
Human
Low AI
High AI
Fully AI
Acc
Macro F1
AUC
(a) Same generator and same attack.
Part I: Benign AI-involved Data generated by GPT-4o
Benign
Benign
0.9467
0.9468
0.9834
0.9889
0.9666
0.9665
0.9996
Level-1
0.8122
0.8604
0.8829
0.9019
0.8712
0.8644
0.9793
Appendix
Table 9: Results on the ROCStory dataset. Upward arrows ( ↑ ) indicate improved AUC relative to the immediately preceding training level under the same testing condition.
Dataset: arXiv
Settings
F1 per Class
Overall Metrics
Train
Test
Human
Low AI
High AI
Fully AI
Acc
Macro F1
AUC
(a) Same generator and same attack.
Part I: Benign AI-involved Data generated by GPT-4o
Benign
Benign
0.8235
0.6000
0.6226
0.8547
0.7286
0.7252
0.9382
Level-1
0.5344
0.3938
0.4405
0.8544
0.5673
0.5558
0.8343
Appendix
Table 10: Results on the Arxiv dataset. Upward arrows ( ↑ ) indicate improved AUC relative to the immediately preceding training level under the same testing condition.
Robustness of machine-generated text detection studies whether AI-text detectors remain reliable across shifts, edits, languages, and adversarial attacks.
An ideal detection system for machine generated content is supposed to work well on any generator as many more advanced LLMs come into existence day by day. Existing systems often struggle with accurately identifying AI-generated content over shorter texts. Further, not all texts might be entirely authored by a human or LLM, hence we focused more over partial cases i.e human-LLM co-authored texts. Our paper introduces a set of models built for the task of token classification which are trained on an extensive collection of human-machine co-authored texts, which performed well over texts of unseen domains, unseen generators, texts by non-native speakers and those with adversarial inputs. We also introduce a new dataset of over 2.4M such texts mostly co-authored by several popular proprietary LLMs over 23 languages. We also present findings of our models' performance over each texts of each domain and generator. Additional findings include comparison of performance against each adversarial method, length of input texts and characteristics of generated texts compared to the original human authored texts.
The rise of human-AI collaborative writing has created a growing need for fine-grained detection methods that support localizing likely LLM-generated content in mixed-authorship documents. Existing methods for detecting LLM-generated text mainly focus on document-level classification and cannot identify which parts of the text are generated by LLMs. This paper introduces a new method to address this urgent need. Our method operates at the token level, the natural unit of modern language models, and builds on existing token-level detection scores. The key idea is to smooth adjacent token scores to reduce their variability, while using an adaptive Lepski-type rule to select the bandwidth according to the local authorship structure. Our method is simple to implement and does not require token-level labeled data for training. Theoretically, we characterize this trade-off and show that the proposed method achieves favorable mean square error performance in estimating the underlying signal. Empirically, we demonstrate strong performance of our method against a wide range of baselines in both synthetic datasets and a realistic dataset. We deploy a publicly accessible website that implements the methods as well.
Yangjun Lu, Hongyi Zhou, Fabian Spill +3
School of Mathematics, University of Birmingham, Birmingham, UK · School of Statistics and Data Science, Shanghai University of Finance and Economics, Shanghai, China · Department of Statistics, The London School of Economics and Political Science, London, UK
Detecting AI-generated text is becoming increasingly challenging as modern language models approach human-level fluency and can evade detectors that rely on surface statistics or likelihood-based signals. We propose AEyeDE, an attribution-driven approach to human-AI authorship detection that leverages model attention as a discriminative signal. Specifically, we extract attention-based attribution matrices for both human- and AI-generated text using a proxy Transformer model with white-box access and train a lightweight Convolutional Neural Network to learn representations from these attribution maps. Across encoder-decoder translation settings, our method consistently outperforms a text-only baseline. In decoder-only settings, it performs strongly in generator-specific detection, remains competitive on standard benchmarks, and shows robustness under cross-dataset transfer and alternative-spelling perturbations. We further show that attention maps exhibit recurring local structures whose relative frequencies differ consistently between human- and AI-generated text across datasets and proxy models. These findings suggest that attention-based attribution maps provide a complementary and interpretable signal for AI-generated text detection. We will make the code publicly available to support future research.
Aria Nourbakhsh, Adelaide Danilov, Christoph Schommer +1
Department of Computer Science, University of Luxembourg