Towards Fine-grained and Robust Detection of Human-AI Co-authorship
Organizations: Stevens Institute of Technology, Hoboken, New Jersey, USA
Abstract
The widespread adoption of large language models (LLMs) is reshaping how textual content is produced, complicating text provenance, and raising concerns for misinformation, plagiarism, and intellectual property. While prior work on AI-generated text detection has progressed, one-dimensional human-AI co-authorship scores lack signals directly relevant to ``AI involvement''. It remains unclear how adversarial paraphrasing distorts detection signals. In this work, we formulate a discrete-class text provenance detection framework. This framework allows an abstraction of realistic co-authorship scenarios into distinct degrees of AI involvement, differentiating the AI polishing on the surface level from the AI updating the discourse structures. This framework reveals the mechanisms of how adversarial paraphrasing blurs the decision boundaries for co-authorship detectors. To improve robustness of the detectors, we augment training data with adversarial paraphrases generated by the state-of-the-art paraphrasing attack model, exposing detectors to challenging meaning-preserving rewrites. Extensive experiments across multiple datasets and generation models demonstrate that adversarial paraphrasing substantially degrades detectors trained only on benign data, while adversarial data augmentation leads to consistent robustness gains and good cross-generator transferability. Overall, our findings highlight the value of discrete-class formulation and principled data augmentation for building reliable detectors of human-AI co-authorship.
Figures & tables
| Generator | Data | Threshold | Low AI (%) | High AI (%) |
| GPT | XSum | 0.233 | 95.00 | 97.67 |
| arXiv | 0.247 | 94.61 | 97.06 | |
| ROCStory | 0.605 | 94.99 | 97.83 | |
| Claude | XSum | 0.298 | 90.00 | 92.50 |
| arXiv | 0.366 | 94.61 | 91.18 | |
| ROCStory | 0.678 | 94.99 | 98.83 |
| Data | GPT-4o | Claude Sonnet 4 | ||||||
| Method | Acc | Macro-F1 | AUC | Method | Acc | Macro-F1 | AUC | |
| XSum | Linguistic | 0.8694 | 0.8682 | 0.9694 | Linguistic | 0.9472 | 0.9471 | 0.9904 |
| Perplexity | 0.5250 | 0.5054 | 0.7836 | Perplexity | 0.5722 | 0.5561 | 0.8253 | |
| BERT | 0.9387 | 0.9388 | 0.9942 | BERT | 0.9526 | 0.9525 | 0.9946 | |
| RoBERTa | 0.9716 | 0.9716 | 0.9995 | RoBERTa | 0.9638 | 0.9638 | 0.9974 | |
| Ours | 0.9805 | 0.9805 | 0.9998 | Ours | 0.9749 | 0.9729 | 0.9979 | |
| Settings | Benign Data Generator: GPT-4o | Settings | Benign Data Generator: Claude | ||||||
| Train | Test | Acc | F1 | AUC | Train | Test | Acc | F1 | AUC |
| (a) Same generator and same attack. | |||||||||
| Benign | Benign | 0.9805 | 0.9805 | 0.9998 | Benign | Benign | 0.9749 | 0.9729 | 0.9979 |
| Level-1 | 0.9861 | 0.9861 | 0.9990 | Level-1 | 0.9694 | 0.9693 | 0.9948 | ||
| Level-2 | 0.9638 | 0.9636 | 0.9973 | Level-2 | 0.9610 | 0.9609 | 0.9960 | ||
| Benign | Level-1 | 0.7695 | 0.7560 | 0.9386 | Benign | Level-1 | 0.6789 | 0.6796 | 0.8999 |
| Augmentation (%) | Accuracy | Macro-F1 | AUC |
| 0% | 0.7695 | 0.7560 | 0.9386 |
| 25% | 0.9078 | 0.9127 | 0.9887 |
| 50% | 0.9348 | 0.9378 | 0.9893 |
| 75% | 0.9396 | 0.9427 | 0.9919 |
| 100% | 0.9475 | 0.9514 | 0.9947 |
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
| Dataset | Gen. | Human words | Low AI/Human | High AI/Human | Fully AI/Human |
| XSum | GPT | 629.6 285.0 | 0.967 0.034 | 0.838 0.151 | 1.012 0.235 |
| XSum | Claude | 629.6 285.0 | 0.984 0.028 | 0.934 0.128 | 1.206 0.221 |
| arXiv | GPT | 758.3 344.3 | 0.964 0.026 | 0.884 0.070 | 1.100 0.244 |
| arXiv | Claude | 758.3 344.3 | 0.944 0.094 | 0.809 0.098 | 0.955 0.169 |
| ROCStory | GPT | 61.9 4.2 | 1.079 0.097 | 1.910 0.249 | 1.659 0.206 |
| ROCStory | Claude | 61.9 4.2 | 1.180 0.115 | 2.717 0.388 | 1.760 0.218 |
| Data | Setting | Original (%) | Recalibrated (%) |
| XSum-GPT-4o | Benign | 47.8 | 95.8 |
| XSum-GPT-4o | Level-1 | 51.9 | 84.1 |
| XSum-Claude | Benign | 48.6 | 92.8 |
| XSum-Claude | Level-1 | 51.9 | 78.9 |
| Dataset: XSum | ||||||||
| Settings | F1 per Class | Overall Metrics | ||||||
| Train | Test | Human | Low AI | High AI | Fully AI | Acc | Macro F1 | AUC |
| (a) Same generator and same attack. | ||||||||
| Part I: Benign AI-involved Data generated by GPT-4o | ||||||||
| Benign | Benign | 0.9775 | 0.9721 | 0.9888 | 0.9836 | 0.9805 | 0.9805 | 0.9998 |
| Level-1 | 0.6824 | 0.7151 | 0.6567 | 0.9697 | 0.7695 | 0.7560 | 0.9386 | |
| Dataset: ROCStory | ||||||||
| Settings | F1 per Class | Overall Metrics | ||||||
| Train | Test | Human | Low AI | High AI | Fully AI | Acc | Macro F1 | AUC |
| (a) Same generator and same attack. | ||||||||
| Part I: Benign AI-involved Data generated by GPT-4o | ||||||||
| Benign | Benign | 0.9467 | 0.9468 | 0.9834 | 0.9889 | 0.9666 | 0.9665 | 0.9996 |
| Level-1 | 0.8122 | 0.8604 | 0.8829 | 0.9019 | 0.8712 | 0.8644 | 0.9793 | |
| Dataset: arXiv | ||||||||
| Settings | F1 per Class | Overall Metrics | ||||||
| Train | Test | Human | Low AI | High AI | Fully AI | Acc | Macro F1 | AUC |
| (a) Same generator and same attack. | ||||||||
| Part I: Benign AI-involved Data generated by GPT-4o | ||||||||
| Benign | Benign | 0.8235 | 0.6000 | 0.6226 | 0.8547 | 0.7286 | 0.7252 | 0.9382 |
| Level-1 | 0.5344 | 0.3938 | 0.4405 | 0.8544 | 0.5673 | 0.5558 | 0.8343 | |
Robustness of Machine-Generated Text Detection
Robustness of machine-generated text detection studies whether AI-text detectors remain reliable across shifts, edits, languages, and adversarial attacks.
See all 104 papers