MixDetect: Word-Level Localization and Quantification of AI Editing

Hongrui Bao; Yubing Ren; Zhendong Pan; Fang Fang; Shi Wang; Yanan Cao

cs.CLSep 26, 2026

MixDetect: Word-Level Localization and Quantification of AI Editing

Authors: Hongrui Bao, Yubing Ren, Zhendong Pan, Fang Fang, Shi Wang, Yanan Cao

Organizations: University of the Chinese Academy of Sciences · Institute of Information Engineering · Shandong University · Visual Information Processing and Learning

Abstract

Large language models are increasingly used to edit human-written text rather than generate entire texts from scratch. Conventional AI-text detectors mainly distinguish human-written from fully AI-generated text, while recent methods for AI-edited text typically provide only a text-level label or editing-degree score. We introduce MixDetect, a word-level framework for localizing and quantifying AI editing. MixDetect separately predicts whether each word has been edited and, conditional on editing, how substantial the edit is, allowing editing scope and editing intensity to be estimated separately. During training, source--edited pairs are aligned to construct word-level supervision, while inference requires only the input text. Experiments show that MixDetect accurately localizes AI-edited words, reflects differences in editing intensity, and reveals different scope--intensity patterns across editing degrees and operations. The overall AI editing magnitude increases under additional AI editing, decreases when AI-generated text is edited by humans, and remains nearly unchanged under ordinary human-to-human editing. The aggregated text-level predictions also perform well on binary and ternary AI-text classification and remain effective under domain and generator shifts. These results show that AI editing can be analyzed beyond a single authorship label or editing-degree score by identifying both where AI editing occurs and how substantial the edits are.

Figures & tables

Figure 1: The same overall AI editing magnitude

S

can arise from different editing patterns. Broad, light editing has high scope

Q

but low intensity

M

, whereas concentrated rewriting has lower scope but higher intensity. Word-level predictions preserve this distinction.

Figure 2: Overview of MixDetect . During training, the source–edited pair is aligned to construct word-level supervision for edit occurrence

q_{i}

and conditional intensity

\mu_{i}

. At inference time, the detector receives only the input text and predicts

\hat{q}_{i}

and

\hat{\mu}_{i}

for each word, with

\hat{s}_{i}=\hat{q}_{i}\hat{\mu}_{i}

Training	AUROC	AUPRC	AP text	$\rho_{Q}$	$\rho_{M}$	$\rho_{S}$
Edit-Only	0.885	0.835	0.679	0.878	0.640	0.861
Full	0.885	0.817	0.682	0.873	0.617	0.852

Table 1: Word-level localization and text-level evaluation on the EditLens test split. AUROC and AUPRC evaluate edit localization over valid words, while AP text computes average precision within each text and then averages across texts.

\rho_{Q}

and

\rho_{M}

are partial Spearman correlations for editing scope and intensity, respectively, and

\rho_{S}

is the Spearman correlation for overall AI editing magnitude.

Figure 3: Predicted intensity increases with edit magnitude.

Figure 4: Predicted editing scope, intensity, and overall AI editing magnitude under different requested editing degrees and proportions on APT-Eval.

Figure 5: Changes in editing scope, intensity, and overall AI editing magnitude across different Grammarly editing operations.

\Delta Q

\Delta M

, and

\Delta S

denote the magnitudes of the paired score changes relative to the corresponding human source.

Detector	Editor	$\Delta S$ (P1)	$\Delta S$ (P2)	$\Delta S$ (P3)
Edit-Only	Llama-3.1-70B	+0.097	+0.150	+0.158
Edit-Only	GPT-4o	+0.063	+0.118	+0.077
Full	Llama-3.1-70B	+0.048	+0.061	+0.114
Full	GPT-4o	+0.030	+0.073	+0.041

Table 2: Paired changes in the overall AI editing magnitude

S

on BEEMO under three AI editing prompts (P1–P3).

	Binary Classification		Ternary Classification
Method	Any AI	Full AI	EditLens	Enron	Unseen Llama
EditLens	0.921	0.943	0.881	0.674	0.860
EchoPrompt	0.722	0.768	0.607	0.660	0.432
Fast-DetectGPT	0.557	0.595	0.414	0.447	0.592
Binoculars	0.566	0.595	0.418	0.468	0.576
LastDE++	0.550	0.540	0.381	0.292	0.382

Table 3: AI-text classification results on the EditLens test split and under domain and generator shifts. All binary and ternary results are reported with macro-F1.

Setting	Prediction	Word AUROC	Text-level $\rho$	Ternary	Separate $Q,M$
Edit-Only	Direct- $s$	0.945 $\pm$ 0.002	0.877 $\pm$ 0.005	–	No
	$q$ only	0.885 $\pm$ 0.005	0.822 $\pm$ 0.009	0.850 $\pm$ 0.013	No
	$\mu$ only	–	0.409 $\pm$ 0.077	0.411 $\pm$ 0.011	No
	$q\mu$	0.885 $\pm$ 0.005	0.861 $\pm$ 0.011	0.848 $\pm$ 0.011	Yes
Full	Direct- $s$	0.945 $\pm$ 0.000	0.880 $\pm$ 0.002	–	No
	$q$ only	0.885 $\pm$ 0.004	0.813 $\pm$ 0.004	0.922 $\pm$ 0.003	No

Table 4: Ablation of the prediction formulation. Word AUROC evaluates word-level edit localization, text-level

\rho

measures the Spearman correlation with the reference overall AI editing magnitude, and Ternary reports macro-F1 for human-written, AI-edited, and AI-generated classification.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Qualitative word-level example from the EditLens test set. Word colors indicate the predicted word-level AI-editing score

\hat{s}_{i}=\hat{q}_{i}\hat{\mu}_{i}

, with text-level

\hat{Q}

\hat{M}

, and

\hat{S}

shown on the right.

AI-Generated Text Detection

AI-generated text detection develops methods to distinguish human-written, machine-generated, and AI-edited text.

See all 167 papers

Operation-Guided Progressive Human-to-AI Text Transformation Benchmark for Multi-Granularity AI-Text Detection
Jun 4, 2026Sondos Mahmoud Bsharat, Jiacheng Liu, Xiaohan Zhao +9AI-Generated Text Detection Human-AI Co-Creation
Mohamed bin Zayed University of Artificial Intelligence · University College London
READER: Reasoning-Enhanced AI-Generated Text Detection
May 24, 2026Pingfan Su, Kai Ye, Shijin Gong +4AI-Generated Text Detection
Department of Statistics, London School of Economics and Political Science · School of Management, University of Science and Technology of China · School of Mathematics, University of Birmingham
ARB: A Matched Authorship-Rewriting Benchmark Dataset for AI-Text Detector Evaluation
Jul 31, 2026Gaetano Perrone, Simon Pietro RomanoAI-Generated Text Detection Robustness of Machine-Generated Text Detection
Department of Electrical Engineering and Information Technology, University of Napoli Federico II, Via Claudio 21, Naples, 80125, Italy

Jun 4, 2026J/K move · Enter open · S save

Operation-Guided Progressive Human-to-AI Text Transformation Benchmark for Multi-Granularity AI-Text Detection

Sondos Mahmoud Bsharat, Jiacheng Liu, Xiaohan Zhao, Tianjun Yao+8

Mohamed bin Zayed University of Artificial Intelligence · University College London

AI-Generated Text Detection Human-AI Co-Creation

Open paper AskPDF arXiv

As AI writing assistants become increasingly integrated into real-world drafting and revision workflows, many documents are no longer purely human-written or AI-generated, but instead result from progressive human-AI co-editing. However, existing AI-text detection benchmarks largely focus on final outputs and provide limited understanding of how AI authorship signals emerge, accumulate, or disappear throughout the revision process. We introduce OpAI-Bench, an operation-guided benchmark for studying progressive human-to-AI text transformation across document, sentence, token, and span granularities. Starting from human-written documents, OpAI-Bench constructs nine sequentially revised versions for each sample under predefined AI coverage levels and five representative AI edit operations, covering four domains while preserving complete authorship provenance at multiple granularities. The benchmark supports comprehensive evaluation with 8 document-level detectors, 7 sentence-level detectors, and 2 fine-grained token/span-level detectors. Experiments reveal that AI-text detectability is governed not only by the proportion of AI-edited content, but also by edit operation, domain, and cumulative revision history. Interestingly, we notice that mixed-authorship intermediate versions are often harder to detect than both fully human and heavily AI-edited endpoints, exposing non-monotonic detection patterns missed by existing benchmarks. OpAI-Bench provides a controlled testbed for analyzing whether, when, and how AI-assisted writing becomes detectable under realistic progressive editing scenarios. Our code and benchmark are available at https://github.com/VILA-Lab/OpAI-Bench.

Abstract

AI-Generated Text Detection

Explore similar work

Operation-Guided Progressive Human-to-AI Text Transformation Benchmark for Multi-Granularity AI-Text Detection

READER: Reasoning-Enhanced AI-Generated Text Detection

ARB: A Matched Authorship-Rewriting Benchmark Dataset for AI-Text Detector Evaluation