cs.CLAug 25, 2026

Linear Probing Provides Robust and Efficient Detection of Machine-Generated Text

Authors: Gerrit Quaremba, Hanqi Yan, Elizabeth Black, Denny Vrandecic, Elena Simperl

Organizations: King’s College London · Wikimedia Foundation

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Distinguishing machine-generated text (MGT) from human-written text (HWT) becomes increasingly important due to potential misuse. However, most supervised detectors often degrade out-of-domain (OOD) and require large, diverse training sets. In this work, we analyze the linearity and quality of MGT representations and show that simple linear probes outperform a wide range of detectors while being substantially more sample-efficient. We first show that MGT and HWT latent representations are linearly separable in low-dimensional space, and provide a plausible explanation for this separability through systematic differences in their representation quality. Motivated by these insights, we train two variants of simple linear probes and evaluate them across 4 benchmarks against 16 baselines. Probes consistently improve OOD detection (+11 AUC), requiring solely <100{<}100 samples to reach near-peak performance. We show that this transferability arises because probes recover a shared latent MGT direction that generalizes across diverse settings. Finally, we demonstrate that probing vectors capture a continuous spectrum of ``machineness'', highlighting their potential for fine-grained estimation of AI-edited text. Overall, our work provides insights into latent-space differences between MGT and HWT and demonstrates the potential of linear probes as as robust and sample-efficient MGT detectors. We release our code on~github.

Figures & tables

Figure 1: Projections of Wikipedia human- and machine-text hidden states onto the first two principal components across layers. Takeaway : Machine- and human-text representations become linearly separable as early as layer 6, and this separation remains consistent through the final layers. Appendix A confirms the same pattern across other domains.
Figure 2: In-domain (top) and OOD (bottom) AUC scores for probing middle-layer representations with MLPs of increasing levels of non-linearity. Takeaway : Increasing model complexity consistently reduces performance, supporting the hypothesis that machine- and human-text representations are linearly separable in latent space.
Figure 3: Representation-quality metrics for human and machine latent representations from the perspectives of information content (Entropy, Effective Rank) and geometric structure (Anisotropy, Intrinsic Dimensionality). Takeaway : Machine-text representations are more compressed, concentrated along fewer dominant directions, and located on simpler manifolds than human-written text representations. These structural differences help explain why the two are linearly separable in latent space.
Figure 4: Cosine similarities of probing vectors at layer 16 across all 16 datasets. Takeaway : Probing vectors exhibit high similarity both within and across datasets, suggesting the existence of a shared latent MGT direction.
Figure 5: In-domain AUC scores for selected datasets as a function of training set size. Takeaway : MGT probes achieve strong performance with only 10–100 training samples.
Figure 6: Edit-strength metrics for samples from APT-Eval Saha and Feizi (2025) plotted against probe projection scores. The dashed line shows the linear regression fit, and rr denotes the Pearson correlation coefficient. Takeaway : Projection scores correlate strongly with edit-strength metrics, suggesting that the latent MGT direction captures a continuous spectrum of “machineness.”
Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Projections of human- and machine-generated text hidden states onto the first two principal components across layers. This figure extends Figure 1 to the Reddit, PeerRead, and ArXiv subsets of M4GT Wang et al. (2024) .
Figure 8: Low-dimensional t-SNE projections of human- and machine-written text from different domains (a) and generators (b) in the M4GT dataset Wang et al. (2024) .
Figure 9: Representation-quality metrics for human and machine latent representations from the perspectives of information content (Entropy, Effective Rank) and geometric structure (Anisotropy, Intrinsic Dimensionality). Extended results to other domains than Wikipedia from M4GT Wang et al. (2024) and selected subsets from DetectRL Wu et al. (2024) , RAID Dugan et al. (2024) , MultiSocial Macko et al. (2025) , and TSM Quaremba et al. (2026) .
Figure 10: OOD AUC scores for baseline detectors (top rows) and MGT probes (bottom row). Rows denote the training subsets, while columns denote the test subsets. Darker colors indicate better performance.
Figure 11: Activation projections of paired DetectRL datasets Wu et al. (2024) in the shared PCA space at layer 16. Dashed lines connect the centroids of the human and machine clusters within each dataset.
Figure 12: In-domain AUC scores for selected datasets as a function of training set size. Compared to Figure 5 , we do not reduce the dimensionality of activations.
Figure 13: Histograms of the similarity measures Semantic (Cosine) Similarity, Jaccard Distance, and Levenshtein Distance for the test sets of APT Saha and Feizi (2025) and EditLens Thai et al. (2026) .
Figure 14: Edit-strength metrics plotted against probe projection scores. The dashed line shows the linear regression fit, and rr denotes the Pearson correlation coefficient. This Figure shows the results for EditLens Thai et al. (2026) .
Figure 15: Layer-wise AUC averaged across benchmarks. Vertical lines denote performance plateaus. Takeaway : MGT signals emerge early and remain stable.
More in this topic

AI-Generated Text Detection

AI-generated text detection develops methods to distinguish human-written, machine-generated, and AI-edited text.

See all 167 papers

Explore similar work

CardsList

  1. Luminol-AIDetect: Fast Zero-shot Machine-Generated Text Detection based on Perplexity under Text Shuffling

    Apr 28, 2026Lucio La Cava, Andrea TagarelliAI-Generated Text Detection

    DIMES Dept., University of Calabria v. P. Bucci 44Z, 87036 Rende, CS, Italy

  2. MGTEVAL: An Interactive Platform for Systemtic Evaluation of Machine-Generated Text Detectors

    Apr 28, 2026Yuanfan Li, Qi Zhou, Chengzhengxu Li +5AI-Generated Text DetectionLanguage Model Generation Evaluation

    Faculty of Electronic and Information Engineering, Xi’an Jiaotong University · Queen Mary University of London

  3. MELD: Multi-Task Equilibrated Learning Detector for AI-Generated Text

    May 7, 2026Chenjun Li, Cheng Wan, Johannes C. PaetzoldAI-Generated Text DetectionDomain Generalization

    Cornell University, Ithaca, NY 14853, USA · Weill Cornell Medicine, New York, NY 10021, USA · Cornell Tech, New York, NY 10044, USA