The rapid evolution of large language models necessitates robust machine-generated text detection. Existing paradigms typically follow two isolated tracks. Training-free methods rely on global statistical scalars such as perplexity, while training-based methods utilize semantic hidden states. Both approaches exhibit fundamental vulnerabilities in adversarial scenarios. Global scalars act as lossy compressions that obscure local probabilistic burstiness in interleaved texts, whereas pure semantic models overfit to specific fingerprints and remain susceptible to spoofing. To expose these flaws, we introduce MOSAIC, a comprehensive adversarial benchmark comprising 16000 samples across a full-granularity attack spectrum. To address these challenges, we propose NeuroStat, an end-to-end framework bridging the statistical and semantic gap. NeuroStat captures uncompressed token-level probabilistic logits alongside deep semantic hidden states from a single causal language model backbone. We fuse these heterogeneous signals through Macro-State Residual Modulation, which adaptively calibrates local convolutional features using global uncertainty indicators. Orthogonal and contrastive losses further ensure the learning of complementary representations. Extensive experiments demonstrate that NeuroStat maintains exceptional robustness on MOSAIC compared to the severe degradation of state-of-the-art methods, establishing a new standard for adversarial text detection. Code and the MOSAIC benchmark are available at https://github.com/TencentBAC/NeuroStat.
Figures & tables
Figure 1: Comparison of detection paradigms under adversarial attacks. Existing methods fail due to their reliance on either (a) easily hijacked global statistical scalars (Training-Free) or (b) overfitted semantic patterns that can be disrupted by style manipulation (Training-Based). (c) NeuroStat robustly detects adversarial AIGC by synergizing uncompressed token-level statistical trajectories and deep semantic representations.
Figure 2: The overall architecture of NeuroStat. A CausalLM backbone simultaneously extracts token logits and hidden states. The Probabilistic Trajectories Branch (top) captures local statistical patterns, which are dynamically calibrated by the MSRM mechanism using global uncertainty indicators. Concurrently, the Semantic Artifacts Branch (bottom) extracts contextual styles via attention pooling. The fused representations are jointly optimized with LOrth and LSupCon objectives to ensure complementary and robust feature learning.
Benchmark
Data Statistics
MGT Tasks
Adversarial Attack Coverage
Summary
Size
#Dom.
Gen.
Pol.
Rew.
Inter- leaved
Stat. Hijack
Trans- lation
Para- phrase
Prompt Inj.
Char./ Enc.
Recur- sive
Dom./ Fmt.
#Cat.
#Atk.
TuringBench Uchendu et al. (2021)
168.6K
1
✓
✗
✗
✗
✗
✗
✗
✗
✗
✗
✗
0
0
HC3 Guo et al. (2023)
85K
3
✓
✗
✗
✗
✗
✗
✗
✗
✗
✗
✗
0
0
M4 Wang et al. (2024)
24.5K
4
✓
✗
✗
✗
✗
✗
✗
✗
✗
✗
✓
1
1
MAGE Li et al. (2024)
29K
5
✓
✗
✗
✗
✗
✗
✓
✗
✗
✗
✓
2
2
RAID Dugan et al. (2024)
628.7K
4
✓
✗
✓
✗
✗
✗
✓
✗
✓
✗
✗
2
11
Table 1: Comparison with existing MGT detection benchmarks. Our benchmark provides 36 sub-methods across 8 linguistic granularity levels with full category coverage, generated via uniform random model assignment over 6 frontier LLMs. ✓ = supported; ✗ = not supported. Comprehensive details are provided in the Appendix Sec. B
MIRAGE-DIG (Disjoint-Input Generation)
Methods
Generate
Polish
Rewrite
AUROC
Accuracy
MCC
TPR@5%
AUROC
Accuracy
MCC
TPR@5%
AUROC
Accuracy
MCC
TPR@5%
Likelihood Solaiman et al. (2019)
0.4936
0.5091
0.0183
0.0147
0.4653
0.5000
0.0000
0.0214
0.4337
0.5000
0.0000
0.0148
LogRank Ippolito et al. (2020)
0.4992
0.5128
0.0260
0.0220
0.4512
0.5000
0.0000
0.0195
0.4225
0.5000
0.0000
0.0132
Entropy Gehrmann et al. (2019)
0.6522
0.6150
0.2543
0.1099
0.5543
0.5417
0.1247
0.0954
0.5805
0.5566
0.1650
0.1189
RoBERTa-Base Liu et al. (2019)
0.5523
0.5397
0.1434
0.1250
0.4859
0.5010
0.0088
0.0460
0.5020
0.5049
0.0293
0.0569
Table 2: Experimental results across three tasks (Generate, Polish, Rewrite). Metrics include AUROC, Accuracy, MCC, and TPR@5%. Best results are highlighted in bold.
Robustness Evaluation on the MOSAIC Adversarial Benchmark
Methods
AUROC across 8 Adversarial Attack Categories
Overall Performance (%)
Interleaved
Stat. Hijack
Translation
Paraphrase
Prompt Inj.
Char./Enc.
Recursive
Dom./Fmt.
AUROC
TPR@5%
Likelihood Solaiman et al. (2019)
49.8
30.6
62.8
47.0
47.8
34.1
46.4
50.0
46.1
3.5
LogRank Ippolito et al. (2020)
50.3
69.9
37.9
53.4
53.1
65.5
54.5
51.2
54.5
9.3
Entropy Gehrmann et al. (2019)
48.2
65.1
40.8
50.3
50.1
60.7
49.9
49.5
51.8
6.9
RoBERTa-Base Liu et al. (2019)
47.6
46.8
55.6
50.0
45.2
46.4
47.9
43.9
47.9
3.0
Table 3: Adversarial robustness evaluation on the MOSAIC benchmark. The left section reports the AUROC scores across all 8 fine-grained attack categories, demonstrating the detectors’ resilience against specific spoofing strategies (e.g., Statistical Hijacking, Interleaved texts). The right section reports the overall detection performance. Best results are highlighted in bold , and second-best results are underlined .
Figure 3: t-SNE visualization of TF, TB, and fused representations. The fusion space shows better class separation.
Figure 4: Reliability diagrams of NeuroStat vs. DetectAnyLLM. Lower ECE indicates better calibration.
Configuration
TF
TB
MSRM
MIRAGE DIG (AUROC)
Polish
Rewrite
Generate
TB-Only (Seq. Cls.)
×
✓
×
0.8990
0.8918
0.9787
TF + TB (Concat)
✓
✓
×
0.9280
0.9145
0.9934
NeuroStat (Full)
✓
✓
✓
0.9576
0.9490
0.9955
Table 4: Ablation on core architectural components. “TB-Only” is a standard sequence classification model.
Configuration
LCE
LSupCon
LOrth
MIRAGE DIG (AUROC)
Polish
Rewrite
Generate
CE Only
✓
×
×
0.8859
0.8798
0.9850
+ LSupCon
✓
✓
×
0.9179
0.9124
0.9922
+ LOrth
✓
×
✓
0.9129
0.9051
0.9865
NeuroStat (Full)
✓
✓
✓
0.9576
0.9490
0.9955
Table 5: Impact of auxiliary losses. All variants use the full architecture, differing only in the loss configuration.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: The full-granularity adversarial attack taxonomy of the MOSAIC benchmark. The sunburst chart illustrates the hierarchical structure of our dataset, encompassing 8 primary attack categories (inner circle) and 36 fine-grained sub-methods (outer circle). This comprehensive spectrum ensures a rigorous evaluation of detectors against both statistical hijacking and semantic obfuscation.
Source Dataset
Domain
Raw Samples
MIRAGE-BENCH DIG
Multi-domain
16,411
MIRAGE-BENCH SIG
Multi-domain
16,388
AI Detection 3000
Mixed
3,354
PubMed
Biomedical
150
SQuAD
Wikipedia/QA
150
Writing Prompts
Creative Writing
150
Appendix
Table 6: Source datasets used for collecting human seed texts.
ImBD Test Dataset (GPT-3.5 & GPT-4o polished)
Methods
GPT-3.5
GPT-4o
XSum
Writing
PubMed
XSum
Writing
PubMed
Likelihood Solaiman et al. (2019)
0.4982
0.8788
0.5528
0.4396
0.8077
0.4596
LogRank Ippolito et al. (2020)
0.4711
0.8496
0.5597
0.4002
0.7694
0.4472
Entropy Gehrmann et al. (2019)
0.6742
0.3021
0.5662
0.6122
0.2802
0.5899
RoBERTa-Base Liu et al. (2019)
0.5806
0.7225
0.4370
0.4921
0.4774
0.2496
Appendix
Table 7: Detection performance (AUROC) on the standard ImBD test dataset. Best results are highlighted in bold , and second-best results are underlined .
Machine-generated text (MGT) detection requires identifying structurally invariant signals across generation models, rather than relying on model-specific fingerprints. In this respect, we hypothesize that while large language models excel at local semantic consistency, their autoregressive nature results in a specific kind of structural fragility compared to human writing. We propose Luminol-AIDetect, a novel, zero-shot statistical approach that exposes this fragility through coherence disruption. By applying a simple randomized text-shuffling procedure, we demonstrate that the resulting shift in perplexity serves as a principled, model-agnostic discriminant, as MGT displays a characteristic dispersion in perplexity-under-shuffling that differs markedly from the more stable structural variability of human-written text. Luminol-AIDetect leverages this distinction to inform its decision process, where a handful of perplexity-based scalar features are extracted from an input text and its shuffled version, then detection is performed via density estimation and ensemble-based prediction. Evaluated across 8 content domains, 11 adversarial attack types, and 18 languages, Luminol-AIDetect demonstrates state-of-the-art performance, with gains up to 17x lower FPR while being cheaper than prior methods.
Lucio La Cava, Andrea Tagarelli
DIMES Dept., University of Calabria v. P. Bucci 44Z, 87036 Rende, CS, Italy
Existing AI-generated text detectors are vulnerable to attacks that manipulate textual characteristics. In this study, we propose a novel Triospect Detection Framework by using additional perspectives of content (core ideas) and expression (stylistic elements) within a given text. Experiments on two benchmarks involving 17 attacks, 12 domains, and 17 source models demonstrate that Triospect is robust against these attacks. It improves the strong baseline by a significant margin of 22.3% (AUROC) and 13% (TPR01) on the Humanize-16K after-attack subset, and by 9.1% (AUROC) and 22% (TPR01) on the adversarial RAID. This framework marks a pioneering effort in statistical methods to enhance detection reliability against attacks. We release our data and code at https://github.com/baoguangsheng/triospect.
Guangsheng Bao, Lihua Rong, Yanbin Zhao +3
Zhejiang University · Zhejiang University of Technology · Shanghai Polytechnic University +2
The rapid proliferation of large language models (LLMs) has created an urgent need for robust and generalizable detectors of machine-generated text. Existing benchmarks typically evaluate a single detector on a single dataset under ideal conditions, leaving open questions about cross-domain transfer, cross-LLM generalization, and adversarial robustness. We present a comprehensive benchmark evaluating diverse detection approaches across two corpora: HC3 (23,363 human-ChatGPT pairs) and ELI5 (15,000 human-Mistral-7B pairs). Methods include classical classifiers, fine-tuned transformer encoders (BERT, RoBERTa, ELECTRA, DistilBERT, DeBERTa-v3), a CNN, an XGBoost stylometric model, perplexity-based detectors, and LLM-as-detector prompting. Results show that transformer models achieve near-perfect in-distribution performance but degrade under domain shift. The XGBoost stylometric model matches performance while remaining interpretable. LLM-based detectors underperform and are affected by generator-detector identity bias. Perplexity-based methods exhibit polarity inversion, with modern LLM outputs showing lower perplexity than human text, but remain effective when corrected. No method generalizes robustly across domains and LLM sources.
Madhav S. Baidya, S. S. Baidya, Chirag Chawla
Indian Institute of Technology (BHU), Varanasi, India · Indian Institute of Technology Guwahati, India