Authorship signals matter in settings where writing style carries identity: digital forensics, plagiarism analysis, account linking, misinformation investigation, and machine-generated text detection. Yet current authorship benchmarks remain fragmented, usually covering only a narrow language set, a single genre, or a limited document-length regime, which makes it difficult to assess whether modern representations truly generalize. We introduce AuthBench, a large-scale multilingual benchmark for authorship representation that is designed to make this evaluation broad, standardized, and realistic. AuthBench contains 428,150 documents written by 153,825 individuals across ten widely used languages, 9 primary genres, 66 fine-grained genres, and four document-length buckets. It supports two complementary tasks: authorship attribution, formulated as same-author retrieval and authorship verification, formulated as same-author binary decision. We benchmark 47 neural models and three non-neural baselines under a unified zero-shot protocol. Results show that authorship representation remains far from solved: the best retrieval model reaches only 0.258 Success@5, while the best verification model achieves 0.076 EER and 0.968 ROC-AUC. The leaderboard also reveals a meaningful task split, with different model families leading retrieval and verification, and large performance differences across languages, genres, and lengths. These findings position AuthBench not only as a new benchmark, but as a diagnostic resource for studying when and why authorship representations succeed or fail. We release AuthBench, its evaluation toolkit, and benchmark data at https://github.com/mao-code/AuthBench and https://huggingface.co/datasets/MaoXun/AuthBench.
Figures & tables
Benchmark
Year
Authors
Docs
Scope
Coverage
Evaluation
Multi- lingual
Multi- genre
Length bins
Standard splits
Attribution
Verification
Blogs50 [ 59 ]
2006
50
66K
English blogs
✗
✗
✗
✗
✓
✗
MUD (adapted corpus) [ 20 , 44 ]
2021
1.07M
321.7M
English Reddit
✗
✗
✗
✗
✗
✗
MARC [ 19 ]
2020
1.15M
1.26M
Multilingual reviews
✓
✗
✗
✓
✗
✗
PAN 2021 AV [ 33 ]
2021
278.2K
73.0K pairs
English fanfiction
✗
✗
✗
✓
✗
✓
PAN 2022 AV [ 34 ]
2022
112
22.7K pairs
English cross-discourse
✗
✓
✗
✓
✗
✓
Table 1: Comparison of AuthBench with representative prior authorship resources.
Category
#Docs
Share
Overall
Total
428,150
100.0%
Languages
10
–
Authors
153,825
–
Avg. docs per author
2.78
–
Genres (fine-grained)
66
–
Table 2: Statistics of our AuthBench.
Figure 1: Overview of the current AuthBench profile for document proportion across languages, genres and document lengths.
Figure 2: AuthBench construction pipeline. Full details are provided in Appendix A .
Figure 3: Examples in our AuthBench.
Model
Model Size
S@5 ↑
R@5 ↑
nDCG@5 ↑
MRR ↑
ROC-AUC ↑
EER ↓
LLMs (instruction-tuned)
llama3-8b-instruct
8B
0.251
0.247
0.206
0.209
0.967
0.080
llama3.1-8b-instruct
8B
0.247
0.243
0.204
0.208
0.968
0.076
qwen2.5-7b-instruct
7.6B
0.207
0.203
0.168
0.172
0.964
0.083
qwen3-4b-instruct
4B
0.197
0.193
0.159
0.163
0.958
0.094
LLMs (base)
Table 3: Main results on AuthBench (test split). The retrieval leaderboard now reports S@5, R@5, nDCG@5, and MRR; the verification leaderboard reports both ROC-AUC and EER. Bold entries mark the best result within each model group for a metric, and underlined entries mark the second best. The full 50-model evaluation, including all three non-neural baselines, is deferred to Appendix B .
Model
Model Size
ar
de
en
es
fr
hi
ja
ko
ru
zh
LLMs (instruction-tuned)
llama3-8b-instruct
8B
0.226
0.244
0.261
0.265
0.269
0.289
0.243
0.173
0.164
0.382
llama3.1-8b-instruct
8B
0.211
0.245
0.259
0.265
0.265
0.297
0.229
0.177
0.165
0.373
qwen2.5-7b-instruct
7.6B
0.182
0.210
0.201
0.208
0.214
0.215
0.215
0.143
0.143
0.343
qwen3-4b-instruct
4B
0.187
0.195
0.187
0.203
0.198
0.229
0.212
0.143
0.143
0.304
LLMs (base)
Table 4: Results by language (S@5). The updated slice table keeps a compact representative subset; complete language-wise S@5 and EER tables are reported in Appendix B .
Model
Model Size
blog
ecommerce_reviews
literature
media_reviews
news
poetry
qna
research_paper
social_media
LLMs (instruction-tuned)
llama3-8b-instruct
8B
0.213
0.107
0.521
0.070
0.302
0.639
0.650
0.769
0.206
llama3.1-8b-instruct
8B
0.220
0.112
0.518
0.035
0.293
0.583
0.618
0.787
0.204
qwen2.5-7b-instruct
7.6B
0.162
0.084
0.496
0.035
0.242
0.583
0.567
0.719
0.167
qwen3-4b-instruct
4B
0.149
0.080
0.462
0.053
0.227
0.611
0.548
0.688
0.160
LLMs (base)
Table 5: Results by primary genre (S@5). Complete genre-wise S@5 and EER tables are reported in Appendix B .
Model
Model Size
short
medium
long
extra_long
LLMs (instruction-tuned)
llama3-8b-instruct
8B
0.150
0.199
0.432
0.539
llama3.1-8b-instruct
8B
0.142
0.196
0.427
0.531
qwen2.5-7b-instruct
7.6B
0.103
0.159
0.379
0.496
qwen3-4b-instruct
4B
0.081
0.150
0.368
0.488
LLMs (base)
Table 6: Results by length bucket (S@5). Complete length-wise S@5 and EER tables are reported in Appendix B .
Appendix figures & tables19 assets
Supplementary material from the paper’s appendix.
Appendix
Model
Model Size
Repository / Implementation
LLMs (instruction-tuned)
llama3-8b-instruct
8B
meta-llama/Meta-Llama-3-8B-Instruct
llama3.1-8b-instruct
8B
meta-llama/Llama-3.1-8B-Instruct
qwen2.5-3b-instruct
3.1B
Qwen/Qwen2.5-3B-Instruct
qwen2.5-7b-instruct
7.6B
Qwen/Qwen2.5-7B-Instruct
deepseek-llm-7b-chat
7B
deepseek-ai/deepseek-llm-7b-chat
Appendix
Table 7: Models and baselines evaluated in AuthBench. We list every system appearing in the updated leaderboard together with its model size and Hugging Face repository or baseline implementation note.
Model
Model Size
S@5 ↑
R@5 ↑
nDCG@5 ↑
MRR ↑
ROC-AUC ↑
EER ↓
LLMs (instruction-tuned)
llama3-8b-instruct
8B
0.251
0.247
0.206
0.209
0.967
0.080
llama3.1-8b-instruct
8B
0.247
0.243
0.204
0.208
0.968
0.076
qwen2.5-3b-instruct
3.1B
0.213
0.209
0.172
0.176
0.962
0.091
qwen2.5-7b-instruct
7.6B
0.207
0.203
0.168
0.172
0.964
0.083
deepseek-llm-7b-chat
7B
0.207
0.204
0.170
0.174
0.940
0.128
Appendix
Table 8: Updated overall zero-shot results on AuthBench. Authorship attribution is evaluated with Success@5 (S@5), Recall@5 (R@5), nDCG@5, and MRR (higher is better). Authorship verification is evaluated with ROC-AUC (higher is better) and EER (lower is better). All 47 neural models and the three non-neural baselines are included.
Figure 4: Overall AuthBench performance by Success@5 (S@5).
Figure 5: Overall AuthBench performance by Recall@5 (R@5).
Figure 6: Overall AuthBench performance by nDCG@5.
Figure 7: Overall AuthBench performance by MRR.
Figure 8: Overall AuthBench performance by ROC-AUC.
Figure 9: Overall AuthBench performance by EER.
Model
Model Size
ar
de
en
es
fr
hi
ja
ko
ru
zh
LLMs (instruction-tuned)
deepseek-llm-7b-chat
7B
0.170
0.203
0.223
0.223
0.213
0.204
0.184
0.135
0.126
0.347
llama3-8b-instruct
8B
0.226
0.244
0.261
0.265
0.269
0.289
0.243
0.173
0.164
0.382
llama3.1-8b-instruct
8B
0.211
0.245
0.259
0.265
0.265
0.297
0.229
0.177
0.165
0.373
qwen2.5-3b-instruct
3.1B
0.207
0.206
0.206
0.215
0.220
0.218
0.223
0.145
0.150
0.336
qwen2.5-7b-instruct
7.6B
0.182
0.210
0.201
0.208
0.214
0.215
0.215
0.143
0.143
0.343
Appendix
Table 9: Language-wise Success@5 on AuthBench (full results).
Model
Model Size
ar
de
en
es
fr
hi
ja
ko
ru
zh
LLMs (instruction-tuned)
deepseek-llm-7b-chat
7B
0.125
0.103
0.141
0.089
0.103
0.071
0.061
0.081
0.123
0.074
llama3-8b-instruct
8B
0.073
0.064
0.099
0.064
0.061
0.051
0.042
0.054
0.112
0.057
llama3.1-8b-instruct
8B
0.073
0.062
0.096
0.066
0.062
0.045
0.042
0.055
0.114
0.059
qwen2.5-3b-instruct
3.1B
0.072
0.072
0.131
0.074
0.068
0.054
0.043
0.060
0.119
0.062
qwen2.5-7b-instruct
7.6B
0.075
0.070
0.118
0.068
0.064
0.048
0.044
0.059
0.120
0.056
Appendix
Table 10: Language-wise EER on AuthBench (full results).
Model
Model Size
blog
ecomm
literature
media
news
poetry
qna
research
social
LLMs (instruction-tuned)
deepseek-llm-7b-chat
7B
0.196
0.110
0.461
0.053
0.243
0.556
0.541
0.713
0.166
llama3-8b-instruct
8B
0.213
0.107
0.521
0.070
0.302
0.639
0.650
0.769
0.206
llama3.1-8b-instruct
8B
0.220
0.112
0.518
0.035
0.293
0.583
0.618
0.787
0.204
qwen2.5-3b-instruct
3.1B
0.171
0.100
0.483
0.035
0.252
0.639
0.573
0.731
0.171
qwen2.5-7b-instruct
7.6B
0.162
0.084
0.496
0.035
0.242
0.583
0.567
0.719
0.167
Appendix
Table 11: Primary-genre Success@5 on AuthBench (full results).
Model
Model Size
blog
ecomm
literature
media
news
poetry
qna
research
social
LLMs (instruction-tuned)
deepseek-llm-7b-chat
7B
0.133
0.094
0.076
0.078
0.111
0.090
0.110
0.029
0.132
llama3-8b-instruct
8B
0.106
0.074
0.036
0.078
0.070
0.064
0.063
0.011
0.081
llama3.1-8b-instruct
8B
0.108
0.064
0.037
0.080
0.065
0.068
0.067
0.012
0.081
qwen2.5-3b-instruct
3.1B
0.137
0.086
0.048
0.079
0.075
0.059
0.098
0.031
0.091
qwen2.5-7b-instruct
7.6B
0.131
0.080
0.051
0.075
0.070
0.061
0.087
0.019
0.085
Appendix
Table 12: Primary-genre EER on AuthBench (full results).
Model
Model Size
short
medium
long
extra_long
LLMs (instruction-tuned)
deepseek-llm-7b-chat
7B
0.106
0.161
0.371
0.524
llama3-8b-instruct
8B
0.150
0.199
0.432
0.539
llama3.1-8b-instruct
8B
0.142
0.196
0.427
0.531
qwen2.5-3b-instruct
3.1B
0.101
0.164
0.385
0.508
qwen2.5-7b-instruct
7.6B
0.103
0.159
0.379
0.496
Appendix
Table 13: Length-bucket Success@5 on AuthBench (full results).
Model
Model Size
short
medium
long
extra_long
LLMs (instruction-tuned)
deepseek-llm-7b-chat
7B
0.185
0.131
0.096
0.072
llama3-8b-instruct
8B
0.109
0.082
0.060
0.055
llama3.1-8b-instruct
8B
0.104
0.078
0.061
0.051
qwen2.5-3b-instruct
3.1B
0.120
0.090
0.073
0.067
qwen2.5-7b-instruct
7.6B
0.110
0.082
0.069
0.051
Appendix
Table 14: Length-bucket EER on AuthBench (full results).
Figure 10: Token-length distribution per language. Each box plot summarizes the document-length distribution for one language in the current AuthBench release, highlighting cross-language differences in median length, spread, and long-tail behavior.
Figure 11: Fine-grained subgenre distribution by language. Each donut chart summarizes one language slice using a shared subgenre palette, while lower-frequency subgenres are grouped into Other to keep the figure legible.
Source
Lang.
Primary genre(s)
Scale
Author labels
Reference
Exorde
Multi
Social / news / forums
65M+/week
Yes (author hash)
[ 12 ]
Babel Briefings
30+
News headlines
4.7M
Partial (publisher / org)
[ 24 , 25 ]
Amazon Reviews Multi (MARC)
6
E-commerce reviews
200k+/lang
Yes (reviewer IDs)
[ 19 , 29 ]
Blog Authorship Corpus
en
Blogs
681k posts
Yes
[ 46 , 5 ]
arXiv Abstracts (metadata snapshot)
en
Research papers
1.7M+
Yes (first author from metadata)
[ 4 , 10 ]
Xiaohongshu / Weibo
zh
Social media
11k+
Yes (user IDs)
[ 58 ]
Appendix
Table 15: Raw sources used for the current AuthBench release. “Scale” reflects either the approximate size of the upstream source or, for crawl-dependent sources, the size of the processed corpus produced by the current collection setting before final benchmark selection.
Source
#Docs
Share
Exorde
94,231
22.0%
Wikisource
78,984
18.4%
Babel Briefings
73,676
17.2%
YouTube comments
71,808
16.8%
Blog Authorship Corpus
22,494
5.3%
Project Gutenberg
18,739
4.4%
Appendix
Table 16: Source composition of the current combined AuthBench export. All 17 configured sources are represented in the materialized benchmark; the table reports their realized document counts after filtering, redundancy reduction, cross-phase merge cleanup, and final selection.
Source
License / terms (summary)
Release mode
Notes / pointer
Exorde
MIT (dataset card)
Tier A
Provider page explicitly states MIT.
Babel Briefings
CC BY-NC-SA 4.0 (dataset card)
Tier A
Attribution, non-commercial use, and share-alike obligations apply.
Amazon Reviews Multi (MARC)
Provider-restricted / non-permissive redistribution; dataset card does not expose a standard open license
Tier B
Safer to require reconstruction from the original release rather than redistribute normalized text.
Blog Authorship Corpus
No clear standardized redistribution license found in the public host materials
Tier B
Conservative manifest-only treatment is safer than direct text redistribution.
arXiv Abstracts (metadata snapshot)
Metadata access is available via arXiv services, but article licenses vary by paper
Tier B
We treat this as metadata-derived text and do not redistribute PDFs or assume one uniform article license.
Xiaohongshu / Weibo
Platform / provider terms for redistribution are unclear
Tier B
Manifest-only unless explicit redistribution permission is documented.
Appendix
Table 17: Licensing / terms and release mode by source. “Release mode” indicates whether we redistribute normalized text (Tier A) or provide manifest-only reconstruction (Tier B). This table covers all 17 sources in the current release.
Our results show that contrastive learning outperforms a classification-based approach to authorship verification under the tested settings. We identify loss function, batch size, training duration, pre-trained model, input context length, and random text span data augmentation as important factors of model performance. Based on these considerations, we develop a ModernBERT Bi-Encoder model that achieves 98.4% accuracy on the PAN21 authorship verification task.
Authorship verification (AV) assumes that an author's writing style remains sufficiently stable to distinguish it from that of other writers. In practice, however, this assumption is challenged by distribution shifts caused by changes in genre, time, and AI-assisted writing. Existing AV benchmarks typically study these factors in isolation and focus predominantly on English, limiting our understanding of model robustness under realistic conditions. We introduce AVShift, the first German benchmark for systematically evaluating AV under multiple distribution shifts. AVShift comprises over 150,000 text pairs spanning three genres and 21 years, enabling controlled evaluation of cross-genre, temporal, and AI-era shifts within a unified framework. We benchmark representative feature-based, embedding-based, and LLM-based approaches. Our experiments show that fine-tuned LLMs generalize best across genres and benefit substantially from stylistically diverse training data. We further demonstrate that temporal drift is one of the strongest factors affecting AV, with performance degrading significantly as the time gap between documents increases. In contrast, we find no evidence of a measurable AI-era distribution shift within AVShift. Finally, our feature analysis reveals stylistic features that remain stable across genres, while their relative importance varies depending on the specific genre transition. We release AVShift and our code for future research.
Personalized text generation aims to make LLMs write in a specific individual's style, yet existing benchmarks measure task accuracy or preference alignment rather than whether the model's output actually resembles the target author's writing. We introduce PersonalBench, a benchmark that evaluates inference-time personalization methods through three independent lenses: LUAR (a trained authorship verification model), an LLM-as-judge, and automated stylometrics. Across 50 authors, 1,000 generations, and two model families (Qwen 3, GLM-4), we find that personalization methods do produce author-differentiated output (LUAR discriminates target authors within generated text at AUC=0.918) but this differentiation never crosses the human-LLM boundary. All methods achieve LUAR similarity to real authors in the range 0.484-0.508, below the cross-author human floor of 0.626 (ceiling 0.756). The LLM's own authorship fingerprint dominates: generated text is more distant from any human author than random humans are from each other. Methods are statistically indistinguishable from each other on LUAR (spread 0.024) despite appearing differentiated on the LLM judge, a discrepancy we trace to circularity between trait extraction and profile extraction. We validate that LUAR reliably measures authorship in our corpus (AUC=0.76 single-post, 0.96 multi-post). We release PersonalBench as a calibrated measuring stick: inference-time personalization modulates the LLM's style but does not bridge the gap to human authorship.