AuthBench: A Large-Scale Multilingual Benchmark for Authorship Representation across Genres and Lengths

MaoXun Huang; Zhenxing Zhang; Claire Cardie

cs.CLSep 6, 2026

AuthBench: A Large-Scale Multilingual Benchmark for Authorship Representation across Genres and Lengths

Authors: MaoXun Huang, Zhenxing Zhang, Claire Cardie

Organizations: Department of Computer Science, Cornell University Ithaca, NY, United States

Abstract

Authorship signals matter in settings where writing style carries identity: digital forensics, plagiarism analysis, account linking, misinformation investigation, and machine-generated text detection. Yet current authorship benchmarks remain fragmented, usually covering only a narrow language set, a single genre, or a limited document-length regime, which makes it difficult to assess whether modern representations truly generalize. We introduce AuthBench, a large-scale multilingual benchmark for authorship representation that is designed to make this evaluation broad, standardized, and realistic. AuthBench contains 428,150 documents written by 153,825 individuals across ten widely used languages, 9 primary genres, 66 fine-grained genres, and four document-length buckets. It supports two complementary tasks: authorship attribution, formulated as same-author retrieval and authorship verification, formulated as same-author binary decision. We benchmark 47 neural models and three non-neural baselines under a unified zero-shot protocol. Results show that authorship representation remains far from solved: the best retrieval model reaches only 0.258 Success@5, while the best verification model achieves 0.076 EER and 0.968 ROC-AUC. The leaderboard also reveals a meaningful task split, with different model families leading retrieval and verification, and large performance differences across languages, genres, and lengths. These findings position AuthBench not only as a new benchmark, but as a diagnostic resource for studying when and why authorship representations succeed or fail. We release AuthBench, its evaluation toolkit, and benchmark data at https://github.com/mao-code/AuthBench and https://huggingface.co/datasets/MaoXun/AuthBench.

Figures & tables

Benchmark	Year	Authors	Docs	Scope	Coverage			Evaluation
					Multi- lingual	Multi- genre	Length bins	Standard splits	Attribution	Verification
Blogs50 [ 59 ]	2006	50	66K	English blogs	✗	✗	✗	✗	✓	✗
MUD (adapted corpus) [ 20 , 44 ]	2021	1.07M	321.7M	English Reddit	✗	✗	✗	✗	✗	✗
MARC [ 19 ]	2020	1.15M	1.26M	Multilingual reviews	✓	✗	✗	✓	✗	✗
PAN 2021 AV [ 33 ]	2021	278.2K	73.0K pairs	English fanfiction	✗	✗	✗	✓	✗	✓
PAN 2022 AV [ 34 ]	2022	112	22.7K pairs	English cross-discourse	✗	✓	✗	✓	✗	✓

Table 1: Comparison of AuthBench with representative prior authorship resources.

Overall
Category	#Docs	Share
Total	428,150	100.0%
Languages	10	–
Authors	153,825	–
Avg. docs per author	2.78	–
Genres (fine-grained)	66	–

Table 2: Statistics of our AuthBench.

Figure 1: Overview of the current AuthBench profile for document proportion across languages, genres and document lengths.

Figure 2: AuthBench construction pipeline. Full details are provided in Appendix A .

Panel of Figure 3 — Figure 3: Examples in our AuthBench.

LLMs (instruction-tuned)
Model	Model Size	S@5 $\uparrow$	R@5 $\uparrow$	nDCG@5 $\uparrow$	MRR $\uparrow$	ROC-AUC $\uparrow$	EER $\downarrow$
llama3-8b-instruct	8B	0.251	0.247	0.206	0.209	0.967	0.080
llama3.1-8b-instruct	8B	0.247	0.243	0.204	0.208	0.968	0.076
qwen2.5-7b-instruct	7.6B	0.207	0.203	0.168	0.172	0.964	0.083
qwen3-4b-instruct	4B	0.197	0.193	0.159	0.163	0.958	0.094
LLMs (base)

Table 3: Main results on AuthBench (test split). The retrieval leaderboard now reports S@5, R@5, nDCG@5, and MRR; the verification leaderboard reports both ROC-AUC and EER. Bold entries mark the best result within each model group for a metric, and underlined entries mark the second best. The full 50-model evaluation, including all three non-neural baselines, is deferred to Appendix B .

LLMs (instruction-tuned)
Model	Model Size	ar	de	en	es	fr	hi	ja	ko	ru	zh
llama3-8b-instruct	8B	0.226	0.244	0.261	0.265	0.269	0.289	0.243	0.173	0.164	0.382
llama3.1-8b-instruct	8B	0.211	0.245	0.259	0.265	0.265	0.297	0.229	0.177	0.165	0.373
qwen2.5-7b-instruct	7.6B	0.182	0.210	0.201	0.208	0.214	0.215	0.215	0.143	0.143	0.343
qwen3-4b-instruct	4B	0.187	0.195	0.187	0.203	0.198	0.229	0.212	0.143	0.143	0.304
LLMs (base)

Table 4: Results by language (S@5). The updated slice table keeps a compact representative subset; complete language-wise S@5 and EER tables are reported in Appendix B .

LLMs (instruction-tuned)
Model	Model Size	blog	ecommerce_reviews	literature	media_reviews	news	poetry	qna	research_paper	social_media
llama3-8b-instruct	8B	0.213	0.107	0.521	0.070	0.302	0.639	0.650	0.769	0.206
llama3.1-8b-instruct	8B	0.220	0.112	0.518	0.035	0.293	0.583	0.618	0.787	0.204
qwen2.5-7b-instruct	7.6B	0.162	0.084	0.496	0.035	0.242	0.583	0.567	0.719	0.167
qwen3-4b-instruct	4B	0.149	0.080	0.462	0.053	0.227	0.611	0.548	0.688	0.160
LLMs (base)

Table 5: Results by primary genre (S@5). Complete genre-wise S@5 and EER tables are reported in Appendix B .

LLMs (instruction-tuned)
Model	Model Size	short	medium	long	extra_long
llama3-8b-instruct	8B	0.150	0.199	0.432	0.539
llama3.1-8b-instruct	8B	0.142	0.196	0.427	0.531
qwen2.5-7b-instruct	7.6B	0.103	0.159	0.379	0.496
qwen3-4b-instruct	4B	0.081	0.150	0.368	0.488
LLMs (base)

Table 6: Results by length bucket (S@5). Complete length-wise S@5 and EER tables are reported in Appendix B .

Appendix figures & tables19 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Model Size	Repository / Implementation
LLMs (instruction-tuned)
llama3-8b-instruct	8B	meta-llama/Meta-Llama-3-8B-Instruct
llama3.1-8b-instruct	8B	meta-llama/Llama-3.1-8B-Instruct
qwen2.5-3b-instruct	3.1B	Qwen/Qwen2.5-3B-Instruct
qwen2.5-7b-instruct	7.6B	Qwen/Qwen2.5-7B-Instruct
deepseek-llm-7b-chat	7B	deepseek-ai/deepseek-llm-7b-chat

Appendix

Table 7: Models and baselines evaluated in AuthBench. We list every system appearing in the updated leaderboard together with its model size and Hugging Face repository or baseline implementation note.

Model	Model Size	S@5 $\uparrow$	R@5 $\uparrow$	nDCG@5 $\uparrow$	MRR $\uparrow$	ROC-AUC $\uparrow$	EER $\downarrow$
LLMs (instruction-tuned)
llama3-8b-instruct	8B	0.251	0.247	0.206	0.209	0.967	0.080
llama3.1-8b-instruct	8B	0.247	0.243	0.204	0.208	0.968	0.076
qwen2.5-3b-instruct	3.1B	0.213	0.209	0.172	0.176	0.962	0.091
qwen2.5-7b-instruct	7.6B	0.207	0.203	0.168	0.172	0.964	0.083
deepseek-llm-7b-chat	7B	0.207	0.204	0.170	0.174	0.940	0.128

Appendix

Table 8: Updated overall zero-shot results on AuthBench. Authorship attribution is evaluated with Success@5 (S@5), Recall@5 (R@5), nDCG@5, and MRR (higher is better). Authorship verification is evaluated with ROC-AUC (higher is better) and EER (lower is better). All 47 neural models and the three non-neural baselines are included.

Figure 4: Overall AuthBench performance by Success@5 (S@5).

Figure 5: Overall AuthBench performance by Recall@5 (R@5).

Figure 6: Overall AuthBench performance by nDCG@5.

Figure 7: Overall AuthBench performance by MRR.

Figure 8: Overall AuthBench performance by ROC-AUC.

Figure 9: Overall AuthBench performance by EER.

Model	Model Size	ar	de	en	es	fr	hi	ja	ko	ru	zh
LLMs (instruction-tuned)
deepseek-llm-7b-chat	7B	0.170	0.203	0.223	0.223	0.213	0.204	0.184	0.135	0.126	0.347
llama3-8b-instruct	8B	0.226	0.244	0.261	0.265	0.269	0.289	0.243	0.173	0.164	0.382
llama3.1-8b-instruct	8B	0.211	0.245	0.259	0.265	0.265	0.297	0.229	0.177	0.165	0.373
qwen2.5-3b-instruct	3.1B	0.207	0.206	0.206	0.215	0.220	0.218	0.223	0.145	0.150	0.336
qwen2.5-7b-instruct	7.6B	0.182	0.210	0.201	0.208	0.214	0.215	0.215	0.143	0.143	0.343

Appendix

Table 9: Language-wise Success@5 on AuthBench (full results).

Model	Model Size	ar	de	en	es	fr	hi	ja	ko	ru	zh
LLMs (instruction-tuned)
deepseek-llm-7b-chat	7B	0.125	0.103	0.141	0.089	0.103	0.071	0.061	0.081	0.123	0.074
llama3-8b-instruct	8B	0.073	0.064	0.099	0.064	0.061	0.051	0.042	0.054	0.112	0.057
llama3.1-8b-instruct	8B	0.073	0.062	0.096	0.066	0.062	0.045	0.042	0.055	0.114	0.059
qwen2.5-3b-instruct	3.1B	0.072	0.072	0.131	0.074	0.068	0.054	0.043	0.060	0.119	0.062
qwen2.5-7b-instruct	7.6B	0.075	0.070	0.118	0.068	0.064	0.048	0.044	0.059	0.120	0.056

Appendix

Table 10: Language-wise EER on AuthBench (full results).

Model	Model Size	blog	ecomm	literature	media	news	poetry	qna	research	social
LLMs (instruction-tuned)
deepseek-llm-7b-chat	7B	0.196	0.110	0.461	0.053	0.243	0.556	0.541	0.713	0.166
llama3-8b-instruct	8B	0.213	0.107	0.521	0.070	0.302	0.639	0.650	0.769	0.206
llama3.1-8b-instruct	8B	0.220	0.112	0.518	0.035	0.293	0.583	0.618	0.787	0.204
qwen2.5-3b-instruct	3.1B	0.171	0.100	0.483	0.035	0.252	0.639	0.573	0.731	0.171
qwen2.5-7b-instruct	7.6B	0.162	0.084	0.496	0.035	0.242	0.583	0.567	0.719	0.167

Appendix

Table 11: Primary-genre Success@5 on AuthBench (full results).

Model	Model Size	blog	ecomm	literature	media	news	poetry	qna	research	social
LLMs (instruction-tuned)
deepseek-llm-7b-chat	7B	0.133	0.094	0.076	0.078	0.111	0.090	0.110	0.029	0.132
llama3-8b-instruct	8B	0.106	0.074	0.036	0.078	0.070	0.064	0.063	0.011	0.081
llama3.1-8b-instruct	8B	0.108	0.064	0.037	0.080	0.065	0.068	0.067	0.012	0.081
qwen2.5-3b-instruct	3.1B	0.137	0.086	0.048	0.079	0.075	0.059	0.098	0.031	0.091
qwen2.5-7b-instruct	7.6B	0.131	0.080	0.051	0.075	0.070	0.061	0.087	0.019	0.085

Appendix

Table 12: Primary-genre EER on AuthBench (full results).

Model	Model Size	short	medium	long	extra_long
LLMs (instruction-tuned)
deepseek-llm-7b-chat	7B	0.106	0.161	0.371	0.524
llama3-8b-instruct	8B	0.150	0.199	0.432	0.539
llama3.1-8b-instruct	8B	0.142	0.196	0.427	0.531
qwen2.5-3b-instruct	3.1B	0.101	0.164	0.385	0.508
qwen2.5-7b-instruct	7.6B	0.103	0.159	0.379	0.496

Appendix

Table 13: Length-bucket Success@5 on AuthBench (full results).

Model	Model Size	short	medium	long	extra_long
LLMs (instruction-tuned)
deepseek-llm-7b-chat	7B	0.185	0.131	0.096	0.072
llama3-8b-instruct	8B	0.109	0.082	0.060	0.055
llama3.1-8b-instruct	8B	0.104	0.078	0.061	0.051
qwen2.5-3b-instruct	3.1B	0.120	0.090	0.073	0.067
qwen2.5-7b-instruct	7.6B	0.110	0.082	0.069	0.051

Appendix

Table 14: Length-bucket EER on AuthBench (full results).

Figure 10: Token-length distribution per language. Each box plot summarizes the document-length distribution for one language in the current AuthBench release, highlighting cross-language differences in median length, spread, and long-tail behavior.

Figure 11: Fine-grained subgenre distribution by language. Each donut chart summarizes one language slice using a shared subgenre palette, while lower-frequency subgenres are grouped into Other to keep the figure legible.

Source	Lang.	Primary genre(s)	Scale	Author labels	Reference
Exorde	Multi	Social / news / forums	65M+/week	Yes (author hash)	[ 12 ]
Babel Briefings	30+	News headlines	4.7M	Partial (publisher / org)	[ 24 , 25 ]
Amazon Reviews Multi (MARC)	6	E-commerce reviews	200k+/lang	Yes (reviewer IDs)	[ 19 , 29 ]
Blog Authorship Corpus	en	Blogs	681k posts	Yes	[ 46 , 5 ]
arXiv Abstracts (metadata snapshot)	en	Research papers	1.7M+	Yes (first author from metadata)	[ 4 , 10 ]
Xiaohongshu / Weibo	zh	Social media	11k+	Yes (user IDs)	[ 58 ]

Appendix

Table 15: Raw sources used for the current AuthBench release. “Scale” reflects either the approximate size of the upstream source or, for crawl-dependent sources, the size of the processed corpus produced by the current collection setting before final benchmark selection.

Source	#Docs	Share
Exorde	94,231	22.0%
Wikisource	78,984	18.4%
Babel Briefings	73,676	17.2%
YouTube comments	71,808	16.8%
Blog Authorship Corpus	22,494	5.3%
Project Gutenberg	18,739	4.4%

Appendix

Table 16: Source composition of the current combined AuthBench export. All 17 configured sources are represented in the materialized benchmark; the table reports their realized document counts after filtering, redundancy reduction, cross-phase merge cleanup, and final selection.

Source	License / terms (summary)	Release mode	Notes / pointer
Exorde	MIT (dataset card)	Tier A	Provider page explicitly states MIT.
Babel Briefings	CC BY-NC-SA 4.0 (dataset card)	Tier A	Attribution, non-commercial use, and share-alike obligations apply.
Amazon Reviews Multi (MARC)	Provider-restricted / non-permissive redistribution; dataset card does not expose a standard open license	Tier B	Safer to require reconstruction from the original release rather than redistribute normalized text.
Blog Authorship Corpus	No clear standardized redistribution license found in the public host materials	Tier B	Conservative manifest-only treatment is safer than direct text redistribution.
arXiv Abstracts (metadata snapshot)	Metadata access is available via arXiv services, but article licenses vary by paper	Tier B	We treat this as metadata-derived text and do not redistribute PDFs or assume one uniform article license.
Xiaohongshu / Weibo	Platform / provider terms for redistribution are unclear	Tier B	Manifest-only unless explicit redistribution permission is documented.

Appendix

Table 17: Licensing / terms and release mode by source. “Release mode” indicates whether we redistribute normalized text (Tier A) or provide manifest-only reconstruction (Tier B). This table covers all 17 sources in the current release.

Authorship Attribution

Authorship attribution identifies or verifies who wrote a text, code sample, or other creative work from patterns in its style and content.

See all 64 papers

Contrastive Learning for Authorship Verification
Sep 23, 2026Peter KirbyContrastive Learning Representation Learning
Georgia Institute of Technology
When Writing Style Drifts: Benchmarking Authorship Verification under Distribution Shifts in Genre, Time and the AI-Era
Aug 18, 2026Lotta Kiefer, Brisca Balthes, Christoph Leiter +3Distribution Shift Robustness Authorship Attribution
University of Technology Nuremberg (UTN)
PersonalBench: Measuring the Authorship Gap in LLM Personalization
Aug 20, 2026Yash Ganpat SawantPersonalized Language Models Personalized Text Generation
Independent AI Researcher

Abstract

Authorship Attribution

Explore similar work

Contrastive Learning for Authorship Verification

When Writing Style Drifts: Benchmarking Authorship Verification under Distribution Shifts in Genre, Time and the AI-Era

PersonalBench: Measuring the Authorship Gap in LLM Personalization