Personalized text generation aims to make LLMs write in a specific individual's style, yet existing benchmarks measure task accuracy or preference alignment rather than whether the model's output actually resembles the target author's writing. We introduce PersonalBench, a benchmark that evaluates inference-time personalization methods through three independent lenses: LUAR (a trained authorship verification model), an LLM-as-judge, and automated stylometrics. Across 50 authors, 1,000 generations, and two model families (Qwen 3, GLM-4), we find that personalization methods do produce author-differentiated output (LUAR discriminates target authors within generated text at AUC=0.918) but this differentiation never crosses the human-LLM boundary. All methods achieve LUAR similarity to real authors in the range 0.484-0.508, below the cross-author human floor of 0.626 (ceiling 0.756). The LLM's own authorship fingerprint dominates: generated text is more distant from any human author than random humans are from each other. Methods are statistically indistinguishable from each other on LUAR (spread 0.024) despite appearing differentiated on the LLM judge, a discrepancy we trace to circularity between trait extraction and profile extraction. We validate that LUAR reliably measures authorship in our corpus (AUC=0.76 single-post, 0.96 multi-post). We release PersonalBench as a calibrated measuring stick: inference-time personalization modulates the LLM's style but does not bridge the gap to human authorship.
Figures & tables
Prompt type
Non-personalized
Few-shot vs. baseline
TMR
SA%
Δ TMR
Win%
Raw first sentence
0.587
50%
−0.080
23%
Content summary
0.384
22%
+0.049
33%
Table 1: Effect of prompt construction on baseline scores. Naïve extraction leaks author voice, inflating the unpersonalized baseline by 28pp on same-author judgments.
Method
LUAR ↑
TMR ↑
SA% ↑
FuncCos ↑
Non-Personalized
0.484 ± .019
0.384 ± .058
22% ± 7
0.741 ± .011
Few-Shot
0.508 ± .020
0.433 ± .061
31% ± 8
0.749 ± .011
Profile Extraction
0.502 ± .019
0.542 ± .060
29% ± 8
0.761 ± .010
Contrastive
0.494 ± .020
0.447 ± .059
36% ± 8
0.752 ± .011
Real Author (ceiling)
0.756
0.427
30%
—
Cross-Author (floor)
0.626
0.390
7%
—
Table 2: Method comparison across 50 authors, 1,000 generations. LUAR similarity uses 5-post aggregation (5 generations per author-method pair compared to 5 training posts) and is the primary authorship metric ( ↑ better). TMR = trait match rate from LLM judge. SA% = same-author rate. FuncCos = function word cosine similarity. Ceiling = real author’s test posts; floor = cross-author random pairs. All methods score below the real-text cross-author floor on LUAR despite appearing differentiated on TMR. CIs use hierarchical bootstrap (resampling authors, then generations within authors) to account for within-author correlation ( B=10,000 ).
Figure 1: LUAR authorship similarity by method (5-post aggregation). All methods score below the real-text cross-author floor (0.626), far from the real-author ceiling (0.756). The total spread across methods is only 0.024. The LLM’s authorship fingerprint is so dominant that generated text sits in a distinct regime below human-to-human similarity.
Figure 2: Distribution of per-generation LUAR similarity scores by method. All four methods produce overlapping distributions centered near 0.44 (1v5 per-generation scores), well below the real-author distribution centered at 0.58.
Figure 3: Calibration across three metrics. Each panel shows method scores (bars) with ceiling (real author, green) and floor (cross-author, red) baselines. On LUAR (left), all methods score below the real-text cross-author floor. On TMR (center), profile extraction exceeds the real-author ceiling—evidence of circularity (§ 4.6.1 ). On SA% (right), contrastive leads but all methods exceed the chance floor.
LUAR
TMR
FuncCos
LUAR
1.00
—
—
TMR
0.013
1.00
—
FuncCos
0.026
0.067
1.00
Table 3: Pearson correlation between evaluation metrics ( n=1,000 ). All pairwise correlations are near zero, indicating the metrics capture fundamentally different constructs. No single metric is a reliable proxy for another.
Figure 4: LUAR similarity vs. TMR for all 1,000 generations, colored by method. No systematic relationship ( r=0.013 ). Profile extraction’s high TMR does not correspond to high LUAR.