FAVoR: Measuring and Mitigating Author-Style Homogenization in Federated Personalized Generation

Lu Han; Jingyao Zhang; Katy Ilonka Gero; Nguyen H. Tran

cs.CLSep 25, 2026

FAVoR: Measuring and Mitigating Author-Style Homogenization in Federated Personalized Generation

Authors: Lu Han, Jingyao Zhang, Katy Ilonka Gero, Nguyen H. Tran

Organizations: School of Computer Science, The University of Sydney Sydney, NSW 2006, Australia

Abstract

Large language models are increasingly used as personalized writing assistants, but adapting a model across many authors can compromise individual writing style by pulling author-specific signals toward a shared register. Federated parameter-efficient fine-tuning (PEFT) offers a data-local setting for this multi-author adaptation problem: clients keep author text local while sharing compact adapter updates. However, we show that standard aggregation can preserve continuation utility while making different authors' generations less distinguishable in style space, a failure mode we define as author-style homogenization. We evaluate author-style retention with Angular Style Classification Encoder (ASCE)-based diagnostics on our main BlogText benchmark and ASCE-independent external authorship verification. Using this protocol, we find that common federated PEFT baselines can preserve semantic utility while averaging out author-specific signals. To address this homogenization, we instantiate FAVoR (Federated Authorial Voice Retention), an author-style residual mechanism for federated PEFT. FAVoR uses a shared-private adapter design: clients upload shared-adapter updates while retaining author-specific residual corrections locally. Across BlogText and external Mythos-Reddit validation, FAVoR improves author-style retention over standard and personalized federated PEFT baselines. These gains come with small continuation-utility trade-offs and are supported by component ablations, external verification, and cold-start transfer.

Figures & tables

Figure 1: Coordinate-level author/topic discriminability. Each point is one scalar coordinate: a hand-crafted stylometric feature on the left or an ASCE embedding dimension on the right. Lower-right and greener points carry more author-than-topic signal; marker size scales with distance from the origin. ASCE shifts the distribution toward stronger author signal and a lower topic-on-author origin-fit slope.

Figure 2: FAVoR workflow. Clients upload only shared-adapter deltas for aggregation; author targets, hidden states, and private residual packs remain local. Private updates use task loss plus style alignment.

Representation	Acc. $\uparrow$	F1 $\uparrow$	AUC $\uparrow$	EER $\downarrow$
RoBERTa	0.635	0.677	0.629	0.421
StyleDist.	0.628	0.660	0.716	0.344
LUAR	0.633	0.677	0.797	0.277
STAR	0.648	0.715	0.856	0.229
ASCE	0.651	0.692	0.863	0.211

Table 1: Compact source-space representation diagnostics. Coordinate-level author/topic discriminability is reported as an appendix diagnostic in Table 8 .

Dataset	Method	Author Acc. $\uparrow$	Macro- F1 $\uparrow$	Verif. AUC $\uparrow$	EER $\downarrow$	BERTScore $\uparrow$	SemSim $\uparrow$
BlogText	FedAvg	0.199	0.164	0.680	0.371	0.819	0.359
	FedProx	0.174	0.161	0.674	0.365	0.817	0.353
	pFedMe	0.271	0.252	0.739	0.321	0.817	0.372
	Ditto	0.278	0.255	0.763	0.297	0.815	0.366
	FedDPA †	0.294	0.271	0.749	0.313	0.818	0.376
	FAVoR	0.403	0.365	0.824	0.245	0.815	0.380

Table 2: Main federated held-out continuation results. † FedDPA is trained with the dynamic-gate setup; reported continuations use the exported per-client mean validation gate, audited in Appendix B.17 .

Method	StyleDist AUC $\uparrow$	StyleDist EER $\downarrow$	MiniLM AUC $\uparrow$	Stylom. AUC $\uparrow$
FedProx	0.724	0.334	0.631	0.651
Pooled PEFT	0.736	0.327	0.650	0.660
Ditto	0.748	0.318	0.650	0.683
FedDPA †	0.747	0.318	0.659	0.692
Local-only + ASCE	0.768	0.301	0.685	0.708
FAVoR	0.771	0.295	0.675	0.714

Table 3: Main BlogText results with external authorship verification.

Variant	Author Acc. $\uparrow$	Macro- F1 $\uparrow$	BERTScore $\uparrow$	StyleDist. AUC $\uparrow$
Global ASCE	0.161	0.140	0.818	0.728
Ditto + ASCE	0.318	0.286	0.816	0.746
FAVoR w/o ASCE	0.377	0.331	0.814	0.767
FAVoR	0.403	0.365	0.815	0.771

Table 4: Core BlogText component ablations with StyleDistance gen

\rightarrow

source verification.

Figure 3: Adapter-space diagnostic for FAVoR on BlogText. Cells show normalized LoRA update energy by layer block and module, averaged over 50 personalized adapters.

Method	Sep. Acc. $\uparrow$	Macro- F1 $\uparrow$	Target Sim. $\uparrow$
Shared only	0.150	0.166	0.191
Random shared init + private	0.308	0.278	0.201
Local-only from base	0.425	0.428	0.259
FAVoR shared init + private	0.641	0.629	0.363

Table 5:

K=8

cold-start author-separability for unseen authors. FAVoR uses no-ASCE private adaptation.

Appendix figures & tables29 assets

Supplementary material from the paper’s appendix.

Appendix

Example	Method	Shortened raw continuation
1 . Prefix cue: “Now I can post about other things. Yes I’m prepared for the hurricane. Had to go back to Lowes today to get some things. My gas …” Gold cue: “They don’t want the kids to get stuck at school and the road flood and then not …”
	FedAvg	“We will be staying home on Monday, Tuesday, Wednesday, and Thursday. Friday is a day off. So we should be fine. …”
	Pooled PEFT	“We will see what happens. I think my mom is going to stay with us until Saturday. She has a lot …”
	FedProx	“We will be staying home on Monday, Tuesday, Wednesday, and Thursday. Friday is a work day but they will probably close …”
	Ditto	“We will see what happens. I am glad you guys made it through ok. It’s nice to hear from you again. …”
	FAVoR	“We will see what happens. I am glad you guys had a good time at the beach. It is nice to …”

Appendix

Table 6: Matched qualitative comparison over anonymized held-out continuation examples. Entries are shortened raw generated continuations from the same seed-author-prompt instances across all listed methods. The table shows three examples from the fixed-seed sample.

Method	FL	Train	Raw Hist.	RAG	Distilled Artifact	Proto. Acc. $\uparrow$	Proto. F1 $\uparrow$	Gen $\rightarrow$ Src AUC $\uparrow$	EER $\downarrow$	Silh. $\uparrow$
FAVoR	✓	✓	✗	✗	✗	0.288	0.208	0.760	0.319	-0.045
Mythos Avg-Author	✗	✗	✗	✗	✗	0.093	0.047	0.627	0.394	-0.184
Mythos RAG-Only	✗	✗	✓	✓	✗	0.131	0.095	0.713	0.372	-0.198
Mythos Sheet + RAG	✗	✗	✓	✓	✓	0.176	0.126	0.689	0.389	-0.145
Mythos Sheet (no persona) + RAG	✗	✗	✓	✓	✓	0.130	0.088	0.698	0.365	-0.158
Mythos Sheet Only	✗	✗	✗	✗	✓	0.150	0.103	0.665	0.378	-0.143

Appendix

Table 7: Prompting and retrieval baselines on BlogText. Author-style columns use the author-disjoint ASCE source-anchored rescore on the shared prompting-aligned subset for this supplemental comparison. Prototype accuracy/F1 classifies generated continuations by the nearest source-train author prototype, and Gen

\rightarrow

Src AUC/EER uses same-author versus different-author generated-to-source pairs. Utility columns report the original generation-quality metrics for the same comparison.

Representation space	Dim.	Author Acc. $\uparrow$	Macro- F1 $\uparrow$	Author-in-topic $\omega^{2}$ $\uparrow$	Topic $\omega^{2}$ $\downarrow$	A/T ratio $\uparrow$	Strict AUC $\uparrow$	EER $\downarrow$
RoBERTa-base	768	0.635	0.677	0.1135	0.0557	1.95	0.629	0.421
StyleDistance	768	0.628	0.660	0.2357	0.0577	4.21	0.716	0.344
LUAR-MUD doc1	512	0.633	0.677	0.1257	0.0364	3.52	0.797	0.277
STAR	1024	0.648	0.715	0.1886	0.0478	4.16	0.856	0.229
ASCE Author-Style Signature Space	256	0.651	0.692	0.1916	0.0499	3.64	0.863	0.211

Appendix

Table 8: Native-dimensional source-space diagnostics.

Method	Author Acc. $\uparrow$	Macro- F1 $\uparrow$	Verif. AUC $\uparrow$	EER $\downarrow$	Silhouette $\uparrow$	Asst. Score $\downarrow$	BERTScore $\uparrow$	SemSim $\uparrow$	MAUVE $\uparrow$
Base only	0.207	0.191	0.695	0.353	-0.141	0.865	0.815	0.353	0.130
FedAvg	0.219	0.170	0.716	0.343	-0.136	0.230	0.811	0.361	0.263
FedProx	0.214	0.177	0.698	0.365	-0.149	0.183	0.808	0.352	0.327
pFedMe	0.293	0.245	0.807	0.263	-0.053	0.625	0.816	0.365	0.327
Ditto	0.368	0.328	0.836	0.243	-0.033	0.424	0.813	0.381	0.417
FAVoR	0.528	0.487	0.899	0.185	0.134	0.251	0.810	0.378	0.334

Appendix

Table 9: Single-seed Qwen2.5-7B base-model sensitivity probe on the full 50-author BlogText held-out continuation split. Bold marks the best listed value; base-only is a non-collaborative reference.

Method	Sep. Acc. $\uparrow$	Macro- F1 $\uparrow$	Verif. AUC $\uparrow$	EER $\downarrow$	Centroid Dist. $\uparrow$	Silhouette $\uparrow$	SemSim $\uparrow$	MAUVE $\uparrow$
FedAvg shared	0.177 $\pm\,0.013$	0.150 $\pm\,0.013$	0.729 $\pm\,0.002$	0.331 $\pm\,0.002$	0.677 $\pm\,0.007$	-0.176 $\pm\,0.010$	0.385 $\pm\,0.006$	0.272 $\pm\,0.040$
Ditto	0.386 $\pm\,0.022$	0.342 $\pm\,0.017$	0.883 $\pm\,0.012$	0.193 $\pm\,0.008$	0.809 $\pm\,0.007$	0.041 $\pm\,0.020$	0.411 $\pm\,0.008$	0.386 $\pm\,0.105$
FAVoR	0.441 $\pm\,0.021$	0.417 $\pm\,0.028$	0.889 $\pm\,0.006$	0.186 $\pm\,0.008$	0.821 $\pm\,0.007$	0.062 $\pm\,0.017$	0.413 $\pm\,0.008$	0.308 $\pm\,0.066$
Local-only (ref.)	0.555 $\pm\,0.006$	0.509 $\pm\,0.006$	0.926 $\pm\,0.011$	0.147 $\pm\,0.018$	0.886 $\pm\,0.003$	0.154 $\pm\,0.006$	0.412 $\pm\,0.008$	0.406 $\pm\,0.042$

Appendix

Table 10: K=100 BlogText scale stress test over three seeds. Values are mean

\pm

sample standard deviation. Bold marks the best collaborative method; local-only is a non-collaborative style-retention reference. Verification AUC/EER uses same-author versus different-author generated-continuation pairs in the K=100 ASCE space.

Allocated word-unit budget	Gen–Gen AUC FAVoR / Ditto	EER FAVoR / Ditto	BERTScore FAVoR / Ditto
25% ( $226{,}095$ )	0.755 / 0.685	0.300 / 0.373	0.8141 / 0.8140
50% ( $452{,}189$ )	0.757 / 0.706	0.305 / 0.336	0.8140 / 0.8147
100% ( $904{,}378$ )	0.753 / 0.699	0.316 / 0.360	0.8149 / 0.8153

Appendix

Table 11: Compute-normalized text-volume sweep. Values are means over two training seeds.

Allocation	Gen–Gen AUC FAVoR / Ditto ( $\Delta$ )	EER FAVoR / Ditto ( $\Delta$ )	BERTScore FAVoR / Ditto
Balanced	0.753 / 0.699 ( $+0.054$ )	0.316 / 0.360 ( $-0.044$ )	0.8149 / 0.8153
Moderate	0.740 / 0.695 ( $+0.045$ )	0.340 / 0.364 ( $-0.024$ )	0.8143 / 0.8147
Strong	0.738 / 0.702 ( $+0.036$ )	0.327 / 0.360 ( $-0.033$ )	0.8154 / 0.8136

Appendix

Table 12: Controlled author-data imbalance analysis. Values are means over two training seeds and

\Delta=\text{FAVoR}-\text{Ditto}

Method	$K$	Sep. Acc. $\uparrow$	Macro- F1 $\uparrow$	Target Sim. $\uparrow$	Copy-5 $\downarrow$	SemSim $\uparrow$	BERTScore $\uparrow$	MAUVE $\uparrow$
Base model, no adaptation	8	0.175	0.210	0.115	0.000	0.282	0.806	0.042
FAVoR shared init + private residual (ASCE)	8	0.538	0.555	0.353	0.154	0.304	0.793	0.071
FAVoR shared init + private residual (no ASCE)	8	0.641	0.629	0.363	0.128	0.301	0.792	0.078
Local-only + ASCE from base	8	0.400	0.377	0.261	0.050	0.331	0.807	0.115
Local-only from base	8	0.425	0.428	0.259	0.075	0.319	0.806	0.078
Random shared init + private residual	8	0.308	0.278	0.201	0.026	0.307	0.809	0.069

Appendix

Table 13: Full cold-start results for unseen authors. The shared adapter is trained on 40 federated training authors and transferred to 10 held-out authors; author-separability metrics use the leakage-free A train -only ASCE encoder and held-out-author prototypes built from the same

K

adaptation examples.

$K$	Fixed $\Delta$ Acc. [95% CI]	Fixed $\Delta m$ [95% CI]
4	$+0.086$ $[-0.081,\ 0.244]$	$+0.046$ $[-0.017,\ 0.103]$
8	$+0.119$ $[0.050,\ 0.203]$	$+0.050$ $[0.003,\ 0.096]$
12	$+0.064$ $[-0.017,\ 0.150]$	$+0.036$ $[0.006,\ 0.065]$
16	$+0.053$ $[-0.022,\ 0.133]$	$+0.032$ $[-0.008,\ 0.073]$
24	$-0.044$ $[-0.172,\ 0.067]$	$+0.027$ $[-0.027,\ 0.076]$

Appendix

Table 14: Nested cold-start

K

-sweep with fixed 32-text evaluation prototypes. Intervals are nominal author-cluster 95% bootstrap confidence intervals.

Source	Items	Votes	Vote acc.	Maj. acc.	Mean conf.
Ditto	20	60	68.3%	70.0%	2.73
FAVoR	20	60	91.7%	95.0%	3.79
Human ref.	10	30	93.3%	90.0%	3.50
Overall	50	150	82.7%	84.0%	3.31

Appendix

Table 15: Small human 2AFC audit. Vote accuracy is over individual annotator votes; majority accuracy is after three-vote item aggregation.

Method	Mean quality	FAVoR $-$ method	Unadjusted 95% CI	Holm-adjusted $p$
FedAvg	2.270	$-0.073$	$[-0.310,\ 0.170]$	1.000
FedProx	2.213	$-0.017$	$[-0.267,\ 0.243]$	1.000
pFedMe	2.133	$+0.063$	$[-0.217,\ 0.323]$	1.000
Ditto	2.370	$-0.173$	$[-0.443,\ 0.103]$	1.000
FedDPA	2.337	$-0.140$	$[-0.380,\ 0.097]$	1.000
FAVoR	2.197	–	–	–

Appendix

Table 16: Reference-free overall-quality judging on the 1–5 scale. Differences are computed from unrounded output-level means.

Method	Evaluator	Source AUC	Gen $\rightarrow$ Src AUC	Gen $\rightarrow$ Src EER	Gen $\rightarrow$ Gen AUC
FedProx	StyleDistance	0.705 $\pm$ 0.000	0.724 $\pm$ 0.011	0.334 $\pm$ 0.006	0.654 $\pm$ 0.021
FedProx	MiniLM	0.635 $\pm$ 0.000	0.631 $\pm$ 0.002	0.414 $\pm$ 0.003	0.532 $\pm$ 0.005
FedProx	Stylometry	0.619 $\pm$ 0.000	0.651 $\pm$ 0.003	0.385 $\pm$ 0.005	0.573 $\pm$ 0.021
Pooled PEFT	StyleDistance	0.705 $\pm$ 0.000	0.736 $\pm$ 0.006	0.327 $\pm$ 0.004	0.691 $\pm$ 0.016
Pooled PEFT	MiniLM	0.635 $\pm$ 0.000	0.650 $\pm$ 0.001	0.406 $\pm$ 0.005	0.574 $\pm$ 0.004
Pooled PEFT	Stylometry	0.619 $\pm$ 0.000	0.660 $\pm$ 0.008	0.384 $\pm$ 0.008	0.562 $\pm$ 0.022

Appendix

Table 17: External-verifier breakdown.

Method	Author Acc. $\uparrow$	Macro- F1 $\uparrow$	Centroid $\uparrow$	Silhouette $\uparrow$	Asst. $\downarrow$	BERTScore $\uparrow$	SemSim $\uparrow$	MAUVE $\uparrow$
FedProx	0.174 $\pm\,0.008$	0.161 $\pm\,0.009$	0.431 $\pm\,0.009$	-0.199 $\pm\,0.004$	0.264 $\pm\,0.041$	0.817 $\pm\,0.002$	0.353 $\pm\,0.005$	0.200 $\pm\,0.005$
Pooled PEFT	0.254 $\pm\,0.021$	0.232 $\pm\,0.027$	0.537 $\pm\,0.005$	-0.184 $\pm\,0.030$	0.219 $\pm\,0.017$	0.802 $\pm\,0.001$	0.359 $\pm\,0.002$	0.389 $\pm\,0.059$
Ditto	0.278 $\pm\,0.011$	0.255 $\pm\,0.002$	0.546 $\pm\,0.004$	-0.097 $\pm\,0.026$	0.328 $\pm\,0.016$	0.815 $\pm\,0.001$	0.366 $\pm\,0.005$	0.378 $\pm\,0.074$
Local-only + ASCE	0.402 $\pm\,0.026$	0.366 $\pm\,0.016$	0.643 $\pm\,0.012$	-0.037 $\pm\,0.025$	0.239 $\pm\,0.015$	0.813 $\pm\,0.001$	0.375 $\pm\,0.009$	0.486 $\pm\,0.053$
FAVoR	0.403 $\pm\,0.004$	0.365 $\pm\,0.009$	0.624 $\pm\,0.006$	-0.039 $\pm\,0.022$	0.260 $\pm\,0.004$	0.815 $\pm\,0.000$	0.380 $\pm\,0.001$	0.431 $\pm\,0.026$

Appendix

Table 18: Additional compact BlogText metric summary.

Audit item	Value
Evaluation authors	50
Topic source	Original BlogText metadata joined by author ID
Topics with $\geq 3$ authors	3 usable; 4 including all buckets
Largest topic bucket	Student: 13/50 (26.0%)
indUnk bucket	11/50 (22.0%)
Dominant-topic fraction	mean 1.0; min 1.0

Appendix

Table 19: Topic and negative-sampling summary.

Comparison	Metric	Support	Fraction	Supporting evaluators
FAVoR $>$ Ditto	Gen $\rightarrow$ Src primary AUC	3/3	1.0	minilm, styledistance, stylometry

Appendix

Table 20: Cross-evaluator agreement.

Method	Seeds	Proto. Acc. $\uparrow$	Proto. F1 $\uparrow$	Gen $\rightarrow$ Src AUC $\uparrow$	Gen $\rightarrow$ Src EER $\downarrow$
FedAvg	3	0.140 $\pm$ 0.009	0.096 $\pm$ 0.001	0.651 $\pm$ 0.023	0.379 $\pm$ 0.008
FedProx	3	0.157 $\pm$ 0.011	0.111 $\pm$ 0.005	0.645 $\pm$ 0.023	0.387 $\pm$ 0.020
pFedMe	3	0.176 $\pm$ 0.014	0.120 $\pm$ 0.010	0.691 $\pm$ 0.019	0.363 $\pm$ 0.018
Ditto	3	0.219 $\pm$ 0.016	0.169 $\pm$ 0.012	0.715 $\pm$ 0.014	0.351 $\pm$ 0.008
Pooled PEFT	3	0.193 $\pm$ 0.008	0.148 $\pm$ 0.012	0.689 $\pm$ 0.012	0.363 $\pm$ 0.016
Local-only + ASCE	3	0.342 $\pm$ 0.023	0.292 $\pm$ 0.032	0.764 $\pm$ 0.006	0.307 $\pm$ 0.013

Appendix

Table 21: Generated-to-source ASCE results with the author-disjoint encoder. The encoder is trained on BlogText authors disjoint from the main 50-author FL roster; its classifier head is not used for this table. Values are the mean

\pm

standard deviation over seeds.

Method	Author Acc. $\uparrow$	Macro-F1 $\uparrow$	Silhouette $\uparrow$	Gen-Gen AUC $\uparrow$	Gen-Gen EER $\downarrow$
FedAvg	0.199 $\pm$ 0.008	0.164 $\pm$ 0.015	-0.194 $\pm$ 0.014	0.680 $\pm$ 0.029	0.371 $\pm$ 0.018
FedProx	0.174 $\pm$ 0.008	0.161 $\pm$ 0.009	-0.199 $\pm$ 0.004	0.674 $\pm$ 0.003	0.365 $\pm$ 0.011
pFedMe	0.271 $\pm$ 0.047	0.252 $\pm$ 0.033	-0.131 $\pm$ 0.005	0.739 $\pm$ 0.011	0.321 $\pm$ 0.014
Ditto	0.278 $\pm$ 0.011	0.255 $\pm$ 0.002	-0.097 $\pm$ 0.026	0.763 $\pm$ 0.025	0.297 $\pm$ 0.018
Ditto + ASCE	0.318 $\pm$ 0.037	0.286 $\pm$ 0.046	-0.115 $\pm$ 0.004	0.761 $\pm$ 0.010	0.309 $\pm$ 0.010
Pooled PEFT	0.254 $\pm$ 0.021	0.232 $\pm$ 0.027	-0.184 $\pm$ 0.030	0.699 $\pm$ 0.020	0.344 $\pm$ 0.009

Appendix

Table 22: Expanded BlogText author-style results under the author-disjoint ASCE encoder. This table expands the BlogText author-style rows in Table 2 with pooled and diagnostic references. Accuracy and F1 use source-train prototypes, while Gen-Gen AUC/EER measure same-author versus different-author verification among generated continuations. Values are mean

\pm

standard deviation over seeds.

Collaborative method	Source-prototype Acc.	Gen $\rightarrow$ Src AUC
FAVoR	0.403 $\pm$ 0.017	0.771 $\pm$ 0.011
FAVoR w/o ASCE	0.347 $\pm$ 0.016	0.766 $\pm$ 0.009
Ditto	0.214 $\pm$ 0.022	0.723 $\pm$ 0.014
FedDPA	0.242 $\pm$ 0.034	0.713 $\pm$ 0.018
pFedMe	0.195 $\pm$ 0.003	0.707 $\pm$ 0.030
FedAvg	0.137 $\pm$ 0.028	0.663 $\pm$ 0.012

Appendix

Table 23: Post-hoc evaluation under independently trained ASCE-B. Archived generations and pair construction are held fixed; values are mean

\pm

sample standard deviation over three seeds.

ASCE training $K_{\mathrm{ASCE}}$	Examples	BlogText Proto-Acc. FAVoR / Ditto / FedAvg	BlogText Gen $\rightarrow$ Src AUC FAVoR / Ditto / FedAvg	Mythos Proto-Acc. [95% CI]	Mythos AUC [95% CI]
10	1,020	0.322 / 0.211 / 0.125	0.726 / 0.696 / 0.648	0.203 $[0.147,0.263]$	0.647 $[0.618,0.677]$
25	2,749	0.378 / 0.241 / 0.147	0.757 / 0.709 / 0.659	0.277 $[0.217,0.337]$	0.712 $[0.680,0.744]$
50	4,959	0.293 / 0.199 / 0.147	0.754 / 0.685 / 0.631	0.220 $[0.160,0.283]$	0.650 $[0.623,0.680]$

Appendix

Table 24: ASCE evaluator-scale sensitivity on fixed BlogText outputs and evaluation-only transfer to unseen Mythos authors.

Method	Sep. Acc. $\uparrow$	Macro- F1 $\uparrow$	Centroid $\uparrow$	Silhouette $\uparrow$	AUC $\uparrow$	EER $\downarrow$	BERTScore $\uparrow$	SemSim $\uparrow$
FedAvg	0.134 $\pm\,0.023$	0.115 $\pm\,0.023$	0.455 $\pm\,0.083$	-0.193 $\pm\,0.024$	0.658 $\pm\,0.024$	0.386 $\pm\,0.026$	0.823 $\pm\,0.000$	0.482 $\pm\,0.009$
FedProx	0.141 $\pm\,0.024$	0.113 $\pm\,0.017$	0.463 $\pm\,0.096$	-0.172 $\pm\,0.016$	0.663 $\pm\,0.029$	0.377 $\pm\,0.026$	0.823 $\pm\,0.000$	0.478 $\pm\,0.008$
pFedMe	0.324 $\pm\,0.010$	0.290 $\pm\,0.015$	0.654 $\pm\,0.040$	-0.054 $\pm\,0.008$	0.762 $\pm\,0.005$	0.305 $\pm\,0.005$	0.823 $\pm\,0.000$	0.477 $\pm\,0.002$
Ditto	0.392 $\pm\,0.022$	0.359 $\pm\,0.029$	0.685 $\pm\,0.030$	-0.018 $\pm\,0.007$	0.818 $\pm\,0.002$	0.253 $\pm\,0.002$	0.825 $\pm\,0.000$	0.487 $\pm\,0.006$
FedDPA †	0.357 $\pm\,0.035$	0.317 $\pm\,0.033$	0.667 $\pm\,0.048$	-0.053 $\pm\,0.021$	0.803 $\pm\,0.013$	0.268 $\pm\,0.010$	0.825 $\pm\,0.000$	0.488 $\pm\,0.005$
FAVoR	0.561 $\pm\,0.024$	0.521 $\pm\,0.029$	0.771 $\pm\,0.021$	0.093 $\pm\,0.017$	0.900 $\pm\,0.012$	0.175 $\pm\,0.011$	0.826 $\pm\,0.000$	0.494 $\pm\,0.005$

Appendix

Table 25: Mythos-Reddit external validation.

Method	Sep. Acc. $\uparrow$	Macro- F1 $\uparrow$	Centroid $\uparrow$	Silhouette $\uparrow$	Phrase $\downarrow$	BERTScore $\uparrow$	SemSim $\uparrow$	Gap-Silh. $\uparrow$	Gap-Cent. $\uparrow$
FedAvg	0.134 $\pm\,0.023$	0.115 $\pm\,0.023$	0.455 $\pm\,0.083$	-0.193 $\pm\,0.024$	1.434 $\pm\,0.367$	0.823 $\pm\,0.000$	0.482 $\pm\,0.009$	-0.237 $\pm\,0.020$	-0.322 $\pm\,0.067$
FedProx	0.141 $\pm\,0.024$	0.113 $\pm\,0.017$	0.463 $\pm\,0.096$	-0.172 $\pm\,0.016$	1.857 $\pm\,0.761$	0.823 $\pm\,0.000$	0.478 $\pm\,0.008$	-0.216 $\pm\,0.011$	-0.313 $\pm\,0.081$
pFedMe	0.324 $\pm\,0.010$	0.290 $\pm\,0.015$	0.654 $\pm\,0.040$	-0.054 $\pm\,0.008$	1.848 $\pm\,0.270$	0.823 $\pm\,0.000$	0.477 $\pm\,0.002$	-0.098 $\pm\,0.013$	-0.122 $\pm\,0.026$
Ditto	0.392 $\pm\,0.022$	0.359 $\pm\,0.029$	0.685 $\pm\,0.030$	-0.018 $\pm\,0.007$	1.697 $\pm\,0.466$	0.825 $\pm\,0.000$	0.487 $\pm\,0.006$	-0.062 $\pm\,0.008$	-0.091 $\pm\,0.014$
FedDPA †	0.357 $\pm\,0.035$	0.317 $\pm\,0.033$	0.667 $\pm\,0.048$	-0.053 $\pm\,0.021$	1.511 $\pm\,0.087$	0.825 $\pm\,0.000$	0.488 $\pm\,0.005$	-0.097 $\pm\,0.016$	-0.110 $\pm\,0.034$
FAVoR	0.561 $\pm\,0.024$	0.521 $\pm\,0.029$	0.771 $\pm\,0.021$	0.093 $\pm\,0.017$	1.670 $\pm\,0.304$	0.826 $\pm\,0.000$	0.494 $\pm\,0.005$	0.049 $\pm\,0.022$	-0.006 $\pm\,0.005$

Appendix

Table 26: Detailed Mythos-Reddit diagnostics.

Metric	Summary gate	Dynamic gate	Dynamic $-$ summary
Author Acc.	0.350	0.350	+0.000
Macro-F1	0.266	0.283	+0.018
Verif. AUC	0.778	0.763	-0.015
EER	0.299	0.303	+0.004
BERTScore	0.8191	0.8178	-0.0013
SemSim	0.3982	0.3827	-0.0155

Appendix

Table 27: Bounded FedDPA gate-sensitivity probe.

Variant	Shared agg.	Personal branch	Private residual	ASCE align.	Author Acc. $\uparrow$	Macro- F1 $\uparrow$	Asst. Score $\downarrow$	BERTScore $\uparrow$
Global ASCE	✓	✗	✗	✓	0.161 $\pm\,0.014$	0.140 $\pm\,0.012$	0.301 $\pm\,0.065$	0.818 $\pm\,0.002$
Ditto + ASCE	✓	✓	✗	✓	0.318 $\pm\,0.037$	0.286 $\pm\,0.046$	0.409 $\pm\,0.040$	0.816 $\pm\,0.002$
FAVoR w/o ASCE	✓	✓	✓	✗	0.377 $\pm\,0.036$	0.331 $\pm\,0.033$	0.306 $\pm\,0.018$	0.814 $\pm\,0.001$
FAVoR	✓	✓	✓	✓	0.403 $\pm\,0.004$	0.365 $\pm\,0.009$	0.260 $\pm\,0.004$	0.815 $\pm\,0.000$

Appendix

Table 28: Full component-ablation summary with seed variability. Panel A reports component configurations and core BlogText metrics; Panel B reports ASCE-independent same-topic gen

\rightarrow

source AUC/EER. Author Acc. and Macro-F1 are generated-text ASCE prototype metrics. Values are mean

\pm

sample standard deviation over three seeds.

Comparison	Metric	Left	Right	Directional $\Delta$	Cluster 95% CI	$p$
FAVoR vs. w/o ASCE	Author acc.	0.403	0.377	+0.026	[-0.012, 0.064]	0.225
FAVoR vs. w/o ASCE	Top-1 cosine	0.642	0.629	+0.013	[0.003, 0.023]	0.011
FAVoR vs. w/o ASCE	Asst. score	0.260	0.306	+0.046	[0.012, 0.081]	0.009
FAVoR vs. w/o ASCE	BERTScore	0.815	0.814	+0.001	[-0.001, 0.002]	0.410
FAVoR vs. w/o ASCE	SemSim	0.380	0.377	+0.003	[-0.006, 0.012]	0.248
FAVoR vs. Global ASCE	Author acc.	0.403	0.161	+0.242	[0.194, 0.291]	$<.001$

Appendix

Table 29: Paired component-ablation significance tests.

Comparison	Evaluator	Metric	Left	Right	Directional $\Delta$	Cluster 95% CI	$p$
FAVoR vs. w/o ASCE	StyleDistance	AUC	0.771	0.767	+0.004	[-0.007, 0.015]	0.498
FAVoR vs. w/o ASCE	StyleDistance	EER	0.295	0.301	+0.006	[-0.007, 0.019]	0.438
FAVoR vs. w/o ASCE	MiniLM	AUC	0.675	0.664	+0.011	[-0.002, 0.025]	0.102
FAVoR vs. w/o ASCE	MiniLM	EER	0.378	0.388	+0.010	[-0.003, 0.025]	0.129
FAVoR vs. w/o ASCE	Stylometry	AUC	0.714	0.717	-0.003	[-0.013, 0.006]	0.492
FAVoR vs. w/o ASCE	Stylometry	EER	0.342	0.334	-0.007	[-0.019, 0.005]	0.233

Appendix

Table 30: External-verifier paired tests for component ablations using ASCE-independent same-topic gen

\rightarrow

source verification. Directional

\Delta

is signed so positive values favor the left variant; for EER, positive directional

\Delta

means lower raw EER for the left variant.

Variant	Private residual	ASCE align.	Shared prox.
Global ASCE	✗	✓	✗
FAVoR w/o ASCE	✓	✗	✓
FAVoR	✓	✓	✓

Appendix

Table 31: Component-ablation configuration check.

Quantity	Value
Base model	Qwen2.5-3B-Instruct
Quantization	4-bit QLoRA, NF4, bfloat16 compute
Trainable PEFT params	29.93M
Serialized shared adapter	114.3 MiB
Rounds	6
Clients per round	12 of 50

Appendix

Table 32: Efficiency and communication cost for the main BlogText FAVoR runs.

Comparison	Metric	FAVoR	Baseline	Directional $\Delta$	Cluster 95% CI	$p$	$n$
FAVoR vs. Ditto	Author acc.	0.403	0.279	+0.125	[0.078, 0.170]	$<.001$	578
FAVoR vs. Ditto	Top-1 cosine	0.642	0.655	-0.013	[-0.026, -0.001]	0.037	578
FAVoR vs. Ditto	SemSim	0.380	0.366	+0.014	[0.004, 0.024]	0.004	578
FAVoR vs. Ditto	BERTScore	0.815	0.815	-0.001	[-0.002, 0.001]	0.405	578
FAVoR vs. Ditto	Asst. score	0.260	0.328	+0.068	[0.023, 0.114]	0.002	578
FAVoR vs. Pooled	Author acc.	0.408	0.255	+0.153	[0.110, 0.198]	$<.001$	569

Appendix

Table 33: Paired tests over matched seed–author–prompt examples. FAVoR and baseline values are recomputed on each method-pair matched subset, so the same method can have slightly different displayed values across comparison blocks and need not exactly equal its full-coverage summary-table mean. Directional

\Delta

is signed, so positive values favour FAVoR; for the assistant score, lower raw values are better. Confidence intervals are cluster-bootstrap intervals over seed–author clusters. For author accuracy,

p

uses McNemar’s exact test; for continuous metrics,

p

uses the two-sided cluster-bootstrap tail probability.

Comparison	Evaluator	Metric	FAVoR	Baseline	Directional $\Delta$	Cluster 95% CI	$p$
FAVoR vs. Ditto	StyleDistance	AUC	0.771	0.748	+0.023	[0.009, 0.038]	0.001
FAVoR vs. Ditto	StyleDistance	EER	0.295	0.319	+0.024	[0.008, 0.037]	0.002
FAVoR vs. Ditto	MiniLM	AUC	0.675	0.650	+0.025	[0.009, 0.042]	0.004
FAVoR vs. Ditto	MiniLM	EER	0.378	0.402	+0.024	[0.005, 0.040]	0.012
FAVoR vs. Ditto	Stylometry	AUC	0.714	0.683	+0.031	[0.009, 0.055]	0.004
FAVoR vs. Ditto	Stylometry	EER	0.342	0.362	+0.021	[0.002, 0.043]	0.026

Appendix

Table 34: External-verifier paired tests for main BlogText comparisons.

Personalized Language Models

Personalized language models adapt responses to individual users’ preferences, histories, writing styles, and changing needs.

See all 89 papers

PersonalBench: Measuring the Authorship Gap in LLM Personalization
Aug 20, 2026Yash Ganpat SawantPersonalized Language Models Personalized Text Generation
Independent AI Researcher
AuthorMix: Modular Authorship Style Transfer via Layer-wise Adapter Mixing
Mar 24, 2026Sarubi Thillainathan, Ji-Ung Lee, Michael Sullivan +1Text Style Transfer Text Generation
Department of Language Science and Technology, Saarland Informatics Campus Saarland University, Saarbrücken, Germany
Communication-Efficient Personalized Adaptation via Federated-Local Model Merging
Feb 20, 2026Yinan Zou, Md Kamran Chowdhury Shisher, Christopher G. Brinton +1Federated Learning Federated Fine-Tuning
Elmore Family School of Electrical and Computer Engineering, Purdue University.

Aug 20, 2026J/K move · Enter open · S save

PersonalBench: Measuring the Authorship Gap in LLM Personalization

Yash Ganpat Sawant

Independent AI Researcher

Personalized Language Models Personalized Text Generation Language Model Generation Evaluation Authorship Attribution

Open paper AskPDF arXiv

Personalized text generation aims to make LLMs write in a specific individual's style, yet existing benchmarks measure task accuracy or preference alignment rather than whether the model's output actually resembles the target author's writing. We introduce PersonalBench, a benchmark that evaluates inference-time personalization methods through three independent lenses: LUAR (a trained authorship verification model), an LLM-as-judge, and automated stylometrics. Across 50 authors, 1,000 generations, and two model families (Qwen 3, GLM-4), we find that personalization methods do produce author-differentiated output (LUAR discriminates target authors within generated text at AUC=0.918) but this differentiation never crosses the human-LLM boundary. All methods achieve LUAR similarity to real authors in the range 0.484-0.508, below the cross-author human floor of 0.626 (ceiling 0.756). The LLM's own authorship fingerprint dominates: generated text is more distant from any human author than random humans are from each other. Methods are statistically indistinguishable from each other on LUAR (spread 0.024) despite appearing differentiated on the LLM judge, a discrepancy we trace to circularity between trait extraction and profile extraction. We validate that LUAR reliably measures authorship in our corpus (AUC=0.76 single-post, 0.96 multi-post). We release PersonalBench as a calibrated measuring stick: inference-time personalization modulates the LLM's style but does not bridge the gap to human authorship.

Abstract

Personalized Language Models

Explore similar work

PersonalBench: Measuring the Authorship Gap in LLM Personalization

AuthorMix: Modular Authorship Style Transfer via Layer-wise Adapter Mixing

Communication-Efficient Personalized Adaptation via Federated-Local Model Merging