The Missing Coefficients: Bayesian Pairwise Merging for Model Personalization

Yaling Shen; Tongtong Wu; Siyuan Yan; Gholamreza Haffari

cs.LGSep 30, 2026

The Missing Coefficients: Bayesian Pairwise Merging for Model Personalization

Authors: Yaling Shen, Tongtong Wu, Siyuan Yan, Gholamreza Haffari

Organizations: Monash University

Abstract

How can we personalize a shared expert library from a user's pairwise choices? Prior work can realize different reward trade-offs by merging reward-specialized experts, given a vector of trade-off weights. In practice, users can more naturally choose between outputs than specify numerical weights. The challenge is therefore to turn these choices into the coefficients required for merging, while accounting for ambiguity when feedback is limited. Our key idea is to treat the unknown reward weights as latent variables: infer a posterior over them from pairwise choices and reward-score differences, and use its mean directly as the merge coefficients. We instantiate this idea as Bayesian Pairwise Merging (BPM), whose posterior also characterizes which reward trade-offs remain plausible given the feedback. We evaluate BPM on radiology summarization, image captioning, and story generation, spanning text-to-text and image-to-text generation. With 100 feedback per simulated persona, BPM achieves macro decided win rates of 91.7%, 77.1%, and 64.3% against uniform merge. For six pairs of simulated personas, each prefers the model fitted to its own feedback, a pattern also observed in a human proof-of-concept. In simulations under BPM's model and prior, its nominal 90% intervals for temperature-scaled reward weights achieve task-averaged marginal coverage of 88.9% and 89.2% with only 10 and 25 comparisons, respectively. BPM thus enables personalization from pairwise feedback without per-user policy training, while characterizing the coefficient ambiguity left by limited feedback.

Figures & tables

Figure 1: Pairwise feedback personalizes a merged model. Two simulated personas provide different choices (a) , inducing distinct posteriors (b) and merged-model outputs (c) . Each persona favors its corresponding model on shared response pairs (d) .

Figure 2: Bayesian Pairwise Merging. A persona’s pairwise feedback updates a shared prior over reward weights into a posterior, whose mean supplies the merge coefficients and whose spread quantifies their uncertainty. Norm. denotes normalization by the infinity norm.

	Summarization		Image Captioning		Story Generation
	$n=100$	$n=500$	$n=100$	$n=500$	$n=100$	$n=500$
RS	91.7 [90.3, 93.1]	88.4 [86.2, 90.4]	77.1 [74.7, 79.3]	78.7 [76.6, 80.7]	64.3 [62.5, 66.1]	76.1 [74.4, 77.8]
Base	76.8 [74.6, 79.0]	76.7 [74.0, 79.3]	74.6 [72.2, 76.7]	76.4 [74.4, 78.2]	71.3 [69.4, 73.1]	81.4 [79.9, 82.8]
RM	74.9 [73.2, 76.7]	84.1 [81.8, 86.2]	72.1 [70.3, 74.0]	75.3 [73.5, 77.0]	57.3 [56.1, 58.6]	71.5 [70.2, 72.7]
MLE	28.4 [26.6, 30.3]	43.9 [38.8, 49.0]	44.7 [43.0, 46.4]	45.4 [43.3, 47.6]	43.4 [42.2, 44.7]	46.3 [45.0, 47.6]
MAP	28.9 [26.6, 31.2]	51.1 [46.1, 56.0]	57.9 [55.5, 60.2]	58.8 [56.4, 61.3]	49.5 [48.2, 50.7]	51.9 [50.8, 53.1]

Table 1: Main results. bpm ’s macro decided win rate (%) against each method, with 95% CIs, averaged over three sampled feedback subsets. Rows are not comparable with each other. Green favors bpm and red favors the compared method, with intensity proportional to the distance from

50

	(a) Expert Training Algorithm			(b) Rubric-based Persona
	GRPO	RLOO	DPO	Uniform Merge	Base Policy	Ridge-Merge	BT-BoN
$n=100$	91.7 [90.3,93.1]	93.9 [92.6,95.1]	77.7 [74.2,81.1]	65.7 [63.7,67.7]	70.8 [69.0,72.8]	60.4 [59.1,61.7]	63.4 [61.7,65.1]
$n=500$	88.4 [86.2,90.4]	97.7 [96.9,98.4]	90.3 [88.2,92.1]	74.7 [72.8,76.5]	77.8 [76.2,79.3]	71.4 [70.2,72.7]	71.1 [69.3,72.7]

Table 2: Robustness results. BPM’s macro-weighted decided win rate (%) with 95% CI. Cell colors as in Table 1 . (a) Experts trained with GRPO, RLOO, or DPO on the summarization task. Each library is compared with its own uniform merge, so columns do not rank training algorithms. (b) Rubric-based personas replace the prompted personas, on Story generation .

Figure 3: Preference behavior and coefficient uncertainty. (a) Each persona’s win rate for its corresponding model ( $n=500$ ). The bottom row uses human judgments ( $n=100)$ . (b) Win rates against uniform merge as feedback preferences shift ( $n=500$ ). (c) Macro win rates across budgets. (d) Coverage (top) and mean width (bottom) of nominal $90\%$ intervals across tasks, whiskers spanning task ranges. (e) Win rates against uniform merge using an alternative LLM for judgements.

Figure 4: Posterior analysis. (a) Left: merge-coefficient variation across feedback subsets. Right: bpm ’s decided win rates against alternative estimators (Appendix F.3 ). (b) Posterior uncertainty

U

versus decided win rate against uniform merging. Each point is one persona, with both quantities averaged over three feedback subsets;

\rho

is the within-task, fixed-budget Spearman correlation.

Appendix figures & tables23 assets

Supplementary material from the paper’s appendix.

Appendix

Symbol	Definition
$x,y$	Input and generated output.
$L,t$	Number of rewards and reward/expert index.
$\bm{\theta},\ \bm{\theta}_{0}$	Model and shared base-model parameters.
$\{\bm{\theta}_{t}\}_{t=1}^{L}$	Shared library of reward-specialized experts.
$\Delta^{L-1}$	Probability simplex in $\mathbb{R}^{L}$ .
$\bm{\lambda}\in\Delta^{L-1}$	Merge coefficients, set to $\widehat{\mathbf{w}}$ in BPM.

Appendix

Table 3: Notation used in Bayesian Pairwise Merging. Reward weights describe preference trade-offs, whereas merge coefficients determine how experts are combined.

Data role	Summarization	Image Captioning	Story Generation
Expert-training pool	85.6k reports	600 images	1,000 prompts
Prior calibration	1,000 patients	1,000 images	1,000 prompts
Pool simulation	250 reports	250 images	250 prompts
Reward standardization	250 reports	250 images	250 prompts
Test	451 reports	500 images	500 prompts

Appendix

Table 4: Data splits. Expert-training inputs are sampled from the available data shown here, with training sample counts given in Table 5 .

Setting	Summarization	Image Captioning	Story Generation
Training inputs per expert	1,600	600	1,000
Optimizer steps	100	100	100
Learning rate	$3\times 10^{-5}$	$3\times 10^{-5}$	$3\times 10^{-5}$
Completions per prompt	8	8	8
Completions per optimizer step	128	128	128
Sampling temperature	1.0	1.0	0.7

Appendix

Table 5: Training configuration for the main GRPO expert libraries.

Persona	Nonzero criterion weights
Literary	Prose elegance $0.30$ ; showing rather than telling $0.25$ ; imagery $0.25$ ; avoidance of purple prose $0.20$ .
Genre	Engagement $0.35$ ; surprise $0.25$ ; dialogue quality $0.20$ ; coherence $0.20$ .
Librarian	Readability $0.35$ ; coherence $0.25$ ; empathy $0.25$ ; avoidance of purple prose $0.15$ .
Flash	Brevity $0.35$ ; surprise $0.35$ ; imagery $0.30$ .
Screenwriter	Dialogue quality $0.45$ ; showing rather than telling $0.35$ ; imagery $0.20$ .

Appendix

Table 6: Weights defining the five rubric-based Story Generation personas. Unlisted criteria have zero weight.

Figure 5: Persona-level personalization performance at $n=500$ . Each cell is bpm ’s decided win rate (%) against the row’s compared method, averaged over three feedback subsets. Colors follow Table 1 .

Description	Comparator	Wins	Losses	Ties	Win rate (%)
Picture Book	Uniform	44	8	48	84.6 [72.5, 92.0]
Magazine Editor	Uniform	84	7	9	92.3 [85.0, 96.2]
Picture Book	Magazine Editor model	89	5	6	94.7 [88.1, 97.7]
Magazine Editor	Picture Book model	93	3	4	96.9 [91.2, 98.9]

Appendix

Table 7: Human evaluation on Image Captioning. The win rate refers to the model fitted to the named preference description. Each pack has

100

images. Intervals are

95\%

Wilson intervals on decided comparisons.

Persona	$n$	Uniform	Base Policy	Ridge-Merge	BT-BoN
Literary	100	69.4 [65.9, 72.9]	80.6 [77.7, 83.5]	61.6 [58.8, 64.4]	67.4 [64.5, 70.7]
	500	82.6 [79.7, 85.2]	90.2 [88.2, 92.2]	77.6 [74.9, 80.1]	81.9 [79.0, 84.7]
Genre	100	65.7 [62.3, 69.0]	68.0 [64.5, 71.2]	60.8 [58.1, 63.5]	64.0 [60.8, 67.1]
	500	68.8 [65.4, 72.1]	70.8 [67.6, 74.2]	66.1 [63.6, 68.5]	65.9 [62.7, 69.0]
Librarian	100	59.6 [55.8, 63.2]	49.8 [45.7, 53.5]	52.3 [49.4, 55.1]	64.4 [61.2, 67.5]
	500	67.2 [63.7, 70.6]	57.7 [53.9, 61.5]	61.1 [58.4, 63.9]	68.9 [65.6, 72.0]

Appendix

Table 8: Rubric-persona decided win rates. BPM’s win rates (%) against each comparator. Entries average three feedback subsets and include

95\%

paired input-bootstrap intervals.

Judge	vs. uniform-weight retrain	vs. bpm merge	vs. other persona
Ward	98.37 [97.20,99.53]	59.09 [36.36,77.27]	100.00 [99.06,100.00]
ED	85.40 [81.37,89.13]	48.76 [41.79,55.72]	97.95 [96.59,99.09]

Appendix

Table 9: Retrained policies at inferred weights: decided win rates (%) under the named persona’s judge. The last column compares each persona’s retrained policy with the other persona’s retrained policy.

	RLOO experts		DPO experts
Persona	$n=100$	$n=500$	$n=100$	$n=500$
Ward	92.4 [89.7, 94.9]	97.4 [96.0, 98.5]	74.9 [68.7, 80.6]	87.2 [83.8, 90.2]
ED	94.8 [93.0, 96.3]	96.4 [94.8, 97.8]	86.8 [83.6, 89.9]	93.4 [90.9, 95.5]
Complete	93.3 [90.8, 95.6]	99.2 [98.5, 99.8]	78.5 [72.7, 83.8]	92.3 [89.5, 94.8]
Concise	96.5 [95.2, 97.7]	97.9 [96.4, 99.1]	83.2 [77.8, 87.9]	92.6 [90.1, 94.7]
Faithful	94.3 [91.9, 96.5]	98.2 [97.0, 99.3]	72.7 [65.9, 79.4]	89.2 [85.4, 92.6]

Appendix

Table 10: Personalization within alternative expert libraries. bpm ’s decided win rates (%) against each library’s own uniform merge, averaged over three feedback subsets, with

95\%

input-bootstrap intervals. Each comparison uses its library’s opponent.

Figure 6: Persona-specific reward-weight distributions. The shared prior and the Literary and Librarian posteriors on Story Generation at

n=500

, for six selected reward dimensions. Dots and horizontal bars denote means and central

95\%

intervals. These are marginal distributions of

w

, rather than intervals for the temperature-scaled coefficients

\beta=w/\tau

	$n=10$		$n=25$
Interval	Coverage $\uparrow$	Width $\downarrow$	Coverage $\uparrow$	Width $\downarrow$
bpm posterior	88.9	0.176	89.2	0.191
Prior only	89.8	0.171	89.9	0.171
MAP bootstrap	8.2	0.024	20.4	0.261
MLE bootstrap	53.1	11.144	61.6	8.974
SS sequential Gaussian	99.9	3.036	99.7	2.797

Appendix

Table 11: Coefficient intervals under limited feedback. Coverage (%) and mean width of nominal

90\%

intervals for

\beta=w/\tau

, averaged equally over three tasks after averaging over coefficient coordinates and simulations within each task. The generating parameters follow bpm ’s prior. MAP, MLE, and RM bootstrap intervals use 200 resamples. The SS row uses sequential Gaussian inference, distinct from its signed point estimator used for merging. Prior-only intervals do not condition on feedback.

Generating weight configurations
Setting	$w$ coverage range	$\beta$ coverage range
One weight $0.8$ , $\tau^{*}=0.5$	93.6–96.6	93.8–96.9
One weight $0.8$ , $\tau^{*}=1$	93.8–96.6	93.8–96.8
One weight $0.8$ , $\tau^{*}=2$	93.8–96.6	93.8–96.6
Three weights $0.3$ , $\tau^{*}=0.5$	81.2–90.8	81.2–91.0
Three weights $0.3$ , $\tau^{*}=1$	81.1–89.9	82.3–90.8

Appendix

Table 12: Sensitivity of interval coverage to modeling assumptions. Each entry is the range of mean nominal

90\%

coverage (%) over the nine task–budget cells in a setting, not a confidence interval. Coverage is averaged over coordinates and simulation replicates within each cell. Separate columns distinguish simplex weights

w

from temperature-scaled coefficients

\beta=w/\tau

. Prior-sensitivity and label-noise settings draw generating parameters from the production prior; multipliers modify the fitted prior.

Task	$n$	$\alpha^{*}$	Shrinkage win rate (%)	Shrinkage variability	BPM variability
Summarization	25	0.70	77.6 [75.6, 79.4]	0.299	0.112
Summarization	100	0.20	69.8 [67.8, 71.7]	0.646	0.465
Image Captioning	100	0.25	41.9 [40.0, 43.8]	0.733	0.313
Story Generation	100	0.60	44.5 [43.3, 45.7]	0.328	0.392

Appendix

Table 13: Shrinkage toward uniform compared with the BPM posterior mean. Win rates are shrinkage against BPM, with 95% input-bootstrap intervals. Variability is the persona-averaged pairwise coefficient

\ell_{1}

distance across three feedback subsets. Variability entries are point estimates; they do not imply equivalent stability when close. Shrinkage selection additionally uses held-out elicitation comparisons.

$n$	Method	Variability [95% CI]	Difference from BPM [95% CI]
25	BPM	0.112 [0.080, 0.142]	–
25	MAP	0.231 [0.098, 0.377]	+0.119 [+0.014, +0.240]
25	MLE	0.998 [0.494, 1.482]	+0.886 [+0.412, +1.346]
25	Shrinkage	0.299 [0.148, 0.445]	+0.187 [+0.067, +0.307]
25	RM	0.331 [0.297, 0.367]	+0.219 [+0.173, +0.263]
25	SS	1.498 [1.216, 1.705]	+1.386 [+1.092, +1.587]

Appendix

Table 14: Coefficient variability on Summarization. Variability intervals use 2,000 persona-bootstrap resamples. Difference intervals are paired (method minus BPM), holding the three feedback subsets fixed. Intervals containing zero do not demonstrate equal stability. SS coefficients are signed.

$n$	Method	Variability [95% CI]	Difference from BPM [95% CI]
25	BPM	0.358 [0.214, 0.495]	–
25	MAP	0.366 [0.205, 0.540]	+0.007 [-0.209, +0.245]
25	MLE	1.592 [1.433, 1.747]	+1.234 [+0.938, +1.511]
25	Shrinkage	0.000 [0.000, 0.000]	-0.358 [-0.495, -0.214]
25	RM	0.323 [0.261, 0.389]	-0.036 [-0.212, +0.157]
25	SS	0.742 [0.174, 1.375]	+0.383 [-0.277, +1.138]

Appendix

Table 15: Coefficient variability on Image Captioning. Variability intervals use 2,000 persona-bootstrap resamples. Difference intervals are paired (method minus BPM), holding the three feedback subsets fixed. Intervals containing zero do not demonstrate equal stability. SS coefficients are signed.

$n$	Method	Variability [95% CI]	Difference from BPM [95% CI]
25	BPM	0.088 [0.059, 0.116]	–
25	MAP	0.119 [0.056, 0.190]	+0.032 [-0.010, +0.079]
25	MLE	1.398 [1.249, 1.554]	+1.310 [+1.152, +1.482]
25	Shrinkage	0.000 [0.000, 0.000]	-0.088 [-0.116, -0.059]
25	RM	0.439 [0.389, 0.493]	+0.351 [+0.297, +0.414]
25	SS	1.306 [1.113, 1.501]	+1.219 [+1.038, +1.405]

Appendix

Table 16: Coefficient variability on Story Generation. Variability intervals use 2,000 persona-bootstrap resamples. Difference intervals are paired (method minus BPM), holding the three feedback subsets fixed. Intervals containing zero do not demonstrate equal stability. SS coefficients are signed.

Task	$n{=}10$	$n{=}25$	$n{=}50$	$n{=}100$	$n{=}500$
Summarization	$32.1$	$17.5$	$22.9$	$24.3$	$50.2$
	$[30.7,33.5]$	$[16.0,19.0]$	$[20.4,25.1]$	$[21.7,26.9]$	$[44.6,55.9]$
Image Captioning	$15.8$	$15.9$	$14.3$	$15.6$	$19.2$
	$[14.7,16.9]$	$[14.7,17.1]$	$[13.1,15.5]$	$[14.5,16.8]$	$[17.7,20.7]$
Story Generation	$38.6$	$34.1$	$33.4$	$33.2$	$33.7$
	$[37.2,39.9]$	$[32.9,35.4]$	$[32.2,34.6]$	$[32.0,34.5]$	$[32.3,35.0]$

Appendix

Table 17: Comparison with the adapted Spectral Souping estimator. bpm ’s macro decided win rate (%), averaged equally over personas and over three feedback subsets, with

95\%

paired input-bootstrap intervals. Values below

50\%

favor the adapted SS models. bpm uses the factor-averaged merge from the main experiments; SS uses the exact delta merge described above.

	BPM exact vs. SS exact		BPM exact vs. BPM factor
Task	$n=100$	$n=500$	$n=500$
Summarization	$27.1\ [24.4,29.8]$	$45.5\ [39.4,52.4]$	$49.8\ [41.6,58.6]$
Image Captioning	$16.9\ [15.6,18.2]$	$20.7\ [19.1,22.1]$	$57.5\ [55.2,59.8]$
Story Generation	$34.7\ [33.3,36.0]$	$36.4\ [35.1,37.7]$	$53.0\ [51.7,54.1]$

Appendix

Table 18: Controlling the merge operator. Macro decided win rates (%) with

95\%

paired input-bootstrap intervals, averaged over three feedback subsets. Each column reports direct preference for the first model over the second. Exact denotes a linear combination of adapter deltas; factor denotes bpm ’s main factor-averaged merge.

	Summarization		Image Captioning		Story Generation
$n$	BPM	SS	BPM	SS	BPM	SS
10	$\mathbf{0.071}$	$1.827$	$\mathbf{0.102}$	$1.014$	$\mathbf{0.035}$	$1.661$
25	$\mathbf{0.112}$	$1.498$	$0.358$	$0.742$	$\mathbf{0.088}$	$1.306$
50	$\mathbf{0.304}$	$1.383$	$0.257$	$0.660$	$\mathbf{0.278}$	$1.132$
100	$\mathbf{0.465}$	$0.935$	$0.313$	$0.563$	$\mathbf{0.392}$	$0.892$
250	$\mathbf{0.398}$	$0.462$	$0.259$	$0.277$	$\mathbf{0.295}$	$0.425$

Appendix

Table 19: Coefficient variation across feedback subsets. Persona-averaged pairwise

\ell_{1}

distance; lower values indicate greater stability. Within each task, bold marks the lower value only when the

95\%

paired persona-bootstrap interval for SS minus BPM excludes zero. BPM uses simplex weights; SS uses signed coefficients with

\ell_{1}

norm at most one.

$n$	BPM $-$ Selection	Selection $-$ Uniform-weight selection	BPM $-$ Uniform-weight selection
10	$+39.3\,[34.5,43.8]$	$+0.5\,[-0.0,1.0]$	$+39.8\,[35.1,44.3]$
25	$+47.5\,[44.6,50.4]$	$+0.5\,[-0.2,1.2]$	$+48.0\,[45.0,51.0]$
50	$+53.6\,[51.4,55.8]$	$+1.7\,[0.2,3.3]$	$+55.3\,[53.1,57.5]$
100	$+53.5\,[51.3,55.6]$	$+2.9\,[1.0,4.9]$	$+56.4\,[54.4,58.3]$
250	$+45.9\,[43.5,48.3]$	$+8.0\,[5.9,10.1]$	$+53.9\,[51.5,56.1]$
500	$+21.8\,[19.9,23.7]$	$+31.3\,[29.0,33.4]$	$+53.1\,[50.7,55.4]$

Appendix

Table 20: Direct merging and candidate selection on Summarization. Differences in macro decided win rate against the uniform merge, in percentage points, with

95\%

paired input-bootstrap intervals. Selection uses inferred reward weights; uniform-weight selection uses equal reward weights. These are differences against a common reference, not direct head-to-head win rates.

task	compared method	Dec.	PP [95% CI]	PP bounds	OI	Dec.	PP [95% CI]	PP bounds	OI
		$n=100$				$n=500$
Summarization	RS (uniform merge)	91.7	76.5 [75.4, 77.7]	71.3–81.8	21.1	88.4	74.0 [72.7, 75.3]	65.3–82.7	34.9
	Base policy	76.8	69.3 [67.8, 70.7]	62.7–75.9	26.4	76.7	68.5 [67.2, 69.9]	59.3–77.7	36.8
	L2W	59.6	63.1 [62.2, 63.9]	59.5–66.6	14.0	81.9	72.5 [71.5, 73.6]	68.4–76.7	16.6
	Ridge-Merge	74.9	62.8 [61.8, 63.8]	57.1–68.4	22.6	84.1	67.9 [66.6, 69.2]	58.9–76.9	35.9
	BT + best-of-8	82.5	70.0 [68.8, 71.2]	64.2–75.8	23.0	85.1	69.6 [68.2, 70.9]	60.4–78.7	36.5

Appendix

Table 21: Scoring sensitivity for seven compared methods. Dec.: decided win rate. PP: supplementary score with ties and order-inconsistent pairs assigned 0.5; brackets give 95% input-bootstrap intervals. PP bounds assign 0.25 or 0.75 to order-inconsistent pairs and are not confidence intervals. OI: percentage of order-inconsistent verdicts. Entries use persona-equal aggregation over the available feedback draws. This specified panel contains 42 task–method–budget comparisons.

Task	Cohen’s $\kappa$	Decided agreement	Max. win-rate change
Summarization	$0.90$ – $0.97$	$99.6$ – $100\%$	$1.5$
Image Captioning	$0.92$ – $0.99$	$100\%$	$2.5$

Appendix

Table 22: Repeatability of persona judgments. Cohen’s

\kappa

uses four outcomes: A preferred, B preferred, tie, and order-dependent. Ranges span personas; decided agreement includes only pairs decided in both runs. Win-rate changes are in percentage points (%).

Task	Compared method	Original judge	gpt-5-mini
Summarization	Uniform merge	$90.4[88.4,92.3]$	$92.8[91.8,93.9]$
	Ridge-Merge	$86.1[84.0,88.1]$	$89.3[87.8,90.6]$
	DPO	$55.4[51.8,58.8]$	$83.3[81.0,85.6]$
Image Captioning	Uniform merge	$79.8[77.4,82.1]$	$74.7[72.3,77.1]$
	Ridge-Merge	$76.3[74.0,78.5]$	$73.6[71.4,75.8]$
	DPO	$15.3[13.8,16.9]$	$15.7[14.1,17.4]$

Appendix

Table 23: Evaluation with another judge family. bpm ’s macro decided win rates (%) with

95\%

CI, using the same generated outputs and one feedback subset at

n=500

. These estimates are distinct from the three-subset averages in Table 1 .

Explore similar work

CoPA: Benchmarking Personalized Question Answering with Data-Informed Cognitive Factors
Apr 16, 2026Hang Su, Zequn Liu, Chen Hu +3Large Language Model Personalization Preference Alignment Learning
East China Normal University, China · Zhongguancun Academy, China
POPI: Personalizing LLMs via Optimized Natural Language Preference Inference
Oct 17, 2025Yizhuo Chen, Xin Liu, Ruijie Wang +7Large Language Model Personalization Personalization
University of Illinois Urbana-Champaign · Amazon · University of Notre Dame
Rethinking Personalized Generation: Test-Time Alignment via Factorized Ranking Models
Sep 28, 2026Qiyao Ma, Junshan Zhang, Zhe ZhaoLarge Language Model Personalization Large Language Model Reranking
University of California, Davis

Apr 16, 2026, cs.CLJ/K move · Enter open · S save

CoPA: Benchmarking Personalized Question Answering with Data-Informed Cognitive Factors

Hang Su, Zequn Liu, Chen Hu, Xuesong Lu+2

East China Normal University, China · Zhongguancun Academy, China

Large Language Model Personalization Preference Alignment Learning Cognitive Science Factor

Open paper AskPDF arXiv

While LLMs have demonstrated remarkable potential in Question Answering (QA), evaluating personalization remains a critical bottleneck. Existing paradigms predominantly rely on lexical-level similarity or manual heuristics, often lacking sufficient data-driven validation. We address this by mining Community-Individual Preference Divergence (CIPD), where individual choices override consensus, to distill six key personalization factors as evaluative dimensions. Accordingly, we introduce CoPA, a benchmark with 1,985 user profiles for fine-grained, factor-level assessment. By quantifying the alignment between model outputs and user-specific cognitive preferences inferred from interaction patterns, CoPA provides a more comprehensive and discriminative standard for evaluating personalized QA than generic metrics. The code is available at https://github.com/bjzgcai/CoPA.

Abstract

Explore similar work

CoPA: Benchmarking Personalized Question Answering with Data-Informed Cognitive Factors

POPI: Personalizing LLMs via Optimized Natural Language Preference Inference

Rethinking Personalized Generation: Test-Time Alignment via Factorized Ranking Models