EpiPersona: Persona Projection and Episode Coupling for Pluralistic Preference Modeling

Yujie Zhang; Weikang Yuan; Zhuoren Jiang; Pengwei Yan

cs.AIMar 30, 2026

EpiPersona: Persona Projection and Episode Coupling for Pluralistic Preference Modeling

Authors: Yujie Zhang, Weikang Yuan, Zhuoren Jiang, Pengwei Yan

Organizations: Zhejiang University.

Abstract

Pluralistic alignment is essential for adapting large language models (LLMs) to the diverse preferences of individuals and minority groups. However, existing approaches often mix stable personal traits with episode-specific factors, limiting their ability to generalize across episodes. To address this challenge, we introduce EpiPersona, a framework for explicit persona-episode coupling. EpiPersona first projects noisy preference feedback into a low-dimensional persona space, where similar personas are aggregated into shared discrete codes. This process separates enduring personal characteristics from situational signals without relying on predefined preference dimensions. The inferred persona representation is then coupled with the current episode, enabling episode-aware preference prediction. Extensive experiments show that EpiPersona consistently outperforms the baselines. It achieves notable performance gains in hard episodic-shift scenarios, while remaining effective with sparse preference data.

Figures & tables

Figure 1: We propose EpiPersona , which first maps the preference feedback space $X$ into a persona space $\mathcal{Z}_{p}$ . Based on an individual’s historical preference information, we model the individual’s latent persona $Z_{u}$ . We further introduce two variants of EpiPersona that couple persona $Z_{u}$ with episode context $e$ for preference prediction: EpiPersona-A , which is tailored to inferring individuals’ episode-specific preferences in a plug-and-play manner, and EpiPersona-B , which is designed for pluralistic reward learning.

Figure 2: Overview of the persona projection mapping.

	Prism		Arena
	GPT-OSS-120B	Llama-3.3-70B	GPT-OSS-120B	Llama-3.3-70B
Random	50%	50%	50%	50%
LLM as a judge (Base model)	56.30 $\pm$ 1.26%	55.90 $\pm$ 1.22%	64.36 $\pm$ 3.91%	64.18 $\pm$ 3.91%
SynthesizeME (GPT-OSS-120B)	57.24 $\pm$ 1.27%	57.67 $\pm$ 1.27%	64.89 $\pm$ 4.11%	64.39 $\pm$ 3.93%
SynthesizeME (Llama-3.1-8B)	57.27 $\pm$ 1.28%	56.86 $\pm$ 1.26%	63.83 $\pm$ 4.26%	64.17 $\pm$ 3.85%
PI (GPT-OSS-120B)	54.00 $\pm$ 1.37%	57.22 $\pm$ 1.28%	61.72 $\pm$ 4.78%	63.28 $\pm$ 3.98%

Table 1: Comparison on LLM-as-a-judge task. Values are percentages with standard deviations (95% CI).

	Prism		Arena
	Llama-3.2-3B	Llama-3.1-8B	Llama-3.2-3B	Llama-3.1-8B
GPO	55.26 $\pm$ 1.25%	56.48 $\pm$ 1.24%	53.87 $\pm$ 4.14%	56.69 $\pm$ 4.05%
VPL	58.26 $\pm$ 1.75%	58.23 $\pm$ 1.75%	56.69 $\pm$ 5.81%	54.93 $\pm$ 5.71%
PAL	56.81 $\pm$ 1.75%	54.23 $\pm$ 1.73%	60.56 $\pm$ 5.63%	56.69 $\pm$ 5.81%
BT	56.58 $\pm$ 1.47%	57.11 $\pm$ 1.55%	51.94 $\pm$ 5.31%	54.06 $\pm$ 5.42%
Ours	59.60 $\pm$ 1.76%	58.54 $\pm$ 1.76%	59.57 $\pm$ 5.67%	57.45 $\pm$ 5.12%

Table 2: Comparison of the pluralistic reward model learning. Values are percentages with standard deviations (95% CI).

	Prism	Arena
Ablation Variant 1: Text-based preference modeling
Global persona prediction
only stable persona gpt-oss-120b	57.54 $\pm$ 1.15%	64.05 $\pm$ 4.13%
Episode-specific preference prediction
with distilled Llama-3.1-8B	56.98 $\pm$ 1.38%	62.52 $\pm$ 4.08%
with gpt-oss-120b	56.82 $\pm$ 1.63%	63.04 $\pm$ 4.15%

Table 3: Ablation study on EpiPersona.

Figure 3: Evaluation of EpiPersona under the influence of different factors. (A) Episode similarity : performance under high vs. low similarity, showing smaller drops for EpiPersona under the episode-shift scenario. (B) Number of preference feedback instances : model performance across users with varying amounts of historical feedback, highlighting advantages in sparse scenarios. (C) Window size : effect of the number of observable preference feedback instances on EpiPersona.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

	Arena		Prism
	User num	Hist/cur pairs	User num	Hist/cur pairs
Train	23	202/103	280	4192/2177
Validation	19	155/80	65	973/514
Test	89	514/ 284	378	5816/ 3033

Appendix

Table 4: Statistics of user numbers and historical/current pairs for Arena and Prism datasets.

Figure 4: The distribution of the history samples per user.

Method	Human-O/I	Machine-U	Compact-C	Global-A
Text-based sum.	✓	-	-	-
Parametrized.	-	✓	✓	✓
Predef.	✓	✓	✓	-
RAG	✓	-	-	-
EpiPersona-A	✓	✓	✓	✓

Appendix

Table 5: Comparing various preference modeling methods in terms of key attributes. Predef.: Predefined dimensions; Text-based sum.: text-based preference summarization; Parametrized.: Parametrized modeling; EpiPersona-A: Our proposed method.

Figure 5: Model architecture details (parameterized abductive reasoning and VQ-based mapping).

Figure 6: Window size experiment on EpiPerona (Arena).

User Preference Modeling

User preference modeling infers what people value from choices, behavior, and feedback to personalize recommendations, predictions, and AI actions.

See all 271 papers

Group Preference Collapse in Personalized Multimodal Large Language Models
Jun 12, 2026Fan Lyu, Wenqi Zhang, Joost van de WeijerMultimodal Large Language Models Personalized Language Models
Computer Vision Center, Universitat Autònoma de Barcelona, Barcelona, Spain, 08193
Toward Robust Personalized Alignment for LLMs: Mitigating Persona Drift in Multi-Turn Dialogue
Sep 11, 2026Youyuan Zhang, Siyuan Li, Fangming Liu +1Belief Revision LLM Alignment
Harbin Institute of Technology, Shenzhen, China · Peng Cheng Laboratory, China · Huazhong University of Science and Technology, China
Learning Dynamic User Personas from Implicit Interaction Streams via Iterative Refinement
Jul 29, 2026Haifeng WuLLM Personalization User Modeling
PayPal

Jun 12, 2026J/K move · Enter open · S save

Group Preference Collapse in Personalized Multimodal Large Language Models

Fan Lyu, Wenqi Zhang, Joost van de Weijer

Computer Vision Center, Universitat Autònoma de Barcelona, Barcelona, Spain, 08193

Multimodal Large Language Models Personalized Language Models User Preference Modeling Text Generation

Open paper AskPDF arXiv

Personalized multimodal large language models (MLLMs) aim to generate user-specific responses, but existing methods mainly rely on profile-level information and overlook diverse user preferences. We identify group preference collapse, where multi-user personalized MLLMs become insensitive to individual preferences and drift toward dominant population-level choices due to suppressed preference signals and unreliable preference use during generation. We propose PrefMoE, a preference-centric framework that separates stable profile information from preference-related representations. PrefMoE decomposes preferences into shared prototypes and personalized residuals, preserves individualized residuals with imbalance-aware learning, counterfactual pseudo-user augmentation, and residual decorrelation, and routes profile and preference factors through separate LoRA adaptation paths. Experiments across multiple MLLM backbones show that PrefMoE improves preference-sensitive personalization while substantially reducing preference collapse. Project page: https://prefmoe.github.io/.

Abstract

User Preference Modeling

Explore similar work

Group Preference Collapse in Personalized Multimodal Large Language Models

Toward Robust Personalized Alignment for LLMs: Mitigating Persona Drift in Multi-Turn Dialogue

Learning Dynamic User Personas from Implicit Interaction Streams via Iterative Refinement