Personalized Language Model Alignment

Follow this topic Ask about this topic

Personalized language model alignment studies how models can infer and follow an individual’s preferences from feedback, conversation history, profiles, or behavior rather than relying only on population-wide preferences. Common approaches include personalized preference optimization such as DPO and reinforcement learning, user-specific adapters, memory and retrieval, and inference-time steering or decoding. Evaluation spans preference prediction and generation, multi-turn personalization, and applied settings such as recommendations, privacy, and cultural or political alignment. A central challenge is balancing faithful personalization with safety, truthfulness, and general capability, especially when user signals are sparse, shifting, or in tension with one another.

Latest papers

61 papers12 papers in the last four weeks, up 140% on the four weeks before. 0.1% of all new papers.

Same Verdict, Different Readers: Personalizing Fact-Checking Explanations from Relative Preferences
Oct 27, 2026Haokun Wen, Xuemeng Song, Mong-Li Lee +1Automated Fact-Checking Personalized Language Model Alignment
National University of Singapore Singapore · Southern University of Science and Technology Shenzhen, China
FATE-5: Faithful Alignment of Trait Expression for Continuous Big Five Personality Control
Oct 26, 2026Gunhee Cho, Hongyeob Kim, Yun-Gyung CheongPersonality Modeling in Language Models Personalized Language Model Alignment
Department of Artificial Intelligence, Sungkyunkwan University, Suwon, South Korea
When Personas Become Harmful: Guiding LLMs with Global Citizenship Values Alignment
Oct 24, 2026Siqi Shen, Lajanugen Logeswaran, Moontae Lee +2LLM Safety LLM Safety Alignment
University of Michigan · LG AI Research · University of Illinois at Chicago
MusicTwins: Training Audio-Language Models to Listen like Their Persona
Oct 23, 2026Milan Liessens Dujardin, Kevin MiaoAudio-Language Models Personalized Language Model Alignment
Bryel Labs, UC Berkeley · Bryel Labs
LaVGuard: Selective Decoding for Contextual Value Drift
Oct 21, 2026Jiale Dai, Liuxian Ma, Hongcan Deng +5LLM Alignment Language Model Decoding
State Key Laboratory of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University · College of Artificial Intelligence, Tsinghua University · University of Chinese Academy of Sciences +1
VISTA: Steering LLMs with Latent User Values Inferred from a Few Survey Answers
Oct 16, 2026Yinuo Xu, Jadie Adams, Brian Hu +3Personalized Language Model Alignment Language Model Steering
University of Michigan · Kitware Inc.
WeFair: Fidelity-Aware Alignment from Human Feedback for Personalized Query Rewriting in Short-Video Search
Oct 12, 2026Yingtao Ren, Aolin Li, Cheng Cheng +6Query Reformulation Personalized Language Model Alignment
Weixin Group, Tencent China
Personalized Federated Direct Preference Optimization via LoRA Adapters
Oct 9, 2026Zhanhong Jiang, Soumik SarkarPersonalized Language Model Alignment Direct Preference Optimization
Translational AI Center, Iowa State University, Ames, IA 50011 USA · Department of Mechanical Engineering, Iowa State University, Ames, IA 50011 USA
Measuring Cultural Alignment Beyond the Average: A Framework for Evaluating Maternal-Health LLM Interactions in Indian Contexts
Oct 8, 2026Umaira Izhar, Gunjan Arora, Pushpendra SinghCultural Alignment Clinical Language Model Evaluation
Indraprastha Institute of Information Technology Delhi
MIRROR: From Imitation to Internalization in LLM Personalization
Oct 7, 2026Huayi Lai, Jicheng Yang, Min Yi +1Personalized Language Models Personalized Language Model Alignment
Baidu Inc.
Collaborative Personalized Preference Alignment for LLMs under Data Deficiency
Oct 5, 2026Liyan Yang, Yige Yuan, Zhiqin YangPersonalized Language Models Personalized Language Model Alignment
City University of Hong Kong · University of Washington · The Hong Kong University of Science and Technology
Groupwise Distortion Guarantees for Preference-Based Alignment
Oct 4, 2026Jacob Brodkey, Roberto Tamez, Aaron RothPairwise Preference Learning Social Choice Theory
University of Pennsylvania
PB-GRPO: Learning Socially Adaptive LLM Agents from Persona-Driven Simulation with Preference-Batched GRPO
Oct 2, 2026Jingquan Wang, Jun Yin, Xu Han +3LLM Agent Training Group Relative Policy Optimization
Amazon
ConvoDrift: A Multi-Turn Conversational Dataset for Modeling Stylistic Tone Evolution
Oct 2, 2026Vihindi Kotalawala, Pamoda Dilranga, Gayani Thoradeniya +1Conversational Agents Personalized Language Model Alignment
Informatics Institute of Technology, Sri Lanka · Independant Researcher, Sri Lanka · National Health and Medical Research Council, Australia +1
From Static Personal Values to Contextualized Personalization: Bayesian Personalized Value Alignment for LLMs
Sep 24, 2026Hanze Guo, Aixuan Song, Jing Yao +4LLM Alignment Personalized Language Models
Gaoling School of Artificial Intelligence, Renmin University of China · Microsoft Research Asia · Beijing Key Laboratory of Research on Large Models and Intelligent Governance +1
Auditing Political Alignment in LLM Assistants: Engagement, Stance, and User Identity
Sep 19, 2026Joan C. TimonedaPolitical Bias in Language Models Language Model Auditing
Assistant Professor, Department of Political Science, Purdue University
Toward Robust Personalized Alignment for LLMs: Mitigating Persona Drift in Multi-Turn Dialogue
Sep 11, 2026Youyuan Zhang, Siyuan Li, Fangming Liu +1Belief Revision LLM Alignment
Harbin Institute of Technology, Shenzhen, China · Peng Cheng Laboratory, China · Huazhong University of Science and Technology, China
Alignment by Stereotyping: How LLMs Sacrifice Individual Distinctiveness for Cultural Adaptation
Sep 5, 2026Qishuai Zhong, Zongmin Li, Siqi Fan +1Personalized Language Model Alignment Demographic Bias in Language Models
Nanyang Technological University, Singapore · University of Electronic Science and Technology of China
Gradient-Aligned Pair Selection for Personalized Preference Optimization
Sep 4, 2026Ruoming Jin, Xinyu Li, Hao Zhou +6Preference Optimization Personalized Language Model Alignment
Kent State University · EvenUp, USA · Auburn University
Personalized Privacy Control in LLMs via Attention Head Intervention
Aug 21, 2026Junseok Kim, Nakyeong Yang, Kyomin JungPrivacy-Preserving Language Models Personalized Language Model Alignment
IPAI, Seoul National University · Max Planck Institute for Software Systems
Behaviorally Grounded User Profiles from the Wild for Personalized Alignment and Multi-Perspective Reasoning
Aug 11, 2026Yuxuan Li, Victor Zhong, Ehsan KamallooPersonalized Language Model Alignment User Modeling
University of Waterloo · ServiceNow AI Research
Cautious Context Steering for Language Model Personalization
Aug 6, 2026Gihoon Kim, Jeyoung Lee, Suhan Woo +4Language Model Steering LLM Personalization
Yonsei University · Hyundai Motors Company · Korea Institute of Science and Technology
Compliance2LoRA: Personalizable On-Demand Safety Alignment on Arbitrary Policy Subsets via Hypernetwork-Generated LoRA Adapters
Jul 30, 2026Pankayaraj Pathmanathan, Furong HuangLLM Alignment LLM Safety Alignment
University of Maryland College Park
Beyond a Global Norm: Personalizing Toxicity Sensitivity in Language Models Without Retraining
Jul 25, 2026Rares A. C. Diaconescu, Iulia Slanina, Alina Florea +5LLM Alignment LLM Safety Alignment
Delft University of Technology
Whose Norms? Disentangling Cultural and Personal Alignment in Large Language Models
Jun 5, 2026Angana Borah, Isabelle Augenstein, Rada MihalceaLLM Alignment Normative Conformity in Language Models
University of Michigan - Ann Arbor · University of Copenhagen
Persona-Guided LLM Agents for Task-Oriented Dialogue
Jun 4, 2026Maryam Shoaeinaeini, Brent Harrison, A. B. SiddiquePersona-Conditioned Generation Task-Oriented Dialogue Systems
University of Kentucky, Lexington, KY, USA
TriAlign: Towards Universal Truth Consistency in Personalized LLM Alignment
Jun 1, 2026Thi-Nhung Nguyen, Linhao Luo, Rollin Omari +3LLM Alignment Algorithmic Fairness
Department of Data Science & AI, Monash University · Defence Science and Technology Group, Australia
Large Language Models Should Learn Personalized Rather Than Aggregated Human Preferences
May 30, 2026Cristina GarbaceaLLM Alignment Social Choice Theory
University of Chicago, Data Science Institute, Chicago, USA.
MATO: Multi-objective Personalized Alignment with Test-time Optimization for Large Language Models
May 25, 2026Linhao Luo, Thuy-Trang Vu, Van-Anh Nguyen +3LLM Alignment Inference-Time Optimization
Monash University · Defence Science and Technology Group, Australia
Personalized Alignment Revisited: The Necessity and Sufficiency of User Diversity
May 9, 2026Enoch Hyunwook KangLLM Alignment Personalized Language Model Alignment
Foster School of Business, University of Washington, Seattle, Washington, USA.

Oct 27, 2026J/K move · Enter open · S save

Same Verdict, Different Readers: Personalizing Fact-Checking Explanations from Relative Preferences

Haokun Wen, Xuemeng Song, Mong-Li Lee, Wynne Hsu

National University of Singapore Singapore · Southern University of Science and Technology Shenzhen, China

Automated Fact-Checking Personalized Language Model Alignment Faithfulness of Language Model Explanations Preference Learning

Open paper AskPDF arXiv

Fact-checking explanations help Web users understand why a claim receives its verdict, yet the same explanation may not satisfy readers with different backgrounds and preferences. We formulate Faithful Personalized Fact-Checking Explanation Rewriting, which adapts an existing explanation to individual readers while preserving factual fidelity. Rather than relying on reader-written reference explanations, we learn personalized rewriting from the relative preferences of LLM-simulated readers. However, this presents two challenges: distinguishing persona-specific preferences from general explanation quality and learning beyond a limited set of predefined explanation styles. To address these challenges, we propose Personalized Explanation Rewriting via Preference Learning (PERPL), a framework consisting of two components. Simulated Preference Construction derives persona-specific supervision from simulated reader preferences through cross-persona reciprocal pairing. Progressive Preference Learning first trains a lightweight rewriter on the constructed supervision and then refines it through exploration-guided preference optimization, enabling it to discover preferred explanations beyond the predefined styles. Extensive experiments demonstrate that PERPL outperforms much larger general-purpose LLMs in personalized preference alignment while maintaining high factual fidelity.