Personalized Language Model Alignment

Overview

Personalized language model alignment studies how models can infer and follow an individual’s preferences from feedback, conversation history, profiles, or behavior rather than relying only on population-wide preferences. Common approaches include personalized preference optimization such as DPO and reinforcement learning, user-specific adapters, memory and retrieval, and inference-time steering or decoding. Evaluation spans preference prediction and generation, multi-turn personalization, and applied settings such as recommendations, privacy, and cultural or political alignment. A central challenge is balancing faithful personalization with safety, truthfulness, and general capability, especially when user signals are sparse, shifting, or in tension with one another.

Latest papers

CardsList

65 papers8 papers in the last four weeks, down 20% on the four weeks before. 0.1% of all new papers.

  1. Sample-efficient Personalization of Large Language Model Alignment with Multidimensional User Preference Data

    Nov 17, 2026Zhen-Yu Zhang, Jerry Huang, Gang Niu +2Preference Learning for LLMsPersonalized Language Model Alignment

  2. Personalization of Basic and Moral Values Guiding LLM-Controlled Assistant Robots

    Nov 13, 2026Mateusz Wozniak, Qi Gao, Agnieszka WykowskaPersonalized Language Model AlignmentSocial Robotics

  3. Adaptive Personalization Under Preference Epistemic Uncertainty

    Oct 29, 2026Priyanka Dey, Rajarshi Das, Vikas Bahirwani +3Personalized Language ModelsPersonalized Language Model Alignment

  4. Same Verdict, Different Readers: Personalizing Fact-Checking Explanations from Relative Preferences

    Oct 27, 2026Haokun Wen, Xuemeng Song, Mong-Li Lee +1Preference Learning for LLMsAutomated Fact-Checking

  5. FATE-5: Faithful Alignment of Trait Expression for Continuous Big Five Personality Control

    Oct 26, 2026Gunhee Cho, Hongyeob Kim, Yun-Gyung CheongPersonality Modeling in Language ModelsPersonalized Language Model Alignment

  6. When Personas Become Harmful: Guiding LLMs with Global Citizenship Values Alignment

    Oct 24, 2026Siqi Shen, Lajanugen Logeswaran, Moontae Lee +2LLM SafetyLLM Safety Alignment

  7. MusicTwins: Training Audio-Language Models to Listen like Their Persona

    Oct 23, 2026Milan Liessens Dujardin, Kevin MiaoAudio-Language ModelsPersonalized Language Model Alignment

  8. LaVGuard: Selective Decoding for Contextual Value Drift

    Oct 21, 2026Jiale Dai, Liuxian Ma, Hongcan Deng +5LLM AlignmentLanguage Model Decoding

  9. Personalized Federated Direct Preference Optimization via LoRA Adapters

    Oct 9, 2026Zhanhong Jiang, Soumik SarkarPersonalized Language Model AlignmentDirect Preference Optimization

  10. Collaborative Personalized Preference Alignment for LLMs under Data Deficiency

    Oct 5, 2026Liyan Yang, Yige Yuan, Zhiqin YangPersonalized Language ModelsPersonalized Language Model Alignment

  11. Groupwise Distortion Guarantees for Preference-Based Alignment

    Oct 4, 2026Jacob Brodkey, Roberto Tamez, Aaron RothPairwise Preference LearningSocial Choice Theory

  12. ConvoDrift: A Multi-Turn Conversational Dataset for Modeling Stylistic Tone Evolution

    Oct 2, 2026Vihindi Kotalawala, Pamoda Dilranga, Gayani Thoradeniya +1Conversational AgentsPersonalized Language Model Alignment

  13. From Static Personal Values to Contextualized Personalization: Bayesian Personalized Value Alignment for LLMs

    Sep 24, 2026Hanze Guo, Aixuan Song, Jing Yao +4LLM AlignmentPersonalized Language Models

  14. Auditing Political Alignment in LLM Assistants: Engagement, Stance, and User Identity

    Sep 19, 2026Joan C. TimonedaPolitical Bias in Language ModelsLanguage Model Auditing

  15. Toward Robust Personalized Alignment for LLMs: Mitigating Persona Drift in Multi-Turn Dialogue

    Sep 11, 2026Youyuan Zhang, Siyuan Li, Fangming Liu +1Belief RevisionLLM Alignment

  16. Alignment by Stereotyping: How LLMs Sacrifice Individual Distinctiveness for Cultural Adaptation

    Sep 5, 2026Qishuai Zhong, Zongmin Li, Siqi Fan +1Personalized Language Model AlignmentDemographic Bias in Language Models

  17. Gradient-Aligned Pair Selection for Personalized Preference Optimization

    Sep 4, 2026Ruoming Jin, Xinyu Li, Hao Zhou +6Preference OptimizationPersonalized Language Model Alignment

  18. Personalized Privacy Control in LLMs via Attention Head Intervention

    Aug 21, 2026Junseok Kim, Nakyeong Yang, Kyomin JungPrivacy-Preserving Language ModelsPersonalized Language Model Alignment

  19. Cautious Context Steering for Language Model Personalization

    Aug 6, 2026Gihoon Kim, Jeyoung Lee, Suhan Woo +4Language Model SteeringLLM Personalization

  20. Beyond a Global Norm: Personalizing Toxicity Sensitivity in Language Models Without Retraining

    Jul 25, 2026Rares A. C. Diaconescu, Iulia Slanina, Alina Florea +5LLM AlignmentLLM Safety Alignment

  21. Whose Norms? Disentangling Cultural and Personal Alignment in Large Language Models

    Jun 5, 2026Angana Borah, Isabelle Augenstein, Rada MihalceaLLM AlignmentNormative Conformity in Language Models

  22. Persona-Guided LLM Agents for Task-Oriented Dialogue

    Jun 4, 2026Maryam Shoaeinaeini, Brent Harrison, A. B. SiddiquePersona-Conditioned GenerationTask-Oriented Dialogue Systems