cs.CLOct 29, 2026

Adaptive Personalization Under Preference Epistemic Uncertainty

Authors: Priyanka Dey, Rajarshi Das, Vikas Bahirwani, Qi Zeng, Julia Kruk, Qi Qi

Organizations: Meta Superintelligence Labs · University of Southern California

Abstract

Human preferences change over time: an established preference may remain valid over time, become uncertain as context shifts, or be explicitly revised. Effective personalization requires conversational AI to adapt its response strategy to evolving preferences rather than relying on passive memory retrieval. We formalize this capability as adaptive personalization under epistemic uncertainty. A human preference study further confirms that people favor state-contingent response strategies as preference uncertainty changes. To evaluate whether language models mirror this adaptation, we create a multilingual benchmark spanning 250 personas, 5 languages, and 25K conversational turns that decouples strategy selection from response execution. Across eight closed- and open-weight models, we uncover a recognition-generation gap: models reliably identify appropriate strategies when selecting among candidate options, but collapse during open-ended generation, suffering a 35-46% drop in strategy calibration. Crucially, this failure is systematic: models default to exploit-first behavior, over-relying on existing preferences regardless of preference epistemic uncertainty. We show this failure is addressable: while prompting interventions fail, targeted training that supervises models' internal strategy representations toward state-conditional human preferences narrows the user acceptability generalization gap by a third on out-of-distribution personas and conversational styles.

Figures & tables

Appendix figures & tables36 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Toward Robust Personalized Alignment for LLMs: Mitigating Persona Drift in Multi-Turn Dialogue

    Sep 11, 2026Youyuan Zhang, Siyuan Li, Fangming Liu +1Belief RevisionLLM Alignment

  2. HyperTrace: Hypothesis-Based Preference Tracing for Online LLM Personalization

    Sep 9, 2026Jianzhi Shen, Keyu Mao, Minghao Shao +7LLM PersonalizationPersonalized Language Models

  3. UPSHIFT: User Preference Steering from Human-AI Interaction Feedback Traces

    Nov 2, 2026Jingping Wang, Zhanhao Li, Shao Zhang +1Personalized Language ModelsLanguage Model Steering