Personalized Reward Modeling

Follow this topic Ask about this topic

Latest papers

10 papers2 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

PerSPL: Personalized Semi-supervised Preference Learning
Oct 21, 2026Xiaoan Xu, Tingzhou Wei, Juncheng Dong +3Personalized Reward Modeling User Preference Modeling
Duke University · George Washington University
Using Context Is Not Enough: Test-Time Training for Personalized Reward Modeling
Sep 28, 2026Bohao Wang, Xiaoyan Zhao, Yang Zhang +4Pairwise Preference Learning Reward Modeling
Zhejiang University · National University of Singapore
To Go Far, Go Together: Diverse Preferences Induce a Curriculum for Reward Optimization
Aug 19, 2026Taehyung Kim, Jongeun ChoiPolicy Optimization Reinforcement Learning
Yonsei University, Seoul, South Korea
PALATE: Personalized Aesthetic Learning through Adaptive Taste Evolution for Multi-User Portrait Retouching
Aug 19, 2026Jingxuan Wang, Yifan Mei, Yuxia Niu +2Personalized Reward Modeling User Preference Modeling
Harbin Institute of Technology · MT Lab, Meitu Inc. · Xiamen University +1
PAFO: Pareto Fairness Optimization for Personalized Reward Modeling
Jun 6, 2026Xiaoyan Zhao, Haoting Ni, Yang Zhang +3Reward Modeling LLM Alignment
National University of Singapore · University of Science and Technology of China · Peking University
Test-Time Personalization: A Diagnostic Framework and Probabilistic Fix for Scaling Failures
May 9, 2026Linhai Zhang, Yulan HeReward Modeling Reward Hacking
King’s College London · The Alan Turing Institute
Personalized RewardBench: Evaluating Reward Models with Human Aligned Personalization
Apr 8, 2026Qiyao Ma, Dechen Gao, Rui Cai +4Personalized Reward Modeling Reward Model Evaluation
University of California, Davis
Personalizing Text-to-Image Generation to Individual Taste
Apr 8, 2026Anne-Sofie Maerten, Juliane Verwiebe, Shyamgopal Karthik +3Personalized Reward Modeling Personalized Image Aesthetics Assessment
Tübingen AI Center, University of Tübingen · Department of Brain and Cognition, KU Leuven
P-Check: Advancing Personalized Reward Model via Learning to Generate Dynamic Checklist
Jan 6, 2026Kwangwook Seo, Dongha LeeReward Modeling Personalized Reward Modeling
Department of Artificial Intelligence,Yonsei University
The Reward Model Selection Crisis in Personalized Alignment
Dec 28, 2025Fady Rezk, Yuangang Pan, Chuan-Sheng Foo +4Personalized Language Model Alignment Personalized Reward Modeling
School of Informatics, University of Edinburgh, UK · Agency for Science, Technology and Research (A*STAR), Singapore · Samsung AI Research Center, Cambridge, UK

Oct 21, 2026J/K move · Enter open · S save

PerSPL: Personalized Semi-supervised Preference Learning

Xiaoan Xu, Tingzhou Wei, Juncheng Dong, Korel Gundem+2

Duke University · George Washington University

Personalized Reward Modeling User Preference Modeling Preference Learning Reward Modeling Pairwise Preference Learning

Open paper AskPDF arXiv

Personalized reward models require user-specific preference comparisons, but collecting such feedback is costly and provides only limited supervision for each user. At the same time, large pools of responses can be generated without user identities or preference labels, yet existing personalized reward models leave this data unused. Directly learning from these responses is difficult because pairwise objectives constrain relative scores within a prompt, whereas pseudo-labeling individual responses requires scores that are comparable across prompts for the same user. We propose PerSPL (Personalized Semi-supervised Preference Learning), a backbone-agnostic framework that uses pair-mean score centering to stabilize this cross-prompt score coordinate and iteratively turns unlabeled responses into user-specific training signals. Against seven representative baselines across four benchmarks---PersonalLLM, HelpSteer2, Reddit TLDR, and Chatbot Arena---PerSPL achieves the highest reported mean accuracy among the evaluated methods for both seen and unseen users. The two latter benchmarks provide real-user preference comparisons in summarization and open-domain chatbot interaction, respectively. These results indicate that unlabeled responses can provide additional supervision for personalized reward modeling when per-user preference labels are scarce.