cs.IRNov 17, 2026

Better Rankers, Worse Retrievers: The Hidden Cost of RL Post-Training in Generative Recommendation

Authors: Yuheng Chen, Fan Zhang, Yixuan Xu, Chao Yang, Yang Xiang, Can Yang

Organizations: The Hong Kong University of Science and Technology, Hong Kong, China · Trip.com Group, Shanghai, China

Abstract

Generative recommenders decode the semantic ID of a user's next item with a language model, and post-training them with reinforcement learning (RL) after supervised fine-tuning (SFT) has become standard practice. The gains are mostly reported on the first ten results, yet in deployment such models hand hundreds of candidates to a downstream ranker or reward model. We show that RL post-training has a hidden cost: it improves accuracy at the top of the list (hit@K for small K) but, on all five datasets we study, lowers the recall of the candidate list (hit@K for K in the hundreds). The model becomes a better ranker and a worse retriever. Controlled experiments trace the loss to the negative advantages that RL assigns to the candidates it samples, the rollout. The root cause is that only one next item is logged per user, so RL treats every other candidate as wrong. Yet these candidates are the model's most plausible alternatives, and often the next items of other users with similar histories. Penalising them pushes down exactly what similar users choose next, and these items drop out of those users' candidate lists. This diagnosis points directly to a remedy: out-of-rollout anchoring, a single additional loss term that preserves the SFT model's ordering of the items outside the rollout. It keeps the gain at the top and restores recall to the SFT level on all five datasets, and it is effective under other RL configurations as well. To our knowledge, this is the first systematic study of this recall loss: we measure it, identify its cause, and show that, once the cause is known, a simple remedy suffices.

Figures & tables

Explore similar work

CardsList
  1. Robust Post-Training for Generative Recommenders: Why Exponential Reward-Weighted SFT Outperforms RLHF

    Mar 10, 2026Keertana Chidambaram, Sanath Kumar Krishnamurthy, Qiuling Xu +2Recommender SystemsGenerative Recommendation

  2. GR2: Generative Reasoning Re-ranker

    Feb 8, 2026Mingfu Liang, Yufei Li, Jay Xu +20RL for Language ModelsRL for Recommendation

  3. Training with Missed Targets in Generative Recommendation: Separating Supervision from Probability Competition

    Oct 7, 2026Xuesi Wang, Yangbin Shi, Xiaolin ZhengGenerative RecommendationLearning to Rank