Generative recommenders decode the semantic ID of a user's next item with a language model, and post-training them with reinforcement learning (RL) after supervised fine-tuning (SFT) has become standard practice. The gains are mostly reported on the first ten results, yet in deployment such models hand hundreds of candidates to a downstream ranker or reward model. We show that RL post-training has a hidden cost: it improves accuracy at the top of the list (hit@K for small K) but, on all five datasets we study, lowers the recall of the candidate list (hit@K for K in the hundreds). The model becomes a better ranker and a worse retriever. Controlled experiments trace the loss to the negative advantages that RL assigns to the candidates it samples, the rollout. The root cause is that only one next item is logged per user, so RL treats every other candidate as wrong. Yet these candidates are the model's most plausible alternatives, and often the next items of other users with similar histories. Penalising them pushes down exactly what similar users choose next, and these items drop out of those users' candidate lists. This diagnosis points directly to a remedy: out-of-rollout anchoring, a single additional loss term that preserves the SFT model's ordering of the items outside the rollout. It keeps the gain at the top and restores recall to the SFT level on all five datasets, and it is effective under other RL configurations as well. To our knowledge, this is the first systematic study of this recall loss: we measure it, identify its cause, and show that, once the cause is known, a simple remedy suffices.
Figures & tables
Figure 1. Change in hit@ K after GRPO post-training on Office and Industrial, relative to the SFT model that RL started from (exact ranking of the full catalogue; mean of three seeds, bands show their range). Line chart of the change in hit at K against the cut-off K on a logarithmic axis for Office and Industrial. Both curves are above zero for small K and fall below zero beyond K of about 20 and 50, reaching minus 5.1 and minus 1.5 points at K equal to 500.
Figure 2. (a) A generative recommender is deployed as the retrieval stage of a cascade (highlighted: the common case), where recall at large K matters, or end to end, where accuracy at the top matters (example systems by the role their papers describe). (b) The effect studied here: after RL post-training, targets near the top move up and targets further down drop out of the top K . Two panels. Panel a shows the classical retrieval and ranking cascade, a generative model replacing the retrieval stage, which is highlighted as the common case, and a generative model replacing the whole cascade. Panel b shows two rankings side by side, under SFT and after RL: a target near the top moves up and a target further down leaves the top K.
System
Platform
Objective
Candidates
Retrieval stage: candidates are passed to a ranker
TSGR ( Zhan et al., 2026 )
Taobao
GRPO
1,500
GrowthGR ( Wang et al., 2026a )
Taobao
GRPO
1,000
EG-GRPO ( Zhu et al., 2026 )
Tmall
GRPO
100
GenR-PO ( Li et al., 2024 )
JD
DPO
100
PinGER ( Li et al., 2026 )
Pinterest
DPO
100
Table 1. RL and preference post-training of generative recommenders in prior work. Candidates: number generated per request (passed to the next stage or, failing that, the largest beam width); n/s: not stated. † Online results are reported with and without a reward model that selects among the generated candidates.
Dataset
Items
Test
Split
In rollout
Office
3,459
4,866
chronological
37%
Industrial
3,686
4,533
chronological
42%
Beauty
12,101
5,000
leave-one-out
24%
Toys
11,924
5,000
leave-one-out
16%
Sports
18,357
5,000
leave-one-out
11%
Table 2. Datasets. “In rollout”: share of training prompts whose rollout contains the target at the start of RL.
NDCG
hit@ K
@10
5
10
50
100
200
500
Office
+0.95
+0.71
+0.44
−1.52
−2.77
−4.14
−5.14
Industrial
+0.43
+0.54
+0.51
+0.02
−1.27
−1.04
−1.51
Beauty
+0.38
+0.28
+0.47
+0.33
−0.31
−0.39
−0.49
Toys
+0.40
+0.30
+0.50
+0.24
−0.40
−0.62
−0.04
Sports
+0.01
+0.20
−0.18
−0.47
−0.25
−0.19
−0.67
Table 3. Change after one epoch of GRPO relative to SFT, in points (mean of three seeds). Table 5 gives the absolute values.
Figure 3. Why recall degrades (illustrative example). (1) A target in the rollout gets a positive advantage and the other candidates negative ones, which improves the top of the list. (2) A plausible item such as black ink is sampled for many users and is rarely the target, so penalties accumulate on it and its score also falls for similar users. (3) Where the rollout misses black ink no gradient counteracts the fall, and for a test user whose target it is, rarely sampled items overtake it. Three cards. Card one shows a rollout for a training user in which the target, colour ink, gets a plus sign and three other candidates including black ink get minus signs. Card two shows sixteen squares of which fifteen are negative, standing for the advantages black ink collects over all rollouts that sample it. Card three shows a test user whose target is black ink: under SFT it is above the top K line, after RL two rarely sampled items have moved above it and it falls below the line.
Office
Industrial
Rank under SFT
median rank
demoted
median rank
demoted
1
1 → 1
13%
1 → 1
19%
2–10
4 → 3
42%
5 → 5
47%
11–50
26 → 44
62%
25 → 30
56%
51–100
72 → 136
64%
72 → 90
57%
101–200
146 → 262
65%
147 → 166
55%
Table 4. Where the targets move under GRPO: test prompts grouped by the rank of their target under SFT, with the median rank before and after RL and the share whose rank worsens (mean of three seeds).
Figure 4. Evidence for the cause (Office; change in hit@ K against SFT; a, c: synced reference, one run). (a) Keeping only the positive advantages removes the loss and the gain; keeping only the negative ones produces a much larger loss. (b) DPO loses recall up to K=500 with negatives from the SFT beam and only at intermediate K with random ones (three seeds, bands: range). (c) Test prompts grouped by the rollout record of their target, computed before RL: the loss falls on targets with a negative record. Three line charts of the change in hit at K against K. Panel a: with positive advantages only the curve stays near zero, GRPO falls to minus 6 points and negative advantages only fall to minus 20 points at K of 500. Panel b: DPO with beam negatives falls to minus 4 points on average, with random negatives it returns to zero at K of 500. Panel c: targets with a strongly negative rollout record lose 27 points at K of 500, mildly negative ones 9 points, and targets with a positive record gain a few points.
Figure 5. Out-of-rollout anchoring on one training prompt. GRPO handles the rollout as before. In addition, M items sampled outside the rollout are scored by the frozen SFT model and by the policy, and the policy is penalised if their relative scores change. A rollout of four candidates, the target with a plus sign and three others with minus signs, labelled inside the rollout, GRPO unchanged. Below a dotted line, five items outside the rollout, three of them highlighted as anchor items that must keep the relative order the SFT model gave them, labelled with the anchor term lambda times a KL divergence.
Figure 6. GRPO with and without the anchor. (a, b) Change in hit@ K against SFT (mean of three seeds, bands show their range). (c) Office test prompts grouped by the rollout record of their target (Section 4.3 ): change in hit@500 against SFT (one seed). Three panels. Panels a and b are line charts of the change in hit at K against K for Office and Industrial. The GRPO curve starts above zero and falls to minus 5.1 points on Office and minus 1.5 on Industrial at K of 500; the curve of GRPO with the anchor starts at the same height and stays near zero, ending at minus 0.7 on Office and plus 0.4 on Industrial. Panel c is a bar chart of the change in hit at 500 for four groups of Office test prompts: for targets with a strongly negative rollout record, minus 23.4 points under GRPO and minus 3.1 with the anchor; for mildly negative ones, minus 5.4 and plus 0.6; for the two groups with a positive record, small gains under both.
hit@ K (%)
NDCG
3
5
10
50
100
200
300
500
@10
Office
SFT
11.76
13.42
15.97
25.73
31.94
39.56
44.88
52.84
11.59
GRPO
12.66
14.13
16.41
24.22
29.16
35.42
40.29
47.69
12.55
GRPO + anchor
12.76
14.41
16.89
26.27
32.17
39.29
44.40
52.14
12.64
anchor effect
+0.10
+0.29
+0.48
+2.05
+3.01
+3.87
+4.11
+4.45
+0.09
Industrial
SFT
8.54
10.21
13.19
23.21
30.20
38.05
43.97
53.23
9.05
Table 5. SFT, GRPO and GRPO with the out-of-rollout anchor (mean of three seeds; best value per dataset and column in bold). Anchor effect: paired difference between the two RL models in points.
Office
Industrial
@50
@100
@500
@50
@100
@500
SFT
25.73
31.94
52.84
23.21
30.20
53.23
Synced reference
23.76
27.81
46.40
24.22
29.36
51.71
+ anchor
25.94
31.28
52.28
23.58
29.38
53.98
anchor effect
+2.18 ∗
+3.47 ∗
+5.88 ∗
−0.64
+0.02
+2.27 ∗
REINFORCE++
20.86
24.39
37.92
21.51
26.34
46.94
Table 6. The anchor on other base configurations: hit@ K in % (one seed). Anchor effect: paired difference in points; a star marks a significant difference. Within each pair the better value is bold when the difference is significant.