EchoRec: Multi-Item Prediction-Empowered Generative Recommendation via Cycle-Consistent Preference Alignment
Authors: Haokai Ma, Aoqi Hu, Yueao Xing, Ruobing Xie, Yonghui Yang, Teng Tu, Lei Meng, Tat-Seng Chua
Organizations: National University of Singapore Singapore, Singapore · Beijing University of Posts and Telecommunications Beijing, China · Tencent Beijing, China · Shandong University Jinan, China
Generative recommendation autoregressively generates the semantic IDs of the target item, unifying preference modeling and index retrieval within the shared token space. Recent attempts have introduced Multi-Token Prediction (MTP) into this field, yet they primarily inherit its efficiency merit, leaving its potential as dense supervision unexplored. Unlocking this potential hinges on whether future behaviors qualify as informative supervision. Our analysis reveals that future behaviors carry a semantic echo of the current one far above that of random pairs, which nevertheless decays along horizons under intent transitions, making them informative yet order-dependent signals. Motivated by this, we propose EchoRec, which empowers MTP with cycle-consistent holistic preference alignment across multi-horizon for generative recommendation. It comprises two synergistic modules. Horizon-aware Preference Generation (HPG) sequentially chains lightweight auxiliary branches upon the base recommender, where each branch conditions on its predecessor to respect preference evolution. Verifiable Holistic-Preference Alignment (VHA) further consolidates them into the holistic preference and echoes it back through cycle-consistent projectors to suppress spurious alignment, with theoretical guarantees that exclude the rank-collapse form of spurious alignment under an invertible transport, enabling the holistic preference to be retained in the decoding representation. All auxiliary components serve as disposable scaffolding discarded at inference, introducing negligible online serving overhead. Extensive experiments on three datasets demonstrate the superiority of our EchoRec, together with its naturally acquired multi-item generation ability. Our code and datasets will be available upon acceptance.
Figures & tables
Figure 1. Conceptual comparison of future-behavior modeling paradigms. (a) Next-item prediction underutilizes future-behavior supervision. (b) Parallel multi-item prediction risks homogenizing the horizons. (c) EchoRec propagates supervision across horizons and cycle-consistently aligns the decoding representation with holistic preference.
Figure 2. Analysis of the usefulness of future information for next-item prediction. (a) shows the SID overlap lift of (in,in+h) over random pairs across three datasets, which remains well above random even at larger offsets. (b) presents the codebook-wise lift on Game, where the semantic echo stays consistent across all codebooks.
Figure 3. Analysis of the homogenization pitfall of parallel MTP structures on Game. (a) reports the inter-horizon Jaccard similarity of the top-10 decoded items, where RPG-Par. exhibits higher overlap than RPG-Seq. on average. (b) compares the performance across horizons, where RPG-Par. merely matches the rollout without horizon-specific supervision yet underperforms at the immediate horizon in+1 .
Figure 4. Overall structure of our EchoRec .
Dataset
#User
#Item
#Interactions
Sparsity
Game
22,970
30,642
310,554
99.96%
Baby
40,501
49,783
485,607
99.98%
Arts
20,965
76,985
280,931
99.98%
Table 1. Statistics of three real-world datasets.
Algorithm
Game
Baby
Arts
H@10
N@10
H@20
N@20
H@10
N@10
H@20
N@20
H@10
N@10
H@20
N@20
EAGER ( Wang et al., 2024b )
0.0266
0.0132
0.0376
0.0160
0.0172
0.0094
0.0241
0.0111
0.0024
0.0016
0.0030
0.0018
TIGER ( Rajput et al., 2023 )
0.0831
0.0443
0.1275
0.0554
0.0220
0.0110
0.0352
0.0143
0.0583
0.0364
0.0791
0.0417
LETTER ( Wang et al., 2024a )
0.0361
0.0179
0.0544
0.0225
0.0210
0.0108
0.0352
0.0144
0.0177
0.0095
0.0262
0.0116
ETEGRec ( Liu et al., 2025b )
0.0631
0.0328
0.0968
0.0413
0.0206
0.0106
0.0330
0.0137
0.0384
0.0214
0.0557
0.0258
RPG ( Hou et al., 2025 )
0.0931
0.0564
0.1249
0.0644
0.0362
0.0243
0.0452
0.0266
0.0625
0.0421
0.0802
0.0466
Table 2. Performance comparison of EchoRec against six baselines across three datasets. ∗ denotes statistically significant improvements of EchoRec over its corresponding backbone ( p \textless 0.01 with paired t-tests).
Figure 5. Results of EchoRec and its ablation versions on Game, Baby, and Arts. All components are effective.
Figure 6. Multi-horizon prediction performance of RPG+HPG against RPG-Seq. and RPG-Par. on Game, in terms of (a) the horizon-wise performance and (b) the inter-horizon overlap of the decoded items at three future horizons.
Figure 7. Sensitivity of EchoRec to the loss weights (a) λtrace and (b) λcyc on Baby, where ⋆ marks the adopted value.
Figure 8. Robustness analysis of EchoRec against RPG on Baby and Arts, where the proportion η of historical interactions is randomly replaced with sampled items.
Dataset
Algorithm
#Tra. (ms)
#Inf. (ms)
#Mem. (MiB)
Game
SETRec
125.61
41.57
1464.07
RPG
30.11
87.44
181.04
EchoRec (RPG)
61.87
85.46
181.04
Baby
SETRec
139.81
46.46
2231.32
RPG
11.25
64.13
171.54
EchoRec (RPG)
32.81
64.21
171.54
Table 3. Efficiency comparison results of EchoRec (RPG) against SETRec and RPG, where “#Tra.” and “#Inf.” denote the average training and inference latency per batch, while “#Mem.” denotes the GPU memory usage during inference.
Figure 9. Case study of EchoRec on Baby, including (a) the next-item recommendation and (b) the multi-horizon recommendation, where the ground-truth items are marked in red.
Generative recommendation retrieves items by autoregressively generating semantic identifiers, but beam search may discard a target before its complete identifier is generated. Our preliminary analysis across three benchmarks shows that most missed targets are pruned within the first two decoding steps, highlighting the importance of early prefix retention. However, retaining a target's first-token branch alone is insufficient if its continuation is pruned at the next step: aligning only the first-token distribution improves first-token survival but yields little gain in complete-path retention. This observation motivates joint supervision of early branches and their continuations. We propose Prefix Alignment with Temporal History (PATH), which aggregates transition statistics from the training corpus over recent interactions with exponential decay to construct history-conditioned two-token prefix targets. PATH aligns the model's joint predictions with these targets through forward KL divergence, using a chain-rule decomposition that enables unbiased Monte Carlo estimation. At inference time, PATH reuses the transition statistics for pointwise mutual information (PMI) calibration, reranking completed candidates relative to global prefix frequency. Experiments on Beauty, Instruments, and Yelp demonstrate the effectiveness of PATH, showing consistent improvements in recommendation performance and higher full-SID survival rates.
Hongliang Sun, Lianjie Li, Bolin Zhang +3
Department of Computer Science, Weihai & Qingdao Research Institute Harbin Institute of Technology Weihai, China
Generative recommendation models that formulate the task as sequence generation overcome the objective fragmentation problem of traditional cascade architectures, yet existing approaches still suffer from flat semantic representations lacking hierarchical structure for multi-step reasoning and an externally constructed chain-of-thought (CoT) that requires expensive annotations and remains disconnected from the generation objective. We propose HoloRec, an endogenous chain-of-thought recommendation mechanism that unifies representation, reasoning, and generation by constructing a hierarchical semantic encoding matrix via multi-granularity nested residual quantization optimized by a holistic reconstruction loss. HoloRec supports two inference modes: a non-thinking mode that uses lightweight multi-granularity supervised alignment for fast prediction, and a thinking mode that employs an interleaved reasoning scheme to generate CoT steps on the fly, directly embedding reasoning into the generation process without external data. Experiments on multiple public recommendation datasets demonstrate that HoloRec consistently outperforms baselines, with especially significant gains in sparse scenarios, and the thinking mode achieves better accuracy than the non-thinking mode with only modest inference overhead.
Shuqi Zhao, Jingsong Su, Xiang Liu +9
Institute of Information Engineering, Chinese Academy of Sciences Beijing, China · School of Artificial Intelligence, Beijing Normal University Beijing, China · JD.com,Beijing,China
Generative Retrieval (GR) offers a promising paradigm for recommendation through next-token prediction (NTP). However, scaling it to large-scale industrial systems introduces three challenges: (i) within a single request, the identical model inputs may produce inconsistent outputs due to the pagination request mechanism; (ii) the prohibitive cost of encoding long user behavior sequences with multi-token item representations based on semantic IDs, and (iii) aligning the generative policy with nuanced user preference signals. We present GenRec, a preference-oriented generative framework deployed on the JD App that addresses above challenges within a single decoder-only architecture. For training objective, we propose Page-wise NTP task, which supervises over an entire interaction page rather than each interacted item individually, providing denser gradient signal and resolving the one-to-many ambiguity of point-wise training. On the prefilling side, an asymmetric linear Token Merger compresses multi-token Semantic IDs in the prompt while preserving full-resolution decoding, reducing input length by ~2X with negligible accuracy loss. To further align outputs with user satisfaction, we introduce GRPO-SR, a reinforcement learning method that pairs Group Relative Policy Optimization with NLL regularization for training stability, and employs Hybrid Rewards combining a dense reward model with a relevance gate to mitigate reward hacking. In month-long online A/B tests serving production traffic, GenRec achieves 9.5% improvement in click count and 8.7% in transaction count over the existing pipeline.
Yanyan Zou, Junbo Qi, Lunsong Huang +7
JD.com Beijing, China · Waseda University Tokyo, Japan