WeGR: Towards Unified Generative Retrieval via Behavior-aware Tokenization and Preference Alignment

Jiajun Han; Guoxin Ma; Felix; Liangsheng Xu; Ruiqi Yu; Yingdi Guo; Zongbing Wang; Xinhua Ye

cs.IROct 12, 2026

WeGR: Towards Unified Generative Retrieval via Behavior-aware Tokenization and Preference Alignment

Authors: Jiajun Han, Guoxin Ma, Felix, Liangsheng Xu, Ruiqi Yu, Yingdi Guo, Zongbing Wang, Xinhua Ye,

Organizations: Tencent Penguin Island

Abstract

Generative retrieval (GR) offers a promising paradigm for industrial search by unifying candidate generation within an autoregressive framework. However, effectively leveraging user behaviors in industrial GR remains challenging: existing SID tokenization methods struggle to preserve collaborative signals while maintaining balanced codebook utilization, whereas reinforcement learning (RL) for prioritizing high-value candidates suffers from sparse rewards when generated SIDs fail to match positive items. To address these challenges, we propose WeGR, a unified GR framework with two complementary components. Specifically, the Collaborative and Collapse-free Tokenizer (CoCo-VAE) injects query-to-item and item-to-item collaborative signals into continuous representations before quantization, preserving behavioral information without disrupting codebook optimization. It further couples first-layer codebook refresh to reactivate underutilized codewords with fidelity anti-collapse regularization, which broadens gradient coverage over relevant codewords and penalizes collisions in deeper layers to yield behavior-aware SIDs with balanced utilization. For preference alignment, Dynamic Horizon Policy Optimization (DHPO) progressively expands reward evaluation and policy optimization horizons to alleviate reward sparsity, enabling coarse-to-fine preference learning from SID prefixes to complete items without auxiliary reward models. Extensive offline experiments validate that WeGR improves retrieval and ranking performance. Moreover, a six-week online A/B test on a large-scale e-commerce search platform achieves relative gains of 2.539% in orders, 1.508% in CTR, and 1.381% in CVR, while reducing CPU and GPU usage by 25.5% and 26.1%. These results highlight the effectiveness and efficiency of WeGR in industrial deployment, supporting multiple search scenarios and tens of millions of daily page views.

Figures & tables

Figure 1. (a) Our proposed End-to-End generative retrieval framework, (b) the traditional multi-stage cascading architecture in E-commerce search.

Figure 2. Overview of the proposed WeGR framework. The upper panels (a–c) illustrate CoCo-VAE for behavior-aware SID construction, including (a) codebook refresh to reactivate underutilized codewords, (b) Q2I and I2I collaborative learning to incorporate behavioral co-occurrence signals into item representations, and (c) fidelity and anti-collapse regularization to preserve quantization quality and improve codebook utilization. The lower panels (d–e) present the preference optimization pipeline, covering (d) session-level training data construction and SFT initialization, and (e) DHPO, which progressively expands reward evaluation and policy optimization horizons for coarse-to-fine preference learning.

Methods	$\mathbf{CUR}_{L1}$	$\mathbf{CUR}_{L1*L2}$	$\mathbf{CUR}_{\mathit{Total}}$	$\mathbf{ICR}$
RQ–Kmeans	100%	95.56%	21.33%	6.92%
RQ–Kmeans + L3 Balance	100%	95.56%	52.30%	24.26%
RQ–OPQ	100%	95.56%	1.33%	22.64%
RQ–VAE	99.96%	68.11%	0.89%	0.09%
R 3 –VAE	18.80%	18.37%	9.19%	1.92%
CoCo–VAE	99.98%	94.85%	22.17%	7.71%

Table 1. Comparison of codebook metrics across different SID methods.

Semantic ID Construction (SFT)
Method	Click (20k)						Order (20k)
Method	HR@6	HR@50	HR@200	MRR@6	MRR@50	MRR@200	HR@6	HR@50	HR@200	MRR@6	MRR@50	MRR@200
RQ–Kmeans	0.2641	0.4120	0.4869	0.1714	0.1815	0.1823	0.4051	0.5890	0.6613	0.2702	0.2833	0.2841
RQ–Kmeans + L3 Balance	0.2491	0.3746	0.4455	0.1657	0.1745	0.1753	0.3832	0.5374	0.6137	0.2602	0.2711	0.2719
RQ–OPQ	0.2585	0.4000	0.4510	0.1694	0.1790	0.1796	0.3897	0.5661	0.6188	0.2621	0.2744	0.2751
RQ–VAE	0.1865	0.2775	0.3185	0.1216	0.1281	0.1285	0.3470	0.4922	0.5416	0.2313	0.2419	0.2425

Table 2. Overall performance comparison on the Click and Order datasets. The best results are highlighted in bold.

Semantic ID Construction
Methods	Click (20k)		Order (20k)
Methods	HR@6	MRR@6	HR@6	MRR@6
CoCo-VAE	0.2759	0.1821	0.4103	0.2776
w/o q2i	0.2650	0.1732	0.3933	0.2604
w/o i2i	0.2704	0.1748	0.4042	0.2693
w/o fid	0.2437	0.1585	0.3944	0.2602

Table 3. Ablation studies of CoCo-VAE and DHPO. The best results in each group are highlighted in bold.

Configuration	Order	CTR	CVR	Latency (ms)
V1.0 (Additive)	+1.579%	+1.108%	+0.520%	—
V1.1 (60% Removal)	+1.962%	+0.884%	+0.950%	$-15.6$
V1.2 (80% Removal)	+2.539%	+1.508%	+1.381%	$-21.7$

Table 4. Online A/B testing results from June 22 to August 4. Order, CTR, and CVR denote relative improvements over the production baseline, with bold values indicating statistical significance (

p<0.05

); the last column reports the absolute change in average serving latency.

Figure 3. The comparisons of online resource savings for baseline and WeGR. online resource savings for baseline and WeGR.

Figure 4. Daily Click and Order HR@200 of WeGR over time. The daily Click and Order HR@200 of WeGR from June 19 to August 19, 2026.

Metric	UG1	UG2	UG3	UG4	UG5	UG6
Order	+1.611%	+5.154%	+1.556%	+4.823%	+9.855%	+6.143%
CTR	+1.125%	+2.323%	+2.112%	+1.638%	+3.448%	+0.524%
CVR	+0.399%	+2.481%	+0.247%	+3.063%	+6.585%	+7.382%

Table 5. Online relative gains across different user groups. All numbers denote relative improvements over the baseline, and bold values indicate statistical significance (

p<0.05

Query bucket	Frequency	Order	CVR	CTR
Low	$0$ – $3$	+3.24%	+2.40%	+0.95%
Medium-low	$4$ – $99$	+3.86%	+3.58%	+0.91%
Medium	$100$ – $999$	$-0.05\%$	$-0.84\%$	+0.99%
High	$\geq 1000$	+3.51%	+1.12%	+2.73%

Table 6. Online relative gains across query-frequency buckets. All numbers denote relative improvements over the baseline, and bold values indicate statistical significance (

p<0.05

Methods	Click (20k)		Order (20k)
Methods	HR@6	MRR@6	HR@6	MRR@6
SFT	0.2759	0.1821	0.4103	0.2776
GRPO-Ranker	0.2790	0.1840	0.4172	0.2835
DHPO	0.2810	0.1851	0.4217	0.2878

Table 7. Comparison of DHPO with ranker-guided GRPO. The best results are highlighted in bold.

Methods	Click (18k)		Order (18k)
Methods	HR@6	MRR@6	HR@6	MRR@6
Semantic ID Construction
RQ-Kmeans	0.2114	0.1428	0.2291	0.1670
CoCo-VAE	0.2289	0.1529	0.2470	0.1815
Improvement	+8.28%	+7.07%	+7.81%	+8.68%
Reinforcement Learning

Table 8. Cross-scenario evaluation of CoCo-VAE and DHPO in live-stream search.

Generative Retrieval

Generative retrieval uses language models to produce relevant document or item identifiers directly from a query.

See all 45 papers

Sep 1, 2026

It Takes Two to Match: Co-Evolving Generative Retriever with Reinforcement Learning

Retrieval is the first stage of modern search and advertising systems, selecting a candidate set from a large item universe for downstream ranking and auction. Recent work increasingly leverages LLMs to improve retrieval through query expansion, data synthesis, and retrieval-feedback training. However, the generative component is typically used for query-side augmentation, while final matching is still delegated to a downstream retriever. We introduce CoGR, a retrieval framework that instead trains LLMs to directly construct retrieval representations on both query and item sides. Each generator produces a compact set of keywords, which are matched directly through an inverted index, preserving compatibility with existing keyword-based retrieval infrastructure. CoGR uses a two-stage training pipeline. Supervised fine-tuning first establishes an aligned keyword space, after which co-evolving reinforcement learning alternately optimizes the query- and item-side generators with GRPO against the opposite side's frozen index. Both sides optimize the same query-to-item retrieval

F_1

objective: the query side receives retrieval

F_1

directly, while the item side receives a counterfactual marginal reward measuring the change in query-side

F_1

caused by its generated keywords. Across 10 representative sparse, dense, and generative baselines, CoGR achieves the best performance on both an internal APP Marketplace dataset and the public WANDS benchmark, improving

F_1

over the strongest baseline by

10.9\%

and

36.1\%

, respectively. Further analysis shows stable co-evolution and increasingly aligned query--item keyword spaces over training.

Runpeng Dai, Kaili Huang, Changsung Kang +1

University of North Carolina at Chapel Hill · Apple

Reinforcement Learning Generative Retrieval Lexical Retrieval RL for IR

Ask PDF

May 14, 2026

Efficient Generative Retrieval for E-commerce Search with Semantic Cluster IDs and Expert-Guided RL

Generative retrieval offers a promising alternative by unifying the fragmented multi-stage retrieval process into a single end-to-end model. However, its practical adoption in industrial e-commerce search remains challenging, given the massive and dynamic product catalogs, strict latency requirements, and the need to align retrieval with downstream ranking goals. In this work, we propose a retrieval framework tailored for real-world recall scenarios, positioning generative retrieval as a recall-stage supplement rather than an end-to-end replacement. Our method, CQ-SID (Category-and-Query constrained Semantic ID), employs category-aware and query-item contrastive learning along with Residual Quantized VAEs to encode items into hierarchical semantic cluster identifiers, significantly reducing beam search complexity. Additionally, we develop EG-GRPO (Expert-Guided Group Relative Policy Optimization), a reinforcement learning approach that aligns generative recall with downstream ranking under sparse rewards by injecting ground-truth samples to stabilize training. Offline experiments on TmallAPP search logs show that CQ-SID achieves up to 26.76% and 11.11% relative gains in semantic and personalized click hitrate over RQ-VAE baselines, while halving beam search size. EG-GRPO further improves multi-objective performance. Online A/B tests confirm gains in GMV (+1.15%) and UCTCVR (+0.40%). The generative recall channel now contributes substantially in production, accounting for over 50.25% of exposures, 58.96% of clicks, and 72.63% of purchases, demonstrating a viable path for deploying generative retrieval in real-world e-commerce systems.

Jianbo Zhu, Xing Fang, Jing Wang +5

Taobao & Tmall Group of Alibaba Hangzhou, China

Semantic ID Recommendation E-Commerce Search Generative Retrieval RL for RAG Information Retrieval E-Commerce

Ask PDF

Oct 20, 2026

Evolving Semantic Indexes for Generative Retrieval

Generative retrieval (GR) casts search and recommendation as autoregressive generation of discrete semantic identifiers (SIDs). SID design largely determines GR quality, but its discrete, interdependent components (SID construction, collision handling, decoding, and training) cannot be optimized end-to-end with gradients and remain largely hand-designed. We introduce a framework that improves these components through LLM-driven evolutionary program search, to our knowledge the first application of this approach to GR. A locally served, open-weight 31B model mutates executable GR components, so no proprietary LLM API is used, and a 1M-parameter proxy decoder provides fast feedback in place of repeatedly training the 1B target generator. Across 21 SID programs on NQ320K, proxy and target Recall@100 show strong rank correlation (Spearman

ρ= 0.90

). Guided by this proxy, evolution improves the seed GR on NQ320K, MSMARCO300K, and a multilingual e-commerce dataset under a fixed training recipe. On NQ320K, jointly evolving SID construction and collision handling improves Recall@1 from 55.8% to 59.3% and Recall@100 from 85.8% to 90.6%. A component study shows that proxy reliability depends on what is evolved: improvements transfer reliably for the semantic index but not for the generator's training objective or the full pipeline. LLM-driven evolutionary search is thus a practical tool for GR design, provided proxy reliability is established for the chosen search space.

Jinming Nian, Jyun-Yu Jiang, Henry Tsai +2

Santa Clara University · Coupang

Generative Retrieval

Ask PDF

Abstract

Generative Retrieval

Explore similar work

It Takes Two to Match: Co-Evolving Generative Retriever with Reinforcement Learning

Efficient Generative Retrieval for E-commerce Search with Semantic Cluster IDs and Expert-Guided RL

Evolving Semantic Indexes for Generative Retrieval