cs.DCNov 9, 2026

RecoGEM: A Serving System for Production-Scale Generative Recommendation

Authors: Xiangrui Yang, Yunfeng Zhao, Jiawei Yang, Fangyu Zhang, Enrui Zhao, Liang Zeng, Cheng Peng, Minhui Xie, 

Organizations: Kuaishou Technology, Beijing, China · Peking University, Beijing, China · Renmin University of China, Beijing, China

Abstract

Generative recommendation (GR) is moving industrial recommender systems (RecSys) from scoring candidate items to autoregressively generating item identifiers. This shift promises higher recommendation quality, but makes serving substantially harder: each request executes a short sequence of dependent decoding steps that explore a large item space and select promising candidates on the latency-critical path. GR not only creates structural mismatches with conventional RecSys pipelines, but also challenges the architectural and optimization assumptions of LLM serving stacks, making direct adoption of existing systems difficult. We present RecoGEM, our production GR serving system. RecoGEM uses a runtime--engine decoupling architecture to move GR-specific serving responsibilities, such as candidate-state transitions and live artifact updates, to a dedicated runtime. Its execution engine exploits GR's short token sequences, large candidate populations, and predictable step shapes to optimize self-attention, candidate selection, and execution planning. RecoGEM has been deployed for over nine months across our production lines, serving applications with hundreds of millions of users and supporting more than ten GR model families. On our OneModel production workloads, RecoGEM reaches over 1,000 QPS per serving instance with P99 latency below 35 ms and a 2.88×\times throughput improvement over representative baselines.

Figures & tables

Explore similar work

CardsList
  1. xGR: Efficient Generative Recommendation Serving at Scale

    Dec 12, 2025Qingxiao Sun, Tongxuan Liu, Shen Zhang +13LLM ServingLLM Inference Acceleration

  2. GenRec: A Preference-Oriented Generative Framework for Large-Scale Recommendation

    Apr 16, 2026Yanyan Zou, Junbo Qi, Lunsong Huang +7RL for RecommendationGenerative Retrieval