Local-commerce and food-delivery marketplaces run large ads businesses: across hundreds of thousands of stores, merchants pay to be surfaced on browse and search surfaces, and a ranker decides which ads each consumer sees. Ranking well demands strong per-consumer personalization, yet a consumer's raw interaction history is sparse and high-cardinality. Our production ranker had personalized only through hand-engineered signals: aggregate engagement counts, categorical embeddings, and order-agnostic pooled history, with no learned per-consumer sequence representation. We learn one by training a GPT-style generative model of the consumer's engagement sequence to predict their next few engagements, and read its last-token hidden state as the consumer embedding. That single frozen vector becomes one shared personalization input for every ad surface: a daily job refreshes it, and each ranker consumes it as an ordinary dense feature. Offline it is the largest single-feature gain in our ranking-AUC ladder, a relative Eval-AUC gain of +0.59% on a browse-driven feed and +0.57% on a query-driven search surface. Online, it shipped as one component of bundled ranker retrains on both surfaces: Discovery (revenue +0.79%, conversion rate +1.17%) and Search (revenue +0.94%, conversion rate +1.50%). Three design choices adapt the generative objective to a repurchase-heavy marketplace, and are the core of our contribution: an exploration-weighted loss that up-weights businesses new to the consumer over ones already engaged; a two-stage curriculum: a click pretrain that keeps the representation broad and exploratory, then a conversion fine-tune that sharpens purchase intent; and a designed negative set that pairs a fixed popular-business pool and businesses shown but not ordered with random and in-batch negatives.
Figures & tables
Figure 1. The generative user representation and how it is reused: a decoder-only GPT trained on each consumer’s store-engagement sequence, whose last-token hidden state is exported as a frozen 256-d feature and reused by the DCNv2 ranker across both ad surfaces. A diagram showing a consumer store-engagement sequence feeding into a decoder-only GPT model, whose last-token hidden state is exported as a 256-dimensional embedding and fed frozen into a DCNv2 ranker that serves both the Discovery and Search ad surfaces.
Figure 2. Model architecture. Per-position store, business, and context embeddings are concatenated (532-d), projected to a 768-d trunk, run through three pre-norm causal decoder blocks, and projected to 256-d. We export the last-token hidden state as the user embedding and the softmax layer’s rows as business embeddings (§ 3.4 ). A diagram of the model architecture showing store, business, and context input embeddings concatenated and projected to a 768-dimensional trunk, passed through three pre-norm causal decoder blocks, then normalized and projected to 256 dimensions to produce per-position hidden states, from which the last-token user embedding and the softmax-layer business embeddings are both extracted.
Design choice
R@10
R@30
MRR
New R@10
+ Hard negatives
+14.3%
+9.0%
+8.2%
–
Exploration weighting 3:1
− 7.0%
− 3.5%
− 5.7%
+18.4%
Exploration weighting 9:1
− 20.4%
− 11.1%
− 19.8%
+29.1%
Distinct sequence
− 67.5%
− 46.5%
− 75.5%
–
Time-decay attn (replace pos.)
− 5.0%
− 3.7%
− 5.4%
–
Time-decay attn ( + pos.)
− 3.7%
− 3.1%
− 3.4%
–
Table 1. Recall ablations (% change). New R@10: recall on previously unseen businesses (“–”: not recorded). Time-decay rows use the full-softmax model; full-softmax rows are vs. sampled softmax.
Component added
% impr.
Store emb. + mean-pooled last- N converted-store emb.
+0.33%
+ user embedding, uniform weighting
≈ 0.00%
+ user embedding, 9:1 weighting, as ϕ(u,c) (Eq. 2 )
+0.59%
+ DIN attention over the raw history
+0.03%
Table 2. Discovery ranking-AUC ladder: each row’s incremental relative Eval-AUC change over the rows above; row 1 is vs. the retrained production DCNv2. The uniform and 9:1 rows are alternatives, each measured on its own rather than stacked. Search: +0.57% for the user embedding. Single runs.
Metric
Discovery
Search
Revenue
+0.79% ∗∗∗
+0.94% ∗∗∗
CVR
+1.17% ∗∗∗
+1.50% ∗∗∗
CTR
− 0.10% (n.s.)
+1.38% ∗∗∗
eCPM
+0.66% ∗∗∗
+1.56% ∗∗∗
ROAS
+0.13% (n.s.)
− 0.01% (n.s.)
Table 3. Online A/B lift by surface (% change). ∗∗∗p<0.001 , ∗∗p<0.01 , ∗p<0.05 ; n.s. = not significant.
Recommendation systems power engagement and monetization across feeds, ads, and short-video platforms, but translating the latest advances in Large Language Models into Recommendation Systems (RecSys) gains remains rare, particularly in advertising and production-scale real-world industry setups. Prior real-world LLM successes typically fall into three buckets: (a) generative retrieval that directly predicts the next items for candidate generation, (b) late-stage re-ranking that uses LLMs, and (c) auxiliary signal enrichment with LLMs. We introduce a complementary paradigm for ads: a fine-tuned open-source LLM used not as a ranker, but as an ads-specific ancillary predictor, forecasting likely advertisers from user profiles and histories. This LLM-driven advertiser prediction augments conventional candidate generation and provides informative priors to downstream ranking. Developed in a large-scale production advertising system, our approach produces substantial offline improvements and measurable online business impact, demonstrating that LLM world knowledge and predictive capacity can be efficiently harnessed. Beyond validating LLMs for ads applications, our results show that targeted ancillary predictions can unlock end-to-end gains across both retrieval and late-stage ranking, offering a practical path to LLM-enhanced recommendation at scale.
Personalized storefronts in large e-commerce marketplaces are often assembled from many independent components: static themes per page section ("placement"), retrieval systems to fetch eligible products per placement, and pointwise rankers to order content. While effective in optimizing for aggregate preferences, this paradigm is rigid and can limit personalization and semantic cohesion across the page. This makes it poorly suited to support dynamic objectives and merchandising requirements over time. To address this, we introduce a cascaded merchandising framework that decomposes storefront construction into two generative tasks: (i) placement-level theme generation and (ii) constrained keyword generation per placement to power product retrieval. Teacher-student fine-tuning is leveraged to improve scalability of this framework under production latency and cost constraints. Fine-tuned model ablations are shown to approach closed-weight LLM performance. We further contribute frameworks for AI-driven content evaluation and quality filtering, enabling safe and automated deployment of dynamic content at scale. Generative output is fused with traditional ranking models to preserve hybrid infrastructure. In online experiments, this framework yields an estimated +2.7% lift in cart adds per page view over a strong baseline.
Generative AI is transforming how people access information, challenging traditional advertising mechanisms built around predefined slots. Towards generation-native advertising, we propose the Latent Advertiser Mixture Auction (LAMA), a token-level advertising mechanism that embeds advertiser influence directly into the generation process. Advertisers report local continuation values that induce advertiser-specific next-token policies, from which the platform decodes through a latent mixture while updating an allocation posterior. We show that LAMA satisfies Markov DSIC and IR, and achieves near-optimal KL-regularized welfare. We further develop a learning-based implementation that reconstructs the required reports online from learned local advantages and root values. Proof-of-concept experiments on real-world commercial-search query splits show that LAMA improves platform welfare and revenue while maintaining user-facing response quality, providing initial evidence for the feasibility of generation-native advertising.
Hanbing Liu, Bowei Zhang, Changyuan Yu +2
Renmin University of China Beijing, China · Beijing, China · Stanford University Stanford, CA, USA