Personalized content recommendation on large marketplace websites must reconcile heterogeneous candidate content (e.g., customer stories, product pages, industry pages, and learning content) with sparse, fast-changing user behavior, placing increasing demands on the retrieval stage to represent user interest well, not just the downstream ranking stage. We present LSER (LLM Session Embedding Retrieval), a production content recommendation system for autodesk.com that derives session- and history-level embeddings from a pretrained large language model (LLM) and adds them as auxiliary user-side features in the retrieval-stage user tower alongside existing learned sequence encoders. To keep user-side embeddings fresh under real-time behavior, we build a nearline serving path that asynchronously refreshes sequential and LLM-derived user vectors for our main content recommendation server. We validate the incremental value of the LLM features through an ablation study across our full multi-content-type catalog. Our results indicate that integrating LLM-derived embeddings as additional retrieval signals, together with production-grade nearline serving, is technically feasible and can improve model performances.
Figures & tables
Figure 1. Overview of the LSER architecture. A nearline serving pipeline (top) asynchronously refreshes user-side embedding components from recent activity and writes them to an embedding store. At serving time, a two-tower retrieval model (bottom) assembles a user embedding eu in the User Tower fuser and an item embedding ei in the Item Tower fitem ; retrieval ranks candidates by cosine similarity (Section 2.1 ) for Top- K results. System diagram with a nearline path that refreshes user embeddings from session and history activity into a store, and a two-tower retrieval path that fuses stored vectors with request-time features to rank items by similarity.
Figure 2. Nearline computation pipeline. New activity triggers asynchronous SageMaker endpoints that refresh nearline user-side embeddings (including gseq and LLM outputs) in an embedding store; a separate real-time endpoint reads those vectors, fuses them in fuser , and scores Top- K candidates via ANN without heavy encoders on the critical path. Flowchart of nearline user-embedding jobs triggered by new activity, writing vectors to a store, and a real-time endpoint that reads those vectors and returns top-ranked recommendations.
Metric
Popular products
Customer success
Δ HR@1
+0.2%
+6.0%
Δ HR@5
+2.2%
+2.4%
Δ HR@10
+9.6%
—
Δ MRR
+0.021
+0.030
Table 1. Offline ablation: adding LLM-derived 64-d session/history vectors as auxiliary user features on the held-out test split. Absolute changes (treatment − baseline): Δ HR@ K in percentage points; Δ MRR on [0,1] . Other surfaces and metrics are not tabulated because they remained essentially flat relative to baseline.
Metric
Relative Lift
Engagement
Homepage Prod&Ind. carousel CTR
+10.3%
AutoCAD customer story CTR
+36.1%
Conversion
Product trial start rate
+17.1%
Product trial install rate
+10.5%
Table 2. Online A/B test under a 50/50 traffic split. Relative lifts of treatment over the rule-based control. Engagement rows are click-through rate on recommended tiles; conversion rows are product-level trial and order rates. All metrics were statistically significant ( p<0.05 ).
Large language models (LLMs) have demonstrated prominent reasoning capabilities in recommendation tasks by transforming them into text-generation tasks. However, existing approaches either disregard or ineffectively model the user-item high-order interactions. To this end, this paper presents an enhanced LLM-based recommender (ELMRec). We enhance whole-word embeddings to substantially enhance LLMs' interpretation of graph-constructed interactions for recommendations, without requiring graph pre-training. This finding may inspire endeavors to incorporate rich knowledge graphs into LLM-based recommenders via whole-word embedding. We also found that LLMs often recommend items based on users' earlier interactions rather than recent ones, and present a reranking solution. Our ELMRec outperforms state-of-the-art (SOTA) methods in both direct and sequential recommendations.
Xinfeng Wang, Jin Cui, Fumiyo Fukumoto +1
Graduate School of Engineering University of Yamanashi, Kofu, Japan · Interdisciplinary Graduate School University of Yamanashi, Kofu, Japan
Adapting large language models (LLMs) for personalized recommendation requires aligning their general-purpose capabilities with user-specific preferences while effectively leveraging both behavioral and semantic signals. Existing approaches typically integrate these signals at either the input level (e.g., injecting behavioral embeddings into the token space) or the output level (e.g., contrastive alignment of separate encoders), suffering from distribution gaps or lack of end-to-end task supervision. In this work, we introduce L2Rec, which unifies behavioral and semantic understanding at the parameter level of LLMs. Our key insight is that the same set of Transformer parameters can serve as a shared medium for both views: by applying view-specific, personalized low-rank perturbations via a Dual-view Personalized Mixture-of-Experts (DPMoE) mechanism, L2Rec enables a single LLM backbone to produce complementary behavioral and semantic adaptations for each user with minimal representation-level misalignment. An adaptive cross-view fusion module further integrates the dual-view outputs into a unified user preference. Experiments on four datasets show that L2Rec consistently outperforms state-of-the-art baselines, and online A/B testing on a large-scale industrial platform validates significant improvements in key engagement metrics.
Large language models (LLMs) are increasingly used as backbone architectures for recommender systems because of their strong sequence modeling and representation learning capabilities. However, most LLM-based recommenders operate primarily on discrete textual tokens, whereas practical recommendation pipelines also rely on continuous numerical features and dense embedding features produced by upstream feature engineering or pretrained encoders. This mismatch limits the ability of LLM-based models to exploit fine-grained non-textual signals. We propose a soft-token fusion framework that maps numerical and embedding features into the LLM embedding space, allowing heterogeneous recommendation signals to be consumed through the standard token interface. We instantiate the framework in a shared-parameter LLM-based two-tower retrieval model and introduce an interaction-based fusion module that refines embedding and numerical soft tokens before they are inserted into the final LLM input. Experiments on three Amazon recommendation benchmarks show that soft-token fusion improves retrieval performance over LLM-based baselines, and that interaction-based fusion is more effective than direct concatenation of heterogeneous soft tokens.