PACERec: Progressive Adaptation of LLM Item Embeddings with Collaborative Evidence for Recommendation

Junzhao Huang; Xi Wu; Zihan Wang; Zijing Wang; Ming Wang; Kang Liu; Xiaocui Yang; Shi Feng

cs.IRNov 2, 2026

PACERec: Progressive Adaptation of LLM Item Embeddings with Collaborative Evidence for Recommendation

Authors: Junzhao Huang, Xi Wu, Zihan Wang, Zijing Wang, Ming Wang, Kang Liu, Xiaocui Yang, Shi Feng,

Organizations: School of Computer Science and Engineering, Northeastern University, Shenyang, China · Ant Group, China

Abstract

Large language model (LLM) embeddings have been increasingly adopted in sequential recommendation to encode static item attributes and alleviate the limitations of item ID embeddings under sparse interactions. However, these embeddings typically capture only static content similarity while neglecting sequential transitions, and since their adaptation quality relies on sparse per-item updates or global geometric calibration, existing methods still struggle to explicitly transfer preceding interaction contexts from frequent to long-tail items under unevenly distributed behavioral evidence, limiting recommendation effectiveness across the catalog. To address this challenge, we propose PACERec, an ID-free progressive adaptation framework that incorporates preceding interaction contexts into semantic representation adaptation by learning a shared velocity field from items with sufficient interactions via frequency-weighted flow matching. In this way, PACERec progressively guides static LLM embeddings toward collaborative behavioral targets while preserving original item semantics, overcoming sparse per-item updates to effectively generalize sequential transition patterns to long-tail items for both user modeling and candidate scoring. Extensive experiments on three real-world datasets across three sequential backbones demonstrate the superiority of our method over competitive baselines in both overall and long-tail recommendation.

Figures & tables

Figure 1. (a) Mean weighted training predecessor coverage on Yelp, Food, and Beauty. Solid curves use the $K$ nearest neighbors in the frozen LLM embedding space. Dashed curves use each item’s $K$ most frequent training predecessors. (b) LLM initialization and ID–LLM fusion versus PACERec’s behavior-guided semantic item adaptation and intent-residual user modeling. Two panels: (a) line chart of mean weighted predecessor coverage on Yelp, Food, and Beauty; (b) architecture comparison between LLM initialization, ID-LLM fusion, and PACERec.

Figure 2. Overview of PACERec at inference. The adaptation flow transforms each semantic source $x_{i}$ into an adapted representation $r_{i}$ for preference scoring, which is fused with semantic similarity for top- $K$ ranking. Architecture diagram of PACERec at inference showing Semantic Encoding, Progressive Adaptation Flow, Preference Scoring with Sequence Backbone and Intent Residual, Semantic Similarity Scoring, and Fusion and Ranking.

Figure 3. Training of the adaptation flow: (a) offline construction of fixed behavioral targets

p_{i}

before training, and (b) flow matching supervision for items with

c_{i}\geq\kappa

during training. Two-part diagram showing (a) offline construction of fixed behavioral targets from preceding interaction contexts and (b) online confidence-weighted flow matching supervision for items meeting the evidence threshold.

		Yelp				Food				Beauty
Backbone	Method	Overall		Tail		Overall		Tail		Overall		Tail
Backbone	Method	H@10	N@10	H@10	N@10	H@10	N@10	H@10	N@10	H@10	N@10	H@10	N@10
SASRec	Base	0.0280	0.0134	0.0056	0.0026	0.0335	0.0170	0.0093	0.0053	0.0339	0.0177	0.0146	0.0085
	LLM-ESR	0.0513	0.0261	0.0027	0.0012	0.0322	0.0159	0.0030	0.0013	0.0226	0.0111	0.0005	0.0002
	AlphaFuse	0.0561	0.0283	0.0087	0.0038	0.0717	0.0382	0.0361	0.0193	0.0566	0.0295	0.0305	0.0159
	EUMR	0.0359	0.0181	0.0027	0.0014	0.0345	0.0178	0.0080	0.0046	0.0338	0.0180	0.0147	0.0089

Table 1. Overall and tail-item recommendation performance on three datasets. H@10 and N@10 denote Hit@10 and NDCG@10, respectively. Base denotes the sequence backbone trained solely on interaction data, and Tail evaluates test instances whose target items fall outside the most frequent 20% of training items. The best result is highlighted in bold , and the best baseline is underlined . Impr. denotes the relative improvement of PACERec over the best baseline.

Method		SASRec				FMLP-Rec				LRURec
		Overall		Tail		Overall		Tail		Overall		Tail
		H@10	N@10	H@10	N@10	H@10	N@10	H@10	N@10	H@10	N@10	H@10	N@10
Structure Variant	w/o-SItem	0.0384	0.0215	0.0109	0.0063	0.0450	0.0267	0.0195	0.0121	0.0394	0.0234	0.0168	0.0107
	w/o-Intent	0.0720	0.0405	0.0581	0.0332	0.0742	0.0411	0.0621	0.0351	0.0659	0.0363	0.0537	0.0302
	w/o-Flow	0.0726	0.0407	0.0683	0.0392	0.0757	0.0430	0.0719	0.0413	0.0714	0.0402	0.0660	0.0380
Loss Variant	w/o- $\mathcal{L}_{\mathrm{rank}}$	0.0113	0.0058	0.0103	0.0053	0.0703	0.0397	0.0745	0.0441	0.0661	0.0396	0.0677	0.0394

Table 2. Ablation results on Beauty. H@10 and N@10 denote Hit@10 and NDCG@10, respectively.

Qwen3-4B
Backbone	Method	Overall		Tail
Backbone	Method	H@10	N@10	H@10	N@10
SASRec	FAERec	0.0510	0.0296	0.0431	0.0258
SASRec	PACERec	0.0668	0.0374	0.0514	0.0297
FMLP-Rec	FAERec	0.0481	0.0282	0.0449	0.0276
FMLP-Rec	PACERec	0.0707	0.0398	0.0584	0.0336

Table 3. Performance comparison with different LLMs and sequence models on the Beauty dataset.

Figure 4. Effect of

\lambda

on NDCG@10 for Yelp, Food, and Beauty. The top and bottom rows show overall and tail performance, respectively. Dashed curves represent fixed

\lambda

values, while horizontal lines indicate performance with learnable

\lambda

. Six subplots comparing fixed and learnable lambda on Yelp, Food, and Beauty. The top row shows overall NDCG@10, and the bottom row shows tail NDCG@10.

Figure 5. NDCG@10 across item frequency groups on Yelp, Food, and Beauty using SASRec. Grouped bar charts comparing Base, ACE, FAERec, and PACERec across six item frequency groups on three datasets.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	# Users	# Items	Sparsity	Avg. length
Beauty	52,204	57,289	99.99%	7.56
Food	32,126	39,264	99.98%	8.57
Yelp	15,720	11,383	99.89%	12.23

Appendix

Table 4. Statistics of the three datasets.

Component	Setting
Architecture	$d=256$ ; hidden widths: 512/384; 2 residual MLP velocity blocks; sinusoidal time dimension: 16; dropout: 0.1
Flow and targets	$S=2$ , $\rho=2$ ; $L=5$ , target decay: 0.8
Dataset-specific	$(\kappa,c_{\mathrm{med}})=(5,47)$ for Yelp; $(2,13)$ for Beauty and Food
Weighting	$c_{0}=10$ ; positive weights: $[0.5,3]$ ; $\alpha$ : initial 0.25, maximum 1.5
Negatives	$M=512$ shared negatives; $\beta=0.75$
History	Intent/semantic windows: 50/10; respective decays: 0.9/0.8

Appendix

Table 5. PACERec hyperparameters, shared unless specified.

Backbone	Overall		Tail		Overall		Tail		Overall		Tail
	Yelp				Food				Beauty
	H@10	N@10	H@10	N@10	H@10	N@10	H@10	N@10	H@10	N@10	H@10	N@10
NARM	0.0456	0.0231	0.0012	0.0005	0.0290	0.0153	0.0035	0.0018	0.0255	0.0146	0.0075	0.0040
w/ PACERec	0.0618	0.0318	0.0249	0.0130	0.0892	0.0488	0.0759	0.0427	0.0729	0.0405	0.0702	0.0403
LightSANs	0.0474	0.0243	0.0027	0.0014	0.0458	0.0259	0.0097	0.0058	0.0440	0.0271	0.0142	0.0094
w/ PACERec	0.0601	0.0312	0.0266	0.0138	0.0918	0.0510	0.0757	0.0439	0.0762	0.0432	0.0731	0.0420

Appendix

Table 6. Performance on additional sequential recommendation backbones. H@10 and N@10 denote Hit@10 and NDCG@10, respectively.

		Yelp				Food				Beauty
		Overall		Tail		Overall		Tail		Overall		Tail
Backbone	Method	H@10	N@10	H@10	N@10	H@10	N@10	H@10	N@10	H@10	N@10	H@10	N@10
SASRec	CITIES	0.0298	0.0140	0.0000	0.0000	0.0291	0.0153	0.0000	0.0000	0.0264	0.0146	0.0012	0.0005
	MELT	0.0386	0.0191	0.0000	0.0000	0.0450	0.0247	0.0001	0.0001	0.0401	0.0236	0.0011	0.0006
	TailRec	0.0316	0.0148	0.0021	0.0007	0.0228	0.0120	0.0027	0.0015	0.0250	0.0143	0.0058	0.0033
	TADA	0.0351	0.0178	0.0006	0.0002	0.0329	0.0186	0.0084	0.0055	0.0303	0.0178	0.0120	0.0074

Appendix

Table 7. Performance comparison with additional long-tail recommendation methods. H@10 and N@10 denote Hit@10 and NDCG@10, respectively. The best and second-best results are shown in bold and underlined, respectively.

Panel of Figure 6 — Figure 6. Item representation distributions and held-out predecessor coverage on Beauty. (a)–(c) PCA projections of SASRec, ACE, and PACERec, displaying the same 3,000 items. Red, blue, and gray denote the most frequent 20%, least frequent 20%, and remaining items according to training interactions, respectively. (d) Weighted test-predecessor coverage of cosine Top- $K$ neighbors in the original representation spaces, averaged equally over 23,902 test target items. Three scatter plots show item representations grouped by training frequency. A line chart compares predecessor coverage at K values from 20 to 100, with PACERec achieving higher coverage than ACE and SASRec. Four subplots on Beauty: (a-c) PCA scatter plots of centered sequence-backbone states and intent residuals for SASRec, FMLP-Rec, and LRURec; (d) boxplots of cross-branch cosine similarity for matched versus shuffled users.

Sequential Recommendation

Sequential recommendation predicts what a user will choose next from ordered interactions, adapting to changing interests and context.

See all 147 papers

Fusion and Alignment Enhancement with Large Language Models for Tail-item Sequential Recommendation
Apr 4, 2026Zhifu Wei, Yizhou Dang, Guibing Guo +2Recommender Systems Sequential Recommendation
Northeastern University, Shenyang, China · Tianjin University, Tianjin, China · Singapore University of Technology and Design, Singapore, Singapore
REPREC: Representation Driven Parameter-Efficient Recommendation System
Jul 24, 2026Harshini Kavuru, Dwipam Katariya, Giri Iyengar +3Sequential Recommendation Large Language Model-Based Recommendation
The Ohio State University, USA · Capital One, AI Foundations, USA
Enhancing High-order Interaction Awareness in LLM-based Recommender Model
Sep 30, 2024Xinfeng Wang, Jin Cui, Fumiyo Fukumoto +1Sequential Recommendation Feature Interaction Modeling
Graduate School of Engineering University of Yamanashi, Kofu, Japan · Interdisciplinary Graduate School University of Yamanashi, Kofu, Japan

Apr 4, 2026J/K move · Enter open · S save

Fusion and Alignment Enhancement with Large Language Models for Tail-item Sequential Recommendation

Zhifu Wei, Yizhou Dang, Guibing Guo, Chuang Zhao+1

Northeastern University, Shenyang, China · Tianjin University, Tianjin, China · Singapore University of Technology and Design, Singapore, Singapore

Recommender Systems Sequential Recommendation Large Language Model-Based Recommendation Latent Space Alignment

Open paper AskPDF arXiv

Sequential Recommendation (SR) learns user preferences from their historical interaction sequences and provides personalized suggestions. In real-world scenarios, most items exhibit sparse interactions, known as the tail-item problem. This issue limits the model's ability to accurately capture item transition patterns. To tackle this, large language models (LLMs) offer a promising solution by capturing semantic relationships between items. Despite previous efforts to leverage LLM-derived embeddings for enriching tail items, they still face the following limitations: 1) They struggle to effectively fuse collaborative signals with semantic knowledge, leading to suboptimal item embedding quality. 2) Existing methods overlook the structural inconsistency between the ID and LLM embedding spaces, causing conflicting signals that degrade recommendation accuracy. In this work, we propose a Fusion and Alignment Enhancement framework with LLMs for Tail-item Sequential Recommendation (FAERec), which improves item representations by generating coherently-fused and structurally consistent embeddings. For the information fusion challenge, we design an adaptive gating mechanism that dynamically fuses ID and LLM embeddings. Then, we propose a dual-level alignment approach to mitigate structural inconsistency. The item-level alignment establishes correspondences between ID and LLM embeddings of the same item through contrastive learning, while the feature-level alignment constrains the correlation patterns between corresponding dimensions across the two embedding spaces. Furthermore, the weights of the two alignments are adjusted by a curriculum learning scheduler to avoid premature optimization of the complex feature-level objective. Extensive experiments across three widely used datasets with multiple representative SR backbones demonstrate the effectiveness and generalizability of our framework.

Abstract

Sequential Recommendation

Explore similar work

Fusion and Alignment Enhancement with Large Language Models for Tail-item Sequential Recommendation

REPREC: Representation Driven Parameter-Efficient Recommendation System

Enhancing High-order Interaction Awareness in LLM-based Recommender Model