Fusion and Alignment Enhancement with Large Language Models for Tail-item Sequential Recommendation

Zhifu Wei; Yizhou Dang; Guibing Guo; Chuang Zhao; Zhu Sun

cs.IRApr 4, 2026

Fusion and Alignment Enhancement with Large Language Models for Tail-item Sequential Recommendation

Authors: Zhifu Wei, Yizhou Dang, Guibing Guo, Chuang Zhao, Zhu Sun

Organizations: Northeastern University, Shenyang, China · Tianjin University, Tianjin, China · Singapore University of Technology and Design, Singapore, Singapore

Abstract

Sequential Recommendation (SR) learns user preferences from their historical interaction sequences and provides personalized suggestions. In real-world scenarios, most items exhibit sparse interactions, known as the tail-item problem. This issue limits the model's ability to accurately capture item transition patterns. To tackle this, large language models (LLMs) offer a promising solution by capturing semantic relationships between items. Despite previous efforts to leverage LLM-derived embeddings for enriching tail items, they still face the following limitations: 1) They struggle to effectively fuse collaborative signals with semantic knowledge, leading to suboptimal item embedding quality. 2) Existing methods overlook the structural inconsistency between the ID and LLM embedding spaces, causing conflicting signals that degrade recommendation accuracy. In this work, we propose a Fusion and Alignment Enhancement framework with LLMs for Tail-item Sequential Recommendation (FAERec), which improves item representations by generating coherently-fused and structurally consistent embeddings. For the information fusion challenge, we design an adaptive gating mechanism that dynamically fuses ID and LLM embeddings. Then, we propose a dual-level alignment approach to mitigate structural inconsistency. The item-level alignment establishes correspondences between ID and LLM embeddings of the same item through contrastive learning, while the feature-level alignment constrains the correlation patterns between corresponding dimensions across the two embedding spaces. Furthermore, the weights of the two alignments are adjusted by a curriculum learning scheduler to avoid premature optimization of the complex feature-level objective. Extensive experiments across three widely used datasets with multiple representative SR backbones demonstrate the effectiveness and generalizability of our framework.

Figures & tables

Figure 1. Visualization of the long-tail item problem (a) and comparison between the existing LLM-based sequential recommendation methods and our FAERec (b, c).

Figure 2. The overall framework of our proposed FAERec.

Dataset	# Users	# Items	Sparsity	Avg.length
Beauty	52,204	57,289	99.99%	7.56
Grocery	32,126	39,264	99.98%	8.57
Yelp	15,720	11,383	99.89%	12.23

Table 1. The statistics of three datasets.

Backbone		Beauty				Grocery				Yelp
		Overall		Tail		Overall		Tail		Overall		Tail
		H@10	N@10	H@10	N@10	H@10	N@10	H@10	N@10	H@10	N@10	H@10	N@10
SASRec	Base	0.0331	0.0180	0.0113	0.0072	0.0332	0.0168	0.0086	0.0055	0.0240	0.0113	0.0060	0.0036
	CITIES	0.0305	0.0159	0.0106	0.0064	0.0286	0.0144	0.0074	0.0045	0.0240	0.0111	0.0049	0.0024
	LOAM	0.0300	0.0158	0.0115	0.0076	0.0285	0.0148	0.0089	0.0059	0.0122	0.0055	0.0060	0.0031
	MELT	0.0178	0.0088	0.0001	0.0000	0.0208	0.0101	0.0004	0.0003	0.0382	0.0188	0.0004	0.0002

Table 2. Performance comparison of different methods on Backbone models. The best performance is bolded. The ‘Impr.’ represents the relative improvement of FAERec over the best baseline method.

Variant	SASRec				FMLP-Rec				LRURec
	Overall		Tail		Overall		Tail		Overall		Tail
	H@10	N@10	H@10	N@10	H@10	N@10	H@10	N@10	H@10	N@10	H@10	N@10
FAERec	0.0642	0.0372	0.0633	0.0368	0.0622	0.0361	0.0645	0.0379	0.0644	0.0374	0.0617	0.0357
w/o AGF	0.0534	0.0267	0.0359	0.0182	0.0624	0.0355	0.0516	0.0291	0.0561	0.0289	0.0369	0.0185
w/o ILA	0.0568	0.0318	0.0473	0.0260	0.0585	0.0343	0.0567	0.0331	0.0519	0.0289	0.0353	0.0192
w/o FLA	0.0626	0.0365	0.0603	0.0355	0.0597	0.0353	0.0630	0.0376	0.0632	0.0369	0.0595	0.0346

Table 3. The ablation study results on the Beauty dataset with three sequential recommendation backbones.

Figure 3. Sensitivity analysis of hyperparameter

\lambda

and

\alpha

on three dataset with SASRec as the backbone.

Figure 4. The group analysis results on three dataset with SASRec as the backbone.

Backbone	Beauty				Grocery				Yelp
	Overall		Tail		Overall		Tail		Overall		Tail
	H@10	N@10	H@10	N@10	H@10	N@10	H@10	N@10	H@10	N@10	H@10	N@10
NARM	0.0197	0.0097	0.0017	0.0010	0.0219	0.0114	0.0016	0.0008	0.0387	0.0195	0.0015	0.0009
w/FAERec	0.0429	0.0226	0.0331	0.0176	0.0540	0.0286	0.0400	0.0213	0.0456	0.0223	0.0051	0.0021
LightSANs	0.0252	0.0143	0.0059	0.0046	0.0291	0.0150	0.0052	0.0038	0.0167	0.0080	0.0024	0.0016
w/FAERec	0.0597	0.0353	0.0614	0.0364	0.0714	0.0406	0.0727	0.0425	0.0254	0.0125	0.0115	0.0057

Table 4. Results on more sequential recommendation backbones.

Figure 5. Visualization of ID and LLM embedding spaces on the Beauty dataset with SASRec as the backbone.

Method	SASRec		FMLP-Rec		LRURec
Method	Cov@10	TCov@10	Cov@10	TCov@10	Cov@10	TCov@10
Base	0.6972	0.6266	0.5273	0.4267	0.6679	0.5920
LOAM	0.6419	0.5634	0.5092	0.4057	0.6768	0.6036
RLMRec-Con	0.5019	0.4141	0.7010	0.6442	0.5477	0.4554
LLM-ESR	0.3513	0.2251	0.5176	0.4215	0.4075	0.2831
FAERec	0.8538	0.8249	0.8571	0.8298	0.8612	0.8315

Table 5. Diversity comparison of different methods on the Beauty dataset with three SR backbones.

Recommender Systems

Recommender systems predict and rank items for people using interaction histories, content, context, and feedback.

See all 502 papers

Jul 13, 2026

SISA-Rec: A Semantically Integrated Sequential Recommender with Contrastive Alignment

Recommendation systems help users recommend relevant items from a large collection of choices. Present work on transformer-based sequential recommendation learns user preferences from interaction logs, but it mostly focuses on item identifiers and doesn't fully use the semantic meaning of items. This limitation becomes a major challenge in sparse and cold-start scenarios where historical interaction data is limited. To solve this problem, we introduce SISA-Rec (Semantically Integrated Sequential Recommendation), a transformer-based framework that embeds semantic context directly into sequential modeling. Our approach fuses item ID embeddings with BERT-based text embeddings via a gated fusion module, injects semantic similarity into the self-attention mechanism, and leverages an attention-based aggregation module to construct comprehensive user representations. Finally, a joint learning objective which combines Bayesian Personalized Ranking (BPR) and contrastive alignment loss, aligns the underlying behavioral and semantic spaces. Experiments were conducted on the two highly sparse Amazon Beauty and Amazon Toys & Games datasets, both having 99.93% sparsity. The results show that SISA-Rec outperforms state-of-the-art baseline models across all evaluation metrics. Compared with the BERT4Rec \cite{petrov2022systematic}, SISA-Rec improves HR@10 by 16.6% and NDCG@10 by 10.3% on Amazon Beauty, and HR@10 by 23.1% and NDCG@10 by 17.9% on Amazon Toys & Games. Cold-start analysis further shows that the proposed model achieves the largest improvements for users with limited interaction historical records. This showcases the value of semantic information when user behavior data is scarce. Overall, the results demonstrate that integrating semantic information into the attention mechanism leads to more accurate and reliable recommendations.

Soohan Abbasi, Shahid Munir Shah, Rafia Shaikh +1

Department of Computer Science, DHA Suffa University, Karachi, Sindh, Pakistan · Department of Computer Science, SZABIST, Pakistan · National University of Computer and Emerging Sciences (FAST), Karachi, Sindh, Pakistan +1

Sequential Recommendation Cold-Start Recommendation Recommender Systems

Ask PDF

Jun 9, 2026

Generative Archetype-Grounded Item Representations for Sequential Recommendation

Sequential recommendation aims to predict users' next interaction with items by analyzing their historical behavior. However, the limited quality of item representations remains a critical bottleneck. While pre-trained large language models (LLMs) can provide rich semantic representations, existing approaches only rely on static encoding of fixed attributes, overlooking the crucial role of target audiences in defining item identity. Moreover, the semantic space struggles to reflect actual user behavior, resulting in a significant gap between semantic representations and behavioral patterns. To address these limitations, we propose GenAIR, a general framework that empowers sequential recommendation with Generative Archetype-grounded Item Representations. Specifically, we first leverage an LLM to analyze item metadata and infer textual description of the Archetype, which represents the conceptual profile of the item's ideal target audience. We then extract the corresponding embeddings in a single forward pass. Further, to ground these generative archetypes in real-world behavior, we introduce a behavioral calibration objective, which explicitly incorporates behavioral signals from actual interactions. This objective adjusts the structure of the embedding space to reflect empirical patterns. GenAIR enables seamless integration with most existing models while maintaining high efficiency. Comprehensive experiments conducted on three real-world datasets demonstrate that GenAIR significantly improves the performance of various sequential recommendation models and consistently outperforms state-of-the-art baseline approaches. Implementation codes are available at https://github.com/AI-Santiago/GenAIR.

Yifan Li, Jiahong Liu, Xinni Zhang +5

The Chinese University of Hong Kong Hong Kong SAR, China · McGill University Montreal, Canada · Tongji University Shanghai, China

Sequential Recommendation Representation Learning Large Language Model-Based Recommendation User Preference Modeling

Ask PDF

Dec 11, 2025

The Best of the Two Worlds: Harmonizing Semantic and Hash IDs for Sequential Recommendation

Conventional Sequential Recommender Systems (SRS) typically assign unique hash IDs (HID) to construct item embeddings, which mainly capture collaborative signals from historical user-item interactions. However, such embeddings are vulnerable in long-tail scenarios where most items are rarely consumed. Recent methods that incorporate auxiliary information often face noisy collaborative sharing from co-occurrence signals or semantic homogeneity caused by flat dense embeddings. In contrast, Semantic IDs (SID), with their support for code sharing and multi-granular semantic modeling, offer a promising alternative. Nevertheless, SID-based methods are hindered by a collaborative overwhelming phenomenon: commonly adopted quantization mechanisms compromise the identifier uniqueness needed to model head items, resulting in a performance trade-off between head and tail items. To address this challenge, we propose H2Rec, a novel framework that harmonizes SID and HID. We design a dual-branch modeling architecture that simultaneously captures the multi-granular semantics of SID while preserving the unique collaborative identity provided by HID. Moreover, we introduce a dual-level alignment strategy to bridge the two representations, enabling effective knowledge transfer and robust preference modeling. Extensive offline experiments on three public benchmarks and online experiments on a large-scale commercial platform demonstrate that H2Rec achieves a better balance between head and tail recommendation quality and consistently outperforms existing baselines.

Ziwei Liu, Yejing Wang, Wanyu Wang +6

City University of Hong Kong Hong Kong, China · Tongji University Shanghai, China · Xi’an Jiaotong University Xi’an, China +3

Semantic ID Recommendation Sequential Recommendation Long-Tail Recommendation Recommender Systems Collaborative Filtering

Ask PDF

Abstract

Recommender Systems

Explore similar work

SISA-Rec: A Semantically Integrated Sequential Recommender with Contrastive Alignment

Generative Archetype-Grounded Item Representations for Sequential Recommendation

The Best of the Two Worlds: Harmonizing Semantic and Hash IDs for Sequential Recommendation