GOD: Enhancing Generalization via Deep Grafting for Sequential Recommendation

WooJoo Kim; JunYoung Kim; JaeHyung Lim; HwanJo Yu

cs.IRAug 17, 2026

GOD: Enhancing Generalization via Deep Grafting for Sequential Recommendation

Authors: WooJoo Kim, JunYoung Kim, JaeHyung Lim, HwanJo Yu

Organizations: Pohang University of Science and Technology Pohang, Republic of Korea

Abstract

Sequential recommenders often struggle with sparse and noisy histories, limiting generalization to unseen interactions. Knowledge distillation mitigates this by transferring dense supervision from a teacher to a student. However, most distillation methods run teacher and student independently, then match student outputs or representations to the teacher. Such supervision entangles student-component effects, blurring whether weak generalization stems from unreliable embeddings, overfitted encoding, or co-adaptation to sparse histories. In this paper, we propose Graft-Oriented Distillation (GOD), a component-level distillation framework for improved generalization through grafting. Grafting denotes replacing selected frozen-teacher components with trainable student counterparts to build hybrid source models. GOD uses these hybrid models to evaluate student embeddings with the teacher encoder and the student encoder with teacher embeddings, providing component-level feedback. At inference, GOD uses only the student, incurring no additional cost. Across three real-world datasets, GOD outperforms state-of-the-art baselines by up to 13.92%.

Figures & tables

Figure 1. Illustration of (a) conventional KD with an independent teacher and (b) KD with grafting. In (b), hybrid source models replace selected teacher components with student counterparts. Inference uses only the non-grafted student.

Panel of Figure 2 — Figure 2. Train/test CKA similarity of teacher-student sequence representations and NDCG@20 gaps with SASRec.

Panel of Figure 3 — Figure 3. Layer-wise CKA heatmaps of teacher-student intermediate representations with 4-layer SASRec.

Figure 4. Illustration of GOD. GOD distills representations of three source models— Non-Grafted Teacher , Embed-Grafted Teacher , and Encoder-Grafted Teacher —into Non-Grafted Student . Identical “shared” indices indicate parameter sharing.

Figure 5. Illustration of Grafted Encoding with teacher-student mutual attention.

Datasets	# Interactions	# Users	# Items	Avg.SeqLen	Sparsity
Amazon Beauty	198,502	22,363	12,101	8.9	99.93 %
Yelp	345,186	29,915	21,572	11.5	99.95 %
MovieLens 1M	999,611	6,040	3,416	165.5	95.16 %

Table 1. Statistics of the datasets.

Dataset	Backbone	Metric	Teacher	Student	RD	CD	DE	RRD	HTD	BD	AdaRec	MSKDIK	EMKD	GOD	$\boldsymbol{Improv.S}$	$\boldsymbol{Improv.B}$
Amazon Beauty	GRU4Rec	HR@10	0.0506	0.0384	0.0392	0.0403	0.0413	0.0426	0.0438	0.0464	0.0450	0.0454	0.0472	0.0508*	32.29 %	7.63 %
		HR@20	0.0788	0.0622	0.0626	0.0639	0.0652	0.0668	0.0682	0.0711	0.0692	0.0703	0.0722	0.0777*	24.92 %	7.62 %
		NDCG@10	0.0257	0.0187	0.0190	0.0196	0.0203	0.0210	0.0217	0.0232	0.0224	0.0227	0.0237	0.0257*	37.69 %	8.44 %
		NDCG@20	0.0322	0.0249	0.0262	0.0266	0.0270	0.0275	0.0281	0.0290	0.0285	0.0286	0.0294	0.0316*	26.91 %	7.48 %
	FMLPRec	HR@10	0.0667	0.0524	0.0540	0.0547	0.0555	0.0564	0.0573	0.0583	0.0583	0.0590	0.0596	0.0634*	20.99 %	6.38 %
		HR@20	0.0959	0.0813	0.0843	0.0849	0.0856	0.0864	0.0871	0.0883	0.0873	0.0886	0.0892	0.0951*	16.97 %	6.61 %

Table 2. Overall performance. Best and second-best results are marked in bold and underlined.

\boldsymbol{Improv.S}

and

\boldsymbol{Improv.B}

denote relative improvements over Student and the strongest baseline. Asterisk (*) indicates statistical significance at

p<0.05

by paired

t

-test over five independent runs against the strongest baseline.

Dataset	Metric	SASRec	RRD	HTD	BD	MSKDIK	EMKD	CL4SRec	MCLRec	CT4Rec	DuoRec	RCL	GOD	$Improv.$
Amazon Beauty	HR@10	0.0727	0.0718	0.0721	0.0749	0.0763	0.0785	0.0796	0.0806	0.0837	0.0856	0.0853	0.0923*	7.83 %
	HR@20	0.1030	0.1010	0.1018	0.1047	0.1062	0.1078	0.1084	0.1099	0.1139	0.1164	0.1172	0.1249*	6.57 %
	NDCG@10	0.0401	0.0396	0.0400	0.0426	0.0438	0.0454	0.0464	0.0474	0.0490	0.0506	0.0517	0.0559*	8.12 %
	NDCG@20	0.0472	0.0461	0.0469	0.0499	0.0513	0.0525	0.0544	0.0554	0.0568	0.0580	0.0596	0.0641*	7.55 %
Yelp	HR@10	0.0568	0.0559	0.0571	0.0574	0.0566	0.0584	0.0588	0.0591	0.0601	0.0606	0.0607	0.0640*	5.44 %
	HR@20	0.0917	0.0903	0.0931	0.0934	0.0917	0.0945	0.0951	0.0957	0.0968	0.0975	0.0978	0.1032*	5.52 %

Table 3. Self-distillation performance with SASRec. Best and second-best results are marked in bold and underlined.

\boldsymbol{Improv.}

denotes the relative improvement over the strongest baseline. Asterisk (*) indicates statistical significance at

p<0.05

by paired

t

-test over five independent runs against the strongest baseline.

Panel of Figure 6 — Figure 6. Performance by sequence length with SASRec.

Panel of Figure 7 — Figure 7. Performance under different noise injection ratios with SASRec.

Panel of Figure 8 — Figure 8. Relative NDCG@20 improvements (%) of GOD over EMKD, the strongest baseline, under different student and teacher dimensions with SASRec.

Panel of Figure 9 — Figure 9. Performance under different teacher training progress with SASRec. Here, 100% denotes the best teacher checkpoint.

Case	(T,T)	(S,T)	(T,S)	Amazon Beauty	Yelp	MovieLens 1M
(1)	$\circ$	$\circ$	$\circ$	0.0474	0.0405	0.1218
(2)	$\circ$	$\circ$		0.0451	0.0375	0.1141
(3)	$\circ$		$\circ$	0.0448	0.0376	0.1135
(4)		$\circ$	$\circ$	0.0410	0.0342	0.1084
(5)	$\circ$			0.0421	0.0349	0.1111
(6)		$\circ$		0.0401	0.0336	0.1074

Table 4. Source model ablation with SASRec. NDCG@20 results are shown for the source models used in KD, where (T,T), (S,T), and (T,S) denote Non-Grafted Teacher , Embed-Grafted Teacher , and Encoder-Grafted Teacher , respectively. Case (1) corresponds to GOD.

Figure 10. Projection design ablation with SASRec. Non-shared uses source model-specific projections, and MLP uses 2-layer projections. Shared Linear corresponds to GOD.

Variant	Amazon Beauty	Yelp	MovieLens 1M
GE	0.0474	0.0405	0.1218
T $\rightarrow$ S only	0.0466	0.0394	0.1184
S $\rightarrow$ T only	0.0459	0.0386	0.1166
GE (half length)	0.0462	0.0396	0.1152
w/o GE	0.0452	0.0379	0.1126

Table 5. GE ablation with SASRec. NDCG@20 is shown by varying teacher-student token interaction. T

\rightarrow

S only allows student-side query tokens to attend to teacher-side key/value tokens, while S

\rightarrow

T only allows the reverse direction. GE (half length) halves each side sequence length so that the concatenated sequence has the original length.

Variant	Amazon Beauty	Yelp	MovieLens 1M
GCL	0.0474	0.0405	0.1218
Source-Target only	0.0460	0.0386	0.1111
Unweighted	0.0455	0.0379	0.1090
Learnable weight	0.0427	0.0359	0.1074

Table 6. GCL ablation with SASRec. NDCG@20 is shown by varying pair construction and weighting. Source-Target only uses three pairs involving

h_{u}^{(S,S)}

, Unweighted uses

\mathcal{L}_{pair}

instead of

\mathcal{L}_{kd}

, and Learnable weight replaces similarity-based weights with trainable pair weights.

Panel of Figure 11 — Figure 11. Dynamics of six GCL pair weights over training. Higher pair similarity yields a lower weight.

Model	Amazon Beauty				MovieLens 1M
Model	Mem.	Time/Epoch	Train	NDCG@20	Mem.	Time/Epoch	Train	NDCG@20
Teacher	0.7 GB	8.35 s	6.12 m	0.0472	1.1 GB	15.88 s	23.82 m	0.1417
Student	0.6 GB	7.75 s	24.03 m	0.0390	0.8 GB	15.13 s	18.66 m	0.1052
BD	1.0 GB	12.98 s	27.69 m	0.0444	1.5 GB	26.67 s	59.12 m	0.1094
EMKD	1.1 GB	15.57 s	34.77 m	0.0447	1.6 GB	32.90 s	51.00 m	0.1098
GOD	0.9 GB	10.06 s	15.26 m	0.0474	1.2 GB	21.00 s	17.85 m	0.1218

Table 7. Efficiency analysis with SASRec. Mem. denotes peak GPU memory, Time/Epoch denotes average training time per epoch, and Train denotes wall-clock training time until the best validation checkpoint.

Panel of Figure 12 — Figure 12. Parameter sensitivity with SASRec.

Sequential Recommendation

Sequential recommendation predicts what a user will choose next from ordered interactions, adapting to changing interests and context.

See all 141 papers

BAHSD: Bridging the Long-tail Gap via Adaptive Distillation in Black-box Sequential Recommendation
Jun 2, 2026Xi Zhou, Famin Wu, Mingming Li +4Adaptive Knowledge Distillation Sequential Recommendation
Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China · Beijing Institute for General Artificial Intelligence, Beijing, China
Pre-trained LLMs Meet Sequential Recommenders: Efficient User-Centric Knowledge Distillation
Apr 23, 2026Nikita Severin, Danil Kartushov, Vladislav Urzhumov +8Sequential Recommendation Language Model Distillation
Independent researcher, Belgrade, Serbia · ITMO University, Saint Petersburg, Russia · Innopolis University, Innopolis, Russia +3
Knowledge Distillation across Item and Code Spaces for Parallel Generative Recommendation
Oct 23, 2026Yixuan Xu, Hailin Liu, Linxiang Peng +1Semantic ID Recommendation Generative Recommendation
The Hong Kong University of Science and Technology, Hong Kong, China · University of Science and Technology of China, Hefei, China

Jun 2, 2026J/K move · Enter open · S save

BAHSD: Bridging the Long-tail Gap via Adaptive Distillation in Black-box Sequential Recommendation

Xi Zhou, Famin Wu, Mingming Li, Hongyue Zhang+3

Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China · Beijing Institute for General Artificial Intelligence, Beijing, China

Adaptive Knowledge Distillation Sequential Recommendation Long-Tail Learning Recommender Systems Knowledge Distillation

Open paper AskPDF arXiv

Sequential recommendation systems are widely adopted but often deployed as black-box APIs, which has driven recent interest in model extraction to replicate their capabilities locally. However, the long-tail distribution induces severe signal heterogeneity: dense head sequences trigger the solidification of teacher preference, biasing extraction toward local patterns, while sparse tail sequences yield flat, noisy predictions. Existing one-size-fits-all extraction overlooks this disparity, resulting in noise overfitting and suboptimal knowledge transfer. We propose BAHSD, a black-box adaptive distillation framework that handles signal heterogeneity via a multi-scale consistency probing mechanism to implicitly quantify signal reliability. Based on this, an adaptive hierarchical objective is designed: dynamic-temperature KL divergence mitigates preference solidification for high-confidence signals, while ranking consistency and InfoNCE contrastive learning provide noise-robust enhancement for low-confidence signals. BAHSD consistently outperforms baselines, achieving up to 4.98% gain over the teacher and 80%+ improvement on tail users, offering a plug-and-play solution for high-fidelity black-box recommendation extraction.

Abstract

Sequential Recommendation

Explore similar work

BAHSD: Bridging the Long-tail Gap via Adaptive Distillation in Black-box Sequential Recommendation

Pre-trained LLMs Meet Sequential Recommenders: Efficient User-Centric Knowledge Distillation

Knowledge Distillation across Item and Code Spaces for Parallel Generative Recommendation