Transformer architectures, capable of capturing sequential dependencies in the history of user interactions, have become the dominant approach in sequential recommender systems. Despite their success, such models consider sequence elements in isolation, implicitly accounting for the complex relationships between them. Graph neural networks, in contrast, explicitly model these relationships through higher order interactions but are often unable to adequately capture their evolution over time, limiting their use for predicting the next interaction. To fill this gap, we present a new framework that combines transformers and graph neural networks and aligns different representations for solving next-item prediction task. Our solution simultaneously encodes structural dependencies in the interaction graph and tracks their dynamic change. Experimental results on a number of open datasets demonstrate that the proposed framework consistently outperforms both pure sequential and graph approaches in terms of recommendation quality, as well as recent methods that combine both types of signals.
Figures & tables
Figure 1. Overview of the CREATE framework. A sequential encoder (top) models users’ local interaction dynamics, while a graph encoder (bottom) captures global item–item context. Their embeddings are aligned via a representation-alignment objective. Training proceeds in two phases: the graph encoder is first pre-trained during a warm-up stage, after which both encoders are jointly optimized end-to-end. Block diagram of CREATE with two encoders (sequential and graph) whose embeddings are aligned; the graph branch is pre-trained in a warm-up stage before joint training.
Figure 2. Visualized training process, with training loss and NDCG@10 scales placed on the left and on the right accordingly. Training curves for Beauty and Sports datasets showing distinct warm-up and joint training phases. In both datasets, training loss (red line, left axis) drops sharply after epoch when the sequential model is activated, then gradually decreases. The validation NDCG@10 metric (blue line, right axis) shows consistent improvement, with Beauty reaching ~0.018 and Sports reaching ~0.020, both stabilizing after approximately 50-100 epochs. The sharp transition at the warm-up boundary demonstrates the model successfully integrating graph-based knowledge with sequential patterns.
Dataset
#Users
#Items
#Interactions
MovieLens-1M
6,040
3,706
1,000,209
Amazon Clothing (5-core)
39,387
23,033
278,677
Amazon Sports (5-core)
35,598
18,357
296,337
Amazon Beauty (5-core)
22,363
12,101
198,502
Yambda-50M (likes)
8283
181,304
881,456
Table 1. Datasets and their statistics used in this work.
Dataset
Metric
SASRec
BERT4Rec
MRGSRec
SVD
Random
PopRnd
LightGCN
UltraGCN
CREATE
Gain
Beauty
NDCG@10
1.87 ( ±0.01 )
1.34
0.41
0.52
0.04
0.07
0.93
0.97
2.15 ( ±0.01 )
+15%
Recall@10
3.83 ( ±0.02 )
2.68
0.88
1.04
0.07
0.15
1.88
1.93
4.65 ( ±0.03 )
+21%
Cov@10
70.04 ( ±1.26 )
21.42
3.24
14.02
99.90
91.24
13.27
17.54
75.67 ( ±1.26 )
+8%
NDCG@100
3.73 ( ±0.02 )
3.17
1.27
1.92
0.18
0.26
2.13
2.25
4.25 ( ±0.03 )
+14%
Recall@100
13.67 ( ±0.08 )
12.19
5.38
7.22
0.82
1.21
8.25
8.34
15.64 ( ±0.13 )
+14%
Cov@100
93.46 ( ±3.29 )
54.88
21.80
72.79
100.00
96.85
46.94
47.33
95.37 ( ±1.04 )
+2%
Table 2. Comparison of models, evaluated using NDCG@K , Recall@K and Coverage@K (in % ), with SASRec underlined as main baseline.
Dataset
Metric
SASRec
GSAU
Self-GNN
LOOM
CREATE
Office
HR@10
1.81
1.95
1.42
2.11
2.16
NDCG@10
0.72
0.78
0.55
0.92
0.96
Beauty
HR@10
1.19
1.35
0.98
1.43
1.49
NDCG@10
0.49
0.54
0.40
0.55
0.57
Table 3. Model comparison in LOOM ( Li et al., 2025 ) setup using HR@10 and NDCG@10 metrics (in % ).
Figure 3. Graph size ablation for MovieLens-1M and Amazon Beauty datasets The plot shows NDCG@10 score for each graph built on n% of latest interactions per user with n% ranging from 10% to 100%. Primary axis depicts the scale for MovieLens-1M and secondary axis corresponds for the values of NDCG@10 on Amazon Beauty. Amazon Beauty has optimal values around 40%, while MovieLens-1M has optimal values n>40%.
Figure 4. Performance (NDCG@10) vs. the number of GCN warm-up epochs. Line plot of NDCG@10 on Amazon Beauty and Sports versus GCN warm-up epochs; performance peaks on 50 for Beauty and 10 epochs for Sports and declines with longer warm-up.
Figure 5. Test ndcg@10 metric by the Barlow coefficient. Line plots showing the impact of the Barlow Twins loss coefficient on NDCG@10 for Beauty and Sports datasets. Both plots show a sharp increase in performance as the coefficient moves from 0.0 to 0.1, confirming that representation alignment significantly improves recommendation quality. For the Beauty dataset, performance remains stable around 0.0215 as the coefficient increases from 0.1 to 1.0. For the Sports dataset, performance peaks around 0.0165 at a coefficient of 0.2 and then slightly declines to approximately 0.0155 as the coefficient approaches 1.0. This demonstrates that while the Barlow Twins loss is crucial, its optimal weight varies slightly by dataset.
Sports
Method
NDCG@10
Cov@10
NDCG@100
Cov@100
Sequential
1.48
50.35
3.04
79.41
CREATE with LightGCN graph encoder
None
1.37
36.58
2.89
66.95
Contrastive
1.49
40.16
3.05
67.59
Barlow
1.56
25.69
3.24
66.02
Table 4. Comparison of our model with different alignment losses and the sequential baseline on the Sports and Clothing datasets, using NDCG@K and Coverage@K metrics (in % ).
Seq. Backbone
Metric
LightGCN
UltraGCN
SASRec
NDCG@10
1.67
1.69
Recall@10
3.02
3.13
BERT4Rec
NDCG@10
1.39
1.43
Recall@10
2.65
2.73
Table 5. Comparison of sequential backbones (SASRec, BERT4Rec) combined with graph encoders (LightGCN, UltraGCN) evaluated by NDCG@10 and Recall@10 .
Information Systems and Machine Learning Lab (ISMLL), University of Hildesheim, Germany · VWFS Data Analytics Research Center (VWFS DARC), University of Hildesheim, Germany · Volkswagen Financial Services AG, Germany