cs.LGMar 1, 2026

GeMi: A Graph-based, Multimodal Recommendation System for Narrative Scroll Paintings

Authors: Haimonti Dutta, Pruthvi Moluguri, Jin Dai, Saurabh Amarnath Mahindre

Organizations: Department of Management Science and Systems, The State University of New York, Jacobs Management Center, Buffalo, 14260, NY, USA. · Institute for Artificial Intelligence and Data Science, The State University of New York, Lockwood Hall, Buffalo, 14260, NY, USA. · Department of Computer Science and Engineering, The State University of New York, Davis Hall, Buffalo, 14260, NY, USA. · Technology, Information, Internet, eBay∗, Embassy Tech Village, Bengaluru, 560103, Karnataka, India.

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Recommendation Systems are effective in managing the ever-increasing amount of multimodal data available today and help users discover interesting new items. These systems can handle various media types such as images, text, audio, and video data, and this has made it possible to handle content-based recommendation utilizing features extracted from items while also incorporating user preferences. Graph Neural Network (GNN)-based recommendation systems are a special class of recommendation systems that can handle relationships between items and users, making them particularly attractive for content-based recommendations. Their popularity also stems from the fact that they use advanced machine learning techniques, such as deep learning on graph-structured data, to exploit user-to-item interactions. The nodes in the graph can access higher-order neighbor information along with state-of-the-art vision-language models for processing multimodal content, and there are well-designed algorithms for embedding, message passing, and propagation. In this work, we present the design of a GNN-based recommendation system on a novel data set collected from field research. Designed for an endangered performing art form, the recommendation system uses multimodal content (text and image data) to suggest similar paintings for viewing and purchase. To the best of our knowledge, there is no recommendation system designed for narrative scroll paintings -- our work therefore serves several purposes, including art conservation, a data storage system for endangered art objects, and a state-of-the-art recommendation system that leverages both the novel characteristics of the data and preferences of the user population interested in narrative scroll paintings.

Figures & tables

Figure 1: An illustration of a panel-by-panel extraction of images from a narrative scroll painting and alignment with the text of an ancient doggerel poem written on handmade paper using black ink. Many parts of the original manuscript of the poem are missing or lost.
Figure 2: The GeMi recommendation system. Fine-tuned Vision Language models are used to extract features, learn latent graph structures, and embeddings using graph convolutions. User-Panel preference matrices are used to augment recommendations.
Figure 3: Distribution of the number of panels per scroll in the two phases of data collection from field surveys.
GeMi VariantParameters
GeMi-GCNGraph encoder: 2-layer Graph Convolutional Network; GCN hidden dimension: 128; GCN dropout: 0.20; Training epochs: 450; Optimizer: Adam; Activation: ReLU; Learning rate: 3×10−43\times 10^{-4} ; Weight decay: 2×10−32\times 10^{-3} ; Gradient clipping: max norm 2.0; Loss (balanced): focal binary cross-entropy with logits; Focal parameters: γ=2.0\gamma=2.0 , α=0.25\alpha=0.25 ; kNN neighbors ( kk ): 30; Tree-edge augmentation neighbors ( ktreek_{tree} ): 25; Tree augmentation max nodes: 2500; Edge dropout: 0.10; k=5k=5 interactions; Preference threshold: 0.2
GeMi-GAEGraph encoder: Graph Auto-Encoder (GAE) with a 2-layer GCN encoder; Hidden dimension: 128; Latent dimension ( dzd_{z} ): 64; Dropout: 0.20; Training epochs: 400; Optimizer: Adam; Activation: ReLU; Learning rate: 2×10−32\times 10^{-3} ; Weight decay: 2×10−42\times 10^{-4} ; Gradient clipping: max norm 2.0; kNN neighbors ( kk ): 30; Edge dropout: 0.10; Self-supervised loss: GAE reconstruction loss Lrec\mathcal{L}_{rec} ; Supervised loss: focal binary cross-entropy with logits; Focal parameters: γ=2.0\gamma=2.0 , α=0.25\alpha=0.25 ; Joint objective: L=Lrec+λsupLsup\mathcal{L}=\mathcal{L}_{rec}+\lambda_{sup}\mathcal{L}_{sup} with λsup=0.60\lambda_{sup}=0.60 ; Augmentation neighbors: kanimal=18k_{animal}=18 , kmyth=18k_{myth}=18 , ktree=35k_{tree}=35 ; Augmentation max nodes per label: 2500; k=5k=5 interactions; Preference threshold: 0.2
GeMi-VGAEGraph encoder: Variational Graph Auto-Encoder (VGAE) with GCN-based parameterization; Hidden dimension: 256; Latent dimension ( dzd_{z} ): 128; Dropout: 0.20; Log-standard-deviation clamp: log⁡σ∈[−10, 10]\log\sigma\in[-10,\,10] ; Training epochs: 500; Optimizer: Adam; Activation: ReLU; Learning rate: 3×10−33\times 10^{-3} ; Weight decay: 5×10−45\times 10^{-4} ; Gradient clipping: max norm 2.0; kNN neighbors ( kk ): 25; Tree augmentation neighbors: ktree=25k_{tree}=25 ; Tree augmentation max nodes: 2500; Edge dropout: 0.10; Reconstruction loss: VGAE reconstruction loss Lrec\mathcal{L}_{rec} ; KL regularization: VGAE KL loss LKL\mathcal{L}_{KL} ; KL annealing: βKL(t)\beta_{KL}(t) linearly increased to βmax⁡=1.0\beta_{\max}=1.0 ; Self-supervised VGAE objective: Lvgae=Lrec+βKLLKL\mathcal{L}_{vgae}=\mathcal{L}_{rec}+\beta_{KL}\mathcal{L}_{KL} ; Supervised loss: focal binary cross-entropy with logits; Focal parameters: γ=2.0\gamma=2.0 , α=0.25\alpha=0.25 ; Joint SSL objective: L=Lvgae+λsslLsup\mathcal{L}=\mathcal{L}_{vgae}+\lambda_{ssl}\mathcal{L}_{sup} with λssl=0.60\lambda_{ssl}=0.60 ; k=5k=5 interactions; Preference threshold: 0.2
Table 1: Parameters of GeMi Variants
ModelAnimalMythologyTree
LATTICE0.57 ±\pm 0.210.47 ±\pm 0.250.20 ±\pm 0.18
PMGT0.52 ±\pm 0.210.46 ±\pm 0.230.33 ±\pm 0.22
HUIGN0.48 ±\pm 0.160.55 ±\pm 0.250.10 ±\pm 0.11
DualGNN0.55 ±\pm 0.210.47 ±\pm 0.260.25 ±\pm 0.21
FREEDOM0.58 ±\pm 0.220.41 ±\pm 0.240.22 ±\pm 0.19
MICRO0.61 ±\pm 0.200.47 ±\pm 0.260.26 ±\pm 0.21
Table 2: Transductive Llama-SigCLIP features, homogeneous graph
ModelAnimalMythologyTree
LATTICE0.53 ±\pm 0.220.54 ±\pm 0.130.21 ±\pm 0.25
PMGT0.47 ±\pm 0.190.52 ±\pm 0.170.25 ±\pm 0.20
HUIGN0.49 ±\pm 0.200.51 ±\pm 0.120.23 ±\pm 0.22
DualGNN0.41 ±\pm 0.180.56 ±\pm 0.320.19 ±\pm 0.27
FREEDOM0.51 ±\pm 0.170.55 ±\pm 0.240.24 ±\pm 0.16
MICRO0.45 ±\pm 0.250.58 ±\pm 0.230.26 ±\pm 0.19
Table 3: Transductive Llama-VAE features, homogeneous graph
ModelAnimalMythologyTree
MambaRec0.50 ±\pm 0.240.60 ±\pm 0.170.12 ±\pm 0.13
PGL0.43 ±\pm 0.260.60 ±\pm 0.200.25 ±\pm 0.18
HPMRec0.47 ±\pm 0.260.63 ±\pm 0.210.34 ±\pm 0.21
COHESION0.41 ±\pm 0.230.59 ±\pm 0.200.27 ±\pm 0.17
SMORE0.40 ±\pm 0.200.66 ±\pm 0.240.24 ±\pm 0.16
CMDL0.41 ±\pm 0.250.67 ±\pm 0.220.23 ±\pm 0.18
Table 4: Transductive Llama-SigCLIP features (user-item)
ModelAnimalMythologyTree
MambaRec0.52 ±\pm 0.190.57 ±\pm 0.260.15 ±\pm 0.28
PGL0.44 ±\pm 0.230.60 ±\pm 0.210.27 ±\pm 0.21
HPMRec0.46 ±\pm 0.180.64 ±\pm 0.220.23 ±\pm 0.14
COHESION0.50 ±\pm 0.240.55 ±\pm 0.110.25 ±\pm 0.26
SMORE0.41 ±\pm 0.540.59 ±\pm 0.130.17 ±\pm 0.31
CMDL0.57 ±\pm 0.280.61 ±\pm 0.150.16 ±\pm 0.25
Table 5: Transductive Llama-VAE features heterogeneous graph
ModelAnimalMythologyTree
GraphSAGE0.48 ± 0.140.56 ± 0.230.34 ± 0.28
PinSAGE0.52 ± 0.210.57 ± 0.220.49 ± 0.14
GATNE-I0.52 ± 0.200.60 ± 0.210.37 ± 0.19
GeMi GCN0.56 ± 0.240.66 ± 0.200.44 ± 0.20
GeMi GAE0.50 ± 0.220.63 ± 0.150.34 ± 0.23
GeMi VGAE0.52 ± 0.220.70 ± 0.210.44 ± 0.20
Table 6: Inductive Llama-SigCLIP features (item-item)
ModelAnimalMythologyTree
GraphSAGE0.56 ± 0.250.58 ± 0.120.33 ± 0.16
PinSAGE0.56 ± 0.250.57 ± 0.120.32 ± 0.17
GATNE-I0.56 ± 0.250.60 ± 0.190.34 ± 0.22
GeMi GCN0.46 ± 0.220.32 ± 0.190.56 ± 0.34
GeMi GAE0.51 ± 0.290.61 ± 0.180.29 ± 0.23
GeMi VGAE0.50 ± 0.270.61 ± 0.230.29 ± 0.23
Table 7: Inductive Llama-VAE features (item-item)
ModelAnimalMythologyTree
GraphSAGE0.63 ± 0.180.59 ± 0.190.26 ± 0.18
PinSAGE0.50 ± 0.210.65 ± 0.170.21 ± 0.15
GATNE-I0.51 ± 0.200.60 ± 0.210.36 ± 0.19
GeMi GCN0.71 ± 0.250.39 ± 0.050.35 ± 0.13
GeMi GAE0.47 ± 0.200.71 ± 0.200.20 ± 0.22
GeMi VGAE0.52 ± 0.230.77 ± 0.110.53 ± 0.19
Table 8: Inductive Llama-SigCLIP features heterogeneous graph
ModelAnimalMythologyTree
GraphSAGE0.58 ± 0.240.52 ± 0.150.26 ± 0.12
PinSAGE0.46 ± 0.200.61 ± 0.110.39 ± 0.22
GATNE-I0.56 ± 0.240.61 ± 0.200.33 ± 0.22
GeMi GCN0.36 ± 0.120.59 ± 0.070.35 ± 0.13
GeMi GAE0.69 ± 0.250.77 ± 0.110.34 ± 0.15
GeMi VGAE0.53 ± 0.190.78 ± 0.100.53 ± 0.19
Table 9: Inductive Llama-VAE features heterogeneous
ModelAnimalMythologyTree
SigCLIP + GCN0.48 ±\pm 0.250.70 ±\pm 0.210.13 ±\pm 0.15
SigCLIP + GAE0.54 ±\pm 0.280.52 ±\pm 0.170.20 ±\pm 0.17
SigCLIP + VGAE0.51 ±\pm 0.250.62 ±\pm 0.190.18 ±\pm 0.18
Table 10: Transductive Base-SigCLIP
ModelAnimalMythologyTree
VAE + GCN0.49 ±\pm 0.230.58 ±\pm 0.200.38 ±\pm 0.24
VAE + GAE0.33 ±\pm 0.200.47 ±\pm 0.210.23 ±\pm 0.20
VAE + VGAE0.34 ±\pm 0.200.53 ±\pm 0.130.20 ±\pm 0.18
Table 11: Transductive Base-VAE features
ModelAnimalMythologyTree
SigCLIP + GCN0.52 ± 0.260.57 ± 0.140.39 ± 0.27
SigCLIP + GAE0.50 ± 0.250.54 ± 0.230.37 ± 0.27
SigCLIP + VGAE0.51 ± 0.350.50 ± 0.270.44 ± 0.21
Table 12: Inductive Base-SigCLIP
ModelAnimalMythologyTree
VAE + GCN0.47 ± 0.270.51 ± 0.280.45 ± 0.23
VAE + GAE0.39 ± 0.230.61 ± 0.220.30 ± 0.25
VAE + VGAE0.51 ± 0.250.33 ± 0.180.51 ± 0.30
Table 13: Inductive Base-VAE features
Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: A toy example from GeMi illustrating two types of item relations – collaborative and semantic. Both can be used to learn the latent graph structures.
ModelParameters
LATTICEGraph convolution layers: 2; Graph mixing coefficient ( λ\lambda ): 0.6; Loss function: Bayesian Personalized Ranking (BPR); Optimizer: Adam; Learning rate: 5×10−45\times 10^{-4} ; Weight decay: 1×10−41\times 10^{-4} ; Activation: linear graph propagation; Embedding normalization: ℓ2\ell_{2} normalization;
PMGTNeighbors per sequence: 32; Hidden size: 128; Transformer layers: 4; Attention heads: 4; Feed-forward dimension: 256; Activation: GELU; Dropout: 0.1; Loss functions: BCE-with-logits + MSE; Masking ratio: 0.16; Random replacement ratio: 0.02; Optimizer: AdamW; Learning rate: 8×10−48\times 10^{-4} ; Weight decay: 1×10−41\times 10^{-4}
HUIGNHierarchical intent layers ( K(1),K(2),K(3)K^{(1)},K^{(2)},K^{(3)} ): [32, 8, 4]; Co-interaction threshold: ≥\geq 2; Loss terms: BPR + entropy regularization + independence regularization + ℓ2\ell_{2} weight decay; Entropy weight ( λ1\lambda_{1} ): 0.1; Independence weight ( λ2\lambda_{2} ): 0.1; Weight decay ( λ3\lambda_{3} ): 1×10−51\times 10^{-5} ; Optimizer: Adam; Learning rate: 1×10−41\times 10^{-4}
DualGNNGraph propagation depth: 2; Graph mixing coefficient ( α\alpha ): 0.6; Contrastive temperature ( τ\tau ): 0.2; Node feature dropout: 0.2; Loss terms: InfoNCE alignment + orthogonality regularization + graph smoothness regularization; Loss weights: λalign=1.0, λortho=0.05, λsmooth=0.05\lambda_{\text{align}}{=}1.0,\,\lambda_{\text{ortho}}{=}0.05,\,\lambda_{\text{smooth}}{=}0.05 ; Optimizer: Adam; Learning rate: 8×10−48\times 10^{-4} ; Weight decay: 1×10−41\times 10^{-4}
FREEDOMMultimodal graph mixing: αv=0.1\alpha_{v}{=}0.1 (image) and αt=0.9\alpha_{t}{=}0.9 (text); Item–Item propagation layers ( LiiL_{ii} ): 1; User–Item propagation layers ( LuiL_{ui} ): 2; Degree-sensitive edge pruning ratio ( ρ\rho ): 0.8; Training loss: BPR + multimodal auxiliary BPR regularizers; Regularization weight ( λ\lambda ): 1×10−31\times 10^{-3} ; Optimizer: Adam; Learning rate: 1×10−31\times 10^{-3} ; Weight decay: 1×10−51\times 10^{-5}
MICROItem-view layers: 1 (per modality); Graph mixing coefficient ( α\alpha ): 0.7; Contrastive objective: InfoNCE (temperature τ=0.5\tau{=}0.5 ); Orthogonality regularizer: cross-view residual decorrelation; Loss weights: βc=0.1\beta_{c}{=}0.1 (contrastive), βs=0.1\beta_{s}{=}0.1 ; Optimizer: Adam; Learning rate: 5×10−45\times 10^{-4} ; Weight decay: 1×10−41\times 10^{-4}
Appendix
Table 14: Transductive Homogeneous Item–Item Baseline Parameters
ModelParameters
MambaRecUI graph layers ( LuiL_{ui} ): 2; Item–Item layers ( LmmL_{mm} ): 2; Dimensionality reduction factor: r=2r=2 ; Normalization + dropout: LayerNorm( dd ) + Dropout(0.2); Loss: BPR pairwise ranking loss + ℓ2\ell_{2} regularization + contrastive InfoNCE + optional MMD alignment; Contrastive weight: 1.0; InfoNCE temperature: 0.2; MMD weight: 1×10−21\times 10^{-2} ; MMD kernel: RBF with bandwidth σ=1.0\sigma=1.0 ; Regularization weight: 1×10−31\times 10^{-3} ; Optimizer: Adam; Learning rate: 1×10−31\times 10^{-3} ; Weight decay: 1×10−41\times 10^{-4}
PGLUI graph layers ( LuiL_{ui} ): 2; Item–Item layers ( LmmL_{mm} ): 2; Dropout: 0.2; Loss: BPR pairwise ranking + contrastive InfoNCE loss; Contrastive temperature ( τ\tau ): 0.2; Regularization weight ( λ\lambda ): 1×10−31\times 10^{-3} on contrastive term; Optimizer: Adam; Learning rate: 1×10−31\times 10^{-3} ; Weight decay: 1×10−41\times 10^{-4}
HPMRecHypercomplex algebra dimension ( AA ): 4; UI graph layers ( LuiL_{ui} ): 2; Item–Item multimodal layers ( LmmL_{mm} ): 2; Hypercomplex interaction: Cayley–Dickson multiplication with scale α=0.1\alpha=0.1 ; Dropout: 0.2; Loss: BPR pairwise ranking + ℓ2\ell_{2} regularization + self-supervised hypercomplex loss; Reg weight: 1×10−31\times 10^{-3} ; SSL weight: 0.05; Optimizer: Adam; Learning rate: 1×10−31\times 10^{-3} ; Weight decay: 1×10−41\times 10^{-4}
COHESIONUI graph layers ( LuiL_{ui} ): 2; Multimodal item–item layers ( LmmL_{mm} ): 2; UI edge dropout: 0.2; Post-fusion stabilizer: LayerNorm + Linear + LeakyReLU; User–User neighborhood aggregation: kuu=40k_{uu}=40 ; Loss: BPR pairwise ranking + ℓ2\ell_{2} regularization; Reg weight: 1×10−31\times 10^{-3} ; Optimizer: Adam; Learning rate: 1×10−31\times 10^{-3} ; Weight decay: 1×10−41\times 10^{-4}
SMOREUI graph layers ( LuiL_{ui} ): 2; Item–Item layers ( LiiL_{ii} ): 2; Spectrum (FFT) convolution: rFFT over feature dimension with learnable complex filters (3 branches: wa,wb,wfw_{a},w_{b},w_{f} ); Nonlinear fusion branch: frequency-domain (FFT(z)⊙FFT(z))⊙wf(\mathrm{FFT}(z)\odot\mathrm{FFT}(z))\odot w_{f} then iFFT; Dropout: 0.2; Auxiliary contrastive loss: InfoNCE; Contrastive weight: 0.15; Temperature ( τ\tau ): 0.2; Regularization: light ℓ2\ell_{2} ; Reg weight: 1×10−41\times 10^{-4} ; Optimizer: Adam; Learning rate: 2×10−32\times 10^{-3} ; Weight decay: 1×10−41\times 10^{-4} ; Gradient clipping: max norm 5.0
CMDLUI graph layers ( LuiL_{ui} ): 2; Content-view augmentation: feature dropout p=0.25p=0.25 + Gaussian noise σ=0.05\sigma=0.05 ; Item content gating; Loss: BPR pairwise ranking + ℓ2\ell_{2} regularization + contrastive alignment; Contrastive weight: 0.1; Contrastive temperatures: τpos=0.2\tau_{pos}=0.2 (items), τneg=0.2\tau_{neg}=0.2 ; Regularization weight: 1×10−31\times 10^{-3} ; Optimizer: AdamW; Learning rate: 1×10−31\times 10^{-3} ; Weight decay: 1×10−41\times 10^{-4}
Appendix
Table 15: Transductive Heterogenous User–Item Baseline Parameters
ModelParameters
GraphSAGEBase percentile: 9090 , GNN layers: 22 , SAGEConv aggregator: max, SAGEConv settings: normalize: False, Hidden channels: 1212 , Output channels: 2, Activation: ReLU (between layers), Loss: cross-entropy on item labels, Optimizer: Adam, Learning rate: 0.00010.0001 .
PinSAGEPercentile threshold: 9090 , Random-walk neighborhood: 5050 , walk length: 55 , new neighbor per node(topm): 2020 , GNN encoder: same as GraphSAGE above (2 layers, hidden 12, out 2, Adam lr 0.00010.0001 ; cross-entropy).
GATNE-IkNN on training items (k = 30), Similarity floor: 0.0, Feature preprocessing: L2-normalize features before cosine, Embedding dim: 128, Negative samples: 10, Optimizer: Adam, Learning rate: 0.0010.001 , Weight decay: 0.0000010.000001 .
Appendix
Table 16: Inductive Homogeneous Item–Item Baseline Parameters
ModelParameters
GraphSAGEGNN layers: 2, SAGEConv aggregator: max, SAGEConv settings: project=True, normalize=False, Hidden channels: 12, Output channels: 2, Activation: ReLU, Loss: cross-entropy on item labels, Optimizer: Adam, Learning rate: 1e-4, Weight decay: default.
PinSAGEPercentile threshold: 90, Random-walk neighborhood: 50, walk length: 5, new neighbor per node(topm): 20, GNN encoder same as GraphSAGE above (2 layers, hidden 12, out 2, Adam, cross-entropy)
GATNE-IkNN on training items (k = 30), Similarity floor: 0.0, Feature preprocessing: L2-normalize features before cosine, Embedding dim: 128, Negative samples: 10, Optimizer: Adam, Learning rate: 1e−31e-3 , Weight decay: 1e−61e-6 . No learning on test interactions.
Appendix
Table 17: Inductive Heterogenous User–Item Baseline Parameters
Figure 5: Effect of the Top- kk recommendation cutoff on Precision@5 across Animal, Mythology, and Tree categories for GeMi variants under different feature backbones (LlamaSigCLIP and LlamaVAE). Results are reported as mean ±\pm standard deviation over multiple runs.
Figure 6: Effect of the focal loss focusing parameter γ\gamma on Precision@5 across Animal, Mythology, and Tree categories for GeMi variants under different feature backbones (LlamaSigCLIP and LlamaVAE). Results are reported as mean ±\pm standard deviation over multiple runs.
Figure 7: Effect of the edge dropout parameter on Precision@5 across Animal, Mythology, and Tree categories for GeMi variants under different feature backbones (LlamaSigCLIP and LlamaVAE). Results are reported as mean ±\pm standard deviation over multiple runs.
Figure 8: RQ4 (kNN ablation): Precision@5 vs. kNN graph construction parameter k for Animal. Error bars denote mean ± 1 std across runs.
Figure 9: RQ4 (kNN ablation): Precision@5 vs. kNN graph construction parameter k for Mythology. Error bars denote mean ± 1 std across runs.
Figure 10: RQ4 (kNN ablation): Precision@5 vs. kNN graph construction parameter k for Tree. Error bars denote mean ± 1 std across runs.
Figure 11: Characteristics of users working with UPFAR
Figure 12: Architecture diagram of UPFAR
Figure 13: Deployed screens from UPFAR
ModelAnimalMythologyTree
SIGCLIP + GCN0.52±0.220.52\pm 0.220.68±0.170.68\pm 0.170.40±0.200.40\pm 0.20
SIGCLIP + GAE0.55±0.220.55\pm 0.220.49±0.140.49\pm 0.140.33±0.220.33\pm 0.22
SIGCLIP + VGAE0.44±0.160.44\pm 0.160.74±0.180.74\pm 0.180.43±0.220.43\pm 0.22
VAE + GCN0.27±0.090.27\pm 0.090.59±0.040.59\pm 0.040.01±0.040.01\pm 0.04
VAE + GAE0.50±0.220.50\pm 0.220.69±0.110.69\pm 0.110.22±0.160.22\pm 0.16
VAE + VGAE0.60±0.110.60\pm 0.110.65±0.090.65\pm 0.090.60±0.210.60\pm 0.21
Appendix
Table 18: Results from Transductive model training using real-world data to guide user-preference matrix generation.
ModelAnimalMythologyTree
SIGCLIP + GCN0.45±0.140.45\pm 0.140.79±0.250.79\pm 0.250.23±0.230.23\pm 0.23
SIGCLIP + GAE0.39±0.200.39\pm 0.200.58±0.200.58\pm 0.200.24±0.260.24\pm 0.26
SIGCLIP + VGAE0.65±0.270.65\pm 0.270.71±0.130.71\pm 0.130.30±0.210.30\pm 0.21
VAE + GCN0.64±0.180.64\pm 0.180.61±0.090.61\pm 0.090.24±0.300.24\pm 0.30
VAE + GAE0.53±0.090.53\pm 0.090.54±0.130.54\pm 0.130.19±0.160.19\pm 0.16
VAE + VGAE0.60±0.200.60\pm 0.200.64±0.190.64\pm 0.190.25±0.190.25\pm 0.19
Appendix
Table 19: Results from Inductive model training using real-world data to guide user-preference matrix generation.
More in this topic

Recommender Systems

Recommender systems predict and rank items for people using interaction histories, content, context, and feedback.

See all 514 papers

Explore similar work

CardsList
May 2, 2026

Dynamic Graph with Similarity-Aware Attention Graph Neural Network for Recommender Systems

Recommender systems are essential components of modern online platforms which presents personalized content in various domain. The traditional collaborative filtering methods depends on static user-item interaction graphs and a limited subset of similarity measures which fail to capture the changing nature of preferences of an individual. Recent graph neural network (GNN) based approaches focus on user-item bipartite graphs which do not use explicit user-user relational modelling and dynamic graph evolution during training. To address these limitations, this paper proposes a Dynamic Graph SimilarityAware Attention Graph Neural Network (DG-SA-GNN) framework that integrates dynamic user similarity graph construction with multi-similarity propagation and attention-based aggregation. The proposed architecture constructs four parallel user similarity graphs using Cosine, Jaccard, Discounted Pearson Correlation Coefficient (Discount PCC), and IPIJ similarity functions, each processed by a dedicated UserGNN module. A Graph Transformer fuses the four graph views, and a CrossAttention module refines user embeddings through interaction with item embeddings. Crucially, the graphs are reconstructed at scheduled epochs during training, enabling the model to adapt to the learned embedding space constituting the dynamic graph component. Mini-batch training with hard negative sampling improves scalability and convergence. Experiments on the MovieLens100K benchmark demonstrate that DG-SA-GNN achieves a Recall@20 of 0.162 and NDCG@20 of 0.065 which is better than the LightGCN baseline in recall. The results validate that dynamic multi-similarity graph construction coupled with attention-based fusion which produce recommendation performance

Aadarsh Senapati, Neha Kujur, Vivek Yelleti

Department of Computer Science and Engineering, SRM University Andhra Pradesh, Mangalagiri, Andhra Pradesh 522502, India

Dynamic Graph LearningRecommender SystemsCollaborative Filtering
AskPDF
Oct 8, 2026

Multimodal Graph Retrieval-Augmented Sequential Recommendation via Collaborative Filtering Paths

Multimodal Large Language Models (MLLMs) have demonstrated strong potential for sequential recommendation through their ability to reason over complex multimodal data. However, existing approaches either rely solely on the target user's own interaction history, neglecting collaborative signals from neighboring users, or incur substantial computational overhead through repeated MLLM inference over long interaction histories. To address these challenges, we propose MGRASRec, a multimodal graph retrieval-augmented framework for sequential recommendation. MGRASRec injects collaborative filtering signals conditioned on the candidate item directly into the MLLM prompt by retrieving structured paths from a user-item interaction graph, extended via multimodal similarity to increase coverage beyond exact co-interaction overlap. This retrieval also surfaces the history items most relevant to the candidate at no additional cost, removing the need for recurrent summarization and keeping inference to a single forward pass per candidate. All components are unified into an augmented prompt for parameter-efficient fine-tuning of an MLLM. Extensive evaluations across three publicly available datasets validate the effectiveness of MGRASRec, achieving the best performance on all metrics with particularly strong gains in ranking quality.

Jason Marcell Setiadi, Xin Cao, Lina Yao

University of New South Wales, Sydney, Australia

Recommender SystemsSequential RecommendationMultimodal RecommendationLarge Language Model-Based Recommendation
AskPDF
Feb 24, 2026

E-MMKGR: A Unified Multimodal Knowledge Graph Framework for E-commerce Applications

Multimodal recommender systems (MMRSs) enhance collaborative filtering by leveraging item-side modalities, but their reliance on a fixed set of modalities and task-specific objectives limits both modality extensibility and task generalization. We propose E-MMKGR, a framework that constructs an e-commerce-specific Multimodal Knowledge Graph E-MMKG and learns unified item representations through GNN-based propagation and KG-oriented optimization. These representations provide a shared semantic foundation applicable to diverse tasks. Experiments on real-world Amazon datasets show improvements of up to 10.18% in Recall@10 for recommendation and up to 21.72% over vector-based retrieval for product search, demonstrating the effectiveness and extensibility of our approach.

Jiwoo Kang, Yeon-Chang Lee

UNIST Ulsan, Korea

Multimodal KGsMultimodal RecommendationMultimodal Graph LearningMultimodal IRCross-Modal Representation Learning
AskPDF