Graph neural networks (GNNs) are typically conceptualized as message-passing neural networks, yet it remains unclear why neighborhood aggregation reliably outperforms node-wise multilayer perceptrons (MLPs). Despite its empirical success, this paradigm can be computationally expensive and sensitive to imperfect graph structures. In this work, we present a retrieval-augmented view of GNNs: each layer makes predictions by applying an MLP to a node representation together with a permutation-invariant summary of retrieved graph context. Motivated by this perspective, we propose RTA, a simple MLP-based framework that replaces structural message passing with label-aware retrieval and propagation. We provide theoretical insights that (i) connect retrieval-based aggregation to softmax-attention message passing, and (ii) establish the robustness of retrieved-context supervision to mis-retrieved outliers. Experiments on multiple text-attributed graph benchmarks show that RTA matches or even outperforms strong GNN and graph LLM baselines while improving efficiency and robustness across diverse scenarios.
Figures & tables
Fig. 1: Technical comparison between GNN and RAG. Both leverage contextual information beyond the raw input.
Fig. 2: Illustrative example on how GNNs are decomposed into RAG-like architectures.
Fig. 3: Overall framework of RTA. RTA recasts graph representation learning as a retrieval-augmented pipeline that avoids explicit multi-hop message passing over the original graph. Specifically, (a) for a query node v0 , we augment its text with a small set of structural neighbors (size k1 ) and corresponding labels as the resulting prompt to obtain the contextual node embedding xv0 . Then, (b) based on the learned embeddings and the original graph structure, we construct a retrieval-defined context graph by jointly considering semantic similarity and structural proximity. Specifically, semantic retrieval is performed in the embedding space (e.g., via FAISS), while structural retrieval exploits graph diffusion information. Finally, (c) we aggregate features and available label embeddings from retrieved contexts and apply an MLP-style update to produce task-specific node representations for prediction.
Dataset
#Nodes
#Edges
#Classes
Type
Children
76,875
1,554,578
24
Homophilic
History
41,551
358,574
12
Homophilic
Computers
87,229
721,081
10
Homophilic
Photo
48,362
500,928
12
Homophilic
Fitness
173,055
1,773,500
13
Homophilic
Arxiv
169,343
1,166,243
40
Homophilic
TABLE I: Datasets statistics. Both homophilic and heterophilic graphs are included in our experiments.
Children
History
Photo
Computers
Fitness
Arxiv
Cornell
Texas
Wisc.
Wash.
MLP
51.2 ±0.2
82.0 ±0.9
60.7 ±0.9
65.3 ±0.4
87.2 ±0.3
46.5 ±0.8
49.5 ±0.7
62.0 ±1.8
46.5 ±1.5
66.0 ±0.9
GCN
56.3 ±0.3
84.4 ±0.3
84.0 ±0.9
87.6 ±0.4
92.1 ±0.7
72.5 ±0.6
52.6 ±0.4
64.8 ±1.5
49.0 ±0.7
69.5 ±0.5
GAT
56.4 ±0.8
84.0 ±0.6
85.3 ±0.5
88.6 ±0.2
92.5 ±0.5
72.7 ±0.8
53.2 ±0.6
65.9 ±1.4
52.3 ±0.4
73.1 ±0.7
GraphSAGE
58.5 ±0.1
86.1 ±0.8
82.9 ±0.7
87.9 ±0.8
92.8 ±0.5
72.9 ±0.8
54.8 ±0.5
64.2 ±1.3
50.1 ±1.0
71.4 ±0.6
DGT
57.4 ±0.7
86.2 ±0.3
85.1 ±0.5
87.3 ±0.3
92.0 ±0.5
74.6 ±0.4
54.5 ±0.3
67.3 ±1.0
51.4 ±1.2
71.5 ±0.5
DGI
54.9 ±0.5
84.9 ±0.3
82.7 ±0.6
86.9 ±0.4
92.9 ±0.7
72.1 ±0.5
14.7 ±0.9
11.2 ±1.4
12.1 ±0.7
21.0 ±1.2
TABLE III: Node classification results (%) on text-attributed graphs. The best and second-best performances are highlighted as boldfaced and underlined , respectively.
Text-Attributed Graphs (TAGs) combine textual semantics with graph structure and are central to many graph learning tasks. However, existing fusion methods often treat text and structure as separate inputs in a shallow, one-way pipeline, which limits deep interaction between modalities and weakens performance under sparse connectivity or cross-graph generalisation. To address this issue, we propose PromptGNN-sim, a bi-directional structure-semantic fusion framework for collaborative GNN-LLM learning. PromptGNN-sim uses a Graph Attention Network (GAT) for semantically aware neighborhood selection by combining structural attention with textual similarity. The selected structural context is then used to generate structure-aware prompts for an LLM, including the target node summary, label categories, and representative keywords from similar neighbors. During training, bi-directional cross-modal contrastive learning and cross-attention are introduced to jointly optimize the GNN and LLM components. Experiments on six public datasets, including Cora, Pubmed, and WikiCS, evaluate accuracy, generalisation, and robustness under cross-task transfer, cross-dataset generalisation, and sparse perturbations. Results show that PromptGNN-sim outperforms classical GNNs, LLMs, and recent GNN-LLM fusion methods, demonstrating the effectiveness of interactive structure-semantic collaboration for text-attributed graph learning.
Text-attributed graphs (TAGs), where each node carries a natural language description, require models to jointly reason over text and graph topology. Existing approaches often handle the two modalities separately: graph neural networks operate on shallow text features, while hybrids of LLMs and graphs use the language model mainly as a text encoder and delegate structure learning to a separate graph module. We propose method that unifies textual reasoning and graph message passing within a masked diffusion language model, a language model with bidirectional attention and generative decoding. For each graph instance, method linearises a sampled local neighbourhood into a token sequence and injects graph structure through a topology attention mask, which realises message passing over the graph. Because the diffusion language model can both interpret and generate text, the method adapts to different tasks simply by changing the prompt, supporting node classification, link prediction, and cross-dataset transfer with no target-specific fine-tuning. Experiments show that method outperforms graph neural networks, graph transformers, and LLM-based baselines on all three TAG benchmarks across two tasks, improving over the strongest baseline by up to 3.9 points.
Text-rich graphs, which integrate complex structural dependencies with abundant textual information, are ubiquitous yet remain challenging for existing learning paradigms. Conventional methods and even LLM-hybrids compress rich text into static embeddings or summaries before structural reasoning, creating an information bottleneck and detaching updates from the raw content. We argue that in text-rich graphs, the text is not merely a node attribute but the primary medium through which structural relationships are manifested. We introduce RAMP, a Raw-text Anchored Message Passing approach that moves beyond using LLMs as mere feature extractors and instead recasts the LLM itself as a graph-native aggregation operator. RAMP exploits the text-rich nature of the graph via a novel dual-representation scheme: it anchors inference on each node's raw text during each iteration while propagating dynamically optimized messages from neighbors. It further handles both discriminative and generative tasks under a single unified generative formulation. Extensive experiments show that RAMP effectively bridges the gap between graph propagation and deep text reasoning, achieving competitive performance and offering new insights into the role of LLMs as graph kernels for general-purpose graph learning.