DynamicGTR: Leveraging Graph Topology Representation Preferences to Boost VLM Capabilities on Graph QAs

Yanbin Wei; Jiangyue Yan; Chun Kang; Yang Chen; Hua Liu; James Kwok; Yu Zhang

cs.CVFeb 25, 2026

DynamicGTR: Leveraging Graph Topology Representation Preferences to Boost VLM Capabilities on Graph QAs

Authors: Yanbin Wei, Jiangyue Yan, Chun Kang, Yang Chen, Hua Liu, James Kwok, Yu Zhang

Organizations: Southern University of Science and Technology · Hong Kong University of Science and Technology · Harbin Institute of Technology (Shenzhen) · Beihang University

Abstract

Vision-Language Models (VLMs) have emerged as versatile solutions for zero-shot question answering (QA) across various domains. However, enabling VLMs to effectively comprehend structured graphs and perform accurate, efficient QA remains challenging. Existing approaches typically rely on one single graph topology representation (GTR), such as fixed-style visual images or unified text descriptions. This ``one-size-fits-all'' strategy often neglects model-specific and task-specific preferences, resulting in inaccurate or over-lengthy responses to graph-related queries. To address this, we propose the $\mbox{DynamicGTR}$ framework, which dynamically selects the optimal GTR for each query during inference, thereby enhancing the zero-shot graph QA capabilities of VLMs with a customizable accuracy and brevity trade-off. Extensive experiments show that DynamicGTR not only improves VLM-based graph algorithm QA performance but also successfully transfers the experience trained from synthetic graph algorithm tasks to real-world applications like link prediction and node classification, without any additional training. Additionally, DynamicGTR demonstrates strong transferability across tasks, domains, and models, suggesting its potential as a flexible solution for broad graph scenarios.

Figures & tables

Figure 1: Illustrating the impacts of diverse GTRs on the same question: suboptimal GTRs may lead to incorrect answers or unnecessarily lengthy responses compared to a preferred GTR.

GTR	Type	Works	Charac.	Enc.	Train
Embeddings	Embed	GraphLLM [ 6 ] , GraphGPT [ 42 ] , GraphToken [ 39 ] , LLaGA [ 9 ]	Partial, Informative	Yes	Yes
Visual Graph	Visual	GITA [ 51 ] , VisionGraph [ 36 ]	Full, Intuitive, Explicit	No	No
Edge Set	Textual	GraphWiz [ 8 ] , NLGraph [ 46 ] , GPT4Graph [ 21 ] , GraphArena [ 43 ] , GCoder [ 58 ]	Full, Sequential, Implicit	No	No
Adjacent List	Textual	InstructGLM [ 56 ] , GraphText [ 59 ]	Full, Sequential, Implicit	No	No
Adjacent Matrix	Textual	GraphDPR [ 36 ]	Full, Redundant, Implicit	No	No

Table 1: Summary of existing GTRs. ‘Charac.’ indicates the characteristics; ‘Enc.’ and ‘Train’ denote whether the GTR is generated by an external encoder and whether extra training is required, respectively. ‘Embed’ is short for ‘Embedding’.

Figure 2: An Illustration of eight candidate GTRs in $\mathcal{R}_{ZS}$ .

Figure 3: Overview of the DynamicGTR framework, where the GTR Router guides the VLM Reasoner to use the most appropriate GTR based on the question.

Model	GTRs	Conn	Cyc	TS	SP	MF	BGM	HP
GPT-4o	1st	$V_{fdp}$ (92.3%)	$V_{sfdp}$ (85.1%)	$T_{set}$ (58.5%)	$T_{set}$ (19.5%)	$T_{list}$ (21.7%)	$V_{dot}$ (34.2%)	$T_{list}$ (30.4%)
	2nd	$V_{neato}$ (92.1%)	$V_{fdp}$ (12.9%)	$T_{list}$ (36.2%)	$V_{neato}$ (18.7%)	$T_{set}$ (20.8%)	$V_{circo}$ (20.3%)	$T_{set}$ (20.3%)
	3rd	$V_{sfdp}$ (91.7%)	$V_{dot}$ (12.2%)	$V_{dot}$ (23.1%)	$T_{list}$ (17.1%)	$T_{mat}$ (16.7%)	$V_{neato}$ (19.8%)	$V_{circo}$ (18.8%)
Gemini 2.5 Pro	1st	$V_{neato}$ (88.8%)	$V_{neato}$ (97.2%)	$T_{set}$ (41.0%)	$T_{list}$ (48.6%)	$T_{mat}$ (40.0%)	$V_{fdp}$ (24.4%)	$T_{list}$ (42.9%)
	2nd	$V_{fdp}$ (88.2%)	$V_{sfdp}$ (93.0%)	$T_{list}$ (30.3%)	$T_{mat}$ (37.6%)	$T_{list}$ (36.0%)	$V_{sfdp}$ (14.5%)	$T_{set}$ (31.7%)
	3rd	$V_{sfdp}$ (88.2%)	$V_{dot}$ (71.8%)	$T_{mat}$ (15.4%)	$T_{set}$ (26.6%)	$T_{set}$ (14.0%)	$V_{neato}$ (14.5%)	$T_{mat}$ (30.2%)

Table 2: Task-preferred Top-3 GTRs with frequency in GTRP dataset (GPT-4o and Gemini-2.5 Pro). ‘Conn’, ‘Cyc’, ‘TS’, ‘SP’, ‘MF’, ‘BGM’, and ‘HP’ denote connectivity, cycle detection, topological sort, shortest path, maximum flow, bipartite graph matching, and Hamilton path. By differing visual or textual GTRs with colors, the special preference patterns of tasks are explicitly exposed .

	Conn		Cyc		TS		SP		MF		BGM		HP
Method
	Acc	Tok	Acc	Tok	Acc	Tok	Acc	Tok	Acc	Tok	Acc	Tok	Acc	Tok
GPT-4o
CoT	92.5	273.3	52.7	480.6	36.6	224.2	54.6	566.0	25.3	362.9	69.5	370.1	50.0	124.9
NLGraph	92.9	296.6	60.2	337.9	36.2	202.6	59.0	533.7	25.6	335.1	62.2	356.3	57.1	176.5
GraphDPR	94.3	412.2	68.7	626.4	40.2	411.0	59.1	496.3	31.2	502.7	76.5	582.1	62.7	475.8

Table 3: Zero-shot capabilities on in-domain graph algorithmic tasks. Acc and Tok refer to task-average accuracy (%) and token consumption, respectively.

	ca-GrQC		ca-HepTh		ogbl-ppa		PolBlog		Cora		ogbn-product
Method
	Acc	Tok	Acc	Tok	Acc	Tok	Acc	Tok	Acc	Tok	Acc	Tok
GPT-4o	66.2	191.2	77.4	229.8	42.6	199.4	59.2	255.7	52.2	228.5	33.2	192.6
+CoT	68.6	244.0	77.8	268.6	48.2	238.1	60.3	311.2	54.4	272.2	33.1	199.9
+GITA	73.1	203.1	83.1	245.9	52.8	266.2	63.6	226.6	56.8	184.4	35.5	233.4
+DynamicGTR	76.5	143.2	86.3	183.4	54.4	206.3	71.9	153.0	64.5	99.2	40.1	156.1

Table 4: Zero-shot capacities on out-of-domain real-world applications across six datasets. Acc and Tok refer to task-average accuracy (%) and token consumption, respectively.

		Conn		Cyc		TS		SP		MF		BGM		HP
GTR	Avg.GRE
		Acc	Tok	Acc	Tok	Acc	Tok	Acc	Tok	Acc	Tok	Acc	Tok	Acc	Tok
$V_{dot}$	1.16	78.5	8.4	80.0	8.0	12.3	346.0	14.5	146.3	7.5	410.2	91.2	244.3	13.3	40.0
$V_{neato}$	1.20	95.1	7.7	87.1	8.0	3.0	30.0	20.0	130.1	10.8	387.4	87.2	280.0	11.1	40.0
$V_{circo}$	1.13	89.7	8.2	70.7	8.0	3.0	30.0	18.2	153.5	8.1	421.3	88.2	271.6	14.4	40.0
$V_{fdp}$	0.88	96.0	8.1	60.5	8.0	3.0	30.0	17.3	150.5	8.1	389.2	82.9	295.3	4.4	74.0

Table 5: Comparison of the DynamicGTR framework with GTR Router versus single GTRs on in-domain tasks (GPT-4o). Additional results for more VLMs in Appendix G show similar trends.

Metric	setting	I.D.	LP	NC
	$\alpha=0$	69.6	73.1	59.3
Acc $\uparrow$	$\alpha=0.5$	69.5	72.4	58.8
	$\alpha=1$	69.3	72.0	58.6
	Best Baseline	61.8	69.7	52.0
	$\alpha=0$	290.8	223.2	168.8
Tok $\downarrow$	$\alpha=0.5$	212.0	177.6	136.1

Table 6: The accuracy and token consumption with setting different

\alpha

values. The base VLM is GPT-4o.

Metric	setting	I.D.	LP	NC
	$k=1$	62.4	70.6	54.2
Acc $\uparrow$	$k=5$	66.2	72.1	57.8
	$k=10$	69.5	72.4	58.8
	Best Baseline	61.8	69.7	52.0
	$k=1$	286.2	198.2	196.6
Tok $\downarrow$	$k=5$	232.4	181.6	158.9

Table 7: The accuracy and token consumption with setting different

k

values. The base VLM is GPT-4o.

Metric	I.D.	LP	NC	Avg.
Transfer Gemini-2.5 Pro’s Router to GPT-4o
$\Delta\text{Acc}\uparrow$	-0.3(+6.1)	-0.1(+3.2)	-1.4(+6.6)	-0.6(+5.3)
$\Delta\text{Tok}\downarrow$	-35.5(-123.4)	-9.2(-56.4)	-40.1(-119.5)	-28.3(-99.8)
Transfer GPT-4o’s Router to Gemini-2.5 Pro
$\Delta\text{Acc}\uparrow$	+0.1(+1.9)	+0.5(+4.0)	-0.1(+9.6)	+0.2(+5.2)
$\Delta\text{Tok}\downarrow$	+14.4(-216.2)	+26.9(-47.5)	+3.2(-115.6)	+14.8(-126.4)

Table 8: Performance differences when a GTR router trained for one VLM is applied to another VLM (i.e., transfer).

Figure D : An Illustration of eight candidate GTRs in $\mathcal{R}_{ZS}$ .

Model	GTRs	Conn	Cyc	TS	SP	MF	BGM	HP
GPT-4o	1st	$V_{fdp}$ (92.3%)	$V_{sfdp}$ (85.1%)	$T_{set}$ (58.5%)	$T_{set}$ (19.5%)	$T_{list}$ (21.7%)	$V_{dot}$ (34.2%)	$T_{list}$ (30.4%)
	2nd	$V_{neato}$ (92.1%)	$V_{fdp}$ (12.9%)	$T_{list}$ (36.2%)	$V_{neato}$ (18.7%)	$T_{set}$ (20.8%)	$V_{circo}$ (20.3%)	$T_{set}$ (20.3%)
	3rd	$V_{sfdp}$ (91.7%)	$V_{dot}$ (12.2%)	$V_{dot}$ (23.1%)	$T_{list}$ (17.1%)	$T_{mat}$ (16.7%)	$V_{neato}$ (19.8%)	$V_{circo}$ (18.8%)
	4th	$V_{circo}$ (84.4%)	$V_{circo}$ (10.5%)	$T_{mat}$ (0.8%)	$V_{fdp}$ (16.3%)	$V_{neato}$ (10.8%)	$V_{sfdp}$ (13.4%)	$V_{dot}$ (17.4%)
	5th	$V_{dot}$ (73.0%)	$V_{fdp}$ (61.1%)	$V_{sfdp}$ (0.8%)	$V_{circo}$ (13.8%)	$V_{fdp}$ (8.3%)	$V_{fdp}$ (9.1%)	$V_{sfdp}$ (15.9%)
	6th	$T_{list}$ (27.0%)	$T_{list}$ (10.8%)	$V_{neato}$ (0.8%)	$V_{sfdp}$ (12.2%)	$V_{dot}$ (7.5%)	$T_{list}$ (7.0%)	$V_{neato}$ (14.5%)

Table I: Complete ranking of GTRs with respect to their label frequency in the GTRP dataset of GPT-4o and Gemini-2.5 Pro. By differing visual or textual GTRs with colors, the special preference patterns of tasks are explicitly exposed .

	Conn	Cyc	TS	SP	MF	BGM	HP
#sample	16,410	4,100	2,910	1,560	1,500	1,860	900
#nodes	25.01	23.42	21.86	13.65	13.90	21.13	13.24
#edges	95.46	23.66	114.10	23.99	49.16	51.03	45.05

Table J : Data Statistics for in-domain graph algorithmic QA tasks.

	ca-GrQC	ca-HepTh	ogbl-ppa	PolBlogs	Cora	ogbn-product
# Nodes	5,242	9,877	576,289	1,490	2,708	2,449,029
# Edges	14,496	25,998	30,326,273	19,025	5,278	61,859,140
domain	collaboration	collaboration	protein association	social	citation	e-commerce
average degree	5.53	5.26	105.25	25.54	3.9	50.52

Table K : Dataset Statistics for out-of-domain real-world applications

	Conn		Cyc		TS		SP		MF		BGM		HP
Method
	Acc	Tok	Acc	Tok	Acc	Tok	Acc	Tok	Acc	Tok	Acc	Tok	Acc	Tok
LLaVA-OneVision-7B
CoT	85.2	312.5	45.3	528.8	30.1	268.4	48.2	612.7	18.7	405.3	60.3	412.8	42.5	156.3
NLGraph	85.7	338.9	52.1	382.4	29.8	241.7	52.6	586.2	19.2	378.5	54.8	398.6	48.9	212.7
GraphDPR	88.6	456.3	60.5	689.2	34.7	458.9	53.9	547.8	25.3	556.2	68.9	635.7	55.2	521.4

Table L: Zero-shot capabilities on in-domain graph algorithmic tasks for LLaVA-OneVision-7B and Qwen3-VL-8B. Acc and Tok refer to task-average accuracy (%) and token consumption, respectively.

	ca-GrQC		ca-HepTh		ogbl-ppa		PolBlog		Cora		ogbn-product
Method
	Acc	Tok	Acc	Tok	Acc	Tok	Acc	Tok	Acc	Tok	Acc	Tok
LLaVA-OneVision-7B	58.4	229.7	69.2	276.3	34.8	241.8	51.5	298.4	44.7	265.9	26.8	235.2
+CoT	60.9	278.5	69.7	312.8	40.3	285.4	52.8	356.7	46.9	312.5	26.7	243.8
+GITA	65.7	245.3	75.8	298.6	45.2	312.9	56.2	269.3	49.2	221.7	29.3	278.6
+DynamicGTR	69.3	172.5	79.5	228.4	47.6	253.7	63.8	189.6	56.3	128.9	33.5	192.4

Table M: Zero-shot capacities on out-of-domain real-world applications for LLaVA-OneVision-7B and Qwen3-VL-8B. Acc and Tok refer to task-average accuracy (%) and token consumption, respectively.

Tasks	Task Instruction
Connectivity	Is there a path between node [P] and node [P] in this undirected graph?
Cycle	Is there a cycle in this undirected graph?
Topological Sort	This representation depicts a directed graph, in which each directed edge from node A to node B signifies that, according to the topological order, node A must precede node B. Q: The topological order of the directed graph is:
Shortest Path	This representation illustrates a directed graph, with each edge’s capacity indicated by a numerical label in close proximity.Q: What is the maximum flow from node 4 to node 0:
Maximum Flow	This representation illustrates a directed graph, with each edge’s capacity indicated by a numerical label in close proximity. Q: What is the maximum flow from node [P] to node [P]:
Bipartite Graph Matching	There are [P] hosts numbered from [P] to [P], and [P] tasks numbered from [P] to [P]. Each host has a set of tasks that it is interested in, represented by arrows from a host to a task in the diagram. However, each host is capable of solving only one task, and similarly, each task can be resolved by just one host. Q: What is the maximum number of hosts that can be assigned a task they are interested in?

Table N : Unified Task Instructions in experimental comparisons, where [P]s are placeholders that will be substituted in specific questions.

Tasks	Task Instruction
Connectivity/ Cycle/ Link Predict	Please put the answer between <answer> and </answer> tags. For example, <answer>Yes</answer> or <answer>No</answer>.
Topological Sort/ Shortest Path/ Hamilton Path	Please put the answer between <answer> and </answer> tags. For example, <answer>0->1->2->3->4</answer> or <answer>0->1->3->7->8->4->6->5->9->2</answer>.
Maximum Flow	Please put the answer between <answer> and </answer> tags. For example, <answer>3</answer> or <answer>8</answer>.
Node Classify	Please put the answer between <answer> and </answer> tags. For example, <answer>Class 1</answer> or <answer>Class 3</answer>.

Table O : Control Task Instructions in experimental comparisons.

		Conn		Cyc		TS		SP		MF		BGM		HP
GTR	Avg.GRE
		Acc	Tok	Acc	Tok	Acc	Tok	Acc	Tok	Acc	Tok	Acc	Tok	Acc	Tok
LLaVA-OneVision-7B
$V_{dot}$	1.11	72.3	9.1	68.5	8.0	10.1	35.2	12.7	168.5	6.2	435.8	80.1	265.9	11.5	45.2
$V_{neato}$	1.09	88.6	7.9	76.3	8.0	2.8	35.2	18.5	152.3	9.5	408.7	83.4	281.7	9.8	42.7
$V_{circo}$	0.99	80.5	8.6	62.7	8.0	2.5	35.2	16.3	175.8	7.8	442.1	78.9	278.4	12.9	42.7

Table P: Comparison of the DynamicGTR framework with GTR Router versus single GTRs on in-domain tasks across diverse leading VLMs.

Tasks	Template
Connectivity/ Cycle/ Hamilton Path	In an undirected graph, (i,j) means that node i and node j are connected with an undirected edge. The nodes are numbered from [P] to [P], and the edges are: ([P], [P]) , ([P], [P])...
Topological Sort	In a directed graph with [P] nodes numbered from [P] to [P]: node [P] should be visited before node [P] node [P] should be visited before node [P] ...
Shortest Path	In an undirected graph, the nodes are numbered from [P] to [P], and the edges are: an edge between node [P] and node [P] with weight [P], an edge between node [P] and node [P] with weight [P], ...
Maximum Flow	In a directed graph, the nodes are numbered from [P] to [P], and the edges are: an edge from node [P] to node [P] with capacity [P], an edge from node [P] to node [P] with capacity [P], ...
Bipartite Graph Matching	There are [P] hosts numbered from [P] to [P], and [P] tasks numbered from [P] to [P]. Each host has a set of tasks that it is interested in: Host [P] is interested in task [P]. Host [P] is interested in task [P]. ...
Link Predict/ Node Classify	In an undirected graph, (i,j) means that node i and node j are connected with an undirected edge. The nodes are numbered from [P] to [P], and the edges are: ([P], [P]) , ([P], [P])... The node attributes are: Node [P], Attribute [P] Node [P], Attribute [P] ...

Table Q : Prompt templates of

T_{set}

, where [P]s are placeholders that will be substituted for specific graph topology.

Tasks	Template
Connectivity/ Cycle/ Hamilton Path	In an undirected graph, (i,j) means that node i and node j are connected with an undirected edge. The nodes are numbered from [P] to [P], and the edges are presented in an adjacent list format: [P] <-> [P], [P], [P],... [P] <-> [P], [P], [P],... ...
Topological Sort	In a directed graph with [P] nodes numbered from [P] to [P]: node [P] should be visited before node [P], [P], [P],... node [P] should be visited before node [P], [P], [P],... ...
Shortest Path	In an undirected graph, the nodes are numbered from [P] to [P], and the edges are presented in an adjacent list format: node [P] is connected to: node [P] with distance: [P], node [P] with distance: [P],... node [P] is connected to: node [P] with distance: [P], node [P] with distance: [P],... ...
Maximum Flow	In a directed graph, the nodes are numbered from [P] to [P], and the edges are presented in an adjacent list format: node [P] is connected to: node [P] with capacity: [P], node [P] with capacity: [P],... node [P] is connected to: node [P] with capacity: [P], node [P] with capacity: [P],... ...
Bipartite Graph Matching	There are [P] hosts numbered from [P] to [P], and [P] tasks numbered from [P] to [P]. Each host has a set of tasks that it is interested in: Host [P] is interested in tasks [P], [P], [P],.... Host [P] is interested in task [P], [P], [P],.... ...
Link Predict/ Node Classify	In an undirected graph, (i,j) means that node i and node j are connected with an undirected edge. The nodes are numbered from [P] to [P], and the edges are presented in an adjacent list format: [P] <-> [P], [P], [P],... [P] <-> [P], [P], [P],... ... The node attributes are: Node [P], Attribute [P] Node [P], Attribute [P] ...

Table R : Prompt templates of

T_{list}

, where [P]s are placeholders that will be substituted for specific graph topology.

Tasks	Template
Connectivity/ Cycle/ Hamilton Path	In an undirected graph, (i,j) means that node i and node j are connected with an undirected edge. The nodes are numbered from [P] to [P], and the edges are represented in an adjacent matrix format : node0 1 2... node0 [P] [P] [P],... node1 [P] [P] [P],... node2 [P] [P] [P],... ...
Topological Sort	In a directed graph with [P] nodes numbered from [P] to [P], the edges are represented in an adjacent matrix format : node0 1 2... node0 [P] [P] [P],... node1 [P] [P] [P],... node2 [P] [P] [P],... ...
Shortest Path/ Maximum Flow	In an undirected graph, the nodes are numbered from [P] to [P], and the edges are represented in an adjacent matrix format with weights : node0 1 2... node0 [P] [P] [P],... node1 [P] [P] [P],... node2 [P] [P] [P],... ...
Bipartite Graph Matching	There are [P] hosts numbered from [P] to [P], and [P] tasks numbered from [P] to [P]. Each host has a set of tasks that it is interested in : Task0 1 2... Host0 [P] [P] [P],... Host1 [P] [P] [P],... Host2 [P] [P] [P],... ...
Link Predict/ Node Classify	In an undirected graph, (i,j) means that node i and node j are connected with an undirected edge. The nodes are numbered from [P] to [P], and the edges are represented in an adjacent matrix format : node0 1 2... node0 [P] [P] [P],... node1 [P] [P] [P],... node2 [P] [P] [P],... ... The node attributes are: Node [P], Attribute [P] Node [P], Attribute [P] ...

Table S : Prompt templates of

T_{mat}

, where [P]s are placeholders that will be substituted for specific graph topology.

Graph Reasoning

Graph reasoning uses relationships among entities to solve multi-step inference, prediction, planning, and question-answering tasks.

See all 64 papers

Teaching LLMs to See Graphs: Unifying Text and Structural Reasoning
May 11, 2026Dario VajdaLLM Reasoning with Graphs Text-Attributed Graph Learning
Faculty of Computer and Information Science University of Ljubljana
Towards a Learnable Retriever for Textual Graph Question Answering
Oct 24, 2026Kaifeng Jin, Yuchen Yan, Zhichen Zeng +4Graph-Based Retrieval Graph Retrieval-Augmented Generation
University of Illinois Urbana-Champaign · Wayne State University · Meta
Is One Token All It Takes? Graph Pooling Tokens for LLM-based GraphQA
Apr 1, 2026Ankit Grover, Lodovico Giaretta, Rémi Bourgerie +1LLM Reasoning with Graphs Graph Pooling
KTH Royal Institute of Technology, Stockholm, Sweden · RISE Research Institutes of Sweden, Stockholm, Sweden

May 11, 2026J/K move · Enter open · S save

Teaching LLMs to See Graphs: Unifying Text and Structural Reasoning

Dario Vajda

Faculty of Computer and Information Science University of Ljubljana

LLM Reasoning with Graphs Text-Attributed Graph Learning Graph Transformer LLM Reasoning LLMs for Graph Learning

Open paper AskPDF arXiv

Using Large Language Models (LLMs) to process graph-structured data is an active research area, yet current state-of-the-art approaches typically rely on multi-step pipelines with Graph Neural Network (GNN) encoders that compress rich textual attributes into solitary tokens, creating a significant semantic bottleneck. In this paper, we introduce the Graph Transformer Language Model (GTLM), a novel architecture that enables pretrained LLMs to natively process graph topologies while entirely eliminating this compressive bottleneck. GTLM is exceptionally parameter-efficient: by injecting graph-aware attention biases directly into the LLM's attention modules, it introduces only 0.015% additional parameters relative to the base model. We theoretically prove that our bidirectional attention prefix preserves node permutation equivariance while maintaining exact backward compatibility with the pretrained base model. Extensive evaluations demonstrate that a 1B-parameter GTLM matches or exceeds the performance of 7B-parameter state-of-the-art models on standard Text-Attributed Graph benchmarks, while significantly surpassing baselines on GraphQA. Finally, we demonstrate that GTLM attention heads implicitly learn to simulate message passing, explaining its superior performance on algorithmic tasks. This paradigm shift enables true algorithmic reasoning within LLMs and provides a scalable foundation for next-generation GraphRAG and relational deep learning.

Abstract

Graph Reasoning

Explore similar work

Teaching LLMs to See Graphs: Unifying Text and Structural Reasoning

Towards a Learnable Retriever for Textual Graph Question Answering

Is One Token All It Takes? Graph Pooling Tokens for LLM-based GraphQA