Towards a Learnable Retriever for Textual Graph Question Answering

Kaifeng Jin; Yuchen Yan; Zhichen Zeng; Xiao Lin; Lihui Liu; Yinglong Xia; Hanghang Tong

cs.CLOct 24, 2026

Towards a Learnable Retriever for Textual Graph Question Answering

Authors: Kaifeng Jin, Yuchen Yan, Zhichen Zeng, Xiao Lin, Lihui Liu, Yinglong Xia, Hanghang Tong

Organizations: University of Illinois Urbana-Champaign · Wayne State University · Meta

Abstract

Textual graph question answering (GraphQA) and graph-based retrieval-augmented generation (GraphRAG) have gained increasing attention as a way to ground large language models (LLMs) on structured knowledge. Despite extensive efforts, existing methods still face two key gaps. The first is an architectural gap: existing methods typically fine-tune an LLM-based generator, but do not also make the retriever itself a learnable LLM for graph evidence selection within one end-to-end retrieve-then-generate pipeline. The second is a retrieval-supervision gap: existing retrievers rarely use gold answer signals to supervise evidence selection. As a result, they may fail to recall evidence nodes required for reasoning, resulting in insufficient coverage of the gold answer set; meanwhile, naive attempts to boost recall often introduce excessive irrelevant information that compromises compactness and overwhelms the LLM. To address these gaps, we propose TaLenT, an LLM-based end-to-end retriever-generator framework for textual GraphQA. To bridge the architectural gap, TaLenT pairs a learnable LLM-based retriever for evidence selection with a fine-tuned LLM-based generator within one unified retrieve-then-generate pipeline. To bridge the retrieval-supervision gap, we leverage gold answer signals as retriever supervision. Specifically, we stratify graph nodes into gold answers, answer-related nodes, and irrelevant noise. We then adopt a weighted pseudo-label loss to prioritize gold answers and preserve answer-related nodes for coverage, while discarding irrelevant noise for compactness. Experimental results across three GraphQA datasets demonstrate that TaLenT consistently improves downstream performance.

Figures & tables

Figure 1: An illustrative example of two limitations of query-based graph retrieval. For the query “What was Isaac Newton’s profession?”, a query-similarity retriever returns a Newton-centric subgraph (blue circle). We partition the nodes into three groups: Gold answers (only a small subset of the gold answer set is retrieved by the subgraph in the blue circle, showing limited coverage ), Answer-related nodes (topically relevant but not exact gold answers), and Irrelevant noise (nodes that match the query tokens such as “Newton” yet are not answer-related). The presence of substantial noise alongside partial gold coverage highlights the compactness issue: improving recall by expanding query-similar candidates can inflate the subgraph with distractors.

Figure 2: Overview of TaLenT , an LLM-based end-to-end retriever-generator framework for textual GraphQA. The full TaLenT pipeline contains three stages: (1) Preprocessing, which applies PCST to construct an initial candidate connected subgraph $\mathcal{S}$ from the knowledge base $\mathcal{G}$ based on query $q$ . (2) Retriever training, which trains a learnable LLM-based retriever $R_{\theta}$ with weighted pseudo-label supervision, given the candidate subgraph $\mathcal{S}$ and query $q$ , to produce $\hat{\mathcal{V}}$ . (3) Generator training, which fine-tunes an LLM-based generator $G_{\phi}$ with standard SFT given retrieved node set $\hat{\mathcal{V}}$ and $q$ , to produce the final answer $a$ .

Dataset (Metrics)	ExplaGraphs (ACC)	WebQSP (Hit@1)	CWQ (Hit@1)
Zero-shot	56.50	41.06	18.58
Zero-CoT ( Kojima et al. 2022 )	57.04	51.30	25.63
CoT-BAG ( Wang et al. 2023a )	57.94	39.60	18.83
KAPING ( Baek et al. 2023 )	62.27	52.64	23.25
Graph-based Inference	33.93	47.22	18.21
Frozen LLM + Prompt Tuning (PT)	57.63	48.34	36.22

Table 1: Benchmark results (%).

Figure 3: WebQSP test-set retrieval results (macro-averaged). We measure how well PCST subgraphs and TaLenT ’s learned retriever output

\hat{\mathcal{V}}

cover the gold answer set

\mathcal{V}^{\star}

. We compare: (1) the PCST candidate subgraph with

k_{v}{=}k_{e}{=}10

(this is the retriever input), (2) a much smaller PCST subgraph with

k_{v}{=}k_{e}{=}3

(a size-matched reference), and (3) the nodes set

\hat{\mathcal{V}}

selected by TaLenT ’s learned retriever. Our retriever keeps coverage close to PCST(

k_{v}{=}k_{e}{=}10

) while using far fewer nodes, and it also covers more gold nodes than the size-matched PCST(

k_{v}{=}k_{e}{=}3

Output subset	Hit@1	Recall
All retrieved nodes	80.28	67.84
Nodes within PCST	75.31	64.19
Nodes out of PCST	10.26	5.46

Table 2: Breakdown of gold answer coverage contributed by different parts of the learned retriever output on WebQSP test. All metrics are macro-averaged over examples and reported as percentages.

Retriever setting	Hit@1	Prec	Recall	F1
TaLenT	80.28	24.55	67.84	26.63
Ablate retriever training	36.61	14.97	26.26	14.97

Table 3: Retriever ablation on WebQSP test set (macro-averaged over examples). “Ablate retriever training” replaces TaLenT ’s learnable retriever by a frozen retriever without LoRA fine-tuning.

System	Hit@1 (%)
TaLenT w/o retriever training	68.37
TaLenT w/o retriever	72.50
TaLenT	76.97

Table 4: QA ablations on WebQSP (Hit@1, %). All systems follow the same pipeline as TaLenT except for the ablated component stated in each row.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	ExplaGraphs	WebQSP	CWQ
#Graphs	2,766	4,737	34,689
Average #Nodes	5.17	1370.89	1255.71
Average #Edges	4.25	4252.37	4000.86
Node Attribute	Commonsense concepts	Entities in Freebase	Entities in Freebase
Edge Attribute	Commonsense relations	Relations in Freebase	Relations in Freebase
Task	Commonsense reasoning	KGQA	KGQA

Appendix

Table 5: Statistics of datasets.

$\lambda$	Hit@1 (%)	Avg. #Retrieved Nodes
0	76.23	10.19
1e-5	76.66	10.38
0.001	76.97	12.79
0.1	76.23	22.42
1.0	75.55	25.25

Appendix

Table 6: Sensitivity analysis on the weak-positive weight

\lambda

on WebQSP, with PCST input fixed to

k_{v}=k_{e}=10

. We report Hit@1 and the average number of retrieved nodes on the test set; smaller retrieved node sets indicate better compactness. Larger

\lambda

values lead to less compact retrieval.

PCST input $k_{v}=k_{e}$	Hit@1 (%)
3	74.51
5	75.43
10	76.97

Appendix

Table 7: Sensitivity analysis on the PCST candidate subgraph size on WebQSP, with

\lambda

fixed to

0.001

Method	LLM-based end-to-end	Coverage-compactness	KGQA	Label-prediction GraphQA
TaLenT	✓	✓	✓	✓
QA-GNN	✗	✗	✗	✗
GNN-RAG	✗	✗	✓	✗
RoG	✗	✗	✓	✗
CG-RAG	✗	✗	✗	✗
SentGraph	✗	✗	✗	✗

Appendix

Table 8: Comparison of TaLenT with answer-related and other recent graph retrieval methods discussed in Appendix D .

Figure 4: WebQSP test-set retrieval results with Llama-3-8B (macro-averaged). We measure how well PCST subgraphs and TaLenT ’s learned retriever output

\hat{\mathcal{V}}

cover the gold answer set

\mathcal{V}^{\star}

. We compare: (1) the PCST candidate subgraph with

k_{v}{=}k_{e}{=}10

(the retriever input), (2) a smaller PCST subgraph with

k_{v}{=}k_{e}{=}5

(a size-matched reference), and (3) the node set

\hat{\mathcal{V}}

selected by TaLenT ’s learned retriever. Our retriever maintains comparable gold-answer coverage with substantially fewer nodes than the original PCST candidate subgraph (

k_{v}{=}k_{e}{=}10

), while achieving higher coverage than the size-matched PCST subgraph (

k_{v}{=}k_{e}{=}5

Method	Hit@1
G-Retriever w/ LoRA	79.36
TaLenT	83.05

Appendix

Table 9: WebQSP downstream QA results with Llama-3-8B, comparing TaLenT against the strongest baseline G-Retriever w/ LoRA in Table 1 (Hit@1, %).

Output subset	Hit@1	Recall
All retrieved nodes	81.45	70.40
Nodes within PCST	76.90	67.35
Nodes out of PCST	8.29	4.28

Appendix

Table 10: Breakdown of gold-answer coverage contributed by different parts of the learned retriever output on the WebQSP test set with Llama-3-8B (

\lambda=0.001

). All metrics are macro-averaged over examples and reported as percentages.

Graph-Based Retrieval

Graph-based retrieval uses links among documents, entities, or vectors to find connected evidence beyond isolated similarity matches.

See all 142 papers

SelfGraphRAG: Bridging the Supervision Gap in Graph-Based RAG with Synthetic QA Generation
Aug 25, 2026Ben Lagnese, Manas GaurGraph Retrieval-Augmented Generation Graph-Based Retrieval
University of Maryland, Baltimore County (UMBC), Baltimore, MD, USA
UniGFM-RAG: Unify Graph Foundation Models and Large Language Models for End-to-End Retrieval and Reasoning
Oct 26, 2026Zicheng Zhao, Linhao Luo, Zongji Yu +4Graph Retrieval-Augmented Generation LLM Reasoning with Graphs
School of Automation and Intelligent Sensing, Shanghai Jiao Tong University, Shanghai 200240, China. · Department of Data Science and AI, Monash University, Melbourne, Australia. · Tencent Youtu Lab, Shanghai 200240, China. +2
Mixture of Demonstrations for Textual Graph Understanding and Question Answering
Mar 23, 2026Yukun Wu, Lihui LiuGraph Retrieval-Augmented Generation Retrieval-Augmented Generation
Wayne State University

Aug 25, 2026J/K move · Enter open · S save

SelfGraphRAG: Bridging the Supervision Gap in Graph-Based RAG with Synthetic QA Generation

Ben Lagnese, Manas Gaur

University of Maryland, Baltimore County (UMBC), Baltimore, MD, USA

Graph Retrieval-Augmented Generation Graph-Based Retrieval Graph SSL Multi-Hop QA

Open paper AskPDF arXiv

Retrieval-augmented generation (RAG) improves large language models by incorporating external knowledge without retraining, but existing methods often underuse the relational structure encoded in knowledge graphs. Graph-based RAG can capture entity relationships, yet supervised graph retrieval typically requires labeled question-answer data that may not be available for newly constructed graphs. We address this limitation with SelfGraphRAG, a framework that generates question-answer pairs directly from knowledge graph structure and uses them to train a query-conditioned graph retriever. The generated questions capture multi-hop paths and local neighborhoods, providing relational supervision without manual annotation. Experiments on multi-hop question answering and classification benchmarks show that SelfGraphRAG improves retrieval precision and downstream reasoning performance over embedding-based baselines. These results suggest that knowledge graph structure can provide useful supervision for training graph retrievers when labeled data are unavailable.

Abstract

Graph-Based Retrieval

Explore similar work

SelfGraphRAG: Bridging the Supervision Gap in Graph-Based RAG with Synthetic QA Generation

UniGFM-RAG: Unify Graph Foundation Models and Large Language Models for End-to-End Retrieval and Reasoning

Mixture of Demonstrations for Textual Graph Understanding and Question Answering