FGTR: Fine-Grained Multi-Table Retrieval via Hierarchical LLM Reasoning

Chaojie Sun; Bin Cao; Tiantian Li; Chenyu Hou; Ruizhe Li; Jing Fan

cs.IRMar 13, 2026

FGTR: Fine-Grained Multi-Table Retrieval via Hierarchical LLM Reasoning

Authors: Chaojie Sun, Bin Cao, Tiantian Li, Chenyu Hou, Ruizhe Li, Jing Fan

Organizations: Zhejiang University of Technology Hangzhou, China · University of Aberdeen Aberdeen, UK

Abstract

With the rapid advancement of large language models (LLMs), growing efforts have been made on LLM-based table retrieval. However, existing studies typically focus on single-table query, and implement it by similarity matching after encoding the entire table. These methods usually result in low accuracy due to their coarse-grained encoding which incorporates much query-irrelated data, and are also inefficient when dealing with large tables, failing to fully utilize the reasoning capabilities of LLM. Further, multi-table query is under-explored in retrieval tasks. To this end, we propose a hierarchical multi-table query method based on LLM: Fine-Grained Multi-Table Retrieval FGTR, a new retrieval paradigm that employs a human-like reasoning strategy. Through hierarchical reasoning, FGTR first identifies relevant schema elements and then retrieves the corresponding cell contents, ultimately constructing a concise and accurate sub-table that aligns with the given query. To comprehensively evaluate the performance of FGTR, we construct two new benchmark datasets based on Spider and BIRD . Experimental results show that FGTR outperforms previous state-of-the-art methods, improving the F_2 metric by 18% on Spider and 21% on BIRD, demonstrating its effectiveness in enhancing fine-grained retrieval and its potential to improve end-to-end performance on table-based downstream tasks.

Figures & tables

Panel of Figure 1 — Figure 1. Illustration of the human cognitive process for table comprehension and the motivation for fine-grained retrieval. (a) Humans typically adopt a hierarchical approach: they first scan schemas to identify relevant columns and then locate the specific rows needed, effectively creating a mental sub-table. (b) Performance comparison on BirdQA demonstrating the necessity of retrieval. Feeding the compact, retrieved sub-table significantly boosts QA accuracy, whereas feeding the full table leads to context window overflow and catastrophic failure.

Figure 2. The overall architecture of our FGTR framework, which consists of an offline preprocessing phase and a two-stage online retrieval process: Schema Retrieval and Cell Retrieval.

Method	Spider				BIRD
Method	R	P	$F_{2}$	SR	R	P	$F_{2}$	SR
Full-schema	100	11.94	100	100	100	8.87	100	100
Gold-Based	100	100	100	100	100	100	100	100
Backbone (Llama3-8b)
C3	70.74	74.83	71.52	32.45	52.00	75.00	55.40	28.34
FGTR	74.12	80.32	75.28	39.54	72.28	76.04	73.00	35.19

Table 1. Schema retrieval results on Spider and BIRD datasets under different backbone models. R, P, and

F_{2}

denote recall, precision, and

F_{2}

, respectively. * indicates results reported in the original papers. †indicates results from our reproduction. @n indicates voting with n predictions. +key indicates schema filling is used.

Method	Spider				BIRD
Method	R	P	$F_{2}$	SR	R	P	$F_{2}$	SR
Full BD	100	8.07	-	100	100	2.94	-	100
Gold-based	100	100	100	100	100	100	100	100
TableRAG	92.84	8.82	31.96	63.08	87.40	5.70	22.60	42.97
FGTR (k=3)	94.98	32.74	68.82	72.73	96.74	26.17	62.85	76.13
FGTR (k=5)	97.73	24.31	69.93	84.43	97.42	18.19	52.06	83.20

Table 2. Cell retrieval results on Spider and BIRD datasets. k indicates retrieval of the top-k most similar values, while adapt refers to our adaptive cell mapping mechanism.

Method	Components		Downstream Accuracy (%)
Method	SR	CR	BirdQA	SpiderQA	TabFACT	WTQ
FGTR	✓	✓	53.76	78.26	76.25	60.27
	✓	✗	–	75.21	74.84	55.47
	✗	✓	46.72	74.80	67.20	56.28
	✗	✗	–	66.08	62.90	53.00
TableRAG	–		45.50	48.61	69.42	56.63

Table 3. Ablation study of the schema retrieval (SR) and cell retrieval (CR) components. Using both components in our FGTR model yields the best performance across all four downstream QA and fact-verification datasets. The performance of two baseline methods, TableRAG and Dater, is included for comparison. The "-" symbol indicates that the input table was too large to fit into the model’s context window.

Method	Spider						BIRD
Method	Schema Retrieval			Cells Retrieval			Schema Retrieval			Cells Retrieval
	R	P	$F_{2}$	R	P	$F_{2}$	R	P	$F_{2}$	R	P	$F_{2}$
FGTR	98.32	70.72	91.20	97.78	84.75	94.86	97.27	72.00	90.89	92.48	90.54	92.09
-w/o column explaining	96.63	68.41	89.27	-	-	-	92.17	70.30	86.84	-	-	-
-w/o question parsing	95.71	59.73	85.42	-	-	-	93.54	65.20	86.06	-	-	-
-w/o schema mapping	91.62	76.38	88.21	-	-	-	89.08	81.33	87.43	-	-	-

Table 4. Ablation study of FGTR on Spider and BIRD datasets for both schema retrieval and cell retrieval.

Figure 3. Performance comparison of FGTR and TableRAG across different table sizes. The x-axis represents table size (number of tokens). The left y-axis and the bar chart show the distribution of table sizes in our datasets. The right y-axis corresponds to the line plots, which compare the EM accuracy of FGTR against TableRAG on four downstream tasks. Each subplot corresponds to a different dataset.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4. Oriainal tables from BirdQA with FGTR relecance heat-map. In the schema retrieval phase, deeper orange indicates higher frequency fields, while in the cell retrieval phase, green intensity distinguishes between constrained (dark) and dependent (light) cells.

Figure 5. Original tables from WTQ with FGTR relecance heat-map.

Dataset	Overall					Per Question
Dataset	# DBs	# Questions	Tables/DB	Cols/Table	Rows/Table	Tables/Q	Cols/Q
SpiderQA	166	965	5.13	5.01	17	1.52	2.99
BIRDQA	95	1,182	7.25	10.66	55,448	2.17	4.95
WTQ	1,876	2,000	1.0	6.41	28.01	1.00	-
TabFACT	1,695	2,000	1.0	6.25	13.43	1.00	-

Appendix

Table 5. Statistical overview of the benchmark datasets. The "Overall" columns describe the general and structural properties of the source databases, including total counts of databases and questions, the average number of tables per database, and the average columns and rows per table. The "Per Question" columns detail the complexity of the ground-truth schema for an average question, reporting the mean number of relevant tables and columns.

Table Retrieval

Table retrieval finds relevant tables and schema elements for question answering, analytics, and text-to-SQL.

See all 19 papers

HELIOS: Harmonizing Early Fusion, Late Fusion, and LLM Reasoning for Multi-Granular Table-Text Retrieval
Feb 25, 2026Sungho Park, Joohyung Yun, Jongwuk Lee +1Information Retrieval Hybrid Retrieval
POSTECH, Republic of Korea · Sungkyunkwan University, Republic of Korea
FollowTable: A Benchmark for Instruction-Following Table Retrieval
May 1, 2026Rihui Jin, Yuchen Lu, Ting Zhang +7Information Retrieval Table Retrieval
Southeast University Nanjing, China Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education · Huawei Noah’s Ark Lab Shenzhen, China
FT-RAG: A Fine-grained Retrieval-Augmented Generation Framework for Complex Table Reasoning
May 2, 2026Zebin Guo, Weidong Geng, Ruichen MaoMultimodal RAG Table QA
Georgia Institute of Technology Atlanta, Georgia, USA · Zhejiang University Hangzhou, Zhejiang, China · Zhejiang Lab Hangzhou, Zhejiang, China

Feb 25, 2026J/K move · Enter open · S save

HELIOS: Harmonizing Early Fusion, Late Fusion, and LLM Reasoning for Multi-Granular Table-Text Retrieval

Sungho Park, Joohyung Yun, Jongwuk Lee, Wook-Shin Han

POSTECH, Republic of Korea · Sungkyunkwan University, Republic of Korea

Information Retrieval Hybrid Retrieval Table QA LLM Reasoning with Graphs

Open paper AskPDF arXiv

Table-text retrieval aims to retrieve relevant tables and text to support open-domain question answering. Existing studies use either early or late fusion, but face limitations. Early fusion pre-aligns a table row with its associated passages, forming "stars," which often include irrelevant contexts and miss query-dependent relationships. Late fusion retrieves individual nodes, dynamically aligning them, but it risks missing relevant contexts. Both approaches also struggle with advanced reasoning tasks, such as column-wise aggregation and multi-hop reasoning. To address these issues, we propose HELIOS, which combines the strengths of both approaches. First, the edge-based bipartite subgraph retrieval identifies finer-grained edges between table segments and passages, effectively avoiding the inclusion of irrelevant contexts. Then, the query-relevant node expansion identifies the most promising nodes, dynamically retrieving relevant edges to grow the bipartite subgraph, minimizing the risk of missing important contexts. Lastly, the star-based LLM refinement performs logical inference at the star graph level rather than the bipartite subgraph, supporting advanced reasoning tasks. Experimental results show that HELIOS outperforms state-of-the-art models with a significant improvement up to 42.6% and 39.9% in recall and nDCG, respectively, on the OTT-QA benchmark.

Abstract

Table Retrieval

Explore similar work

HELIOS: Harmonizing Early Fusion, Late Fusion, and LLM Reasoning for Multi-Granular Table-Text Retrieval

FollowTable: A Benchmark for Instruction-Following Table Retrieval

FT-RAG: A Fine-grained Retrieval-Augmented Generation Framework for Complex Table Reasoning