Enabling question answering over tables and databases in natural language has become a key capability in the democratization of insights from tabular data sources. These systems first require retrieval of data that is relevant to a given natural language query, for which several methods have been introduced. In this work we present and study a table retrieval mechanism devising fine-grained typed query decomposition and global connectivity-awareness (DCTR), to handle the challenges induced by open-domain question answering over relational databases in complex usage contexts. We evaluate the effectiveness of the two mechanisms through the lens of retrieval complexity which we measure along the axes of query- and data complexity. Our analyses over industry-aligned benchmarks illustrate the robustness of DCTR for highly composite queries and densely connected databases.
Figures & tables
Figure 1: Overview of DCTR with component-wise retrieval, group formation, and FK-group expansion.
Setting
Model
BEAVER
FIBEN
BIRD
Baseline
stella
0.181
0.297
0.901
bge-small
0.154
0.198
0.809
e5-small
0.108
0.062
0.737
votek=1 , ngroups=1
stella
0.231 ± .004
0.317 ± .001
0.834 ± .001
bge-small
0.223 ± .005
0.294 ± .003
0.826 ± .002
e5-small
0.199 ± .009
0.257 ± .004
0.817 ± .007
Table 1: Capped Recall@5 (mean ± std) across datasets and models. Bold values indicate the best result per dataset and embedding model.
Setting
Model
BEAVER
FIBEN
BIRD
Baseline
stella
0.277
0.413
0.961
bge-small
0.231
0.262
0.909
e5-small
0.175
0.140
0.850
votek=2 , ngroups=2
stella
0.313 ± .007
0.396 ± .006
0.899 ± .004
bge-small
0.326 ± .002
0.331 ± .006
0.893 ± .003
e5-small
0.273 ± .007
0.346 ± .004
0.882 ± .006
Table 2: Capped Recall@10 (mean ± std) across datasets and models. Bold values indicate the best result per dataset and embedding model.
Setting
Model
BEAVER
FIBEN
BIRD
Baseline
stella
0.413
0.584
0.986
bge-small
0.350
0.461
0.976
e5-small
0.281
0.283
0.949
votek=5 , ngroups=5
stella
0.435 ± .002
0.513 ± .007
0.971 ± .002
bge-small
0.429 ± .003
0.448 ± .001
0.972 ± .002
e5-small
0.408 ± .008
0.516 ± .003
0.965 ± .001
Table 3: Capped Recall@25 (mean ± std) across datasets and models. Bold values indicate the best result per dataset and embedding model.
Figure 2: Query length vs recall@25, aggregated across datasets. Comparison through three embedding models, shows that DCTR outperforms the baseline, especially for longer queries ( ≥ 40 tokens).
Figure 3: Number of components vs Recall@25 across datasets. Comparison through three embedding models shows that DCTR is robust for compositional queries while a single-vector baseline does not resolve compositional queries regardless of model capacity.
Figure 4: Number of tables connected to gold tables vs recall@25, combined across datasets, for different embedding models. DCTR improves more on queries where gold tables are densely connected.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Query length vs recall@k, combined across datasets. Comparison of three embedding models for the proposed method versus the baseline. Across all values of k , the proposed method consistently outperforms the baseline, especially for longer queries ( ≥ 40 tokens).
Figure 6: Number of components vs recall@k, combined across datasets. Comparison of three embedding models for the proposed method versus the baseline. Our method consistently performs better than the baseline for queries with more components. The proposed method consistently outperforms the baseline as the number of query components increases. At k=25 (Figure 6(c) ), our method maintains stable performance for highly compositional queries, whereas the baseline exhibits a performance drop.
Figure 7: Number of tables gold tables vs recall@k, combined across datasets. Comparison of three embedding models for the proposed method versus the baseline. Both methods exhibit a degradation in capped recall as there the number of gold tables increases. The proposed method behaves similarly to the baseline, indicating limited advantage when there is a lot of relevant tables to retrieve.
Figure 8: Number of gold tables join relationships vs recall@k, combined across datasets. Comparison of three embedding models for the proposed method versus the baseline. DCTR improves performance on queries for which gold tables are densely connected.
Figure 9: Distribution of analyzed properties. Results combined for all datasets.
Retrieving the right tables is a prerequisite for Text-to-SQL over realistic databases. Dense table retrievers rank schema elements independently, but this ignores a key source of evidence: some required tables are not mentioned in the question and become identifiable only through their join relationships to already relevant tables. We introduce JOINGR, a join-aware table retrieval method that treats the database join graph as the retrieval space. Columns are represented as graph nodes, while intra-table and foreign-key relationships are represented as typed edges. Given a question, JOINGR selects semantically similar anchor tables, traverses join edges with a query-conditioned scorer, and aggregates the resulting edge deposits into table scores. The scorer is a lightweight MLP on top of frozen query, node, and edge embeddings, trained with a pairwise margin loss over gold tables. On BIRD and Spider datasets, JOINGR is competitive with the strongest retrieval baselines. On BEAVER, a challenging enterprise benchmark with multi-hop table requirements, JOINGR substantially improves recall over dense retrieval and re-ranking baselines. Cross-domain experiments show that the learned scorer transfers across benchmarks, indicating that the method captures reusable joingraph traversal behavior.
Sandipan De, Abhijit Chakraborty, Sambaran Bandyopadhyay +1
With the rapid advancement of large language models (LLMs), growing efforts have been made on LLM-based table retrieval. However, existing studies typically focus on single-table query, and implement it by similarity matching after encoding the entire table. These methods usually result in low accuracy due to their coarse-grained encoding which incorporates much query-irrelated data, and are also inefficient when dealing with large tables, failing to fully utilize the reasoning capabilities of LLM. Further, multi-table query is under-explored in retrieval tasks. To this end, we propose a hierarchical multi-table query method based on LLM: Fine-Grained Multi-Table Retrieval FGTR, a new retrieval paradigm that employs a human-like reasoning strategy. Through hierarchical reasoning, FGTR first identifies relevant schema elements and then retrieves the corresponding cell contents, ultimately constructing a concise and accurate sub-table that aligns with the given query. To comprehensively evaluate the performance of FGTR, we construct two new benchmark datasets based on Spider and BIRD . Experimental results show that FGTR outperforms previous state-of-the-art methods, improving the F_2 metric by 18% on Spider and 21% on BIRD, demonstrating its effectiveness in enhancing fine-grained retrieval and its potential to improve end-to-end performance on table-based downstream tasks.
Chaojie Sun, Bin Cao, Tiantian Li +3
Zhejiang University of Technology Hangzhou, China · University of Aberdeen Aberdeen, UK
Retrieving relevant tables from extensive databases for a given natural language query is essential for accurately answering questions in tasks such as text-to-SQL. Existing table retrieval approaches select a pre-determined set of k tables with the highest similarity to the query. However, the number of required tables varies across queries and cannot be known in advance. Enforcing a fixed number of retrieved tables regardless of the query may either retrieve an undersized set, failing to obtain all necessary evidence, or retrieve an oversized pool, including irrelevant tables. To address this issue, we propose an adaptive table retrieval method that adjusts the number of tables retrieved according to the requirements of each query. Specifically, we utilize an adaptive thresholding mechanism to selectively retrieve tables and integrate a sliding-window reranking algorithm to efficiently process a large table corpus. Extensive experiments on Spider, BIRD, and Spider 2.0 demonstrate that our method effectively addresses the limitations of the top-k retrieval strategy, improving performance in retrieval and downstream tasks. Our code and data are available at https://github.com/sbY99/Adaptive-Table-Retrieval.