Polaris: Learning to Generate Table Descriptions from Retrieval Feedback

Ting Cai; Tuan Minh Phan; AnHai Doan

cs.CLAug 17, 2026

Polaris: Learning to Generate Table Descriptions from Retrieval Feedback

Authors: Ting Cai, Tuan Minh Phan, AnHai Doan

Organizations: University of Wisconsin-Madison

Abstract

Many table-centric NLP tasks such as NL2SQL first retrieve relevant tables from large collections using keyword search. Recent work uses LLMs to generate natural-language table descriptions to improve retrieval, but they are typically optimized for fluency rather than retrieval effectiveness. We present Polaris, a system that trains an LLM to generate table descriptions directly from retrieval feedback. Our key insight is that existing table retrieval benchmarks already contain the supervision needed for this task: given query-table relevance judgments, we generate multiple candidate descriptions for each table, rank them by their BM25 retrieval effectiveness, and use the resulting preference pairs to fine-tune the LLM with Direct Preference Optimization (DPO). Polaris further expands abbreviated table and column names before generation to reduce vocabulary mismatch. Extensive experiments show that Polaris outperforms the state-of-the-art AutoDDG solution, often by a significant margin. More broadly, our results demonstrate that retrieval benchmarks can be repurposed as supervision for training LLMs to generate retrieval-oriented metadata.

Figures & tables

Figure 2: The overall architecture of Polaris.

Figure 3: Using a labeled dataset to generate rankings.

Dataset	Domain	#Tables	#Queries	#Gold
AW	Enterprise	96	15	41
Arctic	Science	251	20	121
LTER	Science	2,015	15	374
ECIR	Government	2,100	12	1,411
WTR	Web	4,634	60	2,190
WikiTables	Web	3,361	57	845

Table 1: Datasets for our experiments. #Gold is the number of gold tables.

Dataset	Method	@5	@10	@20	@50	@100
AW	AutoDDG	.5551	.6192	.6692	.6969	.7030
	Polaris	.7312	.7582	.7869	.7869	.7869
Arctic	AutoDDG	.6274	.5942	.5942	.6360	.6812
	Polaris	.7133	.6743	.6939	.7416	.7678
LTER	AutoDDG	.4372	.4194	.4011	.3656	.3559
	Polaris	.7218	.6546	.6013	.5239	.5145

Table 2: NDCG@K of AutoDDG vs. Polaris

Dataset	Method	@5	@10	@20	@50	@100
AW	AutoDDG	.49	.64	.76	.83	.85
	Polaris	.66	.77	.83	.83	.83
Arctic	AutoDDG	.24	.34	.48	.68	.77
	Polaris	.29	.41	.58	.77	.82
LTER	AutoDDG	.08	.14	.22	.32	.35
	Polaris	.18	.25	.32	.39	.43

Table 3: Recall@K of AutoDDG vs. Polaris.

Figure 4: NDCG@K of multi-field Polaris vs. single-field Polaris.

Appendix figures & tables21 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Table 4 — Table 4: Column name expansion prompt. The prompt uses four few-shot examples (first shown in full above; three omitted for brevity) followed by the target input. Input placeholders are shown in red .

Panel of Table 5 — Table 5: Table name expansion prompt. The prompt uses four few-shot examples (first shown in full above; three omitted for brevity) with expanded column names as additional context. Input placeholders are shown in red .

Parameter	Value
Base model	Llama 3.1 8B Instruct
Quantization	4-bit
LoRA rank / $\alpha$	64 / 64
Trainable parameters	167M / 8.2B (2.05%)
DPO $\beta$	0.1
Learning rate	$5\times 10^{-6}$

Appendix

Table 8: DPO training configuration.

Figure 6: Average length of the descriptions generated by AutoDDG and Polaris on each dataset.

Dataset	Method	@5	@10	@20	@50	@100
AW	AutoDDG	.5551	.6192	.6692	.6969	.7030
	$P{-}(\mathrm{exp{+}DPO})$	.7667	.7948	.8129	.8129	.8129
Arctic	AutoDDG	.6274	.5942	.5942	.6360	.6812
	$P{-}(\mathrm{exp{+}DPO})$	.7041	.6651	.6700	.7008	.7226
LTER	AutoDDG	.4372	.4194	.4011	.3656	.3559
	$P{-}(\mathrm{exp{+}DPO})$	.6042	.5856	.5353	.4358	.4188

Appendix

Table 9: Prompt quality ablation (NDCG@

K

): AutoDDG vs.

P{-}(\mathrm{exp{+}DPO})

P{-}(\mathrm{exp{+}DPO})

= Polaris without DPO training and table/column name expansion. Blue = better per cell.

Dataset	Method	@5	@10	@20	@50	@100
AW	AutoDDG	.49	.64	.76	.83	.85
	$P{-}(\mathrm{exp{+}DPO})$	.69	.78	.82	.82	.82
Arctic	AutoDDG	.24	.34	.48	.68	.77
	$P{-}(\mathrm{exp{+}DPO})$	.29	.40	.55	.74	.78
LTER	AutoDDG	.08	.14	.22	.32	.35
	$P{-}(\mathrm{exp{+}DPO})$	.12	.18	.26	.33	.37

Appendix

Table 10: Prompt quality ablation (Recall@

K

): AutoDDG vs.

P{-}(\mathrm{exp{+}DPO})

P{-}(\mathrm{exp{+}DPO})

= Polaris without DPO training and table/column name expansion. Blue = better per cell.

Dataset	Method	@5	@10	@20	@50	@100
AW	Polaris	.7312	.7582	.7869	.7869	.7869
	$P{-}\mathrm{DPO}$	.6681	.7287	.7794	.7794	.7794
	$P{-}\mathrm{exp}$	.6940	.7485	.7847	.7847	.7847
Arctic	Polaris	.7133	.6743	.6939	.7416	.7678
	$P{-}\mathrm{DPO}$	.7209	.6785	.6891	.7334	.7535
	$P{-}\mathrm{exp}$	.6690	.6493	.6737	.6872	.7106

Appendix

Table 11: Effect of DPO training and table/column name expansion (NDCG@

K

): Polaris vs.

P{-}\mathrm{DPO}

vs.

P{-}\mathrm{exp}

P{-}\mathrm{DPO}

= Polaris without DPO training,

P{-}\mathrm{exp}

= Polaris without name expansion. Blue = best per cell.

Dataset	Method	@5	@10	@20	@50	@100
AW	Polaris	.66	.77	.83	.83	.83
	$P{-}\mathrm{DPO}$	.61	.76	.89	.89	.89
	$P{-}\mathrm{exp}$	.65	.79	.87	.87	.87
Arctic	Polaris	.29	.41	.58	.77	.82
	$P{-}\mathrm{DPO}$	.30	.40	.56	.76	.80
	$P{-}\mathrm{exp}$	.27	.39	.57	.72	.77

Appendix

Table 12: Effect of DPO training and table/column name expansion (Recall@

K

): Polaris vs.

P{-}\mathrm{DPO}

vs.

P{-}\mathrm{exp}

P{-}\mathrm{DPO}

= Polaris without DPO training,

P{-}\mathrm{exp}

= Polaris without name expansion. Blue = best per cell.

Dataset	Method	@5	@10	@20	@50	@100
AW	$P(\mathrm{Qwen})$	.8265	.8390	.8733	.8733	.8733
	Polaris	.7312	.7582	.7869	.7869	.7869
Arctic	$P(\mathrm{Qwen})$	.7303	.6952	.7071	.7425	.7690
	Polaris	.7133	.6743	.6939	.7416	.7678
LTER	$P(\mathrm{Qwen})$	.6269	.5964	.5393	.4439	.4259
	Polaris	.7218	.6546	.6013	.5239	.5145

Appendix

Table 13: Polaris with different LLMs (NDCG@

K

P(\mathrm{Qwen})

vs. Polaris.

P(\mathrm{Qwen})

= Polaris with Qwen2.5-7B, Polaris = Polaris with Llama 3.1 8B. Blue = better per cell.

Dataset	Method	@5	@10	@20	@50	@100
AW	$P(\mathrm{Qwen})$	.73	.81	.88	.88	.88
	Polaris	.66	.77	.83	.83	.83
Arctic	$P(\mathrm{Qwen})$	.29	.42	.60	.77	.82
	Polaris	.29	.41	.58	.77	.82
LTER	$P(\mathrm{Qwen})$	.11	.18	.27	.34	.36
	Polaris	.18	.25	.32	.39	.43

Appendix

Table 14: Polaris with different LLMs (Recall@

K

P(\mathrm{Qwen})

vs. Polaris.

P(\mathrm{Qwen})

= Polaris with Qwen2.5-7B, Polaris = Polaris with Llama 3.1 8B. Blue = better per cell.

Dataset	Method	@5	@10	@20	@50	@100
AW	$A(\mathrm{Qwen})$	.4853	.5286	.5946	.6306	.6306
	$P(\mathrm{Qwen})$	.8265	.8390	.8733	.8733	.8733
Arctic	$A(\mathrm{Qwen})$	.6861	.6586	.6786	.7055	.7437
	$P(\mathrm{Qwen})$	.7303	.6952	.7071	.7425	.7690
LTER	$A(\mathrm{Qwen})$	.3379	.3269	.3275	.3380	.3440
	$P(\mathrm{Qwen})$	.6269	.5964	.5393	.4439	.4259

Appendix

Table 15: Polaris vs. AutoDDG under Qwen2.5-7B (NDCG@

K

A(\mathrm{Qwen})

vs.

P(\mathrm{Qwen})

A(\mathrm{Qwen})

= AutoDDG with Qwen2.5-7B,

P(\mathrm{Qwen})

= Polaris with Qwen2.5-7B. Blue = better per cell.

Dataset	Method	@5	@10	@20	@50	@100
AW	$A(\mathrm{Qwen})$	.42	.55	.73	.83	.83
	AutoDDG	.49	.64	.76	.83	.85
	$P(\mathrm{Qwen})$	.73	.81	.88	.88	.88
	Polaris	.66	.77	.83	.83	.83
Arctic	$A(\mathrm{Qwen})$	.29	.41	.58	.74	.81
	AutoDDG	.24	.34	.48	.68	.77

Appendix

Table 16: Polaris vs. AutoDDG under different LLMs (Recall@

K

A(\mathrm{Qwen})

vs. AutoDDG vs.

P(\mathrm{Qwen})

vs. Polaris.

A(\mathrm{Qwen})

= AutoDDG with Qwen2.5-7B,

P(\mathrm{Qwen})

= Polaris with Qwen2.5-7B; AutoDDG and Polaris use Llama 3.1 8B. Blue = best per cell.

Dataset	Method	@5	@10	@20	@50	@100
AW	$P(\beta{=}0.5)$	.7398	.7662	.8051	.8051	.8051
	Polaris	.7312	.7582	.7869	.7869	.7869
Arctic	$P(\beta{=}0.5)$	.7266	.6874	.6837	.7380	.7585
	Polaris	.7133	.6743	.6939	.7416	.7678
LTER	$P(\beta{=}0.5)$	.6760	.6151	.5806	.4929	.4732
	Polaris	.7218	.6546	.6013	.5239	.5145

Appendix

Table 17: Effect of the DPO regularization parameter

\beta

(NDCG@

K

P(\beta{=}0.5)

vs. Polaris.

P(\beta{=}0.5)

= Polaris with

\beta=0.5

; Polaris uses the default

\beta=0.1

. Blue = better per cell.

Dataset	Method	@5	@10	@20	@50	@100
AW	$P(\beta{=}0.5)$	.68	.77	.86	.86	.86
	Polaris	.66	.77	.83	.83	.83
Arctic	$P(\beta{=}0.5)$	.30	.42	.56	.78	.82
	Polaris	.29	.41	.58	.77	.82
LTER	$P(\beta{=}0.5)$	.13	.20	.29	.37	.40
	Polaris	.18	.25	.32	.39	.43

Appendix

Table 18: Effect of the DPO regularization parameter

\beta

(Recall@

K

P(\beta{=}0.5)

vs. Polaris.

P(\beta{=}0.5)

= Polaris with

\beta=0.5

; Polaris uses the default

\beta=0.1

. Blue = better per cell.

Dataset	Method	@5	@10	@20	@50	@100
AW	$P(5\text{-}\mathrm{desc})$	.8019	.8313	.8601	.8601	.8601
	Polaris	.7312	.7582	.7869	.7869	.7869
Arctic	$P(5\text{-}\mathrm{desc})$	.6843	.6593	.7013	.7446	.7645
	Polaris	.7133	.6743	.6939	.7416	.7678
LTER	$P(5\text{-}\mathrm{desc})$	.6364	.5987	.5567	.4805	.4694
	Polaris	.7218	.6546	.6013	.5239	.5145

Appendix

Table 19: Effect of the number of candidate descriptions (NDCG@

K

P(5\text{-}\mathrm{desc})

vs. Polaris.

P(5\text{-}\mathrm{desc})

= Polaris with 5 candidate descriptions per gold table; Polaris has 3 by default. Blue = better per cell.

Dataset	Method	@5	@10	@20	@50	@100
AW	$P(5\text{-}\mathrm{desc})$	.73	.84	.90	.90	.90
	Polaris	.66	.77	.83	.83	.83
Arctic	$P(5\text{-}\mathrm{desc})$	.29	.41	.61	.80	.84
	Polaris	.29	.41	.58	.77	.82
LTER	$P(5\text{-}\mathrm{desc})$	.13	.20	.28	.35	.39
	Polaris	.18	.25	.32	.39	.43

Appendix

Table 20: Effect of the number of candidate descriptions (Recall@

K

P(5\text{-}\mathrm{desc})

vs. Polaris.

P(5\text{-}\mathrm{desc})

= Polaris with 5 candidate descriptions per gold table; Polaris has 3 by default. Blue = better per cell.

Dataset	Method	@5	@10	@20	@50	@100
AW	$P(3\text{-}\mathrm{epoch})$	.6908	.7522	.7881	.7968	.7968
	Polaris	.7312	.7582	.7869	.7869	.7869
Arctic	$P(3\text{-}\mathrm{epoch})$	.8080	.7131	.7059	.7595	.7951
	Polaris	.7133	.6743	.6939	.7416	.7678
LTER	$P(3\text{-}\mathrm{epoch})$	.7064	.6535	.6020	.5105	.4940
	Polaris	.7218	.6546	.6013	.5239	.5145

Appendix

Table 21: Effect of the number of training epochs (NDCG@

K

P(3\text{-}\mathrm{epoch})

vs. Polaris.

P(3\text{-}\mathrm{epoch})

= Polaris trained for 3 epochs; Polaris uses 1 epoch by default. Blue = better per cell.

Dataset	Method	@5	@10	@20	@50	@100
AW	$P(3\text{-}\mathrm{epoch})$	.65	.83	.91	.93	.93
	Polaris	.66	.77	.83	.83	.83
Arctic	$P(3\text{-}\mathrm{epoch})$	.32	.42	.59	.80	.86
	Polaris	.29	.41	.58	.77	.82
LTER	$P(3\text{-}\mathrm{epoch})$	.16	.23	.31	.38	.42
	Polaris	.18	.25	.32	.39	.43

Appendix

Table 22: Effect of the number of training epochs (Recall@

K

P(3\text{-}\mathrm{epoch})

vs. Polaris.

P(3\text{-}\mathrm{epoch})

= Polaris trained for 3 epochs; Polaris uses 1 epoch by default. Blue = better per cell.

Dataset	Method	@5	@10	@20	@50	@100
AW	AutoDDG	.5551	.6192	.6692	.6969	.7030
	Polaris	.7312	.7582	.7869	.7869	.7869
	$P(\mathrm{multi})$	.8133	.8282	.8677	.8677	.8677
Arctic	AutoDDG	.6274	.5942	.5942	.6360	.6812
	Polaris	.7133	.6743	.6939	.7416	.7678
	$P(\mathrm{multi})$	.7303	.7146	.7330	.7755	.8058

Appendix

Table 23: Multi-field KWS (NDCG@

K

): AutoDDG vs. Polaris vs.

P(\mathrm{multi})

P(\mathrm{multi})

= Polaris using multiple fields, Polaris = single field (description), AutoDDG = single field (description). Blue = best per cell.

Dataset	Method	@5	@10	@20	@50	@100
AW	AutoDDG	.49	.64	.76	.83	.85
	Polaris	.66	.77	.83	.83	.83
	$P(\mathrm{multi})$	.75	.83	.92	.92	.92
Arctic	AutoDDG	.24	.34	.48	.68	.77
	Polaris	.29	.41	.58	.77	.82
	$P(\mathrm{multi})$	.32	.46	.63	.82	.88

Appendix

Table 24: Multi-field KWS (Recall@

K

): AutoDDG vs. Polaris vs.

P(\mathrm{multi})

P(\mathrm{multi})

= Polaris using multiple fields, Polaris = single field (description), AutoDDG = single field (description). Blue = best per cell.

Dataset	Method	@5	@10	@20	@50	@100
AW	$P(\mathrm{multi})$	.8133	.8282	.8677	.8677	.8677
	${-}\mathrm{desc}$	.7594	.7668	.7895	.7895	.7895
	${-}\mathrm{exp}$	.7312	.7582	.7869	.7869	.7869
Arctic	$P(\mathrm{multi})$	.7303	.7146	.7330	.7755	.8058
	${-}\mathrm{desc}$	.7602	.7162	.6726	.7139	.7255
	${-}\mathrm{exp}$	.7133	.6743	.6939	.7416	.7678

Appendix

Table 25: Field ablation (NDCG@

K

) for multi-field

P(\mathrm{multi})

{-}\mathrm{desc}

= w/o description,

{-}\mathrm{exp}

= w/o table/column name expansions,

{-}\mathrm{ctx}

= w/o table context (ECIR, WTR, WikiTables only). Blue = best per cell.

Table Retrieval

Table retrieval finds relevant tables and schema elements for question answering, analytics, and text-to-SQL.

See all 19 papers

CORE-T: COherent REtrieval of Tables for Text-to-SQL
Jan 19, 2026Hassan Soliman, Vivek Gupta, Dan Roth +1Text-to-SQL Information Retrieval
Ubiquitous Knowledge Processing Lab (UKP Lab), Department of Computer Science TU Darmstadt and National Research Center for Applied Cybersecurity ATHENE, Germany · Arizona State University · University of Pennsylvania
PIPER: Content-Based Table Search via profiling and LLM-Generated Pseudoqueries
May 18, 2026Riccardo Terrenzi, Matteo Falconi, Serkan Ayvaz +1Information Retrieval Table Retrieval
Centre for Industrial Software, University of Southern Denmark, Sønderborg, Denmark · Department of Electronics, Information and Bioengineering, Politecnico di Milano, Milano, Italy
TACO: Task-Aware Column Description Generation Using LLMs
Jun 19, 2026Ting Cai, Rakesh R. Menon, Yiru Chen +8LLM-Assisted Annotation
University of Wisconsin-Madison · Adobe · Purdue University

Jan 19, 2026J/K move · Enter open · S save

CORE-T: COherent REtrieval of Tables for Text-to-SQL

Hassan Soliman, Vivek Gupta, Dan Roth, Iryna Gurevych

Ubiquitous Knowledge Processing Lab (UKP Lab), Department of Computer Science TU Darmstadt and National Research Center for Applied Cybersecurity ATHENE, Germany · Arizona State University · University of Pennsylvania

Text-to-SQL Information Retrieval Table Retrieval

Open paper AskPDF arXiv

Realistic text-to-SQL workflows often require joining multiple tables. As a result, accurately retrieving the relevant set of tables becomes a key bottleneck for end-to-end performance. We study an open-book setting where queries must be answered over large, heterogeneous table collections pooled from many sources, without clean scoping signals such as database identifiers. Here, dense retrieval (DR) achieves high recall but returns many distractors, while join-aware alternatives often rely on extra assumptions and/or incur high inference overhead. We propose CORE-T, a scalable, training-free framework that enriches tables with LLM-generated purpose metadata and pre-computes a lightweight table-compatibility cache. At inference time, DR returns top-K candidates; a single LLM call selects a coherent, joinable subset, and a two-step additive adjustment stage restores strongly compatible tables. Across Bird, Spider, MMQA, and Beaver, CORE-T improves over DR by up to 22.7 points in table-selection F1 while returning up to 40% fewer tables, and by up to 24.4 points in multi-table execution accuracy, and uses 1.64-4.20x fewer total selection tokens than LLM-intensive baselines.

Abstract

Table Retrieval

Explore similar work

CORE-T: COherent REtrieval of Tables for Text-to-SQL

PIPER: Content-Based Table Search via profiling and LLM-Generated Pseudoqueries

TACO: Task-Aware Column Description Generation Using LLMs