Retrieval-Feedback-Driven Distillation and Preference Alignment for Efficient LLM-based Query Expansion

Minghan Li; Guodong Zhou

cs.IRMar 14, 2026

Retrieval-Feedback-Driven Distillation and Preference Alignment for Efficient LLM-based Query Expansion

Authors: Minghan Li, Guodong Zhou

Organizations: School of Computer Science and Technology, Soochow University, China

Abstract

Large language models have recently enabled a generative paradigm for query expansion, but their high inference cost makes direct deployment difficult in practical retrieval systems. To address this issue, a retrieval-feedback-driven distillation and preference-alignment framework is proposed to transfer retrieval-friendly expansion behavior from a strong teacher model to a compact student model. Rather than relying on few-shot exemplars at inference time, the framework first leverages two complementary types of teacher-generated expansions, produced under zero-shot and few-shot prompting conditions, as supervision signals for distillation and as candidate pools for preference construction. A retrieval-metric-driven strategy is then introduced to automatically form chosen/rejected expansion pairs according to nDCG@10 differences, and Direct Preference Optimization is applied to explicitly align generation preferences with retrieval objectives. Experiments on TREC DL19/20/21 and MIRACL-zh show that the proposed approach preserves strong retrieval effectiveness while substantially reducing inference cost. In particular, the distilled Qwen3-4B model reaches about 97% of the teacher (DeepSeek-685B) model's nDCG@10 performance on DL19, and remains effective on the Chinese MIRACL-zh benchmark, demonstrating strong practicality across both English and Chinese retrieval settings.

Figures & tables

Figure 1. Overall system architecture of the proposed method.

Dataset	# Test Queries	Corpus Type	Corpus Size
TREC DL19	43	Web (MS MARCO v1)	about 8.8M
TREC DL20	54	Web (MS MARCO v1)	about 8.8M
TREC DL21	53	Web (MS MARCO v2)	about 138M
MIRACL-zh	393 (dev as test)	Wikipedia (Chinese)	4,934,368

Table 1. Statistics of datasets used in the experiments.

Method	nDCG@10	MAP	MRR
BM25 Retriever (DeepSeek Distillation)
BM25	0.5058 †	0.3013 †	0.7036 †
BM25+Rocchio	0.5275 †	0.3474 †	0.6676 †
Original model expansion	0.6164 †	0.4150	0.7592 †
Query2Doc	0.6236	0.4177 †	0.7971
SFT-only	0.6405	0.4269	0.8132

Table 2. Comparison of retrieval effectiveness for different query expansion strategies on TREC DL19.

{\dagger}

indicates a statistically significant difference from SFT+DPO under a paired t-test (

p<0.05

Method	nDCG@10	MAP	MRR
BM25 Retriever (DeepSeek Distillation)
BM25	0.4796 †	0.2856 †	0.6585 †
BM25+Rocchio	0.4910 †	0.3115 †	0.6513 †
Original model expansion	0.5476 †	0.3379 †	0.6766
Query2Doc	0.5801	0.3633 †	0.7120
SFT-only	0.5797	0.3664	0.7258

Table 3. Comparison of retrieval effectiveness for different query expansion strategies on TREC DL20.

{\dagger}

indicates a statistically significant difference from SFT+DPO under a paired t-test (

p<0.05

Method	nDCG@10	MAP	MRR
BM25 Retriever (DeepSeek Distillation)
BM25 (no expansion)	0.4458 †	0.1708 †	0.5060 †
BM25+Rocchio	0.4544 †	0.2151 †	0.5442 †
Original model expansion	0.5186 †	0.2543 †	0.6659
Query2Doc	0.5001 †	0.2407 †	0.6625
SFT-only	0.5271 †	0.2622 †	0.6589

Table 4. Comparison of retrieval effectiveness for different query expansion strategies on TREC DL21.

{\dagger}

indicates a statistically significant difference from SFT+DPO under a paired t-test (

p<0.05

Method	nDCG@10	MAP	MRR
BM25 Retriever (DeepSeek Distillation)
BM25	0.1801 †	0.1486 †	0.2207 †
BM25+Rocchio	0.1740 †	0.1451 †	0.2058 †
Original model expansion	0.3302	0.2720 †	0.3853 †
Query2Doc	0.3232 †	0.2689 †	0.3877 †
SFT-only	0.3251	0.2728	0.3920 †

Table 5. Comparison of retrieval effectiveness for different query expansion strategies on MIRACL-zh.

{\dagger}

indicates a statistically significant difference from SFT+DPO under a paired t-test (

p<0.05

DL19 (Queries = 43)
Setting	Time (s)	s/query
DeepSeek-V3.1 (685B) API	203.24	4.726
OLMo3.1-32B (Teacher, local)	98.77	2.297
Query2Doc (Qwen3-4B, local few-shot)	17.86	0.415
SFT+DPO (Qwen3-4B, local)	4.44	0.103
DL20 (Queries = 54)

Table 6. Efficiency comparison of query expansion generation on DL19 and DL20 under different models and prompting settings.

Figure 4. Peak GPU memory usage for query expansion generation under different settings on DL19 (local inference).

Dataset	Prompting	nDCG@10	MAP	MRR
DL19	zero-shot	0.6741	0.4588	0.8184
DL19	few-shot	0.7051	0.4912	0.8510
DL20	zero-shot	0.6417	0.4316	0.7931
DL20	few-shot	0.6587	0.4570	0.8568
DL21	zero-shot	0.5815	0.2953	0.6865
DL21	few-shot	0.5384	0.2865	0.6798

Table 7. Teacher expansion quality under different prompting configurations (DeepSeek).

Method	nDCG@10	MAP	MRR
OLMo-32B expansion (zero-shot)	0.6444	0.4180	0.8118
OLMo-32B expansion (few-shot)	0.6521	0.4401	0.8595
SFT-only	0.6395	0.4305	0.8482
DPO-only	0.6277	0.4326	0.8587
SFT+DPO	0.6599	0.4388	0.8318

Table 8. Retrieval performance (BM25) on TREC DL19 with OLMo-32B as the teacher model.

DL19 (BGE-base-en-v1.5)
Method	nDCG@10	MAP	MRR
BGE (raw)	0.7016	0.4485 †	0.9419
Original model expansion	0.6680 †	0.4521 †	0.9034
Query2Doc	0.7111	0.4779	0.9535
SFT-only	0.7004	0.4725	0.9536
DPO-only	0.6886	0.4658	0.9420

Table 9. Dense retrieval results with BGE-base-en-v1.5 on TREC DL19 and DL20.

{\dagger}

indicates a statistically significant difference from SFT+DPO under a paired t-test (

p<0.05

Figure 5. Effect of LoRA rank on nDCG@10 performance of the model after SFT on TREC DL19.

Figure 6. Effect of the DPO preference-margin threshold $\Delta$ on nDCG@10 performance on TREC DL19.

Method (SFT Data)	nDCG@10	MAP	MRR
SFT(E1+E2)	0.6405	0.4269	0.8132
SFT(E1-only)	0.6275	0.4222	0.7758
SFT(E2-only)	0.6230	0.4089	0.7585

Table 10. Ablation on SFT distillation data composition (DL19 + BM25).

Language Model Distillation

Language model distillation transfers a larger model’s capabilities to smaller or specialized models using outputs, probabilities, or internal representations.

See all 512 papers

ADORE: Iterative Query Expansion with Retrieval-Grounded Relevance Feedback
Jun 11, 2026Amin Bigdeli, Negar Arabzadeh, Radin Hamidi Rad +3Information Retrieval Query Expansion
University of Waterloo · University of California, Berkeley · Mila – Quebec AI Institute +1
DEPT: Document Embedding Preservation Tuning for Unified Query Expansion and Retrieval
Aug 18, 2026Jingyuan Wang, Richong Zhang, Zhijie Nie +2Dense Retrieval Query Expansion
Beihang University, China
Query Expansion Should Be Coordinated: Dense Expands, Sparse Anchors
Aug 16, 2026Chunran ZhangRetrieval-Augmented Generation RAG Evaluation
School of Computing and Artificial Intelligence Southwest Jiaotong University Chengdu, China

Jun 11, 2026J/K move · Enter open · S save

ADORE: Iterative Query Expansion with Retrieval-Grounded Relevance Feedback

Amin Bigdeli, Negar Arabzadeh, Radin Hamidi Rad, Sajad Ebrahimi+2

University of Waterloo · University of California, Berkeley · Mila – Quebec AI Institute+1

Information Retrieval Query Expansion

Open paper AskPDF arXiv

LLM-based query expansion improves retrieval by enriching the original query with additional context. Yet most methods remain generation-driven, producing plausible pseudo-documents or expansions without checking how the target corpus responds. This can introduce retrieval drift, amplify misleading vocabulary, or miss terms that distinguish relevant from non-relevant documents. We argue that effective expansion requires retrieval-grounded feedback, not just single-pass generation or unverified iteration. We introduce ADORE (ADapt, Observe, Relevance Evaluate), an iterative framework that turns retrieval outcomes into feedback for the next expansion. At each round, an LLM generates pseudo-passages, a retriever exposes the corpus response, and a relevance assessor evaluates retrieved documents against the original query. These judgments identify what to reinforce, what remains undercovered, and what to suppress. Across TREC Deep Learning, BEIR, and BRIGHT, ADORE consistently outperforms strong query expansion baselines with notable improvements across nearly all evaluation settings, improving average nDCG@10 by 24.5% over BM25 and 3.6% over the strongest prior query expansion method on BEIR, and by 122.9% over BM25 and 9.2% over the best query expansion baseline on BRIGHT. Our code and data are publicly available.

Abstract

Language Model Distillation

Explore similar work

ADORE: Iterative Query Expansion with Retrieval-Grounded Relevance Feedback

DEPT: Document Embedding Preservation Tuning for Unified Query Expansion and Retrieval

Query Expansion Should Be Coordinated: Dense Expands, Sparse Anchors