MERGE: Multi-LLM Ensemble for Retrieval via Generative Enrichment

Tzu-I Ho; Yung-Yu Shih; Shang-Yu Su; Dongzhe Wang; Yun-Nung Chen

cs.IRSep 29, 2026

MERGE: Multi-LLM Ensemble for Retrieval via Generative Enrichment

Authors: Tzu-I Ho, Yung-Yu Shih, Shang-Yu Su, Dongzhe Wang, Yun-Nung Chen

Organizations: University of Waterloo · National Taiwan University · Rakuten Group, Inc. · Rakuten Asia Pte. Ltd.

Abstract

Large Language Models (LLMs) are increasingly used to enrich user queries in information retrieval (IR) so that a standard retriever such as BM25 can bridge vocabulary gaps with the target corpus. Any single LLM, however, is limited by its training data and architectural biases, and its enrichment behavior depends on hand-crafted prompts that must be re-engineered for each new model -- an expensive and poorly scalable process. We present MERGE (Multi-LLM Ensemble for Retrieval via Generative Enrichment), a two-stage framework: three heterogeneous 7-8B open-source LLMs independently produce candidate expansions, and a larger LLM generatively synthesizes them into a single query. To make prompt engineering scalable across the ensemble, we integrate a task-grounded Automatic Prompt Optimization (APO) loop into both stages. Unlike APO methods that judge candidates with an LLM evaluator, our loop scores each candidate by its downstream retrieval performance and runs a small tournament between the current champion prompt and optimizer-proposed drafts, terminating once the champion survives two consecutive rounds; a history-augmented variant additionally feeds the recent tournament trajectory back to the optimizer. MERGE is retriever-agnostic and issues a single BM25 pass with no rank fusion, no supervised document expansion, and no re-indexing. On five BEIR benchmarks (NQ, SciFact, FiQA, Touche-2020, DBPedia), MERGE improves BM25 nDCG@10 over the original queries by +2.1 to +14.9 points and matches or outperforms strong LLM-based query-expansion baselines despite using only compact open-source models. Ablations confirm that the Stage-2 ensemble beats any single Stage-1 LLM, and that task-grounded APO converts large seed-prompt regressions into consistent gains without hand-tuning.

Figures & tables

Figure 1: Overview of MERGE. Given an input query, Stage 1 (Diverse Query Expansion) runs three heterogeneous 7–8B instruction-tuned LLMs (Llama-3.1-8B-Instruct, Qwen2.5-7B-Instruct, Mistral-7B-Instruct-v0.3) in parallel to produce candidate expansions $e_{1},e_{2},e_{3}$ ; Stage 2 (Ensemble Synthesis) uses Qwen2.5-14B-Instruct to generatively fuse the candidates into a single merged expanded query, which is then fed to a fixed BM25 retriever. Every prompt—each Stage-1 generator prompt and the Stage-2 ensemble prompt—is specialized via our task-grounded Automatic Prompt Optimization loop (bottom): starting from a hand-crafted seed, the optimizer LLM rewrites the current champion into two drafts, all three are scored on a 20% dev subset via BM25 nDCG@10, and a tournament selects the winner as the next champion. The loop terminates when the champion survives two consecutive rounds.

Method	NQ	SciFact	FiQA	Touché-2020	DBPedia
Vanilla BM25 (original query)	30.48	67.62	23.85	44.23	31.99
+ docT5query	(38.10)	(67.50)	(25.20)	(34.70)	(33.10)
+ BM25 + Exp4Fuse	(39.10)	(68.80)	(24.70)	(51.20)	(36.10)
+ docT5query + Exp4Fuse	(42.80)	( 71.30 )	( 26.30 )	(39.90)	( 38.90 )
+ MERGE w/o APO	34.52	67.79	19.41	38.69	33.59
+ MERGE (APO)	45.42	71.40	25.59	50.16	37.94

Table 1: Retrieval results on five BEIR benchmarks (nDCG@10, %). All rows share the same BM25 retriever and differ only in how the query is expanded. Numbers in parentheses are reproduced from Liu and Zhang (2025) ; docT5query is Nogueira and Lin (2019) . Bold = best result in each column; underline = second best.

Query fed to BM25	NQ	SciFact	FiQA
Original query (no expansion)	30.48	67.62	23.85
Stage-1: Mistral-7B-v0.3	38.79	70.49	24.95
Stage-1: Qwen2.5-7B	34.59	68.69	24.17
Stage-1: Llama-3.1-8B	41.06	69.72	25.00
Stage-2: Qwen2.5-14B (ours)	45.42	71.40	25.59

Table 2: Ablation on the effect of the two-stage architecture. nDCG@10 (%) with BM25 backbone; all methods use MERGE (APO) (M1). Rows 1–3: BM25 fed a single Stage-1 model’s expansion. Row 4: BM25 fed the Stage-2 ensemble expansion.

Regime	nDCG@10	Stage-1 output (Llama-3.1-8B-Instruct)
M0 (no APO)	33.31	“José Martí, Calixto García, Antonio Maceo, Carlos Manuel de Céspedes…” (short entity list, no context)
M1 (APO)	41.06	“The Cuban revolt against Spain was led by José Martí, a Cuban poet and revolutionary. Martí was a key figure in the Cuban War of Independence , which began in 1895 and lasted until 1898 . He advocated for Cuban independence and was a critic of Spanish colonial rule…”
M2 (APO w/ history)	40.14	“The Cuban revolt against Spain was led by José Martí, a Cuban poet, writer, and revolutionary. Martí was a key figure in the Cuban independence movement and played a significant role in organizing the Cuban Revolutionary Party , which aimed to overthrow Spanish rule in Cuba…”

Table 3: APO effect at Stage 1. Input query (NQ): “the cuban revolt against spain was led by” . Model: Llama-3.1-8B-Instruct. We show the enriched query produced under the three APO regimes and its BM25 nDCG@10 on the NQ evaluation split. M0 uses the hand-crafted seed prompt; M1 and M2 use tournament-optimized prompts without / with a 5-round history.

Source	Score	Enriched output
Stage-1: Mistral-7B	38.79	“The Cuban Revolution…was led by a group of individuals who sought to overthrow the Spanish colonial rule and establish an independent Cuban government…armed conflicts from 1868 to 1898 …”
Stage-1: Qwen2.5-7B	34.59	“…led by leaders, figures, insurgents, and activists who fought for, spearheaded, and initiated the…” (generic; no named entities)
Stage-1: Llama-3.1-8B	41.06	“…led by José Martí , a Cuban poet and revolutionary…the Cuban War of Independence , which began in 1895 and lasted until 1898 …”
Stage-2: Qwen2.5-14B	45.42	“…led by a group of Cuban revolutionaries, including José Martí , Máximo Gómez , and Antonio Maceo . The revolt began in 1868 and lasted until 1898 …Martí as the ideological leader and Gómez and Maceo as military leaders…”

Table 4: Ensemble effect at Stage 2 on the NQ query “the cuban revolt against spain was led by” . We show the three Stage-1 candidates (one per LLM in the pool) and the Stage-2 ensemble output, all under MERGE (APO). “Score” is BM25 nDCG@10 on the NQ evaluation split. Long enrichments are elided as “…” but no substantive content is added.

Explore similar work

Jun 11, 2026cs.IR

ADORE: Iterative Query Expansion with Retrieval-Grounded Relevance Feedback

LLM-based query expansion improves retrieval by enriching the original query with additional context. Yet most methods remain generation-driven, producing plausible pseudo-documents or expansions without checking how the target corpus responds. This can introduce retrieval drift, amplify misleading vocabulary, or miss terms that distinguish relevant from non-relevant documents. We argue that effective expansion requires retrieval-grounded feedback, not just single-pass generation or unverified iteration. We introduce ADORE (ADapt, Observe, Relevance Evaluate), an iterative framework that turns retrieval outcomes into feedback for the next expansion. At each round, an LLM generates pseudo-passages, a retriever exposes the corpus response, and a relevance assessor evaluates retrieved documents against the original query. These judgments identify what to reinforce, what remains undercovered, and what to suppress. Across TREC Deep Learning, BEIR, and BRIGHT, ADORE consistently outperforms strong query expansion baselines with notable improvements across nearly all evaluation settings, improving average nDCG@10 by 24.5% over BM25 and 3.6% over the strongest prior query expansion method on BEIR, and by 122.9% over BM25 and 9.2% over the best query expansion baseline on BRIGHT. Our code and data are publicly available.

Amin Bigdeli, Negar Arabzadeh, Radin Hamidi Rad +3

University of Waterloo · University of California, Berkeley · Mila – Quebec AI Institute +1

Relevance Feedback

Ask PDF

Apr 30, 2026cs.IR

A Reproducibility Study of LLM-Based Query Reformulation

Large Language Models (LLMs) are now widely used for query reformulation and expansion in Information Retrieval, with many studies reporting substantial effectiveness gains. However, these results are typically obtained under heterogeneous experimental conditions, making it difficult to assess which findings are reproducible and which depend on specific implementation choices. In this work, we present a systematic reproducibility and comparative study of ten representative LLM-based query reformulation methods under a unified and strictly controlled experimental framework. We evaluate methods across two architectural LLM families at two parameter scales, three retrieval paradigms (lexical, learned sparse, and dense), and nine benchmark datasets spanning TREC Deep Learning and BEIR. Our results show that reformulation gains are strongly conditioned on the retrieval paradigm, that improvements observed under lexical retrieval do not consistently transfer to neural retrievers, and that larger LLMs do not uniformly yield better downstream performance. These findings clarify the stability and limits of reported gains in prior work. To enable transparent replication and ongoing comparison, we release all prompts, configurations, evaluation scripts, and run files through QueryGym, an open-source reformulation toolkit with a public leaderboard.\footnote{https://leaderboard.querygym.com}

Amin Bigdeli, Radin Hamidi Rad, Hai Son Le +4

University of Waterloo Waterloo, Ontario, Canada · Mila – Quebec AI Institute Montreal, Quebec, Canada · Toronto Metropolitan University Toronto, Ontario, Canada +2

Reproducibility Skillgym

Ask PDF

Aug 16, 2026cs.IR

Query Expansion Should Be Coordinated: Dense Expands, Sparse Anchors

Retrieval-augmented generation (RAG) systems rely on retrieval modules to ground large language model (LLM) outputs. LLM-based query expansion enriches retrieval with document-like passages, but evaluations of hybrid retrieval often fuse fixed top-L prefixes of dense and sparse rankings. Because L controls cross-channel contributions and ranking access, it can alter measured expansion gains. We therefore evaluate complete-list effectiveness and record per-channel replay stopping depths required to certify the ordered top-K. This changes the design: because both rankings determine the fused result, their query constructions should be coordinated rather than designed independently. We present DESA (Dense Expansion and Sparse Anchoring), which shares generated references across channels but specializes their integration. Orthogonal residual expansion adds new semantic directions to the dense query, whereas score-product anchoring reorders the original sparse support without admitting expansion-only matches. The same references thus play complementary roles: Dense expands; Sparse anchors. Across seven BEIR datasets, DESA improves nDCG@10 and Recall@20 over the unexpanded query by 3.82% and 2.38%, while reducing dense and sparse replay stopping depths by 36.90% and 36.56%.

Chunran Zhang

School of Computing and Artificial Intelligence Southwest Jiaotong University Chengdu, China

Dense

Ask PDF