The Fusion Tax: A Diagnostic Analysis of When Hybrid Retrieval Hurts

Aaditya Chauhan; Arun Agarwal

cs.IROct 22, 2026

The Fusion Tax: A Diagnostic Analysis of When Hybrid Retrieval Hurts

Authors: Aaditya Chauhan, Arun Agarwal

Organizations: Independent Researcher, Bothell, WA 98012 USA · Independent Researcher, Issaquah, WA 98027 USA

Abstract

Hybrid retrieval, which fuses a sparse lexical retriever with a dense neural one, is widely recommended in the software documentation surveyed in September 2026, usually with reciprocal rank fusion (RRF), on the strength of aggregate gains. We ask what fusion costs on individual queries, treating "do not fuse" as a first-class outcome, in the setting in which hybrid retrieval is deployed: a standard Anserini BM25, five dense encoders spanning 22M-596M parameters and three model generations, a learned-sparse (SPLADE) alternative, and a cross-encoder second stage, on seven BEIR datasets and three fusion functions. Even the best method, dense-weighted score interpolation, ranks relevant documents below the better of its two components on 13-48% of queries and below the deployable dense-only retriever on 8-21% (14-48% and 11-25% at a weight selected under leave-one-corpus-out validation), while beating dense-only in aggregate on every corpus with the primary encoder; the downside is significant under loss-averse risk-sensitive evaluation on six of the seven corpora. We call this loss the fusion tax. It is mostly a ranking effect that preserves recall in aggregate, it is most likely on the queries that only one retriever covers, and it persists across the encoder ladder and the SPLADE substitution. RRF carries the higher tax wherever the dense retriever is clearly stronger and only beats dense-only where the two components are balanced. Fusion failure is weakly predictable within a corpus but transfers unreliably across corpora, because the features that predict it change sign with the sparse-dense balance. A cross-encoder recovers part of the tax only where it is itself reliable. We distill these findings into guidance that keys the fusion decision to the measured sparse-dense balance of the target corpus.

Figures & tables

	Bruch et al. [ 45 ]	Benham & Culpepper [ 4 ]	Fusion-QPP [ 5 , 6 ]	Routing [ 7 , 8 , 9 ]	Concurrent [ 59 , 60 ]	This work
Unit of analysis	aggregate	per-query	per-query	per-query	aggregate	per-query
Loss reference	–	fixed weak BM25 run	absolute fused effectiveness	–	best single system (mean)	$\max$ (BM25, dense) and always-dense
Components	sparse + dense	classical lexical runs	classical lexical runs	sparse / dense / hybrid	sparse + dense	sparse + dense; SPLADE + dense
Fusion functions	RRF, convex, learned	RRF, CombSUM, …	CombSUM/MNZ	convex ( $\alpha$ )	RRF, convex	RRF, convex, CombMNZ
Dense-encoder scale ladder	–	–	–	–	–	✓ 22M–596M, three generations
Learned-sparse substitution	✓	–	–	–	–	✓ (per-component attribution)

TABLE I: Positioning against prior lines of work. “Loss reference” is the quantity a per-query loss is measured against; LOCO = leave-one-corpus-out.

Dataset	Domain	# Docs	# Queries
SciFact	Scientific claims	5,183	300
FiQA	Financial QA	57,638	648
NFCorpus	Medical/nutrition	3,633	323
TREC-COVID	Biomedical	171,332	50
ArguAna	Argument retrieval	8,674	1,406
SciDocs	Scientific citations	25,657	1,000

TABLE II: Dataset statistics (BEIR test splits).

	BM25 (nDCG@10)			Dense (nDCG@10)
Dataset	Anserini (ours)	Anserini (publ.)	rank_bm25	MiniLM	BGE-sm	BGE-base	BGE-lg	Qwen3
SciFact	0.6647	0.6647	0.6295	0.6451	0.7200	0.7376	0.7346	0.6992
FiQA	0.2361	0.2361	0.2016	0.3687	0.3848	0.3909	0.4432	0.4667
NFCorpus	0.3254	0.3254	0.2881	0.3159	0.3371	0.3681	0.3688	0.3552
ArguAna	0.4142	0.4142	0.3561	0.5017	0.5950	0.6362	0.6357	0.6737
SciDocs	0.1581	0.1581	0.1438	0.2164	0.1973	0.2045	0.2001	0.2274

TABLE III: Baseline validation: single-retriever nDCG@10 on all seven datasets. Anserini BM25 (ours) vs. the published Anserini regression values [ 14 ] ; the rank_bm25 runs used for the implementation-sensitivity check; and dense retrieval across the five encoders.

	Selected $\alpha$			nDCG@10				Tax vs. dense (%)			Helped
Dataset	In-sample	LOCO	CV	Dense	$\alpha{=}0.7$	LOCO	CV	$\alpha{=}0.7$	LOCO	CV	LOCO (%)
SciFact	0.7	0.6	0.7	0.720	0.731	0.727	0.721	10.0	12.7	11.3	15.7
FiQA	0.7	0.6	0.7	0.385	0.409	0.405	0.409	14.7	20.7	14.7	23.9
NFCorpus	0.6	0.7	0.6	0.337	0.355	0.355	0.358	18.3	18.3	20.7	33.4
ArguAna	0.8	0.6	0.8	0.595	0.603	0.591	0.605	19.1	25.0	13.9	22.7
SciDocs	0.7	0.6	0.7	0.197	0.207	0.205	0.207	20.9	24.6	20.9	28.1

TABLE IV: Held-out selection of the interpolation weight (BGE-small). In-sample: best

\alpha

on the evaluation queries; LOCO:

\alpha

chosen on the other six corpora; CV: within-corpus 5-fold, mode of the selected weights. Tax vs. dense is the fraction of queries on which fusion scores below dense-only at the given weight; Helped is the fraction it improves at the LOCO weight.

Dataset	BM25	Dense	RRF	Interp(0.3)	Interp(0.5)	Interp(0.7)	CombMNZ
SciFact	0.6647	0.7200	0.7152	0.7113	0.7273	0.7307	0.7230
FiQA	0.2361	0.3848	0.3667	0.3157	0.3886	0.4087	0.3872
NFCorpus	0.3254	0.3371	0.3570	0.3493	0.3560	0.3547	0.3574
ArguAna	0.4142	0.5950	0.5549	0.5115	0.5717	0.6027	0.5702
SciDocs	0.1581	0.1973	0.1954	0.1839	0.1999	0.2070	0.1976
TREC-COVID	0.6559	0.6660	0.8010	0.7077	0.7475	0.7166	0.8045

TABLE V: nDCG@10 across retrieval and fusion methods (BGE-small, Anserini BM25). Best per row in bold.

Fig. 1: Per-query BM25 vs. dense nDCG@10 (BGE-small). For visualization only, each query is colored by the best of the three fusion methods with a 0.01 deadband: green, fusion helped ( $\tau<-0.01$ ); red, fusion hurt ( $\tau>0.01$ ); gray, neutral. Harm concentrates off the diagonal, where the retrievers disagree.

Dataset	Both-Win	Dense-Only	Sparse-Only	Both-Fail
SciFact	76.3%	9.3%	5.3%	9.0%
FiQA	41.5%	22.5%	6.3%	29.6%
NFCorpus	62.5%	7.4%	6.5%	23.5%
ArguAna	65.3%	20.1%	5.0%	9.6%
SciDocs	42.8%	17.0%	6.7%	33.5%
TREC-COVID	100.0%	0.0%	0.0%	0.0%

TABLE VI: Diagnostic quadrant distribution (% of queries, BGE-small, Anserini BM25).

Fig. 2: Distribution of the per-query fusion tax

\tau

(Eq. 1 ) pooled over the seven datasets (BGE-small). Positive values are queries that fusion hurt; the long right tail (log-scale counts) shows that when fusion hurts, the loss can exceed 0.5 nDCG@10.

Dataset	Sh.	Hurt	Help	Net	Sh.	Hurt	Help	Net	Sh.	Hurt	Help	Net	Sh.	Hurt	Help	Net
	Both-Win				Dense-Only				Sparse-Only				Both-Fail
SciFact	76	12	4	-0.024	9	61	7	-0.107	5	94	0	-0.319	9	0	0	+0.000
FiQA	42	22	28	+0.008	23	38	16	-0.029	6	90	5	-0.271	30	0	4	+0.008
NFCorpus	63	45	25	-0.026	7	62	8	-0.035	7	95	5	-0.220	24	0	0	+0.000
ArguAna	65	23	10	-0.031	20	56	12	-0.096	5	87	7	-0.243	10	0	9	+0.027
SciDocs	43	44	24	-0.020	17	65	16	-0.041	7	94	2	-0.142	34	0	2	+0.003

TABLE VII: The interpolation tax by diagnostic quadrant (BGE-small, vs. the per-query oracle). Sh.: quadrant share of queries (%); Hurt/Help: % of the quadrant’s queries hurt/helped; Net: mean

\Delta

nDCG@10 within the quadrant.

Interp(0.7)
	vs. always-dense (deployable)					vs. per-query oracle $\max(\text{BM25},\text{dense})$
Dataset	Hurt %	Loss	Help %	Gain	Net $\Delta$	Hurt %	Loss	Help %	Gain	Net $\Delta$
SciFact	10.0	0.205	14.7	0.213	+0.0107	20.0	0.250	3.7	0.128	-0.0453
FiQA	14.7	0.100	23.1	0.166	+0.0239	23.3	0.157	16.5	0.111	-0.0181
NFCorpus	18.3	0.044	33.4	0.077	+0.0177	39.0	0.103	16.4	0.042	-0.0332
ArguAna	19.1	0.213	21.8	0.221	+0.0077	30.4	0.221	10.3	0.172	-0.0492

TABLE VIII: Symmetric harm/benefit accounting of fusion (BGE-small). Hurt/Help: percentage of queries with lower/higher nDCG@10 than the reference; Loss/Gain: mean absolute change on those queries; Net

\Delta

: mean change over all queries (Eq. 2 ).

Fig. 3: Fusion tax (% of queries hurt) as a function of dense-encoder size (log scale) for interpolation (left) and RRF (right), all seven corpora. The tax does not shrink as encoders scale from 22M to 596M parameters.

		nDCG@10			Fusion tax (% hurt)
Dataset	Encoder	Dense	Interp(0.7)	RRF	Interp(0.7)	RRF
SciFact	MiniLM (22M)	0.645	0.693	0.700	19.3%	21.0%
	BGE-small (33M)	0.720	0.731	0.715	20.0%	23.3%
	BGE-base (110M)	0.738	0.757	0.731	18.0%	21.3%
	BGE-large (335M)	0.735	0.755	0.736	16.7%	22.0%
	Qwen3-Emb (596M, 2025)	0.699	0.728	0.716	19.3%	23.3%

TABLE IX: The fusion tax across five encoders spanning 22M–596M parameters and three model generations, all seven corpora. BGE-small/base/large form a controlled same-family scale ladder; MiniLM and Qwen3-Embedding are cross-family and cross-generation checks. “Fusion tax” is the oracle tax: the percentage of queries each fusion method hurts relative to the better component.

	nDCG@10				Tax % (Interp 0.7)		Tax % (RRF)
Dataset	BM25	SPLADE	Dense	SP+De	SP+De	BM+De	SP+De	BM+De
SciFact	0.665	0.708	0.720	0.737	21.7%	20.0%	22.7%	23.3%
FiQA	0.236	0.356	0.385	0.411	29.6%	23.3%	32.7%	35.8%
NFCorpus	0.325	0.352	0.337	0.362	41.2%	39.0%	46.1%	39.0%
ArguAna	0.414	0.508	0.595	0.619	27.8%	30.4%	33.9%	40.1%
SciDocs	0.158	0.158	0.197	0.203	35.7%	36.2%	39.4%	38.1%

TABLE X: Robustness to the sparse method (BGE-small). SPLADE replaces BM25 as the sparse side; “Tax %” is the oracle tax; SP+De = SPLADE+dense, BM+De = BM25+dense.

Feature	SciFact	FiQA	NFCorpus	ArguAna	SciDocs	TREC-COVID	Quora	$+$ / $-$
dense margin	0.18 -	0.07 -	0.04	0.08 -	0.06	0.14 +	0.35 -	1/4
score-ratio $\alpha$	0.06 -	0.11	0.06 +	0.07 -	0.07	0.10 +	0.05 -	2/3
mean IDF	0.11 -	0.05 +	0.08 -	0.07	0.07	0.09	0.03 -	1/3
WIG (dense)	0.04 +	0.06 +	0.07	0.08 +	0.09 +	0.10 -	0.06 +	5/1
WIG (BM25)	0.09	0.09 +	0.05 +	0.09 +	0.10 +	0.06 +	0.02	5/0
centroid dist.	0.03	0.07	0.12 -	0.07	0.07 +	0.06 +	0.04	2/1

TABLE XI: Per-corpus GBT feature importance for predicting interpolation failure (bold: top-3 per corpus). Superscripts give the sign of the feature’s Spearman association with failure (

|\rho|\geq 0.05

); the last column counts corpora with positive/negative association.

	Interp(0.7) failure		RRF failure
Dataset	In-domain	Cross-corpus	In-domain	Cross-corpus
SciFact	0.70	0.74	0.67	0.71
FiQA	0.58	0.55	0.55	0.57
NFCorpus	0.61	0.49	0.60	0.50
ArguAna	0.59	0.60	0.55	0.55
SciDocs	0.56	0.56	0.58	0.55

TABLE XII: Predicting fusion failure from 17 query features (held-out ROC-AUC, BGE-small). In-domain: 5-fold within the corpus; cross-corpus: trained on the other six.

Dataset	Dense	Interp	Dense	Interp	RRF	Before	After	Net $\Delta$ %	Before	After	Net $\Delta$ %
	First stage		Reranked (nDCG@10)			Tax vs. dense, Interp(0.7)			Tax vs. dense, RRF
SciFact	0.720	0.731	0.739	0.738	0.740	10.0	6.7	33	15.3	8.3	46
FiQA	0.385	0.409	0.426	0.429	0.426	14.7	15.6	-6	30.4	19.9	34
NFCorpus	0.337	0.355	0.340	0.350	0.354	18.3	10.8	41	23.8	21.7	9
ArguAna	0.595	0.603	0.479	0.490	0.548	19.1	1.8	90	32.9	4.4	87
SciDocs	0.197	0.207	0.190	0.190	0.184	20.9	21.0	0	27.6	26.8	3

TABLE XIII: Reranking recovery (BGE-reranker-v2-m3 over the top-30 of each candidate source, BGE-small, Anserini BM25; Quora subsampled to 1,000 queries). “First stage” and “Reranked” give nDCG@10 before and after reranking; Tax vs. dense is the fraction of queries on which the fusion arm scores below the dense arm, before and after reranking, and Net

\Delta

the net change in the prevalence of that tax (

1-\text{after}/\text{before}

); the matched-query transitions behind it are in Supplementary Table S12. On ArguAna and SciDocs the reranker lowers nDCG@10 for every arm, and on Quora it leaves the dense arm unchanged and lowers the interpolated arm, so the collapse of the tax on these corpora reflects the reranker overriding first-stage order, not recovery.

		Index size (MB)			Per-query latency, median (ms)
Dataset	Docs	Lucene	BGE-sm	BGE-lg	BM25	Dense enc.	FAISS	SPLADE enc.	Interp	RRF
SciFact	5k	11	8	21	1.7	6.6	0.1	17.8	0.08	0.05
FiQA	58k	67	88	236	2.0	6.1	1.3	18.1	0.08	0.06
NFCorpus	4k	8	6	15	1.1	5.3	0.1	18.5	0.07	0.05
ArguAna	9k	13	13	36	2.9	19.1	0.2	52.9	0.08	0.06
SciDocs	26k	238	39	105	2.7	6.2	0.6	17.1	0.08	0.05

TABLE XIV: Cost of each component on one consumer workstation (Apple M5, CPU cores; no discrete GPU): index sizes and per-query latencies, median over 200 queries (50 on TREC-COVID, and 50 on every corpus for SPLADE) at batch size one. Dense enc. and FAISS are BGE-small query encoding and exact search; SPLADE enc. is query encoding only; Interp and RRF are the fusion step over two top-100 lists.

Hybrid Retrieval

Hybrid retrieval combines lexical, dense, and structured search signals to find relevant evidence more reliably.

See all 91 papers

Evidence-Unit Fairness and the Limits of Query-Adaptive Sparse-Dense Fusion in Financial Document Retrieval
Jul 31, 2026Chenyu Wu, You LinHybrid IR Information Retrieval
Duke University Durham, NC, USA · Independent Researcher New York, NY, USA
Parameterized Dense-Sparse Fusion for Hybrid Retrieval: Tuning a Rank-Score Mix on BEIR SciFact with Qdrant
Sep 19, 2026Satyanarayan Pati, Srikanth PatilHybrid Retrieval Rank Aggregation
Involead Services Pvt. Ltd.
Do Static Embeddings Add Value to Hybrid Dutch Retrieval?
Aug 3, 2026António Pereira BarataText Embeddings Word Embeddings

Jul 31, 2026J/K move · Enter open · S save

Evidence-Unit Fairness and the Limits of Query-Adaptive Sparse-Dense Fusion in Financial Document Retrieval

Chenyu Wu, You Lin

Duke University Durham, NC, USA · Independent Researcher New York, NY, USA

Hybrid IR Information Retrieval IR Evaluation Evidence Retrieval Finance

Open paper AskPDF arXiv

Retrieval over financial filings is difficult because queries are short and acronym-heavy while the answer-bearing evidence sits inside long, table-dense documents. We study sparse-dense hybrid retrieval on FinDER, a benchmark of expert-annotated questions over corporate 10-K filings. Our first finding is methodological: if the retrieval unit is larger than the dense encoder's input window, the dense model never sees a large share of the labeled evidence, confounding comparison against a full-text sparse baseline. We measure this directly and reduce the imbalance with windows chosen for the encoder budget. On this corpus, fusing BM25 and a compact dense encoder improves reference-level Hit@10 by roughly 28 percent over either component, and training-free, untuned reciprocal rank fusion exceeds the equal-weight blend in an exploratory comparison. We then ask whether choosing the fusion weight per query helps: an oracle over the interpolation-weight grid shows headroom of 21.8 percent, yet none of the three lightweight adaptive routers (a score-confidence heuristic, a random forest over query features, and a ridge regressor over query embeddings) establishes a statistically reliable improvement over the fixed blend under company-grouped cross-validation with cluster-robust inference. Simple fusion is a strong baseline here, and we discuss why per-query weighting does not capture the available headroom.

Abstract

Hybrid Retrieval

Explore similar work

Evidence-Unit Fairness and the Limits of Query-Adaptive Sparse-Dense Fusion in Financial Document Retrieval

Parameterized Dense-Sparse Fusion for Hybrid Retrieval: Tuning a Rank-Score Mix on BEIR SciFact with Qdrant

Do Static Embeddings Add Value to Hybrid Dutch Retrieval?