Static Pruning Across Sparse Retrieval Regimes: What Transfers, What Breaks, and What Still Helps

Zirui Song; Yuye Zhu; Yang Yang

cs.IRAug 17, 2026

Static Pruning Across Sparse Retrieval Regimes: What Transfers, What Breaks, and What Still Helps

Authors: Zirui Song, Yuye Zhu, Yang Yang

Organizations: Amazon Web Services Shanghai, China

Abstract

Static pruning is widely used to accelerate sparse neural retrieval, yet existing studies each validate their conclusions within a single custom pipeline, leaving it unclear which findings transfer to modern engines with different index organizations and dynamic pruning mechanisms. We present the first cross-engine pruning portability study, evaluating static pruning strategies across three engines - a controlled C++ pipeline (exhaustive inverted index), BMP (block-max pruning), and SEISMIC (clustered inverted indexes) - on two benchmarks (MS MARCO, Natural Questions) with two encoders spanning opposite query-density regimes (SPLADE: 44 avg. query terms; V3-GTE: 7 avg. query terms), totaling 1,140 experimental configurations, with an additional deep-judgment validation on TREC DL 2019/2020. We find that index-side pruning (document and posting-list) is portable: it consistently reduces latency (1.2-6.6 $\times$ ) and index size (18-82%) across all engines because sparse retrieval is memory-bound - a conclusion we support with cache-miss, TLB, and IPC profiling. In contrast, query pruning is already internalized by modern engines: it yields 4-11 $\times$ speedup on the exhaustive pipeline but is subsumed by BMP's $β$ and SEISMIC's query_cut. Static pruning complements dynamic pruning: on BMP, combining document and query reduction yields 2.5 $\times$ speedup with NDCG@10 within 0.003 of the exact baseline. Finally, NDCG@10 saturates while Recall@10 is still in the ${\sim}$ 85-95% range across all three engines, providing a portable stopping criterion: practitioners can push pruning to this knee without visible ranking degradation. Together, these findings answer what transfers (index-side pruning), what breaks (query pruning), and what still helps (static atop dynamic pruning).

Figures & tables

Configuration	Query nnz	Doc nnz	Docs	NDCG@10	DL’19/20
MS MARCO + SPLADE	43.95	119.96	8.8M	0.449	0.726
MS MARCO + V3-GTE	6.89	180.41	8.8M	0.428	0.720
NQ + SPLADE	46.97	147.52	2.7M	0.539	—
NQ + V3-GTE	7.20	185.00	2.7M	0.582	—

Table 1. Dataset and encoder statistics. Avg. non-zero counts (nnz) per query and document, corpus size, baseline NDCG@10 on the controlled pipeline, and baseline NDCG@10 on the deep-judgment TREC DL 2019/2020 topics (mean over both years; MS MARCO corpus).

Figure 1. Custom C++ Pareto frontiers on MS MARCO: speedup vs. Recall@10 for all six pruning families ({Doc, Query, Posting}

\times

{Alpha-Mass, Max-Ratio}). Left : SPLADE. Right : V3-GTE. Document pruning provides the most stable high-recall frontier across both encoders. Six Pareto curves comparing pruning families on two encoder regimes. Document pruning provides the most stable frontier in both.

Dataset+Enc.	Pruning	Lat. (ms)	Spd.	Idx (GB)	R@10	NDCG	MRR
MS+SPL	Baseline	70.1	1.0 $\times$	8.15	1.000	0.449	0.383
	Q AM 0.50	11.5	6.1 $\times$	8.15	0.964	0.448	0.383
	Doc AM 0.50	11.7	6.0 $\times$	1.53	0.928	0.443	0.379
	Doc MR 0.30	18.1	3.9 $\times$	2.16	0.974	0.447	0.381
	Post MR 0.30	15.6	4.5 $\times$	2.00	0.979	0.447	0.382
MS+GTE	Baseline	29.5	1.0 $\times$	12.23	1.000	0.428	0.362

Table 2. Controlled pipeline (

\Psi

): representative operating points with two-stage re-rank (

k^{\prime}

= 50). Speedup relative to unpruned baseline. Idx is the in-memory inverted-index size.

Figure 2. SEISMIC Pareto frontiers (qc = 5): speedup vs. Recall@10 under document and query pruning for both encoders. Document pruning provides consistent gains; query pruning overlaps with SEISMIC’s internal query_cut . SEISMIC Pareto curves showing document pruning provides consistent gains while query pruning overlaps with internal query cut.

		Speedup				Recall@10				$\Delta$ NDCG@10				$\Delta$ MRR@10
Config	Pruning	C++	C++ †	BMP	SEIS.	C++	C++ †	BMP	SEIS.	C++	C++ †	BMP	SEIS.	C++	C++ †	BMP	SEIS.
MS+SPL	Doc AM 0.90	1.49 $\times$	1.48 $\times$	1.43 $\times$	1.18 $\times$	1.000	.946	.948	.868	.000	$-$ .003	$-$ .003	$-$ .005	.000	$-$ .003	$-$ .002	$-$ .005
	Post MR 0.10	1.56 $\times$	1.54 $\times$	1.47 $\times$	1.17 $\times$	1.000	.947	.948	.871	.000	$-$ .002	$-$ .001	$-$ .004	.000	$-$ .002	$-$ .002	$-$ .004
	Query MR 0.10	1.68 $\times$	1.65 $\times$	1.91 $\times$ ∗	1.17 $\times$	1.000	.945	.935	.924	.000	.000	.000	+.001	.000	$-$ .001	.000	.000
MS+GTE	Doc AM 0.90	1.78 $\times$	1.61 $\times$	1.23 $\times$	1.42 $\times$	1.000	.969	.969	.937	.000	$-$ .001	$-$ .001	$-$ .002	.000	.000	$-$ .001	$-$ .001
	Post MR 0.10	1.86 $\times$	1.68 $\times$	1.27 $\times$	1.58 $\times$	1.000	.970	.969	.939	.000	$-$ .002	$-$ .002	$-$ .002	.000	$-$ .001	$-$ .002	$-$ .002

Table 3. Portability Matrix: three pruning methods

\times

four configurations, each evaluated on three engines. Speedup is relative to each engine’s unpruned baseline.

\Delta

NDCG and

\Delta

MRR show absolute change vs. the exact baseline (negative = degradation);

\Delta

MRR tracks

\Delta

NDCG throughout. C++ uses two-stage re-ranking (

k^{\prime}

= 50); C++ † shows single-stage (no re-ranking) for direct comparison with BMP/SEISMIC. ∗ BMP external query MR at

\beta

= 1.0.

Figure 3. Per-query latency histogram strips on MS MARCO+SPLADE (controlled pipeline) at four document-pruning levels. The P50/P90/P95 markers show progressive IQR compression: moderate pruning (

\alpha_{d}

= 0.9) narrows the distribution while aggressive pruning (

\alpha_{d}

= 0.5) concentrates it tightly with minimal tail degradation. Histogram strips showing per-query latency distributions at four pruning levels with KDE outlines and percentile markers.

Index	Query	Lat. ( $\mu$ s)	Spd.	R@10	NDCG	MRR	Idx
Full	Full	8,915	1.00 $\times$	1.000	0.449	0.382	15.7
Full	Q MR 0.10	4,680	1.91 $\times$	0.935	0.449	0.382	15.7
Pruned	Full	6,235	1.43 $\times$	0.948	0.446	0.380	10.1
Pruned	Q MR 0.10	3,537	2.52 $\times$	0.917	0.446	0.379	10.1

Table 4. BMP 2

\times

2 factorial on MS MARCO+SPLADE (

\alpha

= 1.0). Speedup relative to the full-index, full-query baseline. Index size in GB.

Figure 4. BMP 2

\times

2 factorial on MS MARCO+SPLADE: latency under four conditions (full/pruned index

\times

full/pruned queries). Recall@10 shown inside each bar. Combined pruning achieves the largest speedup (2.52

\times

) with minimal NDCG loss (

\Delta

NDCG =

-

0.003). Bar chart showing four BMP conditions with latency bars and recall values. Combined pruning achieves the best speedup.

Index	Query	Lat. ( $\mu$ s)	Spd.	R@10	NDCG	MRR	Idx
Full	Full	185	1.00 $\times$	0.977	0.443	0.379	8.1
Full	Q MR 0.10	161	1.15 $\times$	0.924	0.444	0.379	8.1
Pruned	Full	158	1.17 $\times$	0.868	0.438	0.374	6.0
Pruned	Q MR 0.10	133	1.39 $\times$	0.856	0.440	0.376	6.0

Table 5. SEISMIC on MS MARCO+SPLADE (qc = 5, hf = 1.0). Speedup relative to the unpruned baseline at qc = 5. Index size in GB.

Figure 5. Micro-architectural profiling of

\Phi

\Psi

\Xi

with Linux perf on multiple pruning configurations: (a) cache-miss rate, (b) IPC, (c) dTLB-miss rate, (d) per-query latency vs. cache-miss rate (

R^{2}

= 0.84). Latency is tightly coupled to memory locality, not arithmetic throughput. Four-panel profiling figure: bar charts for cache-miss rate, IPC, and dTLB-miss rate, plus a scatter plot of latency versus cache-miss rate with regression line.

Engine	Config	IPC	Cache%	dTLB%	CacheMiss/Q
C++ $\Phi$	scatter-add	1.58	30.0	97.0	740,000
C++ $\Psi$	window-switch	2.50	2.7	2.6	67,000
BMP	$\beta$ =1.0	3.37	1.3	28.4	114,000
BMP	$\beta$ =0.5	3.37	1.8	25.0	89,000
SEISMIC	qc=20	3.01	10.9	40.9	12,500
SEISMIC	qc=5	3.29	7.7	37.7	6,500

Table 6. Micro-architectural profile across engines (MS MARCO+SPLADE). Cache-miss and dTLB-miss are rates; CacheMiss/Q is the absolute count per query driving latency.

Figure 6. Latency vs. quality across three engines (columns) and two dataset–encoder regimes (rows: MS MARCO+SPLADE, top; NQ+V3-GTE, bottom). The dashed line marks the NDCG@10 knee (the most aggressive point within 0.005 of the unpruned baseline); the gold-shaded region to its right is where NDCG@10 has saturated while Recall@10 keeps rising, confirming the metric gap is engine-independent. Six panels: three engines (columns) by two dataset–encoder regimes (rows). A dashed line marks the NDCG knee in each panel; the gold-shaded region to its right highlights where NDCG has saturated while Recall keeps rising.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Config	Pruning	Idx (GB)	Lat ( $\mu$ s)	R@10	NDCG	MRR	Spdup
MS+SPL	None	15.7	8,915	1.000	0.449	0.382	1.00 $\times$
	AM 0.50	4.3	2,869	0.685	0.398	0.335	3.11 $\times$
	AM 0.70	6.4	4,166	0.832	0.431	0.366	2.14 $\times$
	AM 0.90	10.1	6,235	0.948	0.446	0.380	1.43 $\times$
	AM 0.95	11.7	7,111	0.972	0.448	0.381	1.25 $\times$
	MR 0.10	10.4	6,281	0.951	0.446	0.380	1.42 $\times$

Appendix

Table 7. BMP document/posting/query pruning across all four dataset–encoder combinations (

\alpha

=1.0). Speedup relative to each config’s unpruned baseline.

Figure 7. Recall@10 under document pruning across the four dataset–encoder combinations. The two-stage C++ curve shows the effect of full-vector re-ranking, while the no-rerank C++ curve closely tracks BMP at the main operating point (AM=0.90). SEISMIC exhibits a larger Recall drop under the same static pruning threshold, indicating that pruning thresholds have different effective severity under cluster-based traversal. The gray band marks Recall@10

\geq 0.90

. Four-panel figure comparing Recall@10 under document alpha-mass pruning for C++ with reranking, C++ without reranking, BMP, and SEISMIC.

Config	Pruning	Idx (GB)	Lat ( $\mu$ s)	R@10	NDCG	MRR
MS+SPL	None	8.1	186	0.977	0.443	0.379
	AM 0.70	4.3	140	0.769	0.424	0.361
	AM 0.90	6.0	158	0.868	0.438	0.374
	AM 0.95	6.7	167	0.889	0.441	0.378
NQ+SPL	None	7.0	234	0.937	0.533	0.485
	AM 0.70	4.5	167	0.727	0.505	0.460

Appendix

Table 8. SEISMIC document pruning across all configurations (qc=5, hf=1.0).

	SPLADE			V3-GTE
Config	C++	BMP	SEIS.	C++	BMP	SEIS.
Baseline	.726	.726	.721	.720	.718	.721
Doc AM 0.90	.717	.719	.707	.721	.721	.718
Post MR 0.10	.720	.723	.716	.720	.719	.721
Doc AM 0.50	.680	.691	.675	.688	.685	.676
Query prune	.728	.728	.722	.723	.620	.720

Appendix

Table 9. TREC DL 2019/2020 mean NDCG@10 (DL19+DL20 pooled). Query pruning uses each engine’s native mechanism (C++ MR 0.10; BMP

\beta

=0.5; SEISMIC query_cut =5)

Learned Sparse Retrieval

Learned sparse retrieval trains models to expand queries and documents into weighted, indexable terms for efficient semantic search.

See all 18 papers

OPERA: Online Data Pruning for Efficient Retrieval Model Adaptation
Mar 17, 2026Haoyang Fang, Shuai Zhang, Yifei Ma +5Dataset Pruning Dense Retrieval
Amazon Web Services
ColBERTSaR: Sparsified ColBERT Index via Product Quantization
Jun 4, 2026Eugene Yang, Andrew Yates, Dawn Lawrie +3Vector Quantization Multi-Vector Retrieval
Johns Hopkins University, Baltimore, MD, USA
Beyond FLOPs: Benchmarking Real Inference Acceleration of LLM Pruning under a GEMM-Centric Taxonomy
Jun 8, 2026Haozhe Hu, Hao Wu, Anhao Zhao +4LLM Pruning LLM Inference Acceleration
Ningbo Institute of Digital Twin, Eastern Institute of Technology, Ningbo · Department of Computing, The Hong Kong Polytechnic University · Munich Center for Machine Learning, LMU Munich

Mar 17, 2026J/K move · Enter open · S save

OPERA: Online Data Pruning for Efficient Retrieval Model Adaptation

Haoyang Fang, Shuai Zhang, Yifei Ma, Hengyi Wang+4

Amazon Web Services

Dataset Pruning Dense Retrieval

Open paper AskPDF arXiv

Domain-specific finetuning is essential for dense retrievers, yet not all data pairs contribute equally to the learning process. We introduce OPERA, a data pruning framework that exploits this heterogeneity to improve both the effectiveness and efficiency of retrieval model adaptation. We first investigate static pruning (SP), which retains only high-similarity query-document pairs, revealing an intrinsic quality-coverage tradeoff: ranking (NDCG) improves while retrieval (Recall) can degrade due to reduced query diversity. To resolve this tradeoff, we propose a two-stage dynamic pruning (DP) strategy that adaptively modulates sampling probabilities at both query and document levels throughout training, prioritizing high-quality examples while maintaining access to the full training set. Evaluations across eight datasets spanning six domains demonstrate the effectiveness of both approaches: SP improves ranking over standard finetuning (NDCG@10 +0.2 points), while DP achieves the strongest performance on both ranking (NDCG@10 +1.0 points) and retrieval (Recall@20 +0.4 points), with an average rank of 1.38 across all methods. These findings scale to Qwen3-Embedding, an LLM-based dense retriever, confirming architecture-agnostic benefits. Notably, DP reaches comparable performance in less than 50% of the training time required by standard finetuning.

Abstract

Learned Sparse Retrieval

Explore similar work

OPERA: Online Data Pruning for Efficient Retrieval Model Adaptation

ColBERTSaR: Sparsified ColBERT Index via Product Quantization

Beyond FLOPs: Benchmarking Real Inference Acceleration of LLM Pruning under a GEMM-Centric Taxonomy