OPERA: Online Data Pruning for Efficient Retrieval Model Adaptation

Haoyang Fang; Shuai Zhang; Yifei Ma; Hengyi Wang; Cuixiong Hu; Katrin Kirchhoff; Bernie Wang; George Karypis

cs.IRMar 17, 2026

OPERA: Online Data Pruning for Efficient Retrieval Model Adaptation

Authors: Haoyang Fang, Shuai Zhang, Yifei Ma, Hengyi Wang, Cuixiong Hu, Katrin Kirchhoff, Bernie Wang, George Karypis

Organizations: Amazon Web Services

Abstract

Domain-specific finetuning is essential for dense retrievers, yet not all data pairs contribute equally to the learning process. We introduce OPERA, a data pruning framework that exploits this heterogeneity to improve both the effectiveness and efficiency of retrieval model adaptation. We first investigate static pruning (SP), which retains only high-similarity query-document pairs, revealing an intrinsic quality-coverage tradeoff: ranking (NDCG) improves while retrieval (Recall) can degrade due to reduced query diversity. To resolve this tradeoff, we propose a two-stage dynamic pruning (DP) strategy that adaptively modulates sampling probabilities at both query and document levels throughout training, prioritizing high-quality examples while maintaining access to the full training set. Evaluations across eight datasets spanning six domains demonstrate the effectiveness of both approaches: SP improves ranking over standard finetuning (NDCG@10 +0.2 points), while DP achieves the strongest performance on both ranking (NDCG@10 +1.0 points) and retrieval (Recall@20 +0.4 points), with an average rank of 1.38 across all methods. These findings scale to Qwen3-Embedding, an LLM-based dense retriever, confirming architecture-agnostic benefits. Notably, DP reaches comparable performance in less than 50% of the training time required by standard finetuning.

Figures & tables

Figure 1: Comparison of sampling probability distributions across training strategies. Left three panels: Standard finetuning (FT) samples all data pairs uniformly, while static pruning (SP) discards the lowest-similarity ones and up-weights the rest, improving ranking but reducing query coverage. Dynamic pruning (DP) at the start of training retains all pairs with non-zero probability, assigning higher sampling weights to high-similarity pairs through soft thresholding. As training progresses, DP redistributes sampling weights: the model discovers informative examples in the low/mid similarity range and up-weights them. Right panel: Average rank of each method on NDCG@10 and Recall@20 across 8 datasets (lower rank is better); DP achieves the best rank on both metrics. See Section 3.6 for empirical sampling weight evolution on real data.

	Lightweight	Adaptive	Continuous Scoring	Full Coverage	Hierarchical	Scheduled
Curriculum	✓	✗	✗	✗	✗	✗
SPL	✗	✓	✗	✗	✗	✗
Active Learning	✗	✓	✗	✗	✗	✗
InfoBatch	✓	✓	✓	✓	✗	✗
DP (ours)	✓	✓	✓	✓	✓	✓

Table 1: Comparison of adaptive training paradigms.

		NDCG@10					Recall@20
					OPERA					OPERA
Domain	Dataset	PT	FT	IB	SP	DP	PT	FT	IB	SP	DP
Nutrition	NFCorpus	0.451	0.466	0.470	0.491	0.480	0.232	0.300	0.304	0.267	0.304
Medical	TripClick (h)	0.219	0.298	0.295	0.270	0.309	0.189	0.242	0.244	0.219	0.252
Medical	TripClick (t)	0.208	0.245	0.248	0.236	0.249	0.334	0.398	0.396	0.373	0.400
Finance	FiQA	0.489	0.514	0.514	0.516	0.524	0.602	0.639	0.640	0.630	0.639

Table 2: OPERA vs. baselines on bge-large-en-v1.5. Best in bold , second-best underlined . † Datasets seen during pretraining.

		NDCG@10					Recall@20
					OPERA					OPERA
Domain	Dataset	PT	FT	IB	SP	DP	PT	FT	IB	SP	DP
Nutrition	NFCorpus	0.441	0.479	0.478	0.487	0.479	0.211	0.314	0.308	0.265	0.311
Non-Factoid QA	ANTIQUE	0.518	0.496	0.506	0.540	0.520	0.398	0.340	0.329	0.396	0.353
Factoid QA	TriviaQA	0.467	0.489	0.484	0.493	0.504	0.403	0.462	0.453	0.421	0.461
Average		0.475	0.488	0.489	0.507	0.501	0.337	0.372	0.363	0.361	0.375

Table 3: Comparison on Qwen3-Embedding-0.6B. Large-corpus datasets are excluded due to the prohibitive cost of embedding millions of documents with an LLM. Best in bold , second-best underlined .

Table 5

Figure 2: Training efficiency on ANTIQUE (unseen) and FEVER (seen). RP and SP use retention rate

k{=}0.25

Update Interval	Per-step overhead	NDCG@10	Recall@20
FT	0.00%	0.892	0.960
DP ( $I_{u}=1$ )	17.40%	0.902	0.962
DP ( $I_{u}=10$ )	2.82%	0.898	0.963

Table 6: Per-step overhead of DP on FEVER, a large-scale dataset. The per-update score pooling and quantile computation scale with dataset size, but a modest update interval (

I_{u}{=}10

) reduces the overhead to

2.82\%

with on-par performance, confirming scalability.

Metric	Baselines			OPERA (Individual)				OPERA (Two-Stage)
Metric	Pretrained	FT	InfoBatch	SP .25	SP .5	SP .75	DP	SP .25 +DP	SP .5 +DP	SP .75 +DP
NDCG@10	0.543	0.570	0.567	0.560	0.560	0.582	0.587	0.560	0.567	0.582
Recall@20	0.414	0.395	0.390	0.430	0.430	0.419	0.411	0.426	0.433	0.422

Table 7: Denoising evaluation on ANTIQUE with noisy positive samples. The FT baseline uses all training data (

k{=}1.0

); SP k then filters within this set at retention rate

k

, and SP k +DP is the two-stage pipeline. Best in bold , second-best underlined .

Appendix figures & tables20 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Domain	#Docs	Train		Test
Dataset	Domain	#Docs	#q	#pos	#q	#pos
NFCorpus	Nutrition	3,633	2,590	110,575	323	12,334
TripClick (h)	Medical	1,523,878	3,529	55,663	1,175	32,067
TripClick (t)	Medical	1,523,878	105,964	424,820	1,175	6,202
FiQA	Finance	57,638	5,500	14,166	648	1,706
ANTIQUE	Non-Factoid QA	403,666	2,426	19,813	200	2,976

Appendix

Table 8: Dataset statistics. #pos: number of positive query-document pairs. † Datasets seen during pretraining.

Dataset	Negative	Mining	$\mathbf{k}$	Training
	Mining	Range		Iters
NFCorpus	Random	—	0.25	32,000
TripClick (h)	Hard	100–1,100	0.25	32,000
TripClick (t)	Hard	100–1,100	0.25	32,000
FiQA	Hard	10–100	0.50	8,000
TriviaQA	Random	—	0.25	16,000

Appendix

Table 9: Dataset-specific hyperparameters, optimized for the FT baseline.

Dataset	Metric	SP	SP (CBS)
NFCorpus	NDCG@10	0.491	0.463
NFCorpus	Recall@20	0.267	0.305
FiQA	NDCG@10	0.511	0.514
FiQA	Recall@20	0.620	0.635
ANTIQUE	NDCG@10	0.564	0.577
ANTIQUE	Recall@20	0.428	0.403

Appendix

Table 10: SP with cosine similarity vs. CBS as pruning metric. Best per row in bold .

Figure 4: Effect of SP data retention rate

k

on NFCorpus. SP improves NDCG@10 over FT at all retention rates.

Variant	Hierarchy	Schedule	Soft	NDCG@10	Recall@20
FT	—	—	—	0.892	0.960
SP	✗	✗	✗	0.915	0.950
DP (hard cutoff)	✓	✓	✗	0.908	0.957
DP (fixed-strength)	✓	✗	✓	0.900	0.952
DP (full)	✓	✓	✓	0.902	0.962

Appendix

Table 11: Controlled single-component ablation of DP on FEVER. Each variant removes one component: Soft (soft sampling vs. hard cutoff), Schedule (cosine scheduling vs. fixed strength

\alpha{=}4,\beta{=}5,v{=}0.25

), and Hierarchy (query/document two-level sampling). DP is the only variant that improves over FT on both metrics. Best in bold .

Model	Method	NDCG@10	Recall@20
BGE-M3	FT	0.466	0.599
	SP	0.484	0.579
	DP	0.475	0.606

Appendix

Table 12: OPERA on BGE-M3 (FiQA), a multilingual XLM-RoBERTa-based retriever with a different backbone, model size, and pretraining recipe from bge-large-en-v1.5. The same quality-coverage tradeoff reproduces: SP improves ranking but degrades recall, while DP improves both over FT. Best in bold .

Method	NDCG@10	Recall@20
FT	0.466	0.300
SP	0.491	0.267
DP ( $\alpha$ : $2\to 5$ , default)	0.480	0.304
DP ( $\alpha$ : $5\to 2$ , reversed)	0.476	0.310

Appendix

Table 13: Effect of query sampling-strength schedule direction on NFCorpus. The default direction (broad

\to

selective,

\alpha

2\to 5

) and the reversed direction (

\alpha

5\to 2

) both outperform FT; the schedule direction acts as a precision/recall knob. Best in bold .

		NDCG@10			Recall@10
Bucket	n	FT	SP	DP	FT	SP	DP
Hard	216	0.209	0.203	0.212	0.254	0.250	0.254
Medium	216	0.438	0.435	0.447	0.524	0.520	0.522
Easy	216	0.716	0.756	0.736	0.809	0.840	0.845
All	648	0.454	0.465	0.465	0.529	0.537	0.540

Appendix

Table 14: Per-query analysis on FiQA (648 test queries) stratified into difficulty tertiles by the pretrained model’s mean cosine similarity to positive documents (hard

\leq 0.670<

medium

\leq 0.717<

easy), evaluated at 4,000 iterations. DP improves over FT on hard and medium queries (where SP degrades), and both SP and DP improve on easy queries; DP’s gains are distributed across all difficulty levels rather than concentrated on easy queries. Best in bold .

Metric	#iterations	FT	RP25	InfoBatch	SP25	DP
NDCG@10	0	0.45101	0.45101	0.45101	0.45101	0.45101
	500	0.45608	0.45768	0.46132	0.47209	0.46939
	1000	0.46758	0.46583	0.46669	0.47429	0.47479
	2000	0.47059	0.46631	0.47209	0.47979	0.47418
	4000	0.47465	0.47516	0.47438	0.48804	0.48388
	8000	0.47471	0.45486	0.47866	0.49103	0.49008

Appendix

Table 15: Full Results on NFCorpus.

Metric	#iterations	FT	RP50	InfoBatch	SP25	SP50	DP
NDCG@10	0	0.48919	0.48919	0.48919	0.48919	0.48919	0.48919
	500	0.5073	0.50502	0.50971	0.50296	0.50469	0.50404
	1000	0.5085	0.50659	0.50761	0.50241	0.50876	0.50655
	2000	0.51277	0.50653	0.51195	0.50229	0.51293	0.51212
	4000	0.5148	0.5026	0.51559	0.51047	0.51555	0.51812
	8000	0.51359	0.50572	0.5137	0.511	0.5162	0.52418

Appendix

Table 16: Full Results on FiQA.

Metric	#iterations	FT	RP25	InfoBatch	SP25	DP
NDCG@10	0	0.54361	0.54361	0.54361	0.54361	0.54361
	500	0.51341	0.50884	0.51514	0.555	0.54143
	1000	0.53297	0.52488	0.53106	0.56221	0.54905
	2000	0.55202	0.54677	0.55086	0.56385	0.56028
	4000	0.57041	0.56046	0.5673	0.56475	0.5814
	8000	0.5756	0.55501	0.56865	0.56304	0.58932

Appendix

Table 17: Full Results on ANTIQUE.

Metric	#iterations	FT	RP25	InfoBatch	SP25	DP
NDCG@10	0	0.48724	0.48724	0.48724	0.48724	0.48724
	500	0.48569	0.48454	0.48815	0.49358	0.49215
	1000	0.48447	0.48167	0.48317	0.49497	0.49271
	2000	0.48155	0.47685	0.48055	0.4958	0.49108
	4000	0.47759	0.4755	0.47892	0.49768	0.49108
	8000	0.47752	0.47493	0.47722	0.49818	0.48562

Appendix

Table 18: Full Results on TriviaQA.

Metric	#iterations	FT	RP25	InfoBatch	SP25	DP
NDCG@10	0	0.21935	0.21935	0.21935	0.21935	0.21935
	500	0.24666	0.24734	0.24895	0.24686	0.25367
	1000	0.25109	0.24906	0.25135	0.25058	0.25591
	2000	0.25992	0.26174	0.25768	0.25393	0.26853
	4000	0.28174	0.27356	0.27998	0.25811	0.28696
	8000	0.29324	0.27947	0.29348	0.26045	0.30409

Appendix

Table 19: Full Results on TripClick (head).

Metric	#iterations	FT	RP25	InfoBatch	SP25	DP
NDCG@10	0	0.20834	0.20834	0.20834	0.20834	0.20834
	500	0.21923	0.21885	0.21742	0.2224	0.22026
	1000	0.21736	0.21728	0.21675	0.22483	0.22031
	2000	0.21753	0.21787	0.21744	0.22763	0.22117
	4000	0.22105	0.21873	0.22282	0.22932	0.22612
	8000	0.23183	0.22836	0.23209	0.22973	0.23327

Appendix

Table 20: Full Results on TripClick (torso).

Metric	#iterations	FT	RP25	InfoBatch	SP25	DP
NDCG@10	0	0.86791	0.86791	0.86791	0.86791	0.86791
	500	0.87421	0.85787	0.87533	0.89561	0.87832
	1000	0.87297	0.85656	0.87007	0.89834	0.88468
	2000	0.87564	0.86304	0.87542	0.90043	0.88611
	4000	0.88167	0.87094	0.88168	0.90521	0.89146
	8000	0.88552	0.87471	0.88762	0.90846	0.89684

Appendix

Table 21: Full Results on FEVER.

Metric	#iterations	FT	RP25	InfoBatch	SP25	DP
NDCG@10	0	0.78999	0.78999	0.78999	0.78999	0.78999
	500	0.79112	0.78914	0.78982	0.79181	0.79469
	1000	0.79348	0.7896	0.79108	0.79162	0.79549
	2000	0.79611	0.79243	0.79308	0.79228	0.79771
	4000	0.79654	0.79531	0.79701	0.79543	0.80125
	8000	0.80141	0.79758	0.80054	0.79639	0.80721

Appendix

Table 22: Full Results on HotpotQA.

Metric	Pretrained	FT	InfoBatch	SP	DP
MRR@1	0.458	0.502	0.511	0.533	0.517
MRR@5	0.543	0.572	0.580	0.595	0.587
MRR@10	0.550	0.581	0.587	0.605	0.595
MRR@20	0.555	0.584	0.590	0.607	0.598
MRR@50	0.556	0.586	0.591	0.609	0.600
MRR@100	0.557	0.586	0.592	0.609	0.600

Appendix

Table 23: Qwen3-Embedding-0.6B on NFCorpus.

Metric	Pretrained	FT	InfoBatch	SP	DP
MRR@1	0.469	0.392	0.394	0.390	0.384
MRR@5	0.543	0.465	0.466	0.464	0.464
MRR@10	0.555	0.474	0.473	0.473	0.471
MRR@20	0.559	0.479	0.478	0.477	0.476
MRR@50	0.561	0.480	0.480	0.480	0.478
MRR@100	0.561	0.481	0.481	0.480	0.479

Appendix

Table 24: Qwen3-Embedding-0.6B on FiQA. The pretrained model outperforms all finetuned methods, likely due to high-quality financial data in its pretraining corpus.

Metric	Pretrained	FT	InfoBatch	SP	DP
MRR@1	0.615	0.580	0.625	0.640	0.615
MRR@5	0.718	0.682	0.707	0.738	0.696
MRR@10	0.724	0.689	0.716	0.746	0.703
MRR@20	0.727	0.692	0.718	0.747	0.706
MRR@50	0.728	0.693	0.719	0.748	0.707
MRR@100	0.728	0.693	0.719	0.748	0.707

Appendix

Table 25: Qwen3-Embedding-0.6B on ANTIQUE.

Metric	Pretrained	FT	InfoBatch	SP	DP
MRR@1	0.519	0.519	0.517	0.538	0.547
MRR@5	0.602	0.615	0.611	0.620	0.634
MRR@10	0.611	0.624	0.619	0.627	0.642
MRR@20	0.614	0.627	0.622	0.630	0.645
MRR@50	0.616	0.628	0.624	0.631	0.646
MRR@100	0.616	0.628	0.624	0.631	0.647

Appendix

Table 26: Qwen3-Embedding-0.6B on TriviaQA.

Dataset Pruning

Dataset pruning selects compact, informative training subsets to reduce computation while preserving or improving model performance.

See all 22 papers

Towards Dynamic Dense Retrieval with Routing Strategy
Feb 26, 2026Zhan Su, Fengran Mo, Jinghan Zhang +4Prefix Tuning Dense Retrieval
Université de Montréal, Montréal, Québec, Canada · Clemson University, Clemson, South Carolina, USA · University of Washington, America
Align Then Adapt: Label-Efficient Adapter Learning for Asymmetric Dense Retrieval
Apr 3, 2026Seiji Maekawa, Moin Aminnaseri, Pouya Pezeshkpour +1Dense Retrieval Transfer Learning
Megagon Labs
Static Pruning Across Sparse Retrieval Regimes: What Transfers, What Breaks, and What Still Helps
Aug 17, 2026Zirui Song, Yuye Zhu, Yang YangLearned Sparse Retrieval
Amazon Web Services Shanghai, China

Feb 26, 2026J/K move · Enter open · S save

Towards Dynamic Dense Retrieval with Routing Strategy

Zhan Su, Fengran Mo, Jinghan Zhang, Yuchen Hui+3

Université de Montréal, Montréal, Québec, Canada · Clemson University, Clemson, South Carolina, USA · University of Washington, America

Prefix Tuning Dense Retrieval Domain Adaptation Adaptive Model Routing

Open paper AskPDF arXiv

The de facto paradigm for applying dense retrieval (DR) to new tasks involves fine-tuning a pre-trained model for a specific task. However, this paradigm has two significant limitations: (1) It is difficult adapt the DR to a new domain if the training dataset is limited. (2) Old DR models are simply replaced by newer models that are trained from scratch when the former are no longer up to date. Especially for scenarios where the model needs to be updated frequently, this paradigm is prohibitively expensive. To address these challenges, we propose a novel dense retrieval approach, termed dynamic dense retrieval (DDR). DDR uses prefix tuning as a module specialized for a specific domain. These modules can then be compositional combined with a dynamic routing strategy, enabling highly flexible domain adaptation in the retrieval part. Extensive evaluation on six zero-shot downstream tasks demonstrates that this approach can surpass DR while utilizing only 2% of the training parameters, paving the way to achieve more flexible dense retrieval in IR. We see it as a promising future direction for applying dense retrieval to various tasks.

Abstract

Dataset Pruning

Explore similar work

Towards Dynamic Dense Retrieval with Routing Strategy

Align Then Adapt: Label-Efficient Adapter Learning for Asymmetric Dense Retrieval

Static Pruning Across Sparse Retrieval Regimes: What Transfers, What Breaks, and What Still Helps