What Tracks Hard-Negative Transfer? A Study of Lexical and Semantic Signals

Aarush Sinha; Rahul Seetharaman; Aman Bansal

cs.IRMar 22, 2026

What Tracks Hard-Negative Transfer? A Study of Lexical and Semantic Signals

Authors: Aarush Sinha, Rahul Seetharaman, Aman Bansal

Organizations: University of Copenhagen Copenhagen, Denmark · Independent Researcher USA

Abstract

Which properties of hard-negative supervision track dense-retriever transfer before training? We compare lexical noncoverage, frozenencoder similarity statistics, hardness proxies, and combinations of these signals across BM25, dense, LLM-generated, and hybrid sources for MS MARCO training. Evaluation uses three retriever backbones and eleven BEIR datasets. Mean lexical residuality, an IDF-weighted measure of query-token noncoverage, reproduces the aggregate transfer ordering in all six backbone and setting blocks. The individual semantic statistics recover none of the complete orderings, and combining them with the lexical signal does not improve the observed rank agreement. After approximately matching BM25, dense, and LLM diagnostic samples for lexical residuality, none of the tested semantic statistics recovers the observed LLM > Dense > BM25 transfer ordering across three backbones, three matching resolutions, and 100 repetitions. Constructed controls further show that unrelated and false-negative-shaped passages can exhibit favorable lexical and semantic statistics. The findings identify a simple lexical association in this setting while distinguishing source level transfer agreement from the reliability of individual negatives.

Figures & tables

Model	Setting	Mining	Dataset											Mean
Model	Setting	Mining	ArguAna	NFCorpus	Quora	SciDocs	SciFact	TREC COVID	Webis Touche	FiQA	NQ	FEVER	Climate FEVER	Mean
DistilBERT	Non-hybrid	BM25	0.328	0.250	0.827	0.132	0.414	0.490	0.151	0.104	0.342	0.606	0.141	0.344
		Dense	0.464	0.266	0.840	0.125	0.492	0.452	0.159	0.211	0.315	0.620	0.164	0.373
		LLM	0.456	0.262	0.836	0.135	0.488	0.493	0.190	0.221	0.368	0.668	0.197	0.392
	Hybrid	BM25+Dense	0.369	0.258	0.833	0.122	0.424	0.500	0.141	0.108	0.330	0.601	0.150	0.349
		BM25+LLM	0.357	0.258	0.829	0.130	0.435	0.520	0.155	0.124	0.366	0.626	0.145	0.359

Table 1: BEIR nDCG@10 after MS MARCO fine-tuning. Results are grouped by backbone and by non-hybrid versus hybrid mining. Mean averages the listed datasets. Bold marks the best score within each block and column.

Source	Mean $\psi$	D	E	C
BM25	.288	.344	.452	.349
Dense	.387	.373	.465	.397
LLM	.531	.392	.479	.434
BM25+Dense	.338	.349	.439	.334
BM25+LLM	.410	.359	.450	.359
Dense+LLM	.461	.381	.459	.402

Table 2: Lexical residuality and mean BEIR nDCG@10. Mean

\psi

comes from the

5{,}000

-row sensitivity sample with alphanumeric tokens and pooled-document IDF (Table 9 ). D/E/C: DistilBERT, E5-base, and Contriever. Compare within each setting.

Model	Setting	Source	Composite	Mean $\\|g_{j,k}\\|_{2}^{2}$	MNRL proxy	Ambiguity (any-of- $K$ )
DistilBERT	Non-hybrid	BM25	0.1282	27.59	2.416	0.881
		Dense	0.1760	22.96	2.434	0.883
		LLM	0.2610	29.83	2.184	0.733
	Hybrid	BM25+Dense	0.1521	25.41	3.077	0.934
		BM25+LLM	0.1947	28.66	2.955	0.911
		Dense+LLM	0.2193	25.55	2.958	0.915

Table 3: Reported frozen target-backbone diagnostics on

25{,}000

seeded records per source. Ambiguity is the record-level any-of-

K

inversion rate. Gradient energy and MNRL proxy retain their original reported values. The pairwise gradient derivation appears in Appendix B.4 . Bold marks column maxima within backbone and setting.

Variant	Criterion (D/E/C)	Blocks	$r$ (NH)	$r$ (H)
Mean $\psi$ only	n/a	6/6	0.96	0.96
Mean $\rho$ only	n/a	0/6	0.62	0.69
Mean $\eta$ only	n/a	0/6	0.12	0.03
Inversion rate	n/a	0/6	$-0.58$	$-0.67$
Combined attributes (log-det)	no/no/no	6/6	0.93	0.95
Trace form	no/no/no	6/6	0.93	0.95

Table 4: Component and weighting comparisons on

25{,}000

records. Blocks counts complete transfer orderings recovered, out of six.

r

is Pearson correlation with mean BEIR nDCG@10 over three sources per backbone and setting, averaged across backbones (NH: individual, H: hybrid). Control criterion requires all eight controls to rank below all six genuine sources; D/E/C denote the three backbones. Only one of the 36 tested backbone-weighting cells meets it. n/a denotes an untested comparison. Trace includes the zero-residual convention.

Source	DistilBERT	E5-base	Contriever
Easy shuffled	0.602 (1)	0.444 (3)	0.443 (4)
Random passages	0.602 (3)	0.444 (1)	0.444 (3)
Neighbor positive	0.395 (5)	0.360 (5)	0.500 (1)
LLM	0.261 (7)	0.211 (7)	0.181 (7)
BM25	0.128 (13)	0.097 (13)	0.067 (13)
Own-positive mix	0.064 (14)	0.048 (14)	0.034 (14)

Table 5: Selected sources ranked by the combined attribute score at

25{,}000

records, with rank out of fourteen in parentheses. Six controls outrank the strongest genuine source on every backbone. Omitted ranks belong to the random-mixing ladder; rank 2 on every backbone is its

100

percent endpoint, an independent draw from the same distribution as the random-passage control. Values are printed to three decimals; near-ties among the degenerate controls resolve only at higher precision (ranks 1 to 3 on DistilBERT span

0.6017

0.6022

Source	$\overline{\rho}$	$\overline{\eta}$	$\overline{\psi}$	inv.	Flags
BM25	0.602	0.352	0.287	0.378	none
Dense	0.566	0.443	0.386	0.423	none
LLM	0.870	0.375	0.529	0.083	none
BM25+Dense	0.584	0.398	0.337	0.400	none
BM25+LLM	0.736	0.364	0.408	0.230	none
Dense+LLM	0.720	0.409	0.459	0.251	none

Table 6: The audit profile on all fourteen sources, in-family Contriever at

25{,}000

records. Thresholds come from the six real sources only: flag if

\overline{\rho}>0.870

\overline{\eta}>0.443

\overline{\psi}>0.529

, or inversion

<0.083

. Seven of eight controls are flagged and no genuine source is flagged by construction.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Setting	Mining method	$\mathbf{ECI}_{\mathrm{sem}}\uparrow$	$\mathbf{ECI}_{\mathrm{sem}}$ / dim. $\uparrow$
DistilBERT	Non-hybrid	BM25	0.1265	0.000165
		Dense	0.1755	0.000228
		LLM	0.2612	0.000340
	Hybrid	BM25+Dense	0.1510	0.000197
		BM25+LLM	0.1939	0.000252
		Dense+LLM	0.2192	0.000285

Appendix

Table 7: Training-free

\mathrm{ECI}_{\mathrm{sem}}

scores, computed in family, that is, with a frozen copy of the same backbone that is later fine-tuned, over all explicit negatives in each source file. Bold marks the best score within each block.

Setting	$25\%$	$50\%$	$75\%$	$100\%$
Non-hybrid	1.000	1.000	1.000	1.000
Hybrid	1.000	1.000	1.000	1.000

Appendix

Table 8: Rank stability as sampled rows increase, using lexical-residual pairwise

\mathrm{ECI}_{\mathrm{sem}}

. Every fraction reproduces the full-sample order, LLM

>

Dense

>

BM25 for non-hybrid and Dense+LLM

>

BM25+LLM

>

BM25+Dense for hybrid;

1.000

denotes exact agreement with it.

Setting	Source	Base $\mathrm{ECI}_{\mathrm{sem}}$	Tokenizer Range	IDF Range	Mean $C$	Mean $\psi$
Non-hybrid	BM25	0.0885	0.0882 to 0.0934	0.0884 to 0.1020	0.712	0.288
	Dense	0.1086	0.0926 to 0.1118	0.1085 to 0.1131	0.613	0.387
	LLM	0.1860	0.1720 to 0.1900	0.1857 to 0.1866	0.469	0.531
Hybrid	BM25+Dense	0.0986	0.0914 to 0.1026	0.0985 to 0.1076	0.662	0.338
	BM25+LLM	0.1373	0.1312 to 0.1417	0.1371 to 0.1438	0.590	0.410
	Dense+LLM	0.1478	0.1328 to 0.1514	0.1477 to 0.1497	0.539	0.461

Appendix

Table 9: Tokenizer and IDF-corpus sensitivity for

\psi

. Baseline uses alphanumeric tokenization and pooled document IDF. Ranges vary one factor at a time. Computed on

5{,}000

seeded rows per source with frozen prdev/mini-gte embeddings.

Source	$\rho<0.5$	$\eta\leq 0.25$	$C\geq 0.50$	Valid high- $C$	Valid low- $\eta$
BM25	16.8	43.5	89.6	7.9	$<0.1$
Dense	33.5	41.4	69.3	4.6	2.9
LLM	2.1	39.6	38.0	4.1	2.6
BM25+Dense	25.2	42.4	79.5	6.2	1.5
BM25+LLM	9.5	41.6	63.8	6.0	1.3
Dense+LLM	17.7	40.5	53.4	4.3	2.8

Appendix

Table 10: Failure-bucket rates for

\eta

and

\psi

, as percentages of explicit negatives in the

5{,}000

row sensitivity sample, with frozen prdev/mini-gte embeddings. Upper block non-hybrid, lower block hybrid.

Information Retrieval

Information Retrieval studies how to find, rank, and organize relevant information in large collections of data.

See all 471 papers

E-SENS: Exclusion-Sensitive Penalization for Negative-Constraint Retrieval
Aug 31, 2026Yerang Kim, Jiyoon Myung, Joohyung HanLearning to Rank Dense Retrieval
Independent Researcher · Lomin
When Hard Negatives Hurt: Bridging the Generative-Discriminative Gap in Hard Negative Synthesis for Retrieval
May 31, 2026Zhicheng Zhang, Jiwei Tang, Kuicai Dong +9Contrastive Learning Hard Negative Mining
Shenzhen International Graduate School, Tsinghua University Shenzhen, China · Huawei Technologies Co., Ltd. Shenzhen, China · City University of Hong Kong Hong Kong SAR, China +4
Natural Hard-Negative Mining from Positives for Instruction-Following Retrieval
Oct 27, 2026Chaolang Li, Jingyu Li, Zhaocheng Du +3Hard Negative Mining Contrastive Learning

Aug 31, 2026J/K move · Enter open · S save

E-SENS: Exclusion-Sensitive Penalization for Negative-Constraint Retrieval

Yerang Kim, Jiyoon Myung, Joohyung Han

Independent Researcher · Lomin

Learning to Rank Dense Retrieval

Open paper AskPDF arXiv

Retrieval-augmented language models can fail to respect negative constraints when the retriever supplies evidence about concepts the user explicitly excluded. Beyond explicit negation, queries may ask for answers that include one concept while excluding another, or for entities that belong to a category but differ from a closely related instance. Because the excluded concept still appears in the query text, dense retrievers may assign high similarity to documents about that concept even when the user asks to avoid it. We introduce E-SENS, a training-free reranking method for negation-sensitive retrieval. E-SENS extracts a compact trap query for the excluded side and subtracts trap-query similarity from the original-query retrieval score. On ExcluIR, E-SENS shows a clear recall-violation trade-off across four embedding models and reduces trap retrieval at recall-preserving settings.

Abstract

Information Retrieval

Explore similar work

E-SENS: Exclusion-Sensitive Penalization for Negative-Constraint Retrieval

When Hard Negatives Hurt: Bridging the Generative-Discriminative Gap in Hard Negative Synthesis for Retrieval

Natural Hard-Negative Mining from Positives for Instruction-Following Retrieval