Beyond Recall Saturation: Scaling Hard-Negative Reranker Training for Industrial Legal Retrieval

Yunhan Li; Mingjie Xie; Zeyang Shi; Gengshen Wu; Min Yang

cs.CLOct 17, 2026

Beyond Recall Saturation: Scaling Hard-Negative Reranker Training for Industrial Legal Retrieval

Authors: Yunhan Li, Mingjie Xie, Zeyang Shi, Gengshen Wu, Min Yang

Organizations: SIAT-DELI AI and Law Joint Lab, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences Shenzhen, China · City University of Macau Macau, China · City University of Macauy Shenzhen, China

Abstract

Industrial retrieval systems often improve quality by enlarging embedding candidate pools or adopting complex training pipelines. We show that, under a sufficiently large but practical candidate pool, embedding recall exhibits diminishing returns, motivating a closer examination of reranker quality. On Chinese legal retrieval benchmarks, embedding top-300 reaches a high-recall operating region, while relevant evidence may still be ranked below the output cutoff. We compare hard-negative strategies for reranker training, including embedding-mined, self-mined, and dynamic-negative variants, under aligned retrieval protocols that evaluate reranking after candidate retrieval. Among the evaluated staged configurations, an embedding-to-self-mined pipeline provides a favorable trade-off; additional dynamic stages do not improve all held-out metrics. The selected pipeline scales from a 0.6B to a 4B reranker. Relative to the native 4B reranker, the trained 4B Stage2 model improves reported metrics on the STARD and CSAID benchmarks and on an internal development set, and is preferred in 72.4% of non-tied comparisons on 410 de-identified, reconstructed internal operational-query inputs in a latency-sensitive offline preference evaluation. These results support evaluating the retrieval and reranking stages separately when selecting an optimization target for industrial retrieval systems.

Figures & tables

Fig. 1: From candidate coverage to evidence placement. (a) Retrieval places relevant evidence in a candidate pool. (b) Some retrieved evidence remains below the top- $L$ cutoff. (c) The research objective is to improve reranking so that more relevant evidence enters the final list under the same candidate set and output budget. Blue documents denote relevant evidence; gray documents denote other candidates. The rankings and complete coverage shown in this example are illustrative, not measured results or a claim of perfect retrieval or reranking.

Data	Type	Size	Role
Internal legal data	Generated/curated query-evidence pairs	9,221	Training pool
STARD train	Public legal retrieval training data	1,234	Training pool
STARD test	Public held-out benchmark	309	Final benchmark evaluation
CSAID	Public held-out benchmark	118	Final benchmark evaluation
EVAL	Internal development set	–	Configuration comparison; not a final benchmark
Internal operational-query inputs	De-identified and reconstructed	410	Preference evaluation

TABLE I: Overview of the data roles used in this work. Internal validation is used only for training-time checkpoint selection; EVAL is a development set rather than a final benchmark.

Fig. 2: Embedding recall on CSAID and STARD as the candidate pool increases. The curves motivate using a sufficiently large candidate pool for reranker evaluation rather than treating recall expansion as the only optimization target.

Embedding model	Dataset	R@100	R@200	R@250	R@300	$\Delta_{250\rightarrow 300}$
Qwen3-0.6B	STARD	83.31	88.31	90.17	91.89	1.73
Qwen3-0.6B	CSAID	71.48	81.07	84.26	86.03	1.77
Qwen3-4B	STARD	91.47	94.07	94.81	95.34	0.54
Qwen3-4B	CSAID	85.62	92.24	94.33	95.07	0.74

TABLE II: Embedding recall at selected candidate budgets on the test sets. Values are percentages;

\Delta_{250\rightarrow 300}

is the increase in recall from top-250 to top-300, in percentage points.

Fig. 3: Overview of embedding-mined, fixed reranker-mined, and dynamic negative supervision. The columns represent alternative training strategies; the connections to the inference pipeline indicate the use of a reranker trained with one selected strategy, not a model ensemble. Training-set construction retains the original known positives, independently of candidate recall, and combines them with sampled hard, medium-difficulty, and easy negatives after excluding known positives from all negative-sampling pools. Dynamic mining includes epoch-level and online refresh, while the feedback arrow denotes model parameter updates. At inference, embedding retrieval produces top-

K

candidates and the trained reranker returns the top-

L

documents.

Fig. 4: Comparison between the two experimental training designs used in this work. (a) Independent strategy comparison, where each negative-mining strategy is trained separately starting from the same base reranker. (b) Sequential stage composition, where Stage2 inherits Stage1 and the two Stage3 variants are alternative branches from Stage2, rather than consecutive stages.

Dataset	Model	MRR	NDCG	R@10	R@20	R@30
EVAL	Base Reranker	0.6651	0.6814	0.7839	0.8490	0.8717
	Embedding-mined	0.7056	0.7079	0.7829	0.8491	0.8630
	Self-Mined	0.7081	0.7083	0.7882	0.8413	0.8595
	Epoch-dynamic	0.7010	0.7146	0.7826	0.8557	0.8818
	Online-dynamic	0.6897	0.6906	0.7609	0.8056	0.8192
STARD	Base Reranker	0.5778	0.5883	0.6651	0.7543	0.7884

TABLE III: 0.6B negative-mining ablation under the aligned standard evaluation protocol.

Dataset	Model	MRR	NDCG	R@10	R@20	R@30
EVAL	Base Reranker	0.6651	0.6814	0.7839	0.8490	0.8717
	Stage1	0.7066	0.7085	0.7839	0.8472	0.8630
	Stage2	0.7124	0.7132	0.7918	0.8444	0.8597
	Stage3 (epoch-dynamic)	0.7270	0.7199	0.7911	0.8356	0.8463
	Stage3 (online dynamic)	0.7238	0.7188	0.7900	0.8346	0.8460
STARD	Base Reranker	0.5778	0.5883	0.6651	0.7543	0.7884

TABLE IV: Sequential stage progression on the Qwen3-0.6B reranker backbone. Stage2 provides a strong trade-off, while additional Stage3 variants are not uniformly better.

Dataset	Model	MRR	NDCG	R@10	R@20	R@30
EVAL	0.6B Stage2	0.7124	0.7132	0.7918	0.8444	0.8597
	4B Base	0.7356	0.7616	0.8749	0.9301	0.9477
	4B Stage2	0.7982	0.8147	0.9259	0.9542	0.9612
STARD	0.6B Stage2	0.5310	0.4910	0.5494	0.5603	0.5632
	4B Base	0.6783	0.6851	0.7736	0.8384	0.8816
	4B Stage2	0.7126	0.7062	0.7918	0.8579	0.8816

TABLE V: Comparison of the 0.6B Stage2 model, native 4B reranker, and trained 4B Stage2 model under the aligned standard evaluation protocol.

Comparison	$n$	Win	Tie	Lose	Win Rate (95% CI)	$p$
4B Stage2 vs 4B Base	410	239	80	91	72.4% [67.4%, 77.0%]	$3.11\times 10^{-15}$

TABLE VI: LLM-based pairwise preference evaluation on 410 de-identified, reconstructed internal operational-query inputs. Win rate and its 95% Wilson confidence interval (CI) exclude ties; the

p

-value is from a two-sided exact sign test.

LLM Reranking

Large language model reranking uses LLMs to reorder retrieved documents or candidate outputs by relevance, usefulness, or task-specific criteria.

See all 133 papers

Re-Ranking Through an Attribution Lens for Citation Quality in Legal QA
Jun 2, 2026Mohamed Hesham Elganayni, Selim SalehCross-Encoder Reranking Feature Attribution
Technical University of Munich, Munich, Germany
Lychee-Reranker: Fast but Not Late Interaction for Compressed Document Reranking
Jun 22, 2026Xinping Zhao, Jiaxin Xu, Ziqi Dai +9Representation Learning Learning to Rank
Harbin Institute of Technology (Shenzhen) · Shenzhen Loop Area Institute · Soochow University
Legal RAG Bench: an end-to-end benchmark for legal RAG
Mar 2, 2026Abdur-Rahman Butler, Umar ButlerRetrieval-Augmented Generation RAG Evaluation
Isaacus

Jun 2, 2026J/K move · Enter open · S save

Re-Ranking Through an Attribution Lens for Citation Quality in Legal QA

Mohamed Hesham Elganayni, Selim Saleh

Technical University of Munich, Munich, Germany

Cross-Encoder Reranking Feature Attribution Perturbation-Based Feature Attribution LLM Reranking Legal QA RAG Evaluation

Open paper AskPDF arXiv

Retrieval-augmented generation systems for legal question answering typically retrieve passages based on semantic similarity and provide them to a language model, which then generates cited answers. Prior work assumes that highly ranked passages are most likely to be usefully cited by the model. Perturbation-based attribution methods, such as C-LIME, have been used exclusively for post-hoc explanation. However, on the AQuAECHR benchmark, semantic similarity does not correlate with passage attribution. Within a retriever's candidate pool, similarity-based ranking performs worse than random selection at surfacing gold citation paragraphs. To address this limitation, a lightweight cross-encoder is trained on continuous perturbation-based attribution scores to re-rank passages prior to generation. This approach is evaluated on the AQuAECHR benchmark, using two language models and five-fold cross-validation. The re-ranker substantially improves citation faithfulness and alignment with gold expert answers. Notably, two re-rankers trained independently on different models converge beyond their raw attribution agreement. This finding indicates that the cross-encoder reduces model-specific noise and produces a shared relevance signal that partially transfers across models, although same-model re-ranking remains more effective. These results demonstrate that perturbation-based attribution provides a practical, model-agnostic training signal for citation-aware retrieval.

Abstract

LLM Reranking

Explore similar work

Re-Ranking Through an Attribution Lens for Citation Quality in Legal QA

Lychee-Reranker: Fast but Not Late Interaction for Compressed Document Reranking

Legal RAG Bench: an end-to-end benchmark for legal RAG