When Does Dense Retrieval Need Asymmetric Geometry? A Bias-Variance Theory of Shared and Dual Projections

Maojun Sun; Yancheng Yuan; Jian Huang; Ruijian Han

cs.IRSep 26, 2026

When Does Dense Retrieval Need Asymmetric Geometry? A Bias-Variance Theory of Shared and Dual Projections

Authors: Maojun Sun, Yancheng Yuan, Jian Huang, Ruijian Han

Organizations: The Hong Kong Polytechnic University

Abstract

Dense retrieval powers retrieval-augmented generation, semantic search, and question answering, yet the theoretical basis for choosing between shared and dual query-document projections remains unclear. We introduce a bias-variance theory for low-rank bilinear scoring. Shared projections induce positive-semidefinite operators, whereas dual projections realize arbitrary low-rank operators. We derive their exact approximation gap and prove a local Gaussian boundary: dual has lower risk exactly when squared directional signal exceeds the estimation cost of its additional degrees of freedom. This boundary motivates the Cross-fitted Asymmetry Risk Selector (CARS), which estimates reproducible directional signal from training pairs; its Gaussian counterpart admits exact selection-power and regret formulas. Guided by the theory, we run retrieval experiments across multiple datasets and embedding models. The mean Dual-minus-Shared NDCG@10 advantage more than doubles as query rotation increases from 0 degrees to 90 degrees. In the rank-sample-size grids, Shared wins 13 of 16 cells at n=32, whereas Dual wins all 32 cells at n=1024 and n=2048. Consistent with this shift, all 168 comparable operator-risk curves move toward Dual as training data grow. Compared to the two fixed-geometry baselines, CARS reduces held-out regret by 49-96% and achieves 90.1% mean geometry-selection accuracy.

Figures & tables

Figure 1: Shared and dual projections for dense retrieval. A shared map $P$ induces a PSD operator $M=P^{\top}P$ , while dual maps $A$ and $B$ induce a general operator $M=A^{\top}B$ of rank at most $r$ .

Figure 2: Geometry behind the boundary. A: the shared PSD family lies inside the dual family;

\delta

schematically marks a target’s departure from sharing. B: locally, sharing removes

k

dual-only noise coordinates but also discards signal along them;

k

is the extra dimension derived in Section 3 .

Figure 3: A Shared-to-Dual crossover in controlled two-view retrieval. A: Mean Dual-minus-Shared NDCG@10 across mismatch and training size; the gold contour marks zero. B: The same gap versus population distance $G_{\rm PSD}$ to the shared PSD family; bands show 95% normal confidence intervals.

Figure 4: Raw asymmetry is inflated, while correction better predicts held-out fit. A: Training PSD distance exceeds its population value; shading shows a 95%

t

interval. B: Held-out Shared-minus-Dual operator-loss advantage across raw and corrected score deciles. Positive values favor Dual.

Figure 5: Dual-minus-Shared full-corpus NDCG@10 under query rotation. Each bar averages three paired seeds for BGE-base and GTE-base on each dataset.

Figure 6: Rank–sample-size retrieval gaps. Each cell is mean Dual-minus-Shared test NDCG@10 over three paired seeds, in units of $10^{-4}$ ; blue favors Shared and red favors Dual.

Figure 7: Held-out operator fit moves toward Dual with more training queries. A: NQ; B: SciFact. Each point averages ranks and three seeds within one encoder; positive values indicate lower held-out loss for Dual.

Rule	Dataset	BGE-base			Contriever			E5-base-v2			GTE-base
Rule	Dataset	Regret	Accuracy	Wins	Regret	Accuracy	Wins	Regret	Accuracy	Wins	Regret	Accuracy	Wins
Always Shared	ArguAna	0.0950	45.8%	0/5	0.1310	40.0%	0/5	0.2543	30.0%	0/5	0.1294	43.3%	0/5
	MS MARCO	0.0680	75.6%	1/5	0.3254	46.2%	0/5	0.0839	75.0%	1/5	0.0442	79.4%	1/5
	FEVER	0.2490	33.1%	1/5	0.4959	21.9%	0/5	0.3968	25.0%	0/5	0.2999	37.5%	1/5
	NQ	0.0050	85.0%	1/5	0.1915	40.0%	0/5	0.1020	54.3%	1/5	0.0232	74.3%	2/5
	SciFact	0.0226	63.3%	1/5	0.0288	58.3%	0/5	0.0202	65.0%	2/5	0.0374	65.0%	1/5

Table 1: Geometry selection on five datasets and four encoders. Lower held-out operator-risk regret and higher decision accuracy or strict outer-fold wins indicate better performance. Regret and accuracy are averaged over five held-out query folds.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

$\delta^{2}/(\sigma^{2}k)$	Shared risk	Dual risk	$\Pr(\widehat{\pi}=d)$
0.0	32.97/33	63.03/63	0.001/0.001
0.5	48.05/48	63.05/63	0.094/0.093
1.0	62.93/63	62.97/63	0.473/0.474
2.0	93.10/93	63.15/63	0.970/0.969

Appendix

Table 2: Local risk and SURE calibration. Entries are simulation/theory; risks are scaled by

n

Figure 8: Held-out operator-risk trajectories for BGE-base and Contriever (84 curves). Rows are FEVER and NQ (24 curves each), followed by ArguAna and SciFact (18 each). Each line fixes a dataset, encoder, rank, and seed and connects measured nested training sizes. Color denotes rank; line style denotes seed. Positive normalized Shared-minus-Dual loss favors Dual.

Figure 9: The remaining 84 held-out operator-risk trajectories, for E5-base-v2 and GTE-base. Grouping, axes, colors, and line styles match Figure 8 .

Dense Retrieval

Dense retrieval uses learned vector representations to find relevant documents, passages, products, or tools by semantic similarity.

See all 152 papers

The Voronoi Bottleneck: Capacity-Aware Dense Retrieval for Product Search
Jun 9, 2026Charith Chandra Sai Balne, Rithwik Maramraju, Siddharth Pratap Singh +4Contrastive Learning Information Retrieval
Walmart Global Tech USA · Walmart Global Tech2026 USA
Align Then Adapt: Label-Efficient Adapter Learning for Asymmetric Dense Retrieval
Apr 3, 2026Seiji Maekawa, Moin Aminnaseri, Pouya Pezeshkpour +1Dense Retrieval Transfer Learning
Megagon Labs
Test-Time Training for Zero-Resource Dense Retrieval Reranking
May 31, 2026Shiyan Liu, Yichen LiZero-Shot Learning Learning to Rank
Huazhong University of Science and Technology · ByteDance

Jun 9, 2026J/K move · Enter open · S save

The Voronoi Bottleneck: Capacity-Aware Dense Retrieval for Product Search

Charith Chandra Sai Balne, Rithwik Maramraju, Siddharth Pratap Singh, Rohit Upadhyay+3

Walmart Global Tech USA · Walmart Global Tech2026 USA

Contrastive Learning Information Retrieval Dense Retrieval

Open paper AskPDF arXiv

Dense embedding retrieval compresses all relevance information into a single inner product, imposing a fundamental geometric limit -- the Voronoi Bottleneck -- on the number of query-document relevance patterns expressible at fixed embedding dimension (d). We make three contributions. (1) Unified capacity theory. We prove that Voronoi complexity and sign-rank are equivalent for top-1 retrieval, yielding tight dimension bounds and a computable diagnostic, the Capacity Utilization Score (CUS), that predicts per-query retrieval failure with AUC (> 0.8) without relevance labels. (2) Diagnosis. CUS identifies two capacity regimes -- moderate ((δ\gtrsim 1)), where density-aware training yields measurable gains, and vacuous ((δ\ll 1)), where it does not -- giving practitioners an a priori check before investing in retraining. (3) DART training. We introduce AT-DW-InfoNCE, an Adaptive-Temperature Density-Weighted contrastive objective with formally derived optimal weighting (α^* = 2.0). On a 100K-query synthetic product-search corpus with controlled relevance structure, DART improves +1.9 Recall@100 over a same-data InfoNCE baseline ((84.9 \pm 0.0) vs. (83.0 \pm 0.3); 8 seeds, (p < 0.001)), outperforming focal loss and temperature-schedule alternatives. DART requires zero inference-time overhead -- it is a drop-in training objective that improves any dual-encoder system.

Abstract

Dense Retrieval

Explore similar work

The Voronoi Bottleneck: Capacity-Aware Dense Retrieval for Product Search

Align Then Adapt: Label-Efficient Adapter Learning for Asymmetric Dense Retrieval

Test-Time Training for Zero-Resource Dense Retrieval Reranking