Relevance-Guided Residual Calibration for Spectral Compression of Dense Retrieval Embeddings

Yongkang Li; Panagiotis Eustratiadis; Evangelos Kanoulas

cs.IROct 26, 2026

Relevance-Guided Residual Calibration for Spectral Compression of Dense Retrieval Embeddings

Authors: Yongkang Li, Panagiotis Eustratiadis, Evangelos Kanoulas

Organizations: University of Amsterdam Amsterdam, The Netherlands

Abstract

Spectral compression reduces the storage and scoring costs of dense retrieval by projecting embeddings onto a smaller principal subspace. However, its corpus-derived weighting rules tie all retained directions to a shared spectral trend, without directly accounting for their individual contributions to relevance. To address this, we propose Relevance-Guided Residual Calibration (RRC), a closed-form method for refining spectral weights using relevance judgments. RRC estimates margin-based weights from the means and variances of positive--negative score contributions along each retained direction. It retains directions with positive estimated weights, decomposes their log weights into a fitted power-law trend and a residual, and adds the residual to the compressor's log weights on this support. This allows relevance judgments to adjust individual directions while preserving the compressor's overall spectral trend. RRC applies to PCA, Whitening, and SpecTemp, and incorporates the corrections into the existing transformation shared by documents and queries, without iterative model training or an additional inference stage. Experiments with three encoders on nine datasets covering conventional and reasoning-intensive retrieval show that SpecTemp + RRC achieves the highest mean effectiveness among tested compressors on the BEIR group and BRIGHT at every target dimension. On the five evaluated BRIGHT datasets, it improves over SpecTemp by an average of 2.34 nDCG@10 points at 768 dimensions. The gains also persist under scalar and product quantization, improving the trade-off between per-document storage and retrieval effectiveness. Our code is available at: https://anonymous.4open.science/r/rrc-spectral-compression-B5CE/

Figures & tables

Model	Params	Dim	Length	MRL	Reasoning	Release
Qwen3-8B	8.0B	4096	32k	✓	✗	Jun 2025
GTE-7B	7.0B	3584	32k	✗	✗	Jun 2024
DIVER-4B	4.0B	2560	40k	✗	✓	Sep 2025

Table 1. Retrieval model statistics (Params: total parameters; MRL: Matryoshka Representation Learning support).

Method $\downarrow$ $k$ $\rightarrow$	BEIR group					BRIGHT
Method $\downarrow$ $k$ $\rightarrow$	768	512	256	128	Win/Tie/Loss	768	512	256	128	Win/Tie/Loss
Full Dimension	56.52					31.99
PCA ( $k{=}D$ )	56.16				–	30.89				–
PCA ( $k{=}D$ ) + RRC	56.43				8/1/3	33.13				10/0/5
Prefix Truncation	54.31	52.40	46.78	40.05	–	30.94	29.62	25.53	19.81	–
Random Truncation	54.58	53.34	49.79	42.69	–	30.00	29.05	26.09	21.02	–

Table 2. Mean effectiveness (

\times 100

) on the BEIR group (12 encoder–dataset configurations; MRR@10 on MS MARCO, nDCG@10 otherwise) and BRIGHT (15 configurations; nDCG@10). Group means and Win/Tie/Loss counts use configuration scores averaged over three seeds and rounded to two decimal places; ties indicate equality at that precision. Gray cells show full-dimensional references; bold marks the best compressed result in each column. Win/Tie/Loss compares each calibrated method with its corresponding uncalibrated prior. Counts cover 12 BEIR and 15 BRIGHT configurations for PCA (

k{=}D

) + RRC, and are summed over the four target dimensions for compressed methods (48 and 60 comparisons, respectively).

Figure 1. Power-law fit quality versus the gain of SpecTemp + RRC over SpecTemp at

k=768

and

512

. Fit scores use 80 queries per training fold on the BEIR group and 60–114 on BRIGHT; gains use the main experiment’s training budgets. Each point averages three seeds. Colours and shapes denote encoders; filled markers denote BRIGHT and hollow markers the BEIR group. Gains use nDCG@10 (

\times 100

), except MRR@10 on MS MARCO.

Figure 2. Mean nDCG@10 (

\times 100

) over 15 encoder–dataset configurations on BRIGHT, averaged over three seeds, for the power-law prior

\lambda_{i}^{-\gamma}

and the same prior with RRC, at

k=768

and

512

. Shaded regions show the effectiveness gains from RRC; the dashed line is the mean full-dimensional score of the same configurations.

Reference	Weights	$k=768$			$k=512$
Reference	Weights	nDCG	$\Delta$	Wins	nDCG	$\Delta$	Wins
–	SpecTemp	30.19	–	–	29.75	–	–
SpecTemp	SpecTemp on $\mathcal{I}_{+}$	31.73	$+1.54^{*}$	12/15	31.30	$+1.55^{*}$	10/15
	Direct $\widehat{\mathbf{e}}$	31.50	$+1.31$	7/15	31.24	$+1.49$	8/15
	SpecTemp + RRC	32.53	$+2.34^{*}$	11/15	31.96	$+2.21^{*}$	11/15
SpecTemp on $\mathcal{I}_{+}$	SpecTemp + RRC	–	$+0.80$	12/15	–	$+0.66$	11/15

Table 3. Sources of the calibration gain on BRIGHT, averaged over 15 encoder–dataset configurations and 3 seeds (nDCG@10

\times 100

). SpecTemp on

\mathcal{I}_{+}

applies only support selection.

\Delta

and Wins denote the difference between the reported mean scores and the number of outperforming configurations relative to the listed reference; ∗ marks

p<0.05

(Wilcoxon signed-rank test over configurations). All label-based variants share cross-fitting folds,

\widehat{\mathbf{e}}

, and

\mathcal{I}_{+}

Without product quantization		FP32	FP16	SQ8	SQ4	Binary
Full Dimension		31.99	31.99	32.02	31.80	30.21
SpecTemp		30.19	30.18	30.17	30.00	–
SpecTemp + RRC		32.53	32.53	32.49	31.70	–
With product quantization		Bytes per document ( $M$ )
Representation	Codec	16	32	64	128	256
Full Dimension	PQ	4.88	8.14	13.11	20.64	27.63

Table 4. Mean nDCG@10 (

\times 100

) over 15 BRIGHT configurations and three seeds. Spectral methods retain

k=768

dimensions. The upper block compares floating-point and scalar encodings (plus full-dimensional binary); the lower block compares PQ and OPQ at

M

bytes per document.

Figure 3. Storage–effectiveness trade-offs on BRIGHT, averaged over three seeds and 15 encoder–dataset configurations. Colours identify representations and marker shapes identify encodings; spectral methods retain

k=768

dimensions. Lines connect the Pareto-optimal operating points of each representation. Dominated points are faded, except for the binary baseline, which is highlighted for visibility. Full-dimensional PQ at

M=16

and

32

falls below the plotted range. The dashed line marks mean full-dimensional FP32 effectiveness. Storage excludes shared transformation matrices and codebooks, whose costs are amortised over the collection.

Model	Method	MS MARCO				NQ				HotpotQA				FiQA
Model	Method	768	512	256	128	768	512	256	128	768	512	256	128	768	512	256	128
Qwen3-8B	Full Dimension	36.82				65.06				76.84				64.67
	PCA ( $k{=}D$ )	36.64 ${}^{\text{\tiny ns}}$				65.33 ↑				76.54				64.31 ${}^{\text{\tiny ns}}$
	PCA ( $k{=}D$ ) + RRC	36.66 ${}^{\text{\tiny ns}}$				65.13 ${}^{\text{\tiny ns}}$				76.61				64.26 ${}^{\text{\tiny ns}}$
	Prefix Truncation	36.43	35.67	34.52	32.40	63.72	63.18	61.07	57.29	75.27	74.10	70.67	62.83	63.95 ${}^{\text{\tiny ns}}$	63.46	61.59	57.36
	Random Truncation	35.89	35.47	34.28	31.48	63.47	62.49	59.31	53.25	74.71	73.30	69.51	58.94	62.96	61.64	59.38	53.41

Table 5. Retrieval performance on MS MARCO, NQ, HotpotQA, and FiQA. MS MARCO reports MRR@10 on the dev set; the other datasets report nDCG@10 on the test set. Scores are multiplied by 100 and averaged over seeds 1999, 5, and 2026 at target dimensions

k\in\{768,512,256,128\}

. Gray cells show full-dimensional references. ↑ marks a score significantly better than the original full dimension and

{}^{\text{\tiny ns}}

one not significantly different from it (paired t-test,

p<0.05

), in both cases consistently for all three seeds. Unmarked scores are significantly worse or inconsistent across seeds.

Model	Method	Biology				Earth Science				LeetCode				Economics				TheoremQA-T
Model	Method	768	512	256	128	768	512	256	128	768	512	256	128	768	512	256	128	768	512	256	128
Qwen3-8B	Full Dimension	19.60				32.80				34.03				18.52				39.35
	PCA ( $k{=}D$ )	20.08 ${}^{\text{\tiny ns}}$				31.18 ${}^{\text{\tiny ns}}$				35.94 ${}^{\text{\tiny ns}}$				21.93 ↑				40.60 ${}^{\text{\tiny ns}}$
	PCA ( $k{=}D$ ) + RRC	25.17 ↑				28.57				34.42 ${}^{\text{\tiny ns}}$				22.75 ↑				41.96
	Prefix Truncation	18.20 ${}^{\text{\tiny ns}}$	17.53 ${}^{\text{\tiny ns}}$	16.98	14.96	31.95 ${}^{\text{\tiny ns}}$	30.19	28.58	28.58	32.95 ${}^{\text{\tiny ns}}$	34.07 ${}^{\text{\tiny ns}}$	32.58 ${}^{\text{\tiny ns}}$	27.32	19.96 ${}^{\text{\tiny ns}}$	19.55 ${}^{\text{\tiny ns}}$	19.32 ${}^{\text{\tiny ns}}$	15.36	35.72	34.87	31.90	27.32
	Random Truncation	18.64	17.50	15.73	13.88	30.50	29.83	28.26	23.39	32.28	32.24	30.18	26.34	17.33 ${}^{\text{\tiny ns}}$	16.83 ${}^{\text{\tiny ns}}$	16.10	15.89	37.06	36.15	34.44	28.54

Table 6. Retrieval performance (nDCG@10) on the BRIGHT datasets Biology, Earth Science, LeetCode, Economics, and TheoremQA-T. Scores are multiplied by 100 and averaged over seeds 1999, 5, and 2026 at target dimensions

k\in\{768,512,256,128\}

. Gray cells show full-dimensional references. ↑ marks a score significantly better than full dimension and

{}^{\text{\tiny ns}}

one not significantly different from it (paired t-test,

p<0.05

), in both cases consistently for all three seeds. Unmarked scores are significantly worse or inconsistent across seeds.

Embedding Compression

Embedding compression reduces the storage and computation needed for vector representations while preserving task-relevant information.

See all 50 papers

Mar 19, 2026

Spectral Tempering for Embedding Compression in Dense Passage Retrieval

Dimensionality reduction is critical for deploying dense retrieval systems at scale, yet mainstream post-hoc methods face a fundamental trade-off: principal component analysis (PCA) preserves dominant variance but underutilizes representational capacity, while whitening enforces isotropy at the cost of amplifying noise in the heavy-tailed eigenspectrum of retrieval embeddings. Intermediate spectral scaling methods unify these extremes by reweighting dimensions with a power coefficient

γ

, but treat

γ

as a fixed hyperparameter that requires task-specific tuning. We show that the optimal scaling strength

γ

is not a global constant: it varies systematically with target dimensionality

k

and is governed by the signal-to-noise ratio (SNR) of the retained subspace. Based on this insight, we propose Spectral Tempering (SpecTemp), a learning-free method that derives an adaptive

γ(k)

directly from the corpus eigenspectrum using local SNR analysis and knee-point normalization, requiring no labeled data or validation-based search. Extensive experiments demonstrate that Spectral Tempering consistently achieves near-oracle performance relative to grid-searched

γ^*(k)

while remaining fully learning-free and model-agnostic. Our code is publicly available at https://github.com/liyongkang123/SpecTemp.

Yongkang Li, Panagiotis Eustratiadis, Evangelos Kanoulas

University of Amsterdam, Amsterdam, The Netherlands

Embedding Compression Dimensionality Reduction Dense Retrieval

Ask PDF

Sep 17, 2026

MAGIC: Marginal-Guided Compression with Optimal Transport for Efficient Visual Document Retrieval

Recent visual document retrieval (VDR) systems such as ColPali use multi-vector page embeddings, in which patch-level vectors enable fine-grained evidence matching but incur substantial index storage and MaxSim scoring overhead. Post-hoc merging offers a practical route to efficient VDR by reducing this cost without retraining the retriever, but its uniform reconstruction objectives are poorly aligned with the sparse, non-uniform patch usage induced by late-interaction retrieval. Under aggressive compression, this misalignment can preserve rarely used patches while concentrating retrieval activity on too few retained representatives. To address this misalignment, we propose Marginal-Guided Compression with Optimal Transport (MAGIC), a training-free post-hoc compressor for efficient retrieval with frozen multi-vector embeddings. MAGIC derives a MaxSim-induced compression surrogate and optimizes it through a two-marginal entropic optimal-transport formulation, where a retrieval-demand source marginal prioritizes high-use patches and a balanced target marginal regularizes retained-facet usage. Across ViDoRe benchmarks, keep ratios, and retrieval backbones, MAGIC consistently outperforms strong post-hoc compressors, with particularly large gains in the aggressive-compression regime; component ablations verify the complementary effects of its two marginals. We release the code at: https://github.com/xandery-geek/MAGIC.

Xu Yuan, Hua Liu, Wenqi Fan +1

The Hong Kong Polytechnic University, HK SAR · Southern University of Science and Technology, China

Visual Document Retrieval Embedding Compression Multi-Vector Retrieval Late Interaction Retrieval

Ask PDF

May 23, 2026

Spectral Retrieval: Multi-Scale Sinc Convolution over Token Embeddings for Localized Retrieval in LLM Multi-Agent Systems

[Abridged] - Spectral Retrieval is a plug-in re-ranking stage that interpolates between per-token MaxSim and mean-pool retrieval through a multi-scale sinc convolution over token embeddings. In standard dense retrieval each document is one mean-pooled vector; when relevance localises into a short subspan, the signal averages into noise. Spectral Retrieval reuses per-token embeddings from a late-interaction index and convolves them with a normalised sinc kernel at multiple scales. At L=1 the kernel acts as the identity, recovering per-token MaxSim; as L grows it approaches a uniform filter, recovering mean pooling. The maximum cosine over positions and scales yields a score provably no less informative than either endpoint. On a controlled synthetic benchmark with 1,000 documents and planted single-position spikes, mean-pool retrieval sits at chance (Recall@10 ~ 0.02) regardless of spike strength, while Spectral Retrieval reaches Recall@10 = 1.0 once the planted cosine exceeds the corpus-level token noise floor. On LIMIT-small with a frozen all-mpnet-base-v2 encoder, Spectral Retrieval lifts Recall@10 from 0.33 to 0.90, MRR from 0.22 to 0.79, and strict Success@10 from 0.12 to 0.84, without retraining. The method fits naturally into multi-agent LLM systems, where each agent benefits from a tighter, role-specific retrieval window over a shared corpus.

Andrea Morandi

Cisco Systems, Inc.

Late Interaction Retrieval Multi-Vector Retrieval Dense Retrieval

Ask PDF

Abstract

Embedding Compression

Explore similar work

Spectral Tempering for Embedding Compression in Dense Passage Retrieval

MAGIC: Marginal-Guided Compression with Optimal Transport for Efficient Visual Document Retrieval

Spectral Retrieval: Multi-Scale Sinc Convolution over Token Embeddings for Localized Retrieval in LLM Multi-Agent Systems