Quantization Enables Private Dense Retrieval against Malicious Service Providers

Louis Tremblay Thibault; Sofiane Azogagh; Marc-Olivier Killijian; Ulrich Aïvodji

cs.CRSep 28, 2026

Quantization Enables Private Dense Retrieval against Malicious Service Providers

Authors: Louis Tremblay Thibault, Sofiane Azogagh, Marc-Olivier Killijian, Ulrich Aïvodji

Organizations: École de technologie supérieure and Mila · Eurecom · Université du Québec à Montréal

Abstract

Dense retrieval, the key component of Retrieval Augmented Generation (RAG), retrieves the most relevant documents by comparing dense vector representations of queries and passages from a large corpus. In privacy-sensitive applications, the server observes the query and controls which evidence is returned, creating both confidentiality and integrity risks. We formulate private dense retrieval as providing query privacy and retrieval integrity against a malicious server, and develop a two-round cryptographic protocol that provides both guarantees. Our protocol reduces private and verifiable retrieval to multiplication of a committed matrix by an encrypted vector and uses low-bit quantization to make this computation practical. We evaluate the resulting trade-off between cryptographic cost, retrieval quality, and downstream RAG accuracy across six embedding models, four language models, and corpora of up to 2.68 million passages. Our results show that, with a clipped quantizer, three-bit quantization largely preserves retrieval quality and downstream accuracy, while a private query over a corpus the size of a clinical reference requires one to three minutes of server time. These results suggest that private dense retrieval is already practical for moderately sized, privacy-sensitive corpora when minute-scale latency is acceptable.

Figures & tables

	Privacy	Integrity	Exactness	Malicious security
Tiptoe ( Henzinger et al., 2023 )	$\checkmark$	$\times$	$\times$	$\times$
PIR-RAG ( Wang et al., 2025 )	$\checkmark$	$\times$	$\times$	$\times$
PPMI ( Bae et al., 2025 )	$\checkmark$	$\times$	$\checkmark$	$\times$
Compass ( Zhu et al., 2024 )	$\checkmark$	$\times$	$\times$	$\times$
VeriRAG ( Lin et al., 2026 )	$\times$	$\checkmark$	$\times$	$\checkmark$
zkRAG ( Jiang et al., 2026 )	$\times$	$\checkmark$	$\times$	$\checkmark$

Table 1: Private and verifiable retrieval systems. Exactness : scores are exact inner products rather than approximate search. Malicious security : the stated privacy and/or integrity guarantees hold against an actively deviating server.

$b$	$p$	$\sigma_{\max}$	$\lambda_{10}$	$\lambda_{11}$	$d$
$\mathbf{2}$	$2^{13}$	$2^{40.8}$	$\mathbf{138}$	$284$	$\mathbf{2^{10}}$
$3$	$2^{15}$	$2^{37.8}$	$122$	$253$	$2^{11}$
$4$	$2^{17}$	$2^{34.8}$	$110$	$227$	$2^{11}$
$5$	$2^{19}$	$2^{31.8}$	$100$	$206$	$2^{11}$
$8$	$2^{25}$	$2^{22.8}$	$79$	$161$	$2^{11}$

Table 2: Quantization-cryptography exchange rate: Round 1 parameters at

q=2^{64}-2^{32}+1

\kappa=1

, ternary secret and

2^{-64}

failure probability.

\lambda_{10}

and

\lambda_{11}

are the lattice-estimator security at

\sigma=\sigma_{\max}(b)

for LWE dimensions

d=2^{10}

and

2^{11}

, and

d

the smallest dimension at

128

bits.

Corpus	Round 1: $\mathbf{A}$	Round 2: $\mathbf{D}^{\top}$	Client	Server	$/query
			$b{=}2$ / $b{\geq}3$
NFCorpus ( $5{,}761$ chunks)	$5{,}761\times 1{,}024$	$512\times 5{,}761$	$510$ ms / $914$ ms	$1.7$ s / $2.5$ s	$0.003$ / $0.005$
SciFact ( $7{,}510$ chunks)	$7{,}510\times 1{,}024$	$512\times 7{,}510$	$510$ ms / $914$ ms	$1.7$ s / $2.5$ s	$0.003$ / $0.005$
$m=10^{4}$	$10^{4}\times 1{,}024$	$512\times 10^{4}$	$907$ ms / $1.7$ s	$3.0$ s / $5.0$ s	$0.006$ / $0.010$
$m=10^{5}$	$10^{5}\times 1{,}024$	$512\times 10^{5}$	$6.3$ s / $12.5$ s	$21.7$ s / $33.8$ s	$0.043$ / $0.068$
$m=10^{6}$	$10^{6}\times 1{,}024$	$512\times 10^{6}$	$50.4$ s / $1.7$ min	$3.0$ min / $5.4$ min	$0.364$ / $0.650$

Table 3: Measured cost on the evaluation corpora and at scale, at

b{=}2

(Round 1 at

d=2^{10}

) and from

b{=}3

on (Round 1 at

d=2^{11}

); Round 2 runs at

d=2^{12}

with

\ell=512

(Table 5 ). Server time is measured on

48

AMD EPYC 9654 cores; client time is measured on a single core. Prices at the on-demand rate of a

48

-core hpc7a.24xlarge EC2 instance (

7.20$ /h).

	SciFact $k{=}5$		SciFact $k{=}1$		BioASQ $k{=}5$		NQ $k{=}5$ , $2.68$ M
$b$	BGE	arctic	BGE	arctic	BGE	arctic	BGE	arctic
closed book	$0.585$		$0.585$		$0.187$		$0.313$
gold context	$0.814$		$0.809$		$0.680$		$0.740$
float	$0.734$	$0.713$	$0.766$	$0.739$	$0.633$	$0.647$	$0.643$	$0.643$
$8$	$0.734$	$0.718$	$0.771$	$0.739$	$0.633$	$0.643$	$0.657$	$0.637$
$6$	$0.729$	$0.713$	$0.771$	$0.739$	$0.633$	$0.640$	$0.650$	$0.640$

Table 4: Downstream accuracy versus bit-width with the clipped quantizer (reader phi-4 ,

n{=}300

queries per endpoint). SciFact: accuracy on the

188

evidence-bearing claims; BioASQ and NQ: answer containment, NQ retrieving over the full

2.68

M-passage corpus. Closed book: the reader sees no passages; gold context: it sees the labelled relevant passages (at most

k

). Bold marks a significant drop from floating point (exact McNemar,

p<0.05

). Other readers: Appendix E .

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Round	$d$	$\kappa$	$q$	$p$	$\sigma$	$\lambda$	$\Pr[\mathsf{fail}]$
Round 1 ( $b=2$ )	$2^{10}$	$1$	$2^{64}-2^{32}+1$	$2^{13}$	$2^{40.8}$	$138$	$2^{-64}$
Round 1 ( $b=3$ )	$2^{11}$	$1$	$2^{64}-2^{32}+1$	$2^{15}$	$2^{37.8}$	$253$	$2^{-64}$
Round 1 ( $b=4$ )	$2^{11}$	$1$	$2^{64}-2^{32}+1$	$2^{17}$	$2^{34.8}$	$227$	$2^{-64}$
Round 2	$2^{12}$	$1$	$2^{64}-2^{32}+1$	$2^{24}$	$2^{1.7}$	$223$	$2^{-64}$

Appendix

Table 5: GLWE parameters per round: ring dimension

d

, rank

\kappa

, ciphertext modulus

q

, plaintext modulus

p

, noise standard deviation

\sigma

and the resulting security parameter

\lambda

(ternary secret) and maximum failure probability

\Pr[\mathsf{fail}]

per coefficient. Round 2 is given for

m\leq 2^{20}

chunks.

phi-4 (14B)
Endpoint	Encoder	float	$b{=}4$	$b{=}3$	$b{=}2$
SciFact $k{=}5$	BGE	$0.734$	$-1.1$ $[-3.6,+1.5]$ †	$-2.7$ $[-5.4,+0.1]$ †	$-5.9$ $[-10.1,-1.6]$ ∗
SciFact $k{=}5$	arctic	$0.713$	$+0.5$ $[-1.3,+2.3]$ †	$+0.0$ $[-2.6,+2.6]$ †	$-1.6$ $[-4.7,+1.5]$ †
SciFact $k{=}1$	BGE	$0.766$	$+0.5$ $[-0.5,+1.6]$ †	$-0.5$ $[-2.3,+1.3]$ †	$-8.0$ $[-12.1,-3.8]$ ∗
SciFact $k{=}1$	arctic	$0.739$	$+0.0$ $[+0.0,+0.0]$ †	$+1.6$ $[-0.2,+3.4]$ †	$-5.9$ $[-10.1,-1.6]$ ∗
BioASQ	BGE	$0.633$	$+0.3$ $[-1.4,+2.1]$ †	$+1.0$ $[-0.7,+2.7]$ †	$-2.0$ $[-4.6,+0.6]$ †

Appendix

Table 6: Paired accuracy difference from floating point, in points with

95\%

interval, at

b{=}4

3

and

2

. All readers, endpoints and encoders use the clipped quantizer (

n{=}300

queries,

188

evidence-bearing claims for SciFact). ∗ : significant drop, exact McNemar

p<0.05

. † : equivalent to floating point within

\pm 5

points, TOST

p<0.05

. “float” is the floating-point accuracy.

Endpoint	Encoder	Unit	float	$b{=}4$	$b{=}3$	$b{=}2$
SciFact $k{=}5$	BGE	abstracts	$0.734$	$-1.1$ $[-3.6,+1.5]$ †	$-2.7$ $[-5.4,+0.1]$ †	$-5.9$ $[-10.1,-1.6]$ ∗
		chunks	$0.777$	$-2.7$ $[-5.0,-0.4]$ †	$-4.3$ $[-7.1,-1.4]$ ∗	$-6.9$ $[-11.4,-2.5]$ ∗
SciFact $k{=}5$	arctic	abstracts	$0.713$	$+0.5$ $[-1.3,+2.3]$ †	$+0.0$ $[-2.6,+2.6]$ †	$-1.6$ $[-4.7,+1.5]$ †
		chunks	$0.734$	$+1.6$ $[-0.7,+3.9]$ †	$+1.6$ $[-1.9,+5.1]$ †	$-1.1$ $[-5.2,+3.1]$ †
SciFact $k{=}1$	BGE	abstracts	$0.766$	$+0.5$ $[-0.5,+1.6]$ †	$-0.5$ $[-2.3,+1.3]$ †	$-8.0$ $[-12.1,-3.8]$ ∗
		chunks	$0.755$	$-0.5$ $[-2.3,+1.3]$ †	$-2.1$ $[-4.7,+0.4]$ †	$-12.2$ $[-17.6,-6.9]$ ∗

Appendix

Table 7: SciFact with chunks against whole abstracts, reader phi-4 . We give floating-point accuracy on the

188

evidence-bearing claims. We also give the paired difference from floating point at

b{=}4

3

and

2

, in points with

95\%

interval. ∗ : significant drop, exact McNemar

p<0.05

. † : equivalent within

\pm 5

points, TOST

p<0.05

	SciFact						NFCorpus
	low-bit		conventional				low-bit		conventional
$b$	arctic	Cohere	BGE	mxbai	e5	gte	arctic	Cohere	BGE	mxbai	e5	gte
$2$	$92.2$	$89.7$	$86.4$	$88.5$	$82.2$	$86.6$	$88.2$	$88.6$	$85.9$	$86.5$	$78.8$	$83.6$
$3$	$98.7$	$99.1$	$97.4$	$98.7$	$97.3$	$97.5$	$98.2$	$99.3$	$99.0$	$97.4$	$94.1$	$97.6$
$\mathbf{4}$	$99.9$	$99.4$	$99.7$	$99.4$	$100.0$	$99.8$	$99.6$	$99.8$	$99.5$	$98.9$	$96.5$	$100.0$
$8$	$99.9$	$99.5$	$99.9$	$100.0$	$100.0$	$99.8$	$99.7$	$100.0$	$99.7$	$100.0$	$98.7$	$100.0$

Appendix

Table 8: Retrieval quality retained of our two-round protocol under clipped quantization, as the percentage of floating-point nDCG@10 capped at

100

, for

b\in\{2,3,4,8\}

on SciFact and NFCorpus, ranked per chunk and scored per document.

		SciFact					NFCorpus
Encoder	low-bit	float	ret. $4$	ret. $3$	top-1 $4$	top-1 $3$	float	ret. $4$	ret. $3$	top-1 $4$	top-1 $3$
arctic-embed-l-v2.0	$\checkmark$	$0.706$	$100.4$	$99.4$	$0.96$	$0.93$	$0.354$	$99.6$	$98.0$	$0.91$	$0.82$
Cohere embed-v3	$\checkmark$	$0.718$	$99.6$	$98.7$	$0.96$	$0.91$	$0.387$	$99.5$	$98.9$	$0.86$	$0.78$
bge-large-en-v1.5	$\times$	$0.746$	$99.4$	$97.1$	$0.94$	$0.87$	$0.381$	$99.5$	$98.6$	$0.83$	$0.71$
mxbai-embed-large-v1	$\times$	$0.739$	$100.3$	$100.1$	$0.93$	$0.89$	$0.387$	$100.1$	$98.9$	$0.84$	$0.73$
e5-large-v2	$\times$	$0.722$	$99.4$	$96.8$	$0.88$	$0.77$	$0.372$	$96.3$	$90.8$	$0.79$	$0.64$

Appendix

Table 9: The encoder suite under the clipped quantizer. Floating-point nDCG@10, the percentage of it retained at

b{=}4

and

b{=}3

, and top-1 agreement with the floating-point ranking at the same bit-widths.

\checkmark

: trained for low-bit readout.

SciFact
	$b{=}4$		$b{=}3$
Encoder	ret. (%)	top-1	ret. (%)	top-1
arctic-embed-l-v2.0	$100.0\to 100.4$	$0.93\to 0.96$	$97.6\to 99.4$	$0.83\to 0.93$
Cohere embed-v3	$100.0\to 99.6$	$0.93\to 0.96$	$95.4\to 98.7$	$0.74\to 0.91$
bge-large-en-v1.5	$97.1\to 99.4$	$0.79\to 0.94$	$77.3\to 97.1$	$0.54\to 0.87$
mxbai-embed-large-v1	$99.5\to 100.3$	$0.81\to 0.93$	$82.3\to 100.1$	$0.57\to 0.89$

Appendix

Table 10: The quantizer lever. Each cell reads max-abs scaling

\to

clipping at the

99

th percentile. We give the percentage of floating-point nDCG@10 retained and top-1 agreement at

b{=}4

and

b{=}3

. Nothing on the cryptographic side differs between the two columns of each pair.

Encoder	Corpus	$\max\|\mathbf{A}\|$	$s_{\mathbf{A}}$	top dim. (%)	gap	gap/noise	top-1
arctic-embed-l-v2.0	SciFact	$0.200$	$0.0122$	$0.5$	$0.0582$	$11.72$	$0.96$
Cohere embed-v3	SciFact	$0.221$	$0.0125$	$0.8$	$0.0445$	$8.83$	$0.96$
bge-large-en-v1.5	SciFact	$0.305$	$0.0110$	$5.5$	$0.0319$	$7.00$	$0.94$
mxbai-embed-large-v1	SciFact	$0.311$	$0.0111$	$5.4$	$0.0345$	$7.73$	$0.93$
e5-large-v2	SciFact	$0.170$	$0.0105$	$1.9$	$0.0149$	$3.48$	$0.88$
gte-large	SciFact	$0.172$	$0.0107$	$1.7$	$0.0189$	$4.38$	$0.94$

Appendix

Table 11: Score gap against rounding error at

b{=}4

, clipped quantizer.

\max|\mathbf{A}|

: largest corpus coordinate.

s_{\mathbf{A}}

: clipped step. top dim.: share of the squared norm in the largest coordinate. gap: median gap between the two best floating-point scores. gap/noise: median ratio of that gap to the rounding error. top-1: top-1 agreement with floating point.

SciFact
	Trained for low-bit readout		Conventional
$b$	arctic	Cohere	BGE	mxbai	e5	gte
$2$	$91.8$	$92.1$	$87.0$	$90.4$	$84.0$	$87.6$
$3$	$99.4$	$98.7$	$97.1$	$100.1$	$96.8$	$97.8$
$\mathbf{4}$	$100.4$	$99.6$	$99.4$	$100.3$	$99.4$	$99.7$
$5$	$100.7$	$99.8$	$99.0$	$99.9$	$101.3$	$99.2$

Appendix

Table 12: Table 8 on whole abstracts instead of chunks. This checks comparability against published document-level baselines. We give the percentage of floating-point nDCG@10 retained under clipped quantization for each bit-width

b

			Recall@5 retained (%)
Encoder	$m$	Recall@5 (float)	$b{=}2$	$b{=}3$	$b{=}4$
arctic-embed-l-v2.0	$10^{4}$	$0.988$	$99.2$	$99.9$	$100.0$
	$10^{5}$	$0.967$	$97.0$	$99.9$	$99.9$
	$10^{6}$	$0.848$	$93.1$	$99.5$	$99.7$
	$2.68{\cdot}10^{6}$	$0.737$	$89.9$	$99.0$	$99.8$
bge-large-en-v1.5	$10^{4}$	$0.982$	$97.7$	$99.9$	$100.1$

Appendix

Table 13: Recall@5 retained under the clipped quantizer versus corpus size on BEIR Natural Questions (

3{,}452

queries), at

b\in\{2,3,4\}

. Floating-point Recall@5 is given for reference. Generated by implem/analysis/scale_recall5_table.py .

		nDCG@10 ret. (%)		required $b$
Encoder	$m$	$b{=}3$	$b{=}4$	(top- $k$ )
arctic-embed-l-v2.0	$10^{4}$	$100.0$	$99.9$	$3$
	$10^{5}$	$99.9$	$100.1$	$3$
	$10^{6}$	$99.1$	$99.9$	$3$
	$2.68{\cdot}10^{6}$	$98.5$	$99.7$	$4$
bge-large-en-v1.5	$10^{4}$	$99.5$	$99.9$	$3$

Appendix

Table 14: Quality versus corpus size on BEIR Natural Questions with the clipped quantizer. We give nDCG@10 retained at

b{=}3

and

b{=}4

. We also give the smallest

b

that retains

99\%

of floating-point nDCG@10, or

>8

when no

b

up to

8

does. Generated by implem/analysis/scale_analysis.py .

Explore similar work

Spruce: Scalable Private Outsourced Retrieval Using Compact Embeddings
Sep 3, 2026Peichun Hua, Yunming XiaoText Corpora Trees
The Chinese University of Hong Kong, Shenzhen · State Key Laboratory of Internet Architecture, Tsinghua University
PILLAR: Private Inverted-Index Lexical Lookup for Augmented Retrieval
Sep 28, 2026Truong Son Nguyen, Daniel Blackley, Ni Trieu +1Privacy-Preserving Machine Learning
Arizona State University · Department of Computer Science Arizona State University Tempe, AZ 85281, USA · George Mason University +1
GoldenRetriever: Non-Interactive Homomorphic Encrypted Retrieval for Privacy-Preserving RAG
Jul 31, 2026Yang Gao, Gang Quan, Scott Piersall +3Agentic Retrieval-Augmented Generation Systems Homomorphic Encryption
Department of Computer Science, University of Central Florida, Orlando, FL, USA · Electrical and Computer Engineering Department, Florida International University, Miami, FL, USA · College of Design, Construction and Planning, University of Florida, Gainesville, FL, USA

Sep 3, 2026, cs.CRJ/K move · Enter open · S save

Spruce: Scalable Private Outsourced Retrieval Using Compact Embeddings

Peichun Hua, Yunming Xiao

The Chinese University of Hong Kong, Shenzhen · State Key Laboratory of Internet Architecture, Tsinghua University

Text Corpora Trees

Open paper AskPDF arXiv

Retrieval-Augmented Generation (RAG) has made dense retrieval over large document collections a standard building block. Organizations increasingly outsource vector indexes to untrusted clouds, exposing proprietary corpora and user queries. Cryptographic protection is challenging because each query searches corpus-scale state, causing computation, correlated randomness, and communication to grow with the corpus. At million-document scale, a naive secure implementation takes minutes and about 90 GB of communication per query. Even recent optimized systems require 10--22 seconds. We propose Spruce (Scalable Private Outsourced Retrieval Using Compact Embeddings), which co-designs representations with the cryptographic protocol. Spruce learns compact binary codes that preserve candidates for full-precision reranking, replacing corpus-wide embedding scoring with efficient Hamming-distance computation under two-server multi-party computation (MPC). A corpus-calibrated fixed-radius protocol avoids multi-round candidate selection while preserving retrieval quality. Spruce also provides private cluster pruning, which trades minor quality loss for substantially less computation, and a one-core owner-operated dealer that removes cloud OT preprocessing bottlenecks. Across four corpora containing 383K--5.42M documents, Spruce preserves the original search quality with median candidate sets of only 382--1,952. At 10 Gbps inter-server bandwidth, full scans take 0.21--2.97 seconds,

4.8

6.7\times

faster than the closest measured prior work. Private pruning takes 0.06--1.09 seconds, achieves

13.1

22.9\times

speedups, and retains

93.9\%

97.3\%

of full-float NDCG. On the largest corpus, pruning and the dealer jointly improve sustained throughput by

31.5\times

at 1 Gbps per link.

Abstract

Explore similar work

Spruce: Scalable Private Outsourced Retrieval Using Compact Embeddings

PILLAR: Private Inverted-Index Lexical Lookup for Augmented Retrieval

GoldenRetriever: Non-Interactive Homomorphic Encrypted Retrieval for Privacy-Preserving RAG