NanoVDR: Distilling a 2B Vision-Language Retriever into a 70M Text-Only Encoder for Visual Document Retrieval

Zhuchenyang Liu; Yao Zhang; Yu Xiao

cs.IRMar 13, 2026

NanoVDR: Distilling a 2B Vision-Language Retriever into a 70M Text-Only Encoder for Visual Document Retrieval

Authors: Zhuchenyang Liu, Yao Zhang, Yu Xiao

Organizations: Aalto University, Espoo, Finland

Abstract

Vision-Language Model (VLM) based retrievers have advanced visual document retrieval (VDR) to impressive quality. They require the same multi-billion parameter encoder for both document indexing and query encoding, incurring high latency and GPU dependence even for plain-text queries. We observe that this design is unnecessarily symmetric: documents are visually complex and demand strong visual understanding, whereas queries are just short text strings. NanoVDR exploits this query--document asymmetry by decoupling the two encoding paths: a frozen 2B VLM teacher indexes documents offline, while a distilled text-only student as small as 69M parameters encodes queries at inference. The key design choice is the distillation objective. Through systematic comparison of six objectives across three backbones and 22 ViDoRe benchmark datasets, we find that pointwise cosine alignment on query text consistently outperforms ranking-based and contrastive alternatives, while requiring only pre-cached teacher query embeddings and no document processing during training. Furthermore, we identify cross-lingual transfer as the primary performance bottleneck, and resolve it cheaply by augmenting training data with machine-translated queries. The resulting NanoVDR-S-Multi (DistilBERT, 69M) retains 95.1% of teacher quality and outperforms DSE-Qwen2 (2B) on v2 and v3 with 32 $\times$ fewer parameters and 50 $\times$ lower CPU query latency, at a total training cost under 13 GPU-hours.

Figures & tables

Figure 1: Motivation and deployment advantage of NanoVDR. (a) Symmetric vs. asymmetric retrieval: Current VDR systems (top) use the same heavy VLM encoder (2B) for both offline document indexing and online query encoding ( $>$ 2,000 ms per query). NanoVDR (bottom) decouples the two: the frozen VLM teacher encodes documents offline, while a distilled text-only student (70M) encodes queries online in $\sim$ 50 ms on CPU. (b) Performance vs. latency: On the ViDoRe benchmark (mean NDCG@5 across v1/v2/v3), NanoVDR models achieve near-teacher accuracy (dashed line) at 50–143 $\times$ lower CPU latency. Bubble size is proportional to model parameter count; the star ( $\bigstar$ ) marks NanoVDR-S-Multi, the multilingual-augmented variant (§ 3.3 ).

Figure 2: Query-centric distillation training of NanoVDR. Left: The frozen VLM teacher pre-caches training query embeddings via text-only inference. Right: The student text encoder is trained to minimize $\mathcal{L}_{\text{align}}=1-\cos(\mathbf{v}^{Q}_{t},\mathbf{v}^{Q}_{s})$ , requiring no document images or negative sampling.

Model	Backbone	Params	Scoring	ViDoRe v1 (10)	ViDoRe v2 (4)	ViDoRe v3 (8)
Multi-vector VLMs (MaxSim scoring, multi-token document representation)
Tomoro-8B	ColQwen3-8B	8.8B	MaxSim	90.6	65.0	59.0
Tomoro-4B	ColQwen3-4B	4.0B	MaxSim	90.2	65.2	57.6
ColNomic-7B	ColQwen2.5-7B	7.8B	MaxSim	89.8	60.4	55.9
ColPali	PaliGemma-3B	3.0B	MaxSim	84.2	54.7	42.0
ColModernVBert	ModernBERT-base	250M	MaxSim	76.7	33.4	17.4

Table 1: Main results (NDCG@5

\times

100) on the ViDoRe benchmark. Each benchmark version is averaged over the indicated number of datasets. Params = total model parameters; for NanoVDR this includes the backbone + MLP projector. Scoring = retrieval scoring method: MaxSim (token-level late interaction) or Cosine (single-vector dot product).

\dagger

: Qwen3-VL-Embedding-2B Li et al. (2026) , our frozen teacher used for offline document indexing. Best per group in bold . Subscripts on NanoVDR rows indicate teacher retention (%, student/teacher).

\ddagger

: SERVAL Nguyen et al. (2025) generates a textual description of every page offline with a VLM and indexes those descriptions with a standard text encoder. Its ViDoRe v2 score is recomputed from the per-dataset results reported in that paper; see Appendix J .

Model	Params	Dim	Tok/Doc	Size	Scoring	Encode	Ret/1K	Ret/10K	Index/1M
Multi-vector VLMs (MaxSim scoring)
Tomoro-8B	8,768M	320	1,280	35.1 GB	MaxSim	8,225 ms	706 ms	7,124 ms	819 GB
ColNomic-7B	7,829M	128	1,000	31.3 GB	MaxSim	7,471 ms	234 ms	2,385 ms	256 GB
ColPali	2,964M	128	1,030	11.9 GB	MaxSim	7,284 ms	269 ms	2,786 ms	264 GB
ColModernVBert	259M	128	1,000	1.0 GB	MaxSim	183 ms	242 ms	2,553 ms	256 GB
Single-vector models (Cosine scoring)

Table 2: Deployment cost comparison (query-time). All latency measured on a single CPU thread (batch size 1). Params = total parameters. Dim = embedding dimension per vector. Tok/Doc = tokens stored per document page (1 = single-vector). Size = query encoder checkpoint (float32); for NanoVDR this is the student only. Encode = single-query encoding latency. Ret/

N

= scoring latency against

N

candidate documents. Index/1M = index storage for 1M documents (float32 for single-vector, float16 for multi-vector). NanoVDR shares the teacher’s (Qwen3-VL-Embedding-2B) pre-computed document index. The teacher itself is excluded as it is used only for offline GPU-based document indexing.

$\lambda_{a}$	$\lambda_{r}$	v1 (10)	v2 (4)	v3 (8)
1	0	82.2	61.4	44.1
1	0.5	81.6	59.8	42.8
1	1	81.5	59.1	42.5
0.5	1	81.5	58.6	42.1
0	1	81.1	57.4	41.6
InfoNCE		71.5	39.8	30.0

Table 3: Loss ablation (NDCG@5

\times

100, averaged over 3 backbones).

\lambda_{a}

\lambda_{r}

= weights for

\mathcal{L}_{\text{align}}

and

\mathcal{L}_{\text{rank}}

. All trained under identical settings (20 epochs, lr=2e-4).

Language	Train%	#Queries	Teacher	Student	Ret.
English	68.7	6,237	64.0	60.3	94.3%
French	7.6	3,074	56.2	51.7	92.1%
Italian	7.6	2,419	49.0	44.1	90.0%
Spanish	8.1	2,694	51.4	46.1	89.7%
German	8.0	2,694	49.3	42.3	85.7%
Portuguese	0.0	2,419	48.7	36.8	75.6%

Table 4: Distillation retention by query language (NanoVDR-S). Train% = proportion of 711K training pairs in each language. Teacher and Student are mean NDCG@5 (

\times

100). Ret. = Student / Teacher. All 22 ViDoRe datasets pooled (19,537 unique queries after exact-duplicate removal).

Language	#Queries	Teacher	Original	+Aug	$\Delta$	Ret.
English	6,237	64.0	60.3	60.3	+0.0	94.3%
French	3,074	56.2	51.7	53.2	+1.5	94.7%
Italian	2,419	49.0	44.1	45.7	+1.6	93.3%
Spanish	2,694	51.4	46.1	47.8	+1.8	93.1%
German	2,694	49.3	42.3	45.4	+3.1	92.0%
Portuguese	2,419	48.7	36.8	46.1	+9.3	94.6%

Table 5: Effect of multilingual query augmentation by language (NDCG@5

\times

100). Original = NanoVDR-S trained on 711K pairs; +Aug = retrained on 1.49M pairs with translated queries.

\Delta

= +Aug

-

Original. Ret. = +Aug / Teacher. Same 19,537 queries as Table 4 .

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

	v1	v2	v3
Datasets	10	4	8
Query languages	2	4	6
Query source	VQA + LLM	LLM + human	LLM + expert
SOTA NDCG@5	$>$ 90	59–65	57–59

Appendix

Table 6: Cross-version comparison of the ViDoRe benchmark.

Dataset	Ver.	Document Domain	Doc	Query	Source
DocVQA	v1	Industrial documents	EN	EN	H
ArXivQA	v1	Scientific papers	EN	EN	H
InfoVQA	v1	Infographics	EN	EN	H
TatDQA	v1	Financial tables	EN	EN	H
TabFQuAD	v1	Tables in French PDFs	FR	FR	H
SyntheticDocQA-AI	v1	AI-related documents	EN	EN	L

Appendix

Table 7: All 22 ViDoRe evaluation datasets used in this work. Doc = document corpus language. Query = languages in which queries are provided. Source: H = human-authored (from VQA benchmarks or expert annotation); L = LLM-generated; L+H = LLM-generated with human review. Six languages = EN/FR/ES/DE/IT/PT.

Source	Subset	Train	%
VisRAG-Synthetic	—	233,817	32.9
ColPali	—	109,044	15.3
VisRAG-InDomain	—	94,016	13.2
VDR-Multilingual	Spanish	57,491	8.1
	German	56,994	8.0
	French	54,079	7.6

Appendix

Table 8: Training data composition after deduplication. VDR-Multilingual is split by language.

	English								French
Model	ArXivQA	TatDQA	InfoVQA	DocVQA	Syn-AI	Syn-En.	Syn-Gov.	Syn-Hlt.	Shift.	TabFQ.	Avg
Tomoro-8B	91.3	81.5	94.6	66.2	98.9	96.2	96.9	98.7	87.6	94.3	90.6
Tomoro-4B	90.9	80.8	94.2	66.0	98.5	95.9	95.7	98.9	87.4	94.0	90.2
ColNomic-7B	87.9	80.5	91.9	60.9	100.0	96.8	95.8	99.1	88.7	96.0	89.8
ColPali	82.7	70.3	85.4	58.9	96.7	94.2	96.0	96.9	75.2	85.9	84.2
ColModernVBert	72.7	71.3	83.5	47.2	96.7	89.0	93.0	94.7	50.1	68.9	76.7

Appendix

Table 9: Per-dataset NDCG@5 (

\times

100) on ViDoRe v1.

\dagger

: Qwen3-VL-Embedding-2B (our teacher). Best overall in bold , best among NanoVDR variants underlined .

Model	ESG	BioMed	Econ	ESG-H	Avg
Tomoro-8B	60.4	65.4	59.5	74.7	65.0
Tomoro-4B	62.6	65.6	56.0	76.8	65.2
ColNomic-7B	53.6	63.2	55.6	69.4	60.4
ColPali	54.9	55.9	49.4	58.7	54.7
ColModernVBert	30.5	26.6	27.2	49.3	33.4
DSE-Qwen2	54.9	55.9	52.0	60.0	55.7

Appendix

Table 10: Per-dataset NDCG@5 (

\times

100) on ViDoRe v2.

\dagger

: Qwen3-VL-Embedding-2B (our teacher). Best overall in bold , best among NanoVDR variants underlined .

Model	Fin-EN	Fin-FR	CS	HR	Energy	Indust.	Pharma	Physics	Avg
Tomoro-8B	62.8	45.9	72.4	61.5	65.8	52.5	64.3	46.7	59.0
Tomoro-4B	61.3	42.9	72.5	57.3	63.3	52.7	64.0	46.6	57.6
ColNomic-7B	55.2	42.8	74.1	57.0	62.4	49.4	61.1	45.0	55.9
ColPali	34.9	21.8	63.6	43.4	45.7	34.6	52.7	39.5	42.0
ColModernVBert	18.3	11.6	25.9	12.9	22.9	11.4	19.4	17.0	17.4
DSE-Qwen2	40.1	22.4	60.8	40.8	45.0	30.4	51.8	39.0	41.3

Appendix

Table 11: Per-dataset NDCG@5 (

\times

100) on ViDoRe v3.

\dagger

: Qwen3-VL-Embedding-2B (our teacher). Best overall in bold , best among NanoVDR variants underlined .

Config	$\lambda_{a}$	$\lambda_{r}$	S	M	L	S	M	L	S	M	L
			ViDoRe v1			ViDoRe v2			ViDoRe v3
Align	1	0	82.2	82.1	82.4	60.5	62.2	61.5	43.5	44.7	44.2
Align-dom	1	0.5	81.4	81.6	81.8	59.5	59.7	60.2	41.9	43.5	43.1
Combined	1	1	81.2	81.4	82.0	58.6	59.4	59.3	41.4	42.9	43.0
Rank-dom	0.5	1	81.1	81.6	81.8	57.3	58.6	59.9	40.9	42.6	42.8
Rank	0	1	80.3	81.1	81.9	56.5	57.0	58.9	40.0	41.8	42.9

Appendix

Table 12: Loss ablation (NDCG@5

\times

100) per variant. S = NanoVDR-S, M = NanoVDR-M, L = NanoVDR-L.

Requirement	Align-only	With ranking
Teacher query encoding (text)	✓	✓
Teacher doc encoding (images)	—	✓
Query encoding time	0.8 GPU-h	0.8 GPU-h
Doc encoding time	—	24 GPU-h
Total pre-cache cost	0.8 GPU-h	24.8 GPU-h
Embedding storage	2.9 GB	5.8 GB

Appendix

Table 13: Pre-caching and training cost comparison for align-only (

\lambda_{r}\!=\!0

) vs. ranking-based (

\lambda_{r}\!>\!0

) distillation on 711K training pairs with a 2B VLM teacher.

Figure 3: Data efficiency of NanoVDR-S. NDCG@5 vs. fraction of 711K training pairs, with teacher upper bounds (dashed). Percentages indicate retention (student/teacher). Diminishing returns are pronounced after 25% of training data.

Dataset	Ver.	EN	FR	ES	DE	IT	PT
ArXivQA	v1	94.9	—	—	—	—	—
TatDQA	v1	93.1	—	—	—	—	—
InfoVQA	v1	97.9	—	—	—	—	—
DocVQA	v1	92.8	—	—	—	—	—
SyntheticDocQA-AI	v1	97.8	—	—	—	—	—
SyntheticDocQA-En.	v1	99.3	—	—	—	—	—

Appendix

Table 14: Per-language retention (NanoVDR-S NDCG@5/ Teacher NDCG@5

\times

100) across all 22 ViDoRe datasets. “—” = language not present in that dataset. V1 datasets are single-language (8 English, 2 French).

	Teacher	$N$	$\bar{c}$
$r$ with Ret.	+0.607	$-$ 0.566	+0.094

Appendix

Table 15: Pearson correlations between corpus-level features and English-only retention (NanoVDR-S, 19 datasets). Teacher = mean teacher NDCG@5;

N

= corpus size;

\bar{c}

= mean student–teacher query cosine similarity.

Language	Existing	Translated	Total	Model
Portuguese	0	200,000	200,000	en-ROMANCE
Italian	53,787	146,213	200,000	en-it
French	54,079	145,921	200,000	en-fr
German	56,994	143,006	200,000	en-de
Spanish	57,491	142,509	200,000	en-es
Total		777,649

Appendix

Table 16: Translation statistics per target language. Existing = count already in the 711K training set. Translated = newly machine-translated queries. Target = 200K per language.

		SERVAL reports		This paper
ViDoRe v2	in Nguyen et al. (2025)	32B+7B	ColNomic	ColNomic
ESG	SRSM	59.5	57.3	53.6
BioMed	SMBTIM	63.0	64.2	63.2
Econ	SEMEM	55.5	56.7	55.6
ESG-H	RERB	70.7	73.9	69.4
Average		62.2	63.0	60.4

Appendix

Table 17: Recomputing SERVAL’s ViDoRe v2 score. Scores are NDCG@5

\times

100, taken from Table 2 of Nguyen et al. (2025) ; the short names are that paper’s own abbreviations, resolved to ViDoRe datasets through the Hugging Face repository redirects. ColNomic-7B is the one model both papers evaluate, and its two columns bound how far the protocols differ.

Dataset	$n$	min	mean	median	p95	p99	max	$>$ 512
ViDoRe v1
ArxivQA	500	9	25.3	23	42	58	69	0
DocVQA	500	6	11.8	11	18	23	28	0
InfoVQA	500	6	15.7	15	25	32	42	0
TabFQuAD	280	12	31.5	31	45	52	59	0
TAT-DQA	1,663	7	16.9	16	26	32	48	0

Appendix

Table 18: DistilBERT-tokenized query lengths across the 22 ViDoRe datasets. The longest single query in the entire benchmark (ViDoRe v3 Computer Science, 182 tokens) uses 35.5% of the 512-token budget; 99% of queries fit within 78 tokens (15.2%).

Visual Document Retrieval

Visual document retrieval finds relevant pages or regions in image-based documents using visual and multimodal representations.

See all 65 papers

ColNanoVDR: Document-Free Query Distillation for Multi-Vector Visual Document Retrieval via Optimal Transport
Sep 28, 2026Zhuchenyang Liu, Ziyi Wang, Yao Zhang +1Cross-Tokenizer Knowledge Distillation Multi-Vector Retrieval
Aalto University, Finland · Independent Researcher, Netherlands
DistilVDR: A Compact End-to-End Visual Document Retriever via Dual-Student Distillation
Aug 11, 2026Zhuchenyang Liu, Ziyi Wang, Yao Zhang +1VLM Distillation Visual Document Retrieval
Aalto University, Finland · Independent Researcher, Netherlands
Beyond Bag-of-Patches: Learning Global Layout via Textual Supervision for Late-Interaction Visual Document Retrieval
May 8, 2026Pascal Tilli, Mohsen MesgarMultimodal Embedding Multi-Vector Retrieval
University of Stuttgart, Stuttgart, Germany · Bosch Center for Artificial Intelligence, Renningen, Germany

Sep 28, 2026J/K move · Enter open · S save

ColNanoVDR: Document-Free Query Distillation for Multi-Vector Visual Document Retrieval via Optimal Transport

Zhuchenyang Liu, Ziyi Wang, Yao Zhang, Yu Xiao

Aalto University, Finland · Independent Researcher, Netherlands

Cross-Tokenizer Knowledge Distillation Multi-Vector Retrieval Visual Document Retrieval Optimal Transport

Open paper AskPDF arXiv

Multi-vector retrievers built on vision-language models lead visual document retrieval (VDR), but they run a multi-billion-parameter query encoder on every search. Distilling this encoder into a small student that queries the teacher's existing index would remove the bottleneck. The standard recipe, however, matches the teacher's MaxSim scores and so requires encoding and caching every training page, which can reach terabytes of page tokens. NanoVDR avoids pages entirely by training on the teacher's query embeddings alone, but only for single-vector retrievers. We present ColNanoVDR, to our knowledge the first framework to bring this document-free distillation to multi-vector VDR. Its objective, OTW (Optimal Transport with Learned Weights), aligns the student's query tokens with the teacher's by entropic optimal transport, with a learned weight for each student token, and needs no correspondence between the two tokenizations. We prove that the resulting alignment cost bounds the MaxSim score difference on every page. Distilled from five state-of-the-art teachers, the 149M text-only students retain about 95% of their teachers' NDCG@5 on ViDoRe v1-v3 while encoding queries up to 26x faster. Under identical training, OTW matches score distillation while encoding no page and reading 12.6x less cached teacher data.

Abstract

Visual Document Retrieval

Explore similar work

ColNanoVDR: Document-Free Query Distillation for Multi-Vector Visual Document Retrieval via Optimal Transport

DistilVDR: A Compact End-to-End Visual Document Retriever via Dual-Student Distillation

Beyond Bag-of-Patches: Learning Global Layout via Textual Supervision for Late-Interaction Visual Document Retrieval