Multi-vector retrieval models such as ColBERT often outperform single-vector models of comparable scale by matching queries and documents at the token level. However, retaining a vector for every token substantially increases storage requirements and slows retrieval compared to single-vector bi-encoders, limiting their scalability. We introduce FLINT (Fixed-budget Late INTeraction), an architecture that uses attention-based pooling to represent each query and document with a fixed number of embedding vectors. Unlike architectures that pool only document representations while retaining token-level queries, FLINT applies pooling symmetrically to both sides, limiting document storage and the number of vector comparisons needed during retrieval. FLINT achieves comparable ranking effectiveness to the closest baselines on BEIR and outperforms them on several benchmarks with longer and more complex queries, including BrowseComp+, CRUMB, and TREC Tip-of-the-Tongue 2025. We demonstrate that FLINT's effectiveness-efficiency benefits extend across encoder architectures, testing it on two separate backbones (BERT and ModernBERT). On long-query corpora, FLINT delivers 1.5-2.8 times faster end-to-end retrieval than ConstBERT, the closest baseline. FLINT also reduces index size by 5.3-9.3 times relative to token-level late-interaction models such as ColBERTv2.
Figures & tables
Figure 1: FLINT pooling module.
Figure 2: Architecture comparison across ColBERT, ColBERTv2, ConstBERT, and Symmetric FLINT. Tensor shapes [seq,h] track the sequence dimension through each pipeline.
Figure 3: Mean Response Time decomposed into query-encode and PLAID-search time (BERT backbone), FLINT vs. ConstBERT, per corpus. Bars stack encoding (purple) and search (orange) time.
NFCorpus 3.6K docs, qˉ : 7
CRUMB Paper 363K docs, qˉ : 180
CRUMB Clinical 915K docs, qˉ : 183
TREC-ToT 2025 6.4M docs, qˉ : 204
Architecture
Enc.
Srch.
MRT ↓
Enc.
Srch.
MRT ↓
Enc.
Srch.
MRT ↓
Enc.
Srch.
MRT ↓
BERT
FLINT ( V=32 )
8.5
4.1
12.6
13.8
12.1
25.9
9.2
6.9
16.1
12.6
13.5
26.1
ConstBERT
8.3
4.3
12.6
12.1
30.3
42.4
8.7
29.5
38.2
11.9
62.4
74.3
ColBERTv2
8.4
6.8
15.2
11.8
259.0
270.8
8.4
574.3
582.7
11.8
763.1
774.9
ModernBERT
Single-Vector
16.9
1.0
17.9
22.0
1.1
23.1
17.2
1.0
18.2
22.7
1.4
24.1
FLINT ( V=16 )
19.1
4.4
23.5
26.0
7.7
33.7
20.5
6.7
27.2
27.6
12.1
39.7
Table 2: Mean response time (ms). Query encoding (Enc.) plus search (Srch.) gives the mean response time (MRT). For each backbone and corpus, bold marks the lowest MRT and underline the second lowest. qˉ is the mean query length in tokens, and V is the number of vectors per query and document.
Figure 4: Response-time decomposition on TREC-ToT 2025 (BERT backbone). Query-encode (left), PLAID-search (middle), and total MRT (right) against query length for FLINT and ConstBERT, with linear fits (slopes in the legend); dashed drops mark where the fits cross.
Figure 5: Mean activated PLAID cells and candidate documents per query for FLINT and ConstBERT on long-query corpora.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Ratio of ConstBERT’s to FLINT’s query-vector participation ratio vs. query length, one line per corpus. Above the dotted line, ConstBERT’s query vectors are more spread out than FLINT’s.
Figure 7: Query spread and PLAID search on BERT. (a) Participation ratio vs. PLAID cells activated. (b) Cells activated vs. candidate documents scored. (c) Participation ratio vs. PLAID search time (CRUMB Paper). One point per query ( n=3,248 in (a)–(b), 1,008 in (c)); all axes are log except participation ratio. ρ is pooled Spearman, r is within-corpus Pearson.
Figure 8: PLAID cells activated at an equal number of query vectors. FLINT’s 32 pooled vectors compared with 32 of ConstBERT’s token vectors sampled at random per query, and with all of ConstBERT’s token vectors.
MS MARCO
CRUMB
Architecture
MRR@10
Legal
SetOps
BERT
FLINT ( V=32 )
0.380
0.138
0.184
ConstBERT
0.387
0.136
0.232
ColBERTv2
0.395
0.158
0.290
ModernBERT
Single-Vector
0.363
0.138
0.044
FLINT ( V=16 )
0.395
0.154
0.177
Appendix
Table 4: Results on MS MARCO (dev, MRR@10) and the CRUMB Legal and SetOps tasks (NDCG@10), which are not included in Table 1 . Bold indicates the best result in each column and underline the second best.
CRUMB Paper
TREC-ToT
Architecture
Pooling
Qeff
Deff
Qeff
Deff
Reference systems
ColBERTv2
token-level
4.28
4.17
3.92
4.02
ConstBERT
doc-side only
4.24
3.52
3.54
2.75
FLINT-symmetric (primary)
both sides
3.54
3.83
3.23
3.72
BERT-FLINT variants
Appendix
Table 5: Mean query and document effective rank. Participation-ratio effective ranks averaged over a fixed sample of 5,000 documents and all available queries (CRUMB Paper: 72; TREC-ToT train: 143). Lower values indicate more redundant multi-vector sets. FLINT-symmetric applies the shared perceiver to both sides (primary configuration). Variant rows below are document-side pooling changes unless marked query-side; CA = cross-attention and SA = self-attention in the pooling module; (1CA+2SA)×3 denotes three stacked (1 CA+2 SA) stages.
Suite
D
Q
dˉ
qˉ
Task focus
Query characteristics
BEIR 12 corpora set
3.6K–5.4M
49–10K
14–384
7–27
Zero-shot heterogeneous retrieval across tasks including QA, fact-checking, and biomedical search.
Mostly short, keyword-like queries that evaluate transfer across heterogeneous in-domain and out-of-domain collections.
BrowseComp+
100K
830
3810
117
Deep-research web-agent evaluation with human-verified supporting evidence.
Elaborate, multi-step information needs with hard distractors; much of the difficulty arises from identifying relevant evidence within long documents.
CRUMB 6 long task subset
24K–1.1M
69–3.7K
180–575
97–229
Unified multi-task suite for multi-aspect and reasoning-intensive retrieval.
Complex natural-language queries containing multiple explicit and implicit constraints, domain-specific terminology, and limited lexical overlap with relevant documents.
TREC-ToT 2025
6.4M
1.6K
654
204
Known-item entity re-finding from vague natural-language descriptions.
Very long, conversational, and unconstrained queries that may contain false memories, relative comparisons, and minimal direct lexical overlap with the target entity.
Appendix
Table 8: Evaluation benchmarks details. D and Q denote the numbers of documents and queries, while dˉ and qˉ denote their mean lengths in ColBERT tokens. Ranges indicate the minimum and maximum values across each dataset.
Late interaction retrieval matches query and document token vectors after encoding to preserve fine-grained relevance while supporting efficient search.
Multi-vector dense retrieval models, such as ColBERT, achieve strong retrieval effectiveness by modelling fine-grained token-level interactions between queries and documents. Methods such as PLAID use centroid-based quantisation of each token's vector to reduce the index size and speed up retrieval while maintaining strong effectiveness. In this work, we introduce PLAID-PRF, a method that performs Pseudo-Relevance Feedback (PRF) over PLAID to reformulate ColBERT's query vectors based on the top-retrieved results. In contrast with prior methods that perform PRF on multi-vector retrieval models, PLAID-PRF keeps computational costs low by leveraging the internal PLAID centroid vectors, treating them similarly to tokens in traditional PRF methods. The method selects a small and diverse set of high-utility expansion vectors and appends them to the original query, rerunning PLAID to refine both candidate generation and final scoring. Extensive experiments on the standard in-domain MSMARCO and four out-of-domain BEIR benchmarks show that PLAID-PRF consistently improves retrieval effectiveness over various baselines. In particular, PLAID-PRF improves over PLAID by up to 4.3% nDCG@10 and 7.3% MRR@10, while introducing substantially less computation overhead than prior PRF methods. The results demonstrate that our proposed centroid-aware PRF method offers an effective and lightweight mechanism to improve the quality of top-ranked retrieved results. Overall, this work enables effective and efficient feedback-aware late-interaction retrieval without expensive query-time document-token clustering.
Xiao Wang, Sean MacAvaney, Craig Macdonald
University of International Business and Economics Beijing, China · University of Glasgow Glasgow, United Kingdom
Document retrieval identifies relevant documents but does not provide fine-grained evidence cues, such as specific relevant spans. A possible solution is to apply an LLM after retrieval; however, this introduces significant computational overhead and limits practical deployment. We propose FGR-ColBERT, a modification of ColBERT retrieval model that integrates fine-grained relevance signals distilled from an LLM directly into the retrieval function. Experiments on MS MARCO show that FGR-ColBERT (110M) achieves a token-level F1 of 64.5, exceeding the 62.8 of Gemma 2 (27B), despite being approximately 245 times smaller. At the same time, it preserves retrieval effectiveness (99% relative Recall@50) and remains efficient, incurring only a ~1.12x latency overhead compared to the original ColBERT.
While ColBERT is an effective neural retrieval architecture, it requires a heavy index structure to support candidate set retrieval based on approximated token embeddings, gathering and decompressing document token embeddings, and applying the MaxSim operation. Indexes in PLAID and similar ColBERT implementations require five to ten times the disk storage of the original raw text, which limits their scalability. Furthermore, prior work has identified that the gathering and decompression stages are the primary inefficiencies at query time. Limiting the number of document tokens that must be gathered by thresholding and score approximation does not eliminate the need for the entire index to support ad hoc queries. In this work, we propose an embedding quantization approach that turns a ColBERT index into a true inverted index. We show that, theoretically, ColBERT with embedding quantization is equivalent to learned-sparse retrieval except for the scoring mechanism. Empirically, we demonstrate that our index is 50-70% smaller than a one-bit PLAID index while retaining retrieval effectiveness.