FLINT: Fixed-budget Late Interaction

Himank Handa; Negar Arabzadeh; Matei Zaharia

cs.IROct 19, 2026

FLINT: Fixed-budget Late Interaction

Authors: Himank Handa, Negar Arabzadeh, Matei Zaharia

Organizations: University of Illinois Urbana-Champaign · University of California, Berkeley

Abstract

Multi-vector retrieval models such as ColBERT often outperform single-vector models of comparable scale by matching queries and documents at the token level. However, retaining a vector for every token substantially increases storage requirements and slows retrieval compared to single-vector bi-encoders, limiting their scalability. We introduce FLINT (Fixed-budget Late INTeraction), an architecture that uses attention-based pooling to represent each query and document with a fixed number of embedding vectors. Unlike architectures that pool only document representations while retaining token-level queries, FLINT applies pooling symmetrically to both sides, limiting document storage and the number of vector comparisons needed during retrieval. FLINT achieves comparable ranking effectiveness to the closest baselines on BEIR and outperforms them on several benchmarks with longer and more complex queries, including BrowseComp+, CRUMB, and TREC Tip-of-the-Tongue 2025. We demonstrate that FLINT's effectiveness-efficiency benefits extend across encoder architectures, testing it on two separate backbones (BERT and ModernBERT). On long-query corpora, FLINT delivers 1.5-2.8 times faster end-to-end retrieval than ConstBERT, the closest baseline. FLINT also reduces index size by 5.3-9.3 times relative to token-level late-interaction models such as ColBERTv2.

Figures & tables

Panel of Figure 1 — Figure 1: FLINT pooling module.

Panel of Figure 2 — Figure 2: Architecture comparison across ColBERT, ColBERTv2, ConstBERT, and Symmetric FLINT. Tensor shapes $[\mathrm{seq},h]$ track the sequence dimension through each pipeline.

Figure 3: Mean Response Time decomposed into query-encode and PLAID-search time (BERT backbone), FLINT vs. ConstBERT, per corpus. Bars stack encoding (purple) and search (orange) time.

		NFCorpus 3.6K docs, $\bar{q}$ : 7			CRUMB Paper 363K docs, $\bar{q}$ : 180			CRUMB Clinical 915K docs, $\bar{q}$ : 183			TREC-ToT 2025 6.4M docs, $\bar{q}$ : 204
	Architecture	Enc.	Srch.	MRT $\downarrow$	Enc.	Srch.	MRT $\downarrow$	Enc.	Srch.	MRT $\downarrow$	Enc.	Srch.	MRT $\downarrow$
BERT	FLINT ( $V{=}32$ )	8.5	4.1	12.6	13.8	12.1	25.9	9.2	6.9	16.1	12.6	13.5	26.1
	ConstBERT	8.3	4.3	12.6	12.1	30.3	42.4	8.7	29.5	38.2	11.9	62.4	74.3
	ColBERTv2	8.4	6.8	15.2	11.8	259.0	270.8	8.4	574.3	582.7	11.8	763.1	774.9
ModernBERT	Single-Vector	16.9	1.0	17.9	22.0	1.1	23.1	17.2	1.0	18.2	22.7	1.4	24.1
	FLINT ( $V{=}16$ )	19.1	4.4	23.5	26.0	7.7	33.7	20.5	6.7	27.2	27.6	12.1	39.7

Table 2: Mean response time (ms). Query encoding (Enc.) plus search (Srch.) gives the mean response time (MRT). For each backbone and corpus, bold marks the lowest MRT and underline the second lowest.

\bar{q}

is the mean query length in tokens, and

V

is the number of vectors per query and document.

Figure 4: Response-time decomposition on TREC-ToT 2025 (BERT backbone). Query-encode (left), PLAID-search (middle), and total MRT (right) against query length for FLINT and ConstBERT, with linear fits (slopes in the legend); dashed drops mark where the fits cross.

Figure 5: Mean activated PLAID cells and candidate documents per query for FLINT and ConstBERT on long-query corpora.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Ratio of ConstBERT’s to FLINT’s query-vector participation ratio vs. query length, one line per corpus. Above the dotted line, ConstBERT’s query vectors are more spread out than FLINT’s.

Figure 7: Query spread and PLAID search on BERT. (a) Participation ratio vs. PLAID cells activated. (b) Cells activated vs. candidate documents scored. (c) Participation ratio vs. PLAID search time (CRUMB Paper). One point per query (

n{=}3{,}248

in (a)–(b),

1{,}008

in (c)); all axes are log except participation ratio.

\rho

is pooled Spearman,

r

is within-corpus Pearson.

Figure 8: PLAID cells activated at an equal number of query vectors. FLINT’s 32 pooled vectors compared with 32 of ConstBERT’s token vectors sampled at random per query, and with all of ConstBERT’s token vectors.

		MS MARCO	CRUMB
	Architecture	MRR@10	Legal	SetOps
BERT	FLINT ( $V{=}32$ )	0.380	0.138	0.184
	ConstBERT	0.387	0.136	0.232
	ColBERTv2	0.395	0.158	0.290
ModernBERT	Single-Vector	0.363	0.138	0.044
	FLINT ( $V{=}16$ )	0.395	0.154	0.177

Appendix

Table 4: Results on MS MARCO (dev, MRR@10) and the CRUMB Legal and SetOps tasks (NDCG@10), which are not included in Table 1 . Bold indicates the best result in each column and underline the second best.

		CRUMB Paper		TREC-ToT
Architecture	Pooling	$Q_{\mathrm{eff}}$	$D_{\mathrm{eff}}$	$Q_{\mathrm{eff}}$	$D_{\mathrm{eff}}$
Reference systems
ColBERTv2	token-level	4.28	4.17	3.92	4.02
ConstBERT	doc-side only	4.24	3.52	3.54	2.75
FLINT-symmetric (primary)	both sides	3.54	3.83	3.23	3.72
BERT-FLINT variants

Appendix

Table 5: Mean query and document effective rank. Participation-ratio effective ranks averaged over a fixed sample of 5,000 documents and all available queries (CRUMB Paper: 72; TREC-ToT train: 143). Lower values indicate more redundant multi-vector sets. FLINT-symmetric applies the shared perceiver to both sides (primary configuration). Variant rows below are document-side pooling changes unless marked query-side; CA = cross-attention and SA = self-attention in the pooling module;

(1\mathrm{CA}{+}2\mathrm{SA}){\times}3

denotes three stacked (1 CA+2 SA) stages.

Suite	D	Q	$\boldsymbol{\bar{d}}$	$\boldsymbol{\bar{q}}$	Task focus	Query characteristics
BEIR 12 corpora set	3.6K–5.4M	49–10K	14–384	7–27	Zero-shot heterogeneous retrieval across tasks including QA, fact-checking, and biomedical search.	Mostly short, keyword-like queries that evaluate transfer across heterogeneous in-domain and out-of-domain collections.
BrowseComp+	100K	830	3810	117	Deep-research web-agent evaluation with human-verified supporting evidence.	Elaborate, multi-step information needs with hard distractors; much of the difficulty arises from identifying relevant evidence within long documents.
CRUMB 6 long task subset	24K–1.1M	69–3.7K	180–575	97–229	Unified multi-task suite for multi-aspect and reasoning-intensive retrieval.	Complex natural-language queries containing multiple explicit and implicit constraints, domain-specific terminology, and limited lexical overlap with relevant documents.
TREC-ToT 2025	6.4M	1.6K	654	204	Known-item entity re-finding from vague natural-language descriptions.	Very long, conversational, and unconstrained queries that may contain false memories, relative comparisons, and minimal direct lexical overlap with the target entity.

Appendix

Table 8: Evaluation benchmarks details. D and Q denote the numbers of documents and queries, while

\bar{d}

and

\bar{q}

denote their mean lengths in ColBERT tokens. Ranges indicate the minimum and maximum values across each dataset.

Late Interaction Retrieval

Late interaction retrieval matches query and document token vectors after encoding to preserve fine-grained relevance while supporting efficient search.

See all 28 papers

PLAID-PRF: Pseudo-Relevance Feedback with Centroid-like Tokens in PLAID
Jul 21, 2026Xiao Wang, Sean MacAvaney, Craig MacdonaldLate Interaction Retrieval Multi-Vector Retrieval
University of International Business and Economics Beijing, China · University of Glasgow Glasgow, United Kingdom
FGR-ColBERT: Identifying Fine-Grained Relevance Tokens During Retrieval
Mar 31, 2026Antonín Jarolím, Martin FajčíkEvidence Retrieval Late Interaction Retrieval
Brno University of Technology, Czech Republic
ColBERTSaR: Sparsified ColBERT Index via Product Quantization
Jun 4, 2026Eugene Yang, Andrew Yates, Dawn Lawrie +3Vector Quantization Multi-Vector Retrieval
Johns Hopkins University, Baltimore, MD, USA

Jul 21, 2026J/K move · Enter open · S save

PLAID-PRF: Pseudo-Relevance Feedback with Centroid-like Tokens in PLAID

Xiao Wang, Sean MacAvaney, Craig Macdonald

University of International Business and Economics Beijing, China · University of Glasgow Glasgow, United Kingdom

Late Interaction Retrieval Multi-Vector Retrieval Dense Retrieval

Open paper AskPDF arXiv

Multi-vector dense retrieval models, such as ColBERT, achieve strong retrieval effectiveness by modelling fine-grained token-level interactions between queries and documents. Methods such as PLAID use centroid-based quantisation of each token's vector to reduce the index size and speed up retrieval while maintaining strong effectiveness. In this work, we introduce PLAID-PRF, a method that performs Pseudo-Relevance Feedback (PRF) over PLAID to reformulate ColBERT's query vectors based on the top-retrieved results. In contrast with prior methods that perform PRF on multi-vector retrieval models, PLAID-PRF keeps computational costs low by leveraging the internal PLAID centroid vectors, treating them similarly to tokens in traditional PRF methods. The method selects a small and diverse set of high-utility expansion vectors and appends them to the original query, rerunning PLAID to refine both candidate generation and final scoring. Extensive experiments on the standard in-domain MSMARCO and four out-of-domain BEIR benchmarks show that PLAID-PRF consistently improves retrieval effectiveness over various baselines. In particular, PLAID-PRF improves over PLAID by up to 4.3% nDCG@10 and 7.3% MRR@10, while introducing substantially less computation overhead than prior PRF methods. The results demonstrate that our proposed centroid-aware PRF method offers an effective and lightweight mechanism to improve the quality of top-ranked retrieved results. Overall, this work enables effective and efficient feedback-aware late-interaction retrieval without expensive query-time document-token clustering.

Abstract

Late Interaction Retrieval

Explore similar work

PLAID-PRF: Pseudo-Relevance Feedback with Centroid-like Tokens in PLAID

FGR-ColBERT: Identifying Fine-Grained Relevance Tokens During Retrieval

ColBERTSaR: Sparsified ColBERT Index via Product Quantization