FGR-ColBERT: Identifying Fine-Grained Relevance Tokens During Retrieval

Antonín Jarolím; Martin Fajčík

cs.IRMar 31, 2026

FGR-ColBERT: Identifying Fine-Grained Relevance Tokens During Retrieval

Authors: Antonín Jarolím, Martin Fajčík

Organizations: Brno University of Technology, Czech Republic

Abstract

Document retrieval identifies relevant documents but does not provide fine-grained evidence cues, such as specific relevant spans. A possible solution is to apply an LLM after retrieval; however, this introduces significant computational overhead and limits practical deployment. We propose FGR-ColBERT, a modification of ColBERT retrieval model that integrates fine-grained relevance signals distilled from an LLM directly into the retrieval function. Experiments on MS MARCO show that FGR-ColBERT (110M) achieves a token-level F1 of 64.5, exceeding the 62.8 of Gemma 2 (27B), despite being approximately 245 times smaller. At the same time, it preserves retrieval effectiveness (99% relative Recall@50) and remains efficient, incurring only a ~1.12x latency overhead compared to the original ColBERT.

Figures & tables

Panel of Figure 1 — Figure 1: (a) ColBERT retrieval followed by LLM span extraction vs. our approach with integrated LLM knowledge transfer. (b) Newly proposed late interaction with an added token-level relevance scoring, preserving document-level relevance.

Table 2

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Computation Step	Expression	FLOPs
Up-Scaling	$A=E^{R}_{d}\cdot W_{1}$	$2nhh_{2}$
Element-wise ReLU	$A^{\prime}=\operatorname{ReLU}(A)$	$nh_{2}$
Down-Scaling	$B=A^{\prime}\cdot W_{2}$	$2nhh_{2}$
Residual Connection	$E^{I}_{d}=E^{R}_{d}+B$	$nh$
Total FLOPs		$4nhh_{2}+nh_{2}+nh$

Appendix

Table 3: FLOP breakdown for the FFN network transforming embeddings for fine-grained relevance extraction.

Figure 2: Three positive passage-query pairs and corresponding token-level scores (highlighted; darker is higher) derived from the FGR-ColBERT model and fine-grained relevance cues provided by Gemma 2 (bold).

Evidence Retrieval

Evidence retrieval finds and organizes source-grounded information to answer questions, verify claims, and support decisions.

See all 161 papers

PLAID-PRF: Pseudo-Relevance Feedback with Centroid-like Tokens in PLAID
Jul 21, 2026Xiao Wang, Sean MacAvaney, Craig MacdonaldLate Interaction Retrieval Multi-Vector Retrieval
University of International Business and Economics Beijing, China · University of Glasgow Glasgow, United Kingdom
FLINT: Fixed-budget Late Interaction
Oct 19, 2026Himank Handa, Negar Arabzadeh, Matei ZahariaLate Interaction Retrieval Multi-Vector Retrieval
University of Illinois Urbana-Champaign · University of California, Berkeley
Can Language Models Actually Retrieve In-Context? Drowning in Documents at Million Token Scale
Jul 1, 2026Siddharth Gollapudi, Nilesh Gupta, Prasann Singhal +1In-Context Retrieval Length Generalization
UC Berkeley · UT Austin

Jul 21, 2026J/K move · Enter open · S save

PLAID-PRF: Pseudo-Relevance Feedback with Centroid-like Tokens in PLAID

Xiao Wang, Sean MacAvaney, Craig Macdonald

University of International Business and Economics Beijing, China · University of Glasgow Glasgow, United Kingdom

Late Interaction Retrieval Multi-Vector Retrieval Dense Retrieval

Open paper AskPDF arXiv

Multi-vector dense retrieval models, such as ColBERT, achieve strong retrieval effectiveness by modelling fine-grained token-level interactions between queries and documents. Methods such as PLAID use centroid-based quantisation of each token's vector to reduce the index size and speed up retrieval while maintaining strong effectiveness. In this work, we introduce PLAID-PRF, a method that performs Pseudo-Relevance Feedback (PRF) over PLAID to reformulate ColBERT's query vectors based on the top-retrieved results. In contrast with prior methods that perform PRF on multi-vector retrieval models, PLAID-PRF keeps computational costs low by leveraging the internal PLAID centroid vectors, treating them similarly to tokens in traditional PRF methods. The method selects a small and diverse set of high-utility expansion vectors and appends them to the original query, rerunning PLAID to refine both candidate generation and final scoring. Extensive experiments on the standard in-domain MSMARCO and four out-of-domain BEIR benchmarks show that PLAID-PRF consistently improves retrieval effectiveness over various baselines. In particular, PLAID-PRF improves over PLAID by up to 4.3% nDCG@10 and 7.3% MRR@10, while introducing substantially less computation overhead than prior PRF methods. The results demonstrate that our proposed centroid-aware PRF method offers an effective and lightweight mechanism to improve the quality of top-ranked retrieved results. Overall, this work enables effective and efficient feedback-aware late-interaction retrieval without expensive query-time document-token clustering.

Abstract

Evidence Retrieval

Explore similar work

PLAID-PRF: Pseudo-Relevance Feedback with Centroid-like Tokens in PLAID

FLINT: Fixed-budget Late Interaction

Can Language Models Actually Retrieve In-Context? Drowning in Documents at Million Token Scale