Late Interaction Retrieval

Ask about this topic

Overview

Late interaction retrieval represents queries and documents as sets of contextual token vectors, then compares them during search so that a relevant phrase or passage can match without requiring whole-text embeddings to align. ColBERT-style MaxSim is a common scoring method, paired with techniques such as approximate nearest-neighbor search, quantization, token selection, and vector compression to control indexing and query costs. The approach is used in text search and increasingly in visual documents, video, biomedical linking, and long-term memory, with evaluations including BEIR, MS MARCO, and ViDoRe. A central challenge is balancing fine-grained matching quality against storage and latency, while managing sensitivity to document length and ensuring performance transfers to longer or unfamiliar queries.

Latest papers

CardsList

28 papers5 papers in the last four weeks, down 17% on the four weeks before. 0.0% of all new papers.

  1. pplx-embed-v2-late Technical Report

    Oct 21, 2026Markus Krimmel, Louis Milliken, Ekaterina Glazkova +8Multimodal EmbeddingMultimodal IR

    Perplexity AI

  2. FLINT: Fixed-budget Late Interaction

    Oct 19, 2026Himank Handa, Negar Arabzadeh, Matei ZahariaLate Interaction RetrievalMulti-Vector Retrieval

    University of Illinois Urbana-Champaign · University of California, Berkeley

  3. Simple Linear Corrections for Compression in Late Interaction

    Oct 9, 2026Mihir Agarwal, Sabyasachi Basu, Viraj AgrawalLate Interaction RetrievalMulti-Vector Retrieval

    Microsoft Research India

  4. Beyond the Final Embedding: Retrieving Memories from Token-Level Evidence

    Oct 9, 2026Han Luo, Zhan Zhuang, Xu He +5Late Interaction RetrievalMulti-Vector Retrieval

    Southeast University · MemoraX AI · Leiden University +1

  5. Re-ranking and Late Interaction Drive Retrieval Quality: A Controlled Comparison of RAG Strategies for Scientific Question Answering

    Sep 29, 2026Bhagyesh Rathi, Eshan Chawla, William B. AndreopoulosLate Interaction RetrievalScientific QA

    San Jose State University, Department of Computer Science

  6. SALI: Shot-Aware Late Interaction for Cross-Shot Relation Matching in Text-to-Video Retrieval using Film-Grammar Knowledge

    Sep 24, 2026Toya Oyama, Rainer Lienhart, Shin'ichi SatohLate Interaction Retrieval

    The University of Tokyo · National Institute of Informatics · University of Augsburg

  7. BELXTR: Biomedical Entity Linking via Contextualized Token Retrieval

    Sep 22, 2026Samuele Garda, Ulf LeserLate Interaction RetrievalBiomedical Entity Linking

    Computer Science, Humboldt-Universität zu Berlin, Rudower Chaussee 25, 12489, Berlin, Germany

  8. MAGIC: Marginal-Guided Compression with Optimal Transport for Efficient Visual Document Retrieval

    Sep 17, 2026Xu Yuan, Hua Liu, Wenqi Fan +1Visual Document RetrievalEmbedding Compression

    The Hong Kong Polytechnic University, HK SAR · Southern University of Science and Technology, China

  9. EigenLI: Spectral Approximations to Late Interaction

    Sep 7, 2026Archish S, Sabyasachi Basu, Ankit Garg +2Late Interaction RetrievalMulti-Vector Retrieval

    Temple · Microsoft Research India

  10. Query-Aware Token Budgeting for Efficient Late-Interaction Visual Document Retrieval

    Sep 7, 2026PS Rishi, Rajeev Ranjan Dwivedi, Vinod K KurmiLate Interaction RetrievalToken Pruning

    Indian Institute of Science Education and Research Bhopal (IISER Bhopal), India

  11. ViSAR: Training-Free Adaptive-kk Retrieval for Visual Document Question Answering

    Sep 2, 2026Adrien Mialland, Marc Plantevit, Julien Gallois +1Adaptive RetrievalVisual Document Retrieval

    INSA Lyon, CNRS, LIRIS UMR 5205, F-69621 Villeurbanne, France · EPITA Research Laboratory (LRE), FR-94276, Le Kremlin-Bicêtre, France · Lowit, FR-69003 Lyon, France

  12. MoganColBERT-TR: A Late-Interaction Multi-Vector Retrieval Model for Turkish

    Aug 26, 2026Furkan Yilmaz, Habibe Aleyna Tasdemir, Muhammed Faruk GozayInformation RetrievalLate Interaction Retrieval

  13. VaRS-Doc: Interpretation-Aware Variant Representations via Latent Self-Probing for Visual Document Retrieval

    Aug 2, 2026Haocheng Wang, Tongkun Guan, Wei Shen +1Visual Representation LearningLate Interaction Retrieval

    MoE Key Lab of Artificial Intelligence, AI Institute, School of Computer Science, Shanghai Jiao Tong University, China

  14. DenseOn with the LateOn: Fully Open Dense and Late-Interaction Models for Multilingual, Long-Context, and Code Search

    Jul 29, 2026Raphaël Sourty, Antoine Chaffin, Paulo Roberto Moura Junior +1Late Interaction RetrievalCode Search

    1LightOn

  15. H+ Embedding: Harmonizing Global and Token-Level Retrieval with Context-Dependent Phrases

    Jul 29, 2026Shusen Zhang, Junyi Hu, Ye Feng +6Late Interaction RetrievalMulti-Vector Retrieval

    Alibaba Health

  16. PLAID-PRF: Pseudo-Relevance Feedback with Centroid-like Tokens in PLAID

    Jul 21, 2026Xiao Wang, Sean MacAvaney, Craig MacdonaldLate Interaction RetrievalMulti-Vector Retrieval

    University of International Business and Economics Beijing, China · University of Glasgow Glasgow, United Kingdom

  17. Training-Free Lexical-Dense Fusion for Conversational-Memory Retrieval

    Jun 2, 2026Christian LysenstøenLate Interaction RetrievalLexical Retrieval

    Inland Norway University of Applied Sciences · Visiting student, University of California, Berkeley

  18. PROTOCOL: Late Interaction Retrieval for Protein Homolog Search

    May 27, 2026Gabrielle Cohn, Rohan Gumaste, Minh Hoang +1Late Interaction RetrievalProtein Language Models

    MIT · Princeton University

  19. Your Embedding Model is SMARTer Than You Think

    May 24, 2026Jianrui Zhang, Hyun Jung Lee, Sukanta Ganguly +3Multimodal IREfficient Multimodal Inference

    Korea University · NetApp, Inc. · UW-Madison

  20. Spectral Retrieval: Multi-Scale Sinc Convolution over Token Embeddings for Localized Retrieval in LLM Multi-Agent Systems

    May 23, 2026Andrea MorandiLate Interaction RetrievalMulti-Vector Retrieval

    Cisco Systems, Inc.

  21. LFRAG: Layout-oriented Fine-grained Retrieval-Augmented Generation on Multimodal Document Understanding

    Apr 18, 2026Yifan Zhu, Yu Mi, Yue Lu +2Multimodal RAGLate Interaction Retrieval

    Zhejiang University · Hangzhou, China · Institute of Blockchain and Data Security +1

  22. Reproduction Beyond Benchmarks: ConstBERT and ColBERT-v2 Across Backends and Query Distributions

    Apr 11, 2026Utshab Kumar Ghosh, Ashish David, Shubham ChatterjeeMulti-Vector RetrievalLate Interaction Retrieval

    Missouri University of Science and Technology, Rolla, MO, USA

  23. Spike Hijacking in Late-Interaction Retrieval

    Apr 6, 2026Karthik Suresh, Tushar Vatsa, Tracy King +2Late Interaction RetrievalMulti-Vector Retrieval

    Adobe, 345 Park Ave, San Jose, CA 95110, USA

  24. FGR-ColBERT: Identifying Fine-Grained Relevance Tokens During Retrieval

    Mar 31, 2026Antonín Jarolím, Martin FajčíkEvidence RetrievalLate Interaction Retrieval

    Brno University of Technology, Czech Republic

  25. Working Notes on Late Interaction Dynamics: Analyzing Targeted Behaviors of Late Interaction Models

    Mar 27, 2026Antoine Edy, Max Conti, Quentin MacéLate Interaction RetrievalMulti-Vector Retrieval

    Illuin Technology · CentraleSupélec, Paris-Saclay

  26. Evo-Retriever: LLM-Guided Curriculum Evolution with Viewpoint-Pathway Collaboration for Multimodal Document Retrieval

    Mar 17, 2026Weiqing Li, Jinyue Guo, Yaqi Wang +4Multimodal IRCross-Modal Retrieval

    Alibaba Cloud Computing

  27. Training for Compositional Sensitivity Reduces Dense Retrieval Generalization

    Mar 16, 2026Radoslav Ralev, Aditeya Baral, Iliya Zhechev +2Information RetrievalDense Retrieval

    Redis, Bulgaria and Redis, USA

  28. AMES: Approximate Multi-modal Enterprise Search via Late Interaction Retrieval

    Mar 13, 2026Tony Joseph, Carlos Pareja, David Lopes Pegna +1Multimodal IRInformation Retrieval

    Apple, Cupertino, CA, USA