Multi-Vector Retrieval

Multi-vector retrieval represents a query and each candidate document with multiple embeddings, allowing matches to focus on relevant tokens, image patches, or other local features rather than one pooled vector. A common approach is late interaction, such as ColBERT-style MaxSim scoring, alongside methods that group or compress vectors and train representations through contrastive learning or distillation. Work spans text search and visual-document retrieval, with benchmarks such as MS MARCO, BEIR, and ViDoRe, and uses indexes including graphs, inverted indexes, and quantized representations to speed up search. The central tension is preserving fine-grained matching quality while controlling storage and latency; compression effectiveness varies across models and tasks, and robustness to long or mismatched queries remains a concern.