Learned Sparse Retrieval

Follow this topic Ask about this topic

Learned sparse retrieval represents queries and documents as a small set of weighted terms or concepts, combining lexical matching with some ability to match related meanings. Common approaches include SPLADE-style masked-language-model expansion, sparse autoencoders that learn latent vocabularies, and subword features, typically served through inverted indexes. Work spans text, code, and multimodal search, with benchmarks such as BEIR and MTEB used to assess effectiveness alongside latency and index size. A central tension is how to make sparse representations more expressive and robust across domains and languages without sacrificing the speed, compactness, and interpretability that make them useful.

Latest papers

18 papers2 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

Weighted State Souping for Efficient Long-Context Retrieval
Oct 19, 2026William L. Tong, Aryo Lotfi, Emmanuel Abbe +3State Space Language Models Long-Context Retrieval
Harvard University · Apple
Generated Query Expansion Still Helps Strong Sparse Retrieval: A Controlled Study with SPLADE-v3
Sep 29, 2026Ryan C. Barron, Cade W. Trotter, Maksim E. Eren +3Learned Sparse Retrieval Lexical Retrieval
Computational Intelligence & Modeling, Los Alamos National Laboratory, Los Alamos, New Mexico, USA. · Modeling and Observations of Earth Systems, Los Alamos National Laboratory, Los Alamos, New Mexico, USA. · Fluid Dynamics and Solid Mechanics, Los Alamos National Laboratory, Los Alamos, New Mexico, USA. +2
Concepts Complement Dense Semantics: Learning Compact Sparse Spaces for Text-Image Retrieval
Sep 26, 2026Yoonseo Kim, Jungwoo Choi, Cheonyoung Park +3Cross-Modal Retrieval Vision-Language Retrieval
Sungkyunkwan University South Korea · KT Corporation South Korea
Static Pruning Across Sparse Retrieval Regimes: What Transfers, What Breaks, and What Still Helps
Aug 17, 2026Zirui Song, Yuye Zhu, Yang YangLearned Sparse Retrieval
Amazon Web Services Shanghai, China
UEmbed: Unified Sparse and Dense Multimodal Embeddings
Aug 3, 2026Tingyu Song, Mingxin Li, Yanzhao Zhang +5Multimodal IR Learned Sparse Retrieval
CASIA · University of Chinese Academy of Sciences · Alibaba Group +1
Rescaling MLM-Head for Neural Sparse Retrieval
Jun 17, 2026Youngjoon Jang, Seongtae Hong, Jonah Turner +1Masked Language Modeling Learned Sparse Retrieval
Korea University Seoul, South Korea · Independent Seoul, South Korea
uva-irlab-conv at SemEval-2026 Task 8: Multi-Turn RAG with Learned Sparse Retrieval and Listwise Reranking
Jun 10, 2026Simon Lupart, Kidist Amde Mekonnen, Zahra Abbasiantaeb +1Retrieval-Augmented Generation Conversational QA
University of Amsterdam Amsterdam, The Netherlands
Inference-Free Multimodal Learned Sparse Retrieval for Production-Scale Visual Document Search
May 29, 2026Gyu-Hwung Cho, Youngjune Lee, Kiyoon Jeong +5Multimodal IR Learned Sparse Retrieval
NAVER Corp., Republic of Korea · Seoul National University, Republic of Korea · NAVER Corp. +6
No More K-means: Single-Stage Sparse Coding for Efficient Multi-Vector Retrieval
May 28, 2026Lixuan Guo, Yifei Wang, Tiansheng Wen +3Learned Sparse Retrieval Multi-Vector Retrieval
Stony Brook University · Xidian University · Amazon AGI SF Lab +4
Latent Terms: Dense Retrievers Contain Trivially Extractable BM25-ready Zipfian Vocabularies
May 28, 2026Benjamin Clavié, Sean Lee, Aamir Shakir +1Learned Sparse Retrieval Information Retrieval
Mixedbread AI · National Institute of Informatics (NII) · University of Tsukuba
Surface-Form Neural Sparse Retrieval: Robust Fuzzy Matching for Industrial Music Search
May 18, 2026Paul Greyson, Zhichao Geng, Wei Zhang +1Learned Sparse Retrieval Music Information Retrieval
Amazon
From Tokens to Concepts: Leveraging SAE for SPLADE
Apr 23, 2026Yuxuan Zong, Mathias Vast, Basile Van Cooten +2Learned Sparse Retrieval Sparse Autoencoders
Sorbonne Université, CNRS, ISIR · Sinequa by ChapsVision · CNRS, Sorbonne Université, ISIR
Why Advanced Encoders Lag on Sparse Retrieval? The Answer and an Approach to Bridging Vocabulary Gaps
Apr 20, 2026Zhichao Geng, Yang YangLearned Sparse Retrieval Information Retrieval
Amazon Web Service · Shanghai, China
Associative-State Universal Transformers: Sparse Retrieval Meets Structured Recurrence
Apr 1, 2026Liu XiaoRecurrent Transformers Autoregressive Language Modeling
Liu Xiao
On the Challenges and Opportunities of Learned Sparse Retrieval for Code
Mar 23, 2026Simon Lupart, Maxime Louis, Thibault Formal +2Learned Sparse Retrieval Information Retrieval
NAVER LABS Europe
Visual Words Meet BM25: Sparse Auto-Encoder Visual Word Scoring for Image Retrieval
Mar 6, 2026Donghoon Han, Eunhwan Park, Seunghyeon SeoLearned Sparse Retrieval Fine-Grained Image Retrieval
Moreh · Seoul National University
Learning Retrieval Models with Sparse Autoencoders
Feb 27, 2026Thibault Formal, Maxime Louis, Hervé Dejean +1Learned Sparse Retrieval Multilingual IR
NAVER LABS Europe
Revisiting Text Ranking in Deep Research
Feb 25, 2026Chuan Meng, Litu Ou, Sean MacAvaney +1Deep Research Agents Learning to Rank
The University of Edinburgh Edinburgh, United Kingdom · University of Glasgow Glasgow, United Kingdom

Oct 19, 2026J/K move · Enter open · S save

Weighted State Souping for Efficient Long-Context Retrieval

William L. Tong, Aryo Lotfi, Emmanuel Abbe, Josh Susskind+2

Harvard University · Apple

State Space Language Models Long-Context Retrieval Long-Context Modeling Language Modeling Learned Sparse Retrieval

Open paper AskPDF arXiv

State space models (SSMs) combine fast inference and fixed state size with competitive language-modeling performance. Recently, state souping has been proposed as a procedure whereby an SSM independently encodes chunks of tokens like documents, tool definitions, or memory records. The resulting state encodings may be cached and recombined to cheaply construct a context for downstream queries. However, both state souping and vanilla SSMs deteriorate on long, crowded contexts, limiting their utility for long-horizon domains. To address this issue, we propose weighted state souping, a simple method that learns query-dependent weights for composing cached states. The resulting models improve long-context performance substantially across both single- and multi-hop document question answering, needle-in-a-haystack retrieval, variable tracking, and tool use. Our method can be further adapted into an efficient sparse variant, yielding near-constant encoding cost with respect to context length while preserving (and sometimes improving) performance. We also present simple theoretical arguments that illustrate why souping works and when it may fail, and discuss informally the advantages of weighted souping. Weighted state souping thus combines reusable context representations with selective retrieval, offering an efficient approach to language modeling over large collections of documents, tools, and other forms of structured context that increasingly dominate modern agentic workloads.