Lexical Retrieval

Follow this topic Ask about this topic

Lexical retrieval studies how to find and rank documents using words and their frequencies, from classic TF-IDF and BM25 to learned sparse models such as SPLADE. Researchers improve matching with query expansion, term weighting, and hybrid combinations with dense retrieval, applying these methods to web search, retrieval-augmented generation, and conversational memory. Inverted indexes make lexical search efficient and scalable, and recent work explores its use in agentic search, private retrieval, and very large collections. A central tension is that exact term matching can be fast and effective but miss relevant documents with different wording, while semantic methods may add cost and do not consistently outperform well-tuned lexical systems.

Latest papers

27 papers7 papers in the last four weeks, up 75% on the four weeks before. 0.0% of all new papers.

What the Query Remembers: Query Abstraction Governs Lexical and Dense Retrieval of Classical Arabic Poetry
Oct 22, 2026Omar Farouk Zouak, Selma Mani, Larbi Said Chikh +1Lexical Retrieval Dense Retrieval
National School of Artificial Intelligence
Modeling bilingual naming performance through laterally-connected SOMs
Oct 16, 2026Divya Prabhakaran, Uli Grasemann, Erie Shivers +3Lexical Retrieval Computational Cognitive Modeling
Department of Computer Science, The University of Texas at Austin · Center for Brain Recovery, Boston University
SAG2: Embedding-Free Multi-Hop Retrieval with BM25 and JEV
Oct 15, 2026Yuchao WuMulti-Hop Retrieval Information Retrieval
Zleap AI
Reference Wording Shapes Retrieval of Robot Task Instructions
Oct 13, 2026Jahnavi YelamanchiLexical Retrieval IR Evaluation
Generated Query Expansion Still Helps Strong Sparse Retrieval: A Controlled Study with SPLADE-v3
Sep 29, 2026Ryan C. Barron, Cade W. Trotter, Maksim E. Eren +3Learned Sparse Retrieval Lexical Retrieval
Computational Intelligence & Modeling, Los Alamos National Laboratory, Los Alamos, New Mexico, USA. · Modeling and Observations of Earth Systems, Los Alamos National Laboratory, Los Alamos, New Mexico, USA. · Fluid Dynamics and Solid Mechanics, Los Alamos National Laboratory, Los Alamos, New Mexico, USA. +2
PILLAR: Private Inverted-Index Lexical Lookup for Augmented Retrieval
Sep 28, 2026Truong Son Nguyen, Daniel Blackley, Ni Trieu +1Retrieval-Augmented Generation Privacy-Preserving RAG
Arizona State University · Department of Computer Science Arizona State University Tempe, AZ 85281, USA · George Mason University +1
Retrieving Biblical Intertextual References in Karen Blixen's Seven Gothic Tales
Sep 28, 2026András Kovács, Alexander Conroy, Daniel Hershcovich +1Computational Literary Studies Lexical Retrieval
Department of Nordic Studies and Linguistics, University of Copenhagen, Copenhagen, Denmark · Department of Computer Science, University of Copenhagen, Copenhagen, Denmark
SignTrace: Describe a Sign, Find the Word
Sep 13, 2026Zengji Tu, Xingye Zhu, Ningjing Wang +3Lexical Retrieval
Peking University, Beijing 100871, China · Sign Language Department, Loving Heart Society, Peking University, Beijing 100871, China · Institute of Population Research, Peking University, Beijing 100871, China
TF-IDF and BM25 Are Exact KL Divergences
Sep 12, 2026Ivan SilajevKullback-Leibler Divergence Information Retrieval
Independent Researcher
Inspicio: Open-Vocabulary, LLM-Based Sense Retrieval for Historical Languages
Sep 1, 2026Michele CilettiLexical Retrieval Low-Resource Language Processing
Department of Humanities, University of Foggia / Via Arpi 176, 71121 Foggia, Italy
It Takes Two to Match: Co-Evolving Generative Retriever with Reinforcement Learning
Sep 1, 2026Runpeng Dai, Kaili Huang, Changsung Kang +1Reinforcement Learning Generative Retrieval
University of North Carolina at Chapel Hill · Apple
SQLite is Enough. Lexical, Semantic, and Hybrid Search with scrydb
Aug 25, 2026Timo BreuerHybrid Retrieval Semantic Search
TH Köln – University of Applied Sciences, 50968 Köln, Germany
When Your Agent Opens the Chat App: Agent-Controlled Search over Raw Chat Logs Rivals Structured Memory
Aug 13, 2026Ruizhe Li, Licheng Zhang, Benfeng Xu +3Conversational Memory Agentic Search
University of Science and Technology of China · MetaStone Technology
CeQe: Grounding Lexical Retrieval in Semantic Evidence
Aug 1, 2026Adam Kahirov, Umesh Deshpande, Swaminathan SundararamanLexical Retrieval Information Retrieval
IBM Research
Hierarchical BM25: Lexical Search at Billion-Document Scale
Jul 31, 2026Umesh Deshpande, Swaminathan SundararamanLexical Retrieval Information Retrieval
IBM Research, San Jose, USA
BM25 Wins at Scale: A Scaling Study of Retrieval-Augmented Generation Paradigms
Jul 29, 2026Pengyu Wang, Benfeng Xu, Shaohan Wang +5Retrieval-Augmented Generation Graph Retrieval-Augmented Generation
University of Science and Technology of China, Hefei, China · Metastone Technology, Beijing, China · Information Technology Research Center, Beijing Academy of Agriculture and Forestry Sciences, Beijing, China
SkillSight: Calibrating Generic Content Bias for Skill Retrieval
Jul 21, 2026Jinying Xiao, Bin Li, Xiaopeng Li +7Lexical Retrieval LLM Agent Skill Retrieval
1National University of Defense Technology · 2Qinghai Normal University · 3Xizang University
STORM: Stepwise Token Optimization with Reward-Guided Beam Search
Jun 9, 2026Arthur Satouf, Giulio D'Erasmo, Yuxuan Zong +3Language Model Decoding Lexical Retrieval
MILA – Quebec AI Institute & ILLS, Canada · 5Sapienza, University of Rome, Italy · 4Sorbonne Université & ISIR & CNRS, France +2
Training-Free Lexical-Dense Fusion for Conversational-Memory Retrieval
Jun 2, 2026Christian LysenstøenLate Interaction Retrieval Lexical Retrieval
Inland Norway University of Applied Sciences · Visiting student, University of California, Berkeley
AgentIR: A Workload-Adaptive Cascade Retrieval Substrate for Long-Term Conversational Memory
May 24, 2026Aojie Yuan, Haiyue Zhang, Shahin NazarianLexical Retrieval Long-Term Conversational Memory
University of Southern California Los Angeles, CA, USA
Is Grep All You Need? How Agent Harnesses Reshape Agentic Search
May 14, 2026Sahil Sen, Akhil Kasturi, Elias Lumer +2Agentic Search LLM Agent Evaluation
PricewaterhouseCoopers, U.S.
Rethinking Agentic Search with Pi-Serini: Is Lexical Retrieval Sufficient?
May 11, 2026Tz-Huan Hsu, Jheng-Hong Yang, Jimmy LinTool-Augmented Language Model Agents Agentic Search
University of Waterloo · Stencilzeit
Reproducing Complex Set-Compositional Information Retrieval
May 5, 2026Vincent Degenhart, Dewi Timman, Arjen P. de Vries +2Query Complexity Lexical Retrieval
Radboud University · Nijmegen, NL
SmartSearch: How Ranking Beats Structure for Conversational Memory Retrieval
Mar 16, 2026Jesper Derehag, Carlos Calva, Timmy GhiurauConversational Memory Long-Term Conversational Memory
Midbrain.
Retrieval or Representation? Reassessing Benchmark Gaps in Multilingual and Visually Rich RAG
Mar 4, 2026Martin Asenov, Kenza Benkirane, Dan Goldwater +1RAG Evaluation Multimodal IR
Parexel AI Labs London, United Kingdom · Parexel AI Labs San Francisco, United-States
RankEvolve: Automating the Discovery of Retrieval Algorithms via LLM-Driven Evolution
Feb 18, 2026Jinming Nian, Fangchen Li, Dae Hoon Park +1Information Retrieval Automated Algorithm Discovery
Santa Clara University, Santa Clara, CA, USA · Independent Researcher, Bothell, WA, USA · Walmart Global Tech, Sunnyvale, CA, USA
How Do LLM-Generated Texts Impact Term-Based Retrieval Models?
Aug 25, 2025Wei Huang, Keping Bi, Yinqiong Cai +3Information Retrieval Lexical Retrieval
State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences, University of Chinese Academy of Sciences, Beijing, China · Baidu Inc., Beijing, China

Oct 22, 2026J/K move · Enter open · S save

What the Query Remembers: Query Abstraction Governs Lexical and Dense Retrieval of Classical Arabic Poetry

Omar Farouk Zouak, Selma Mani, Larbi Said Chikh, Raid Abderrezak Ouahioune

National School of Artificial Intelligence

Lexical Retrieval Dense Retrieval IR Evaluation Arabic NLP

Open paper AskPDF arXiv

How well a poetry archive can be searched depends on what the user still remembers, not only on what the model understands. We show this for Classical Arabic poetry, benchmarking nine dense encoders and BM25 over two corpora, three indexing modes and two evaluation protocols. Which retriever wins is governed by how much of a poem's wording the query still carries. On tip-of-the-tongue queries BM25 leads every dense encoder, including an 8B state-of-the-art model (0.746 vs. 0.717 Recall@5); on abstract queries it falls to eighth out of ten (0.392 vs. 0.748). Editing the wording of the same queries shows that this ranking largely reflects query composition: adding one remembered half-verse raises BM25 by 38 points, rewriting the quoted words by hand leaves BM25 level with the 8B encoder, and BM25's sensitivity to remembered wording is significantly steeper than that of the strongest dense encoder. The same axis controls explanation augmentation, whose effect swings from +37 to -39 points. The mechanism splits by system type: lexical containment predicts BM25's behaviour, while embedding proximity predicts dense encoders. Finally, the field's two standard protocols, known-item retrieval and thematic relevance, dissociate on the same systems, a result reproduced by two blind human annotators and by two open-weights judges, one of them under a rubric with no literal-match criterion. A benchmark that reports one protocol, or one query register, can give a materially wrong picture. Code and data are available at https://github.com/Ellzo/what-the-query-remembers.