RAG Evaluation

RAG: Retrieval-Augmented Generation

Follow this topic Ask about this topic

Momentum

28 papers in the last four weeks, up 87% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 224

All topics

The Coverage Illusion: From Pre-retrieval Routing Failure to Post-retrieval Cascades in a Production RAG System
May 26, 2026Zafar Hussain, Kristoffer NielboLLM Inference Efficiency Retrieval-Augmented Generation
Aarhus University, Denmark
Detecting Is Not Resolving: The Monitoring Control Gap in Retrieval Augmented LLMs
May 26, 2026Zhe Yu, Wenpeng Xing, Chen Ye +4Language Model Safety Evaluation Knowledge Conflicts in Language Models
Binjiang Institute of Zhejiang University · Zhejiang University · Hangzhou Dianzi University +1
The Attribution Blind Spot: Detecting When Language Models Rely on Memory Rather Than Retrieved Context
May 26, 2026Zhe Yu, Wenpeng Xing, Yunzhao Wei +4Memorization in Language Models LLM Interpretability
Binjiang Institute of Zhejiang University · Zhejiang University · National Fintech Evaluation Center +2
FAB-Bench: A Framework for Adaptive RAG Benchmarking in Semiconductor Manufacturing
May 26, 2026Jingbin Qian, Congwen Yi, Min Xia +3Retrieval-Augmented Generation Benchmark Design
FutureFab.AI
AuthTrace: Diagnosing Evidence Construction in Thematically Dense Single-Author Corpora
May 25, 2026Xiaoqing Wu, Feifei Li, Haoliang Ming +1Evidence Selection RAG Evaluation
WeChat, Tencent Inc., Beijing, China
RAGe: A Retrieval-Augmented Generation Evaluation Framework
May 23, 2026Larissa Guder, João Pedro de Moura, Arthur Accorsi +5Retrieval-Augmented Generation RAG Evaluation
School of Technology, Pontifical Catholic University of Rio Grande do Sul, Av. Bento Gonçalves, Porto Alegre, Rio Grande do Sul, Brazil.
A Systematic Evaluation of Retrieval-Augmented Generation and Language Models for Space Operations
May 23, 2026Ruben Belo, Marta Guimarães, Cláudia SoaresRetrieval-Augmented Generation Information Retrieval
1NOVA LINCS, Costa da Caparica, Almada, Portugal · 2Neuraspace, Coimbra, Portugal · 3Technical University of Munich, Germany
Prominence-Stratified Failure Modes in Retrieval-Augmented Commercial Recommendation: A 37,000-Run Audit
May 22, 2026Will Jack, Noah Lehman, Keller Maloney +1Generative Engine Optimization Recommender Systems
Unusual
Evaluating Commercial AI Chatbots as News Intermediaries
May 21, 2026Mirac Suzgun, Emily Shen, Federico Bianchi +5LLM Evaluation Question Answering
1Stanford University · 2Independent Researcher · 3Together AI.
A Comparative Study of Language Models for Khmer Retrieval-Augmented Question Answering
May 21, 2026Sereiwathna Ros, Phannet Pov, Ratanaktepi Chhor +3Retrieval-Augmented Generation Multilingual QA
Department of Computer Science, Chungbuk National University, Cheongju, South Korea · Department of Big Data, Chungbuk National University, Cheongju, South Korea · General Department of Information and Communication Technology, Ministry of Post and Telecommunications, Phnom Penh, Cambodia +2
Claim-Selective Certification for High-Risk Medical Retrieval-Augmented Generation
May 21, 2026Shao KanClaim Verification Healthcare
Jinglue Technology Development (Nanjing) Co., Ltd.
When Cases Get Rare: A Retrieval Benchmark for Off-Guideline Clinical Question Answering
May 20, 2026Doeun Lee, Muge Zhang, Yi Yu +11LLM Grounding Healthcare
The Ohio State University · The Ohio State University Wexner Medical Center · University of Chicago Medical Center
Fine-grained Claim-level RAG Benchmark for Law
May 20, 2026Souvick Das, Sallam Abualhaija, Domenico BianculliLegal IR Legal QA
University of Luxembourg
MTR-Suite: A Framework for Evaluating and Synthesizing Conversational Retrieval Benchmarks
May 20, 2026Junhao Ruan, Abudukeyumu Abudula, Bei Li +8Conversational Search Synthetic Benchmark Generation
School of Computer Science and Engineering, Northeastern University, Shenyang 110819, China · 2Meituan Inc., 3NiuTrans Research, Shenyang, China · 4Tsinghua University, Beijing, China
Graph-Augmented Retrieval for Cross-Entity Financial Sentiment Analysis: A Comparative Study
May 19, 2026Rajan Bastakoti, Sagar Bhetwal, Nirajan Acharya +1Graph Retrieval-Augmented Generation Financial Sentiment Analysis
DePaul University · Chicago, IL, USA · Youngstwon State University +2
EngiAI: A Multi-Agent Framework and Benchmark Suite for LLM-Driven Engineering Design
May 19, 2026Gioele Molinari, Florian Felten, Soheyl Massoudi +1Multi-Agent LLM Systems LLM Agent Evaluation
IDEAL Chair of Artificial Intelligence in Engineering Design ETH Zurich Zurich, Switzerland · IDEAL, ETH Zurich, Zurich, Switzerland · Autom8.build, Brussels, Belgium
Vector RAG vs LLM-Compiled Wiki: A Preregistered Comparison on a Small Multi-Domain Research
May 18, 2026Theodore O. CochranLLM Inference Efficiency AI-Assisted Scientific Research
AI for Altruism (A4A)
MARQUIS: A Three-Stage Pipeline for Video Retrieval-Augmented Generation
May 17, 2026Debashish Chakraborty, Dengjia Zhang, Jialiang Jin +7Multimodal RAG Retrieval-Augmented Generation
Johns Hopkins University · Human Language Technology Center of Excellence
ConflictRAG: Detecting and Resolving Knowledge Conflicts in Retrieval Augmented Generation
May 17, 2026Chenyu Wang, Yueyuan Li, Yingmin Liu +1Retrieval-Augmented Generation Knowledge Conflicts in Language Models
School of Computer and Artificial Intelligence, Zhengzhou University, Zhengzhou, China · College of Environment and Ecology, Taiyuan University of Technology, Taiyuan, China · Zhejiang University, Hangzhou, China
Why Neighborhoods Matter: Traversal Context and Provenance in Agentic GraphRAG
May 14, 2026Riccardo Terrenzi, Maximilian von Zastrow, Serkan AyvazData Provenance Graph Retrieval-Augmented Generation
Centre for Industrial Software, University of Southern Denmark, Alsion 2, 6400 Sønderborg, Denmark
Deepchecks: Evaluating Retrieval-Augmented Generation (RAG)
May 14, 2026Assaf Gerner, Netta Madvil, Nadav Barak +11LLM Evaluation RAG Evaluation
Deepchecks, Ramat Gan, Israel · Ben-Gurion University, Beer Sheva, Israel
When Retrieval Hurts Code Completion: A Diagnostic Study of Stale Repository Context
May 14, 2026Haojun Weng, Qianqian Yang, Hao Fu +2Retrieval-Augmented Generation Retrieval Robustness
Independent Researcher, California, USA · Independent Researcher, Beijing, China
Does RAG Know When Retrieval Is Wrong? Diagnosing Context Compliance under Knowledge Conflict
May 14, 2026Yihang Chen, Pin Qian, Su Wang +4Retrieval-Augmented Generation Knowledge Conflicts in Language Models
1Georgia Institute of Technology · 2Carnegie Mellon University · University of California San Diego
When Evidence Conflicts: Uncertainty and Order Effects in Retrieval-Augmented Biomedical Question Answering
May 13, 2026Yikun Han, Mengfei Lan, Halil KilicogluSelective Prediction Healthcare
University of Illinois Urbana-Champaign
Overview of the MedHopQA track at BioCreative IX: track description, participation and evaluation of systems for multi-hop medical question answering
May 12, 2026Rezarta Islamaj, Joey Chan, Robert Leaman +13Multi-Hop QA Healthcare
National Library of Medicine (NLM), National Institutes of Health (NIH), MD, 20894, Bethesda, USA · University of Illinois at Urbana Champaign · Korea University +7
Caraman at SemEval-2026 Task 8: Three-Stage Multi-Turn Retrieval with Query Rewriting, Hybrid Search, and Cross-Encoder Reranking
May 12, 2026David-Maximilian Caraman, Gheorghe Cosmin SilaghiCross-Encoder Reranking Conversational Search
Babeş-Bolyai University Cluj-Napoca, Romania
Rethinking Evaluation for LLM Hallucination Detection: A Desiderata, A New RAG-based Benchmark, New Insights
May 11, 2026Wenbo Chen, Veena Padmanabhan, Tootiya Giyahchi +2Hallucination Detection Hallucination in Language Models
Amazon · Carnegie Mellon University
ClinicalBench: Stress-Testing Assertion-Aware Retrieval for Cross-Admission Clinical QA on MIMIC-IV
May 11, 2026Alex StinardHealthcare Clinical QA
Department of Clinical Sciences, College of Medicine · University of Central Florida, Orlando, FL 32816
ASTRA-QA: A Benchmark for Abstract Question Answering over Documents
May 11, 2026Shu Wang, Shansong Zhou, Xinyang Wang +3Question Answering Document QA
The Chinese University of Hong Kong, Shenzhen · Data Science Group, Huolala
Trust or Abstain? A Self-Aware RAG Approach
May 11, 2026Xi Zhu, Ziqi Wang, Kai Mei +5Retrieval-Augmented Generation Knowledge Conflicts in Language Models
Rutgers University · University of Illinois Urbana-Champaign

May 26, 2026, cs.CLJ/K move · Enter open · S save

The Coverage Illusion: From Pre-retrieval Routing Failure to Post-retrieval Cascades in a Production RAG System

Zafar Hussain, Kristoffer Nielbo

Aarhus University, Denmark

LLM Inference Efficiency Retrieval-Augmented Generation RAG Evaluation LLM Routing

Open paper AskPDF arXiv

In modern RAG pipelines, query augmentation methods such as HyDE and query expansion are applied to every query, resulting in substantial LLM inference costs and increased end-to-end latency. The empirical justification for this overhead in real production traffic remains largely unexplored. We present a case study of the Danish National Encyclopedia, evaluating five retrieval workflows over 20,000 query-workflow pairs from production traffic and synthetic conditions. In this system, synthetic queries suggest that LLM augmentation is needed for over 90% of queries to achieve high retrieval coverage. However, under our production deferral policy, only 27.8% of real user queries need LLM augmentation. We call this gap the Coverage Illusion and attribute it to a structural mismatch between synthetic and real query distributions. Pre-retrieval routing cannot resolve this gap, as the need for LLM augmentation is only revealed after searching the index, a result confirmed by our evaluation of four machine learning paradigms. The coverage gap, undetectable from the query alone, motivates a post-retrieval cascade that runs workflows in cheapest-first order and escalates to LLM augmentation only when a step returns no documents. Operating entirely without training overhead or secondary serving infrastructure, the cascade improves quality by +0.140 Composite Overall points over Always-HyDE, reduces latency by 31.8%, and serves 72.2% of real user queries without LLM augmentation.

Momentum

Related topics

Latest papers 224