RAG Evaluation

RAG: Retrieval-Augmented Generation

Ask about this topic

Overview

RAG evaluation studies whether systems find relevant evidence, use it faithfully, and produce correct, appropriately cited answers—including when they should abstain. Common approaches combine retrieval metrics such as Recall and nDCG with answer-level checks for correctness, coverage, faithfulness, citation support, and uncertainty; component ablations and error analysis help locate failures. Benchmarks increasingly test multi-hop and multi-turn questions, long or multimodal documents, domain-specific tasks, and noisy, conflicting, or adversarial evidence, alongside latency and resource costs. A central open tension is that better retrieval scores do not reliably yield better answers, while automated judges and metrics may be costly, unstable, or fail to transfer across datasets and domains.

Latest papers

CardsList

355 papers39 papers in the last four weeks, up 3% on the four weeks before. 0.2% of all new papers.

  1. Beyond Similarity Search: Knowledge Graph Metadata Enrichment for Enterprise Retrieval-Augmented Generation

    Oct 27, 2026Fakabbir AminRetrieval-Augmented GenerationGraph Retrieval-Augmented Generation

  2. Adaptive Retrieval, Selective Answers: Evaluating Conformal Factuality in Agentic RAG

    Oct 26, 2026Muhammad Aimal Rehman, Chi-Kuang YehRAG EvaluationAgentic RAG

    Department of Mathematics & Statistics, Georgia State University, Atlanta, GA, USA

  3. Can System-1-Style Decision Models Replace Dedicated Rerankers in RAG? A Systems Evaluation

    Oct 26, 2026Qiaomu Li, Ying XieLLM RerankingRAG Evaluation

    College of Computing and Software Engineering, Kennesaw State University, Marietta, GA, USA

  4. Topic-Level Fairness in Retrieval-Augmented Generation

    Oct 26, 2026Jaeyoung Park, Hantian Zhang, Kexin Rong +1Group FairnessRetrieval-Augmented Generation

    KAIST · Meta · Georgia Institute of Technology

  5. Ranking Is Not Deciding: Benchmark Shortcuts and Operating-Point Collapse in Query-Time RAG Poisoning Defenses

    Oct 24, 2026Pushkal Kumar, Arvin SahaymRAG SecurityRAG Robustness

    North Creek High School · Washington State University

  6. How Ground Truth Shapes Retrieval Scores: Evidence-Aware Evaluation of a Retrieval-Augmented Generation System for Breast Lesions

    Oct 22, 2026Miguel Mochizuki Silva, Samuel Ismael, Mateus C. Dias +3RAG EvaluationBiomedical IR

    Universidade Federal da Paraíba (UFPB), João Pessoa, PB, Brazil · Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA

  7. Chunk Recomputation under Real Document Revisions: Measuring Update Cost and Retrieval Quality of Fixed, Recursive, and Content-Defined Chunking

    Oct 22, 2026Yuning Bian, Lele Xu, Yifan Yuan +2Document ChunkingRAG Evaluation

    School of Computer Science and Technology, Anhui University of Technology, Ma’anshan, China

  8. Retrieval Augmented Generation Compositionality

    Oct 21, 2026Ido Levy, Segev Shlomov, Leshem ChoshenRetrieval-Augmented GenerationRAG Evaluation

    IBM · Weizmann Institute of Science

  9. Protection Is Not One Event: A Survey of RAG Protection Research by Measured Endpoint

    Oct 21, 2026Peiyang Liu, Xi Wang, Di Liang +3RAG SecurityRAG Evaluation

    National Engineering Research Center for Software Engineering, Peking University, Beijing, China · School of Software and Microelectronics, Peking University, Beijing, China · Tencent Technology, Shenzhen, China +1

  10. Scan or Index? large Evidence Coverage, Answer Quality, and Cost in Long-Document RAG

    Oct 20, 2026Juan Terven, Diana-Margarita Córdova-Esparza, Julio-Alejandro Romero-González +5Long-Context RetrievalEvidence Retrieval

    CICATA Qro., Instituto Politécnico Nacional, México · Facultad de Informática, Universidad Autónoma de Querétaro, México · CIDETEC, Instituto Politécnico Nacional, México +3

  11. DeclineCraft: A RAG Abstention Benchmark for Jointly Measuring Combined Retrieval and Parametric Knowledge

    Oct 20, 2026Atharva Thakur, Abulhair Saparov, Bruno RibeiroRAG EvaluationLLM Abstention

    Purdue University

  12. RAG-NIMBF: A Non-Invasive, Modular Benchmarking Framework for Automated RAG Configuration Analysis

    Oct 20, 2026Niclas Hart, Nicolas WeegerRAG EvaluationRetrieval-Augmented Generation

    Ansbach University of Applied Sciences

  13. Corpus Quality Improves, Retrieval Utility Does Not: An Ablation of Preprocessing and Refinement in a Speech-to-QA Pipeline

    Oct 20, 2026Wonsup Shin, Jingu KimTraining Data CurationSpeech Corpus Construction

    Flitto

  14. Large-Scale Biomedical RAG: Corpus Scaling, Document Representation, and Efficiency Trade-offs

    Oct 19, 2026Adrián Sánchez-Freire, Gonzalo Pérez-Gómez, Aitziber Atutxa +1Retrieval-Augmented GenerationRAG Evaluation

    HiTZ Basque Center for Language Technology, University of the Basque Country (EHU), P.º Rafael Moreno "Pitxitxi", 2, Bilbao, 48013, Biscay, Spain

  15. Benchmark for Structured Retrieval-Augmented Generation in Perovskite Photovoltaics

    Oct 18, 2026Qiwei Han, Chi Zhou, Gaoxiang YuanStructured RAGRAG Evaluation

    Department of Chemistry, Duke University, Durham, NC 27708, USA · Department of Computer Science, Georgia Institute of Technology, Atlanta, GA 30332, USA · Department of Electrical and Computer Engineering, Tandon School of Engineering, New York University, Brooklyn, NY 11201, USA

  16. The Bottleneck Is Retrieval, Not Reading: Diagnosing Evidence-Based Generation with RAG for Long Context Question Answering

    Oct 18, 2026Debopriyo Banerjee, Owais Aijaz, Ahmer Tabassum +3RAG EvaluationEvidence-Grounded Generation

    Inception42 · MBZUAI

  17. Toward Agentic Retrieval-Augmented Generation for Enterprise Retail Workflows over Heterogeneous Data Sources

    Oct 18, 2026Jeonghwan Choi, Jibin Hwang, Hayoon Park +2Agentic RAGRAG Evaluation

    Korea Advanced Institute of Science and Technology (KAIST) · GS Retail · Cluvion

  18. Characterising and Analysing Retrieval-Augmented Generation Techniques

    Oct 16, 2026Emmanuel Igbekele, Olawande DaramolaRetrieval-Augmented GenerationRAG Evaluation

    Department of Informatics, University of Pretoria, Pretoria 0083, South Africa · Department of Computer Science, Landmark University, Kwara State, Nigeria

  19. When Does Uncertainty Measure RAG Hallucination Risk?

    Oct 15, 2026Rajeel Ahmad Ansari, Tautik AgrahariHallucination in RAGLLM Hallucination Detection

    Independent Researcher, Delhi, India · Independent Researcher, Bengaluru, India

  20. Beyond Retrieval Failure: Understanding Why LLMs Miss Evidence in Multi-Hop RAG

    Oct 15, 2026Jiaqi Wei, Wenxuan Huang, Chengdong Xu +2Retrieval-Augmented GenerationEvidence Retrieval

    Zhejiang University Hangzhou, China · Shanghai AI Laboratory Shanghai, China · Fudan University Shanghai, China +3

  21. Analyzing performance of Retrieval Augmented Generation models for Question Answering Tasks

    Oct 14, 2026Fazle Mohammed Tasfiq, Sabab Attin, Sheikh Md. Tahmid Hossain +6RAG EvaluationMultilingual QA

    Dept. of Computer Science and Engineering BRAC University Dhaka, Bangladesh

  22. Improving the Evaluation of Biomedical Question Answering with Few Human Preference Signals

    Oct 12, 2026Shakila Mahjabin Tonni, Xiang Dai, Sarvnaz Karimi +3RAG EvaluationAutomated Evaluation

    Suncorp Bank, Australia · CSIRO, Australia · Monash University, Australia +1

  23. GeopoliticalQA: Benchmarking Retrieval and Generation over Intelligence Reports

    Oct 8, 2026Veronika Solopova, Ivan Havlytskyi, Viktoria Skorik +1Question AnsweringDocument QA

  24. RAG-Stress: Probing the Limits of Evidence Reliance in Retrieval-Augmented Generation

    Oct 8, 2026Shunyuan Zhou, Hao Chen, Tianyu Wang +3Retrieval-Augmented GenerationRAG Evaluation

    Beijing University of Posts and Telecommunications · Humanlaya Data · Beijing Key Laboratory of Key Technologies for AI+ Domain Applications +1

  25. TopoGraphRAG-Bench: Evaluating Multimodal GraphRAG on Layout-Grounded Evidence Reasoning

    Oct 7, 2026Ruochi Li, Jianzhe Lin, Haoxuan Zhang +4Graph Retrieval-Augmented GenerationMultimodal RAG

    Department of Computer Science, North Carolina State University, USA · Independent Researcher · Department of Information Science, University of North Texas, USA +2

  26. Agentic AutoRAG: RAG Pipeline Optimization through Reasoning-Driven Agents

    Oct 6, 2026Lasse B. Strand, Robert Jakob, Kevin O'Sullivan +1Retrieval-Augmented GenerationLarge Language Model-Guided Optimization

    ETH Zurich

  27. Leveraging a four-quadrant approach for evaluating Redpine Science

    Oct 6, 2026Filip Dorm, Leonora VesterbackaScientific QARAG Evaluation

    Redpine, Sweden.

  28. MemStrata: 95% and 90.91% Source-Aware Accuracy on LongMemEval-500 and LoCoMo-1540 with a Local Qwen 3.8 27B Q4_K_M Reader

    Oct 4, 2026Neeraj YadavLLM EvaluationMemory-Augmented Language Models

    Called It Inc.

  29. RAGStress: A controlled benchmark for evaluating retrieval-augmented generation under knowledge-base degradation

    Oct 3, 2026Shiqi Yang, Jiekai Ma, Gaoyuan DuRetrieval-Augmented GenerationRAG Evaluation

    Amazon

  30. Grounding Probes: Generator-Independent Hallucination Detection from Observer Model Hidden States

    Oct 3, 2026Michael Rathmayr, Ádám Kovács, Gábor RecskiHallucination in RAGLLM Grounding

    TU Wien, Austria · KR Labs