RAG Evaluation

RAG: Retrieval-Augmented Generation

Overview

RAG evaluation studies whether systems find relevant evidence, use it faithfully, and produce correct, appropriately cited answers—including when they should abstain. Common approaches combine retrieval metrics such as Recall and nDCG with answer-level checks for correctness, coverage, faithfulness, citation support, and uncertainty; component ablations and error analysis help locate failures. Benchmarks increasingly test multi-hop and multi-turn questions, long or multimodal documents, domain-specific tasks, and noisy, conflicting, or adversarial evidence, alongside latency and resource costs. A central open tension is that better retrieval scores do not reliably yield better answers, while automated judges and metrics may be costly, unstable, or fail to transfer across datasets and domains.

Latest papers

CardsList

368 papers24 papers in the last four weeks, down 35% on the four weeks before. 0.2% of all new papers.

  1. TUSI: A Citation-Grounded Graph-Based Benchmark

    Nov 16, 2026Marzieh Karimi, Mehran Safayani, Alireza BasiriRAG EvaluationLegal Language Model Evaluation

  2. Attribute or Abstain: Evaluating Evidence-Grounded Retrieval on the Low-Resource African Business Web

    Nov 15, 2026Bakary Gibba, Muhammad Abba Mandara, Karim Ismail Abd-Ellah MohamedRAG EvaluationEvidence-Grounded QA

  3. RMIT-ADM+S at TREC RAG 2026: Research Agents, Retriever Ablations, and Open-Weight Replication

    Nov 12, 2026Oleg Zendel, Kun Ran, Chenglong Ma +2Autonomous Research AgentsRAG Evaluation

  4. Two-Sided Conformal Evidence Sets for Retrieval-Augmented Generation

    Nov 7, 2026Ben Jenkins, Mihaela CardeiEvidence SelectionRAG Evaluation

  5. EviCast: Forecasting Retrieval Gains Through Shared Response Calibration

    Nov 2, 2026Yibo Wang, Jisheng Dang, Bimei Wang +9RAG EvaluationRetrieval-Augmented Generation

  6. From Logs to Faults: Reference-Free Fault Localization for Retrieval-Augmented Generation Pipelines

    Nov 1, 2026Hira Naveed, Scott Barnett, Stefanus Kurniawan +4Fault LocalizationRAG Evaluation

  7. RAG-TRACE: Tracing Measurement Evidence and Diagnostic Completeness in AI-Generated ECG Reports

    Nov 1, 2026Elyas Irankhah, Shahed Salehzehi, Parisa HosseiniMedical Report GenerationRAG Evaluation

  8. Reading Between the Chapters: A Graph-Free RAG Rebuilt With AI and Human Feedback

    Oct 31, 2026Dipto Sumit, Syeda Fabiayee Jahan, Farig SadequeRetrieval-Augmented GenerationQuestion Answering

  9. Prevented is Not Protected: What Attack Success Rate Hides about RAG

    Oct 29, 2026Chahana Dahal, Zuobin XiongRAG Poisoning AttacksRAG Security

  10. Adaptive Retrieval, Selective Answers: Evaluating Conformal Factuality in Agentic RAG

    Oct 26, 2026Muhammad Aimal Rehman, Chi-Kuang YehRAG EvaluationAgentic RAG

  11. Can System-1-Style Decision Models Replace Dedicated Rerankers in RAG? A Systems Evaluation

    Oct 26, 2026Qiaomu Li, Ying XieLLM RerankingRAG Evaluation

  12. Topic-Level Fairness in Retrieval-Augmented Generation

    Oct 26, 2026Jaeyoung Park, Hantian Zhang, Kexin Rong +1Group FairnessRetrieval-Augmented Generation

  13. Ranking Is Not Deciding: Benchmark Shortcuts and Operating-Point Collapse in Query-Time RAG Poisoning Defenses

    Oct 24, 2026Pushkal Kumar, Arvin SahaymRAG SecurityRAG Robustness

  14. How Ground Truth Shapes Retrieval Scores: Evidence-Aware Evaluation of a Retrieval-Augmented Generation System for Breast Lesions

    Oct 22, 2026Miguel Mochizuki Silva, Samuel Ismael, Mateus C. Dias +3RAG EvaluationBiomedical IR

  15. Chunk Recomputation under Real Document Revisions: Measuring Update Cost and Retrieval Quality of Fixed, Recursive, and Content-Defined Chunking

    Oct 22, 2026Yuning Bian, Lele Xu, Yifan Yuan +2Document ChunkingRAG Evaluation

  16. Retrieval Augmented Generation Compositionality

    Oct 21, 2026Ido Levy, Segev Shlomov, Leshem ChoshenRetrieval-Augmented GenerationRAG Evaluation

  17. Protection Is Not One Event: A Survey of RAG Protection Research by Measured Endpoint

    Oct 21, 2026Peiyang Liu, Xi Wang, Di Liang +3RAG SecurityRAG Evaluation

  18. Scan or Index? large Evidence Coverage, Answer Quality, and Cost in Long-Document RAG

    Oct 20, 2026Juan Terven, Diana-Margarita Córdova-Esparza, Julio-Alejandro Romero-González +5Long-Context RetrievalEvidence Retrieval

  19. Large-Scale Biomedical RAG: Corpus Scaling, Document Representation, and Efficiency Trade-offs

    Oct 19, 2026Adrián Sánchez-Freire, Gonzalo Pérez-Gómez, Aitziber Atutxa +1Retrieval-Augmented GenerationRAG Evaluation

  20. Benchmark for Structured Retrieval-Augmented Generation in Perovskite Photovoltaics

    Oct 18, 2026Qiwei Han, Chi Zhou, Gaoxiang YuanStructured RAGPerovskite Solar Cells

  21. Toward Agentic Retrieval-Augmented Generation for Enterprise Retail Workflows over Heterogeneous Data Sources

    Oct 18, 2026Jeonghwan Choi, Jibin Hwang, Hayoon Park +2Agentic RAGRAG Evaluation

  22. Characterising and Analysing Retrieval-Augmented Generation Techniques

    Oct 16, 2026Emmanuel Igbekele, Olawande DaramolaRetrieval-Augmented GenerationRAG Evaluation

  23. When Does Uncertainty Measure RAG Hallucination Risk?

    Oct 15, 2026Rajeel Ahmad Ansari, Tautik AgrahariHallucination in RAGLLM Hallucination Detection