RAG Evaluation

RAG: Retrieval-Augmented Generation

Overview

RAG evaluation studies whether systems find relevant evidence, use it faithfully, and produce correct, appropriately cited answers—including when they should abstain. Common approaches combine retrieval metrics such as Recall and nDCG with answer-level checks for correctness, coverage, faithfulness, citation support, and uncertainty; component ablations and error analysis help locate failures. Benchmarks increasingly test multi-hop and multi-turn questions, long or multimodal documents, domain-specific tasks, and noisy, conflicting, or adversarial evidence, alongside latency and resource costs. A central open tension is that better retrieval scores do not reliably yield better answers, while automated judges and metrics may be costly, unstable, or fail to transfer across datasets and domains.

Latest papers

CardsList

368 papers24 papers in the last four weeks, down 35% on the four weeks before. 0.2% of all new papers.

  1. Investigating LLM Capabilities on Long Context Comprehension for Clinical Question Answering

    Oct 21, 2025Feras AlMannaa, Talia Tseriotou, Jenny Chim +1Clinical QAEHR QA

  2. SciRerankBench: Benchmarking Rerankers Towards Scientific Retrieval-Augmented Generation with LLMs

    Aug 12, 2025Haotian Chen, Qingqing Long, Meng Xiao +6LLM RerankingRAG Evaluation

  3. Chunk Twice, Embed Once: A Systematic Study of Segmentation and Representation Trade-offs in Chemistry-Aware Retrieval-Augmented Generation

    Jun 13, 2025Mahmoud Amiri, Thomas BocklitzDocument ChunkingText Embedding Evaluation

  4. ParamMute: Suppressing Knowledge-Critical FFNs for Faithful Retrieval-Augmented Generation

    Feb 21, 2025Pengcheng Huang, Zhenghao Liu, Yukun Yan +8Retrieval-Augmented GenerationKnowledge Conflicts in Language Models

  5. XRAG: eXamining the Core -- Benchmarking Foundational Components in Advanced Retrieval-Augmented Generation

    Dec 20, 2024Qili Zhang, Qianren Mao, Yangyifei Luo +15Retrieval-Augmented GenerationRAG Evaluation

  6. Abstention vs. Hallucination: Benchmarking LLM Source Attribution for Scientific Citations

    May 3, 2024Deepa Tilwani, Yash Saxena, Seyedali Mohammadi +5Citation VerificationRAG Evaluation

  7. HiQA: A Hierarchical Contextual Augmentation RAG for Multi-Documents QA

    Feb 1, 2024Xinyue Chen, Pengyu Gao, Jiangjiang Song +1Retrieval-Augmented GenerationMulti-Document QA