RAG Evaluation

RAG: Retrieval-Augmented Generation

Follow this topic Ask about this topic

Momentum

28 papers in the last four weeks, up 87% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 224

All topics

RAG-Stress: Probing the Limits of Evidence Reliance in Retrieval-Augmented Generation
Oct 8, 2026Shunyuan Zhou, Hao Chen, Tianyu Wang +3Retrieval-Augmented Generation RAG Evaluation
Beijing University of Posts and Telecommunications · Humanlaya Data · Beijing Key Laboratory of Key Technologies for AI+ Domain Applications +1
TopoGraphRAG-Bench: Evaluating Multimodal GraphRAG on Layout-Grounded Evidence Reasoning
Oct 7, 2026Ruochi Li, Jianzhe Lin, Haoxuan Zhang +4Graph Retrieval-Augmented Generation Multimodal RAG
Department of Computer Science, North Carolina State University, USA · Independent Researcher · Department of Information Science, University of North Texas, USA +2
Agentic AutoRAG: RAG Pipeline Optimization through Reasoning-Driven Agents
Oct 6, 2026Lasse B. Strand, Robert Jakob, Kevin O'Sullivan +1Retrieval-Augmented Generation Large Language Model-Guided Optimization
ETH Zurich
Leveraging a four-quadrant approach for evaluating Redpine Science
Oct 6, 2026Filip Dorm, Leonora VesterbackaScientific QA RAG Evaluation
Redpine, Sweden.
MemStrata: 95% and 90.91% Source-Aware Accuracy on LongMemEval-500 and LoCoMo-1540 with a Local Qwen 3.8 27B Q4_K_M Reader
Oct 4, 2026Neeraj YadavLLM Evaluation Memory-Augmented Language Models
Called It Inc.
Mapping the RAG Landscape: A Four Axis Taxonomy of Efficiency, Defense, Interactivity, and Reasoning
Oct 1, 2026Meghana Sunil, Shravya V, Shravan Venkatraman +1Retrieval-Augmented Generation RAG Security
School of Computer Science and Engineering, Vellore Institute of Technology, Chennai, India. · Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE.
AGO AI Quality Gate: Evidence-First Release Decisions for Retrieval-Augmented Generation
Oct 1, 2026Giulio Zeloni, Enrico Lo Conte, Salvatore Rionero +3LLM-as-a-Judge RAG Evaluation
Protom Group S.p.A., Napoli, Italy
LLM-as-a-Judge for Low-Resource Languages: Adapting Ragas and Comparative Ranking for Romanian
Sep 30, 2026Claudiu Creanga, Liviu P. DinuLLM Evaluation LLM-as-a-Judge
Interdisciplinary School of Doctoral Studies, University of Bucharest, Romania · HLT Research Center, University of Bucharest, Romania · Faculty of Mathematics and Computer Science, University of Bucharest, Romania
RAGScope: A Leakage-Controlled, Cost-Aware Evidence-Gating Protocol for RAG Hallucination Triage
Sep 30, 2026Zeming Liu, Qibai Chen, Jingtao Zhang +1Selective Prediction Hallucination Detection
Brown University, United States · Independent Researcher, United States · Georgia Institute of Technology, United States
Re-ranking and Late Interaction Drive Retrieval Quality: A Controlled Comparison of RAG Strategies for Scientific Question Answering
Sep 29, 2026Bhagyesh Rathi, Eshan Chawla, William B. AndreopoulosLate Interaction Retrieval Scientific QA
San Jose State University, Department of Computer Science
BITEM at the NTCIR-19 R2C2 Task: Predicting Confidence from Agentic RAG Pipeline Signals
Sep 29, 2026Julien Knafou, Luc Mottin, Alexandre Flament +3Retrieval-Augmented Generation Agentic RAG
HES-SO, Geneva, Switzerland · SIB Swiss Institute of Bioinformatics, Geneva, Switzerland
Retrieve, Reproduce, Reveal: Dissecting Retrieval-Augmented Software Vulnerability Detection
Sep 29, 2026Sabrina Kaniewski, Tim Krämer, Julius Bächle +3Software Vulnerability Detection Software Engineering Benchmarks
Institute for Secure Networked Systems, Esslingen University, Germany · Esslingen University, Germany · Institute for Intelligent Systems, Esslingen University, Germany +1
Lost in Conversation or Lost in Translation? Diagnosing Multi-Turn Degradation in RAG
Sep 29, 2026Pranav Handa, Ariful AzadRetrieval-Augmented Generation LLM Reliability
Texas A&M University
ARCagent: An Adaptive Retrieval Calibration Agent for Clinical Question Answering
Sep 28, 2026Yuyan ChenClinical QA Agentic RAG
Page-Aware Retrieval-Augmented Generation for EvalLLM 2026: A Five-Variant Study on French PDFs
Sep 28, 2026Abdelhak keliousRetrieval-Augmented Generation Question Decomposition
Capsens, France
RAGWarrant: Evidence-Preserving Governance for RAG Policy Promotion Under Quality, Cost, Latency, and Risk Constraints
Sep 28, 2026Richard Krueger, Lucas Krause, Zach PocquetteRetrieval-Augmented Generation RAG Evaluation
Independent open-source research project
Return or Revise? Learning When Revision Helps Retrieval-Augmented QA
Sep 24, 2026Nicholas Kashani Motlagh, Tim Anderson, Jeremy Gwinnup +1Retrieval-Augmented Generation LLM Answer Verification
DCS Corp · Air Force Research Laboratory
Automated Regulatory Compliance Question Answering in Financial Services with Domain-Adapted Retrieval-Augmented Generation
Sep 24, 2026Tobias Deußer, Abhishek Pillai, Aurelio F. Bariviera +5Retrieval-Augmented Generation Legal QA
University of Bonn, Bonn, Germany · Lamarr-Institute for Machine Learning and Artificial Intelligence, Bonn, Germany · Universitat Rovira i Virgili, Reus, Spain +1
CRISS: A Retrieval-Augmented AI Chatbot for Assisting Cancer Registrars
Sep 24, 2026Vani Seth, Mohammad Beheshti, Anirudh Kambhampati +4Retrieval-Augmented Generation Human-in-the-Loop AI
Department of Electrical Engineering and Computer Science, University of Missouri, Columbia, USA · Missouri Cancer Registry and Research Center, Department of Public Health, College of Health Sciences, University of Missouri, Columbia, USA · MU Institute for Data Science and Informatics, University of Missouri, Columbia, USA
Evaluating Open-Weight LLMs for Turkish Domain Documents Under Retrieval and Hardware Constraints
Sep 23, 2026Imtiaz Ul Hassan, Öykü Akbulut, Onur Kaya +4Multilingual Language Model Evaluation LLM Evaluation
Edge Hill University, United Kingdom · Ermetal Otomotiv ve E¸sya San. Tic. A.¸S., Türkiye
Re:CAP - Auditing Retrieval Coverage in Production RAG Pipelines
Sep 21, 2026Aviral Joshi, Hanoz Bhathena, Max Nelson +1RAG Evaluation IR Evaluation
Machine Learning Center of Excellence, JPMorgan Chase & Co.
RAFT: A Stateful Retrieval-Augmented Framework for Troubleshooting Agents
Sep 17, 2026Mingxuan Zhang, Xiaowen Wang, Anupma Sharan +4Retrieval-Augmented Generation State Tracking
Microsoft, Redmond, WA, USA
Version- and Scope-Aware Question Answering over Normative Documents: A Deployed System and an End-to-End Evaluation at Production Scale
Sep 16, 2026Liuyin Wang, Shuaipeng Jin, Jiwei Shi +1Document QA RAG Evaluation
Beijing Caizhi Technology Co., Ltd., Beijing, China · dknownAI, Beijing, China
When Is Graph Structure Worth Its Cost? The Case for Structure Pricing in Retrieval-Augmented Generation
Sep 16, 2026Yuzhong Zhang, Haoyang Ma, Chao Peng +3Retrieval-Augmented Generation Graph Retrieval-Augmented Generation
The Chinese University of Hong Kong, Shenzhen · The Hong Kong University of Science and Technology · University of Edinburgh +1
AquiLLM: Evaluating Faithfulness in Open-Weight RAG-LLM Systems for Scientific Research
Sep 15, 2026Bernie Boscoe, Srinath Saikrishnan, Vikram Seenivasan +6AI-Assisted Scientific Research Open-Weight Language Models
Dept. of Computer Science, Southern Oregon University, USA · Dept. of Computer Science, University of California, Los Angeles, USA · Dept. of Physics & Astronomy, University of California, Los Angeles, USA +1
CiteGuard-RAG: A Validation-Centered AI System for Evidence-Grounded Question Answering
Sep 14, 2026Sumit Barua, Guan Hong, Halil Dursunoglu +2Retrieval-Augmented Generation Citation Verification
Department of Computer Science, Western Michigan University, Kalamazoo, MI, 49008, USA.
Empirical Evaluation of Open-Source Large Language Models for Retrieval-Augmented Generation in ESG Domain
Sep 14, 2026Motaz Saad, Anna Borrelli, Ivan Gentile +3LLM Evaluation LLM Grounding
University of Salento · University of Naples Federico II · IFAB Foundation
WinSyn: An Automated Pipeline for Realistic Enterprise Question-Answering Evaluation
Sep 14, 2026Amey Varhade, Ananya Sutradhar, Ravishankar Krishnaswamy +1Synthetic Data Generation AI Agent Evaluation
Microsoft Research India
EAR: Entity-Aware Partitioning Approach for Retrieval-Augmented Generation Development
Sep 14, 2026Cenab Batu Bora, Oylum Alatl\i, Sebnem Bora +1Document Chunking Retrieval-Augmented Generation
Georgia Institute of Technology, Atlanta, GA, USA · Ege University, Izmir, Turkey
Debiasing as a Measurement Intervention: Calibrated Ties and Resolution Loss in LLM-as-a-Judge Evaluation
Sep 14, 2026Liang Zhao, Yong Wang, Jiangzhe ChenLLM-as-a-Judge LLM Agent Evaluation
Shanghai University of International Business and Economics

Oct 8, 2026, cs.CLJ/K move · Enter open · S save

RAG-Stress: Probing the Limits of Evidence Reliance in Retrieval-Augmented Generation

Shunyuan Zhou, Hao Chen, Tianyu Wang, Goose Lin+2

Beijing University of Posts and Telecommunications · Humanlaya Data · Beijing Key Laboratory of Key Technologies for AI+ Domain Applications+1

Retrieval-Augmented Generation RAG Evaluation RAG Robustness Knowledge Conflicts in Language Models

Open paper AskPDF arXiv

Following retrieved evidence does not guarantee factual correctness: misleading evidence can induce a model to replace an answer it previously gave correctly. Standard accuracy measures obscure this behavior by combining answer replacement with preexisting errors. We introduce RAG-Stress, a controlled diagnostic protocol for examining the limits of evidence reliance in retrieval-augmented generation. The protocol holds the question and reference answer fixed, edits one assertion to support a designated incorrect answer, and crosses two source priority policies with three positions of the answer span within the evidence text. We measure misleading rate (MR) on each model's subset of questions answered correctly without retrieval, alongside clean accuracy on the full evaluation set. We evaluate fifteen systems spanning API models, open models, and search agents trained with reinforcement learning on TriviaQA-RC, HotpotQA, and SearchQA, with additional English and Chinese MedQA evaluations. Instructions that prioritize documents consistently produce higher MR than those permitting reliance on prior knowledge. Averaged over models and positions, the gap ranges from 10.9 to 13.5 percentage points across the three QA datasets. Mean MR follows End

>

Beginning

>

Middle under both policies, although individual models do not uniformly follow this ordering. A separate paired audit of 500 questions and two checkpoints supports increased harmful override without establishing a corresponding improvement in beneficial correction. These findings distinguish evidence adherence from factual reliability and motivate evaluating whether retrieved evidence preserves, replaces, or corrects a model's answers.

Momentum

Related topics

Latest papers 224