RAG Evaluation

RAG: Retrieval-Augmented Generation

Follow this topic Ask about this topic

Momentum

28 papers in the last four weeks, up 87% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 224

All topics

RAG-Safety-Bench: Reliable Evaluation of Retrieval-Augmented LLM Safety
Sep 11, 2026Adithiyan Rajan Indira Saravanan, Kathleen C. FraserLLM Safety Benchmarks LLM Safety
Faculty of Engineering University of Ottawa Ottawa, Canada
Q2D-Web: A Large-Scale Benchmark for Retrieval in Agentic RAG Systems
Sep 8, 2026Maximilian Schall, Sedigheh Eslami, Markus Krimmel +4Agentic Retrieval Multilingual IR
Perplexity AI
Better Together: Complementary Query Rewriting Under a Strong RAG Baseline
Sep 4, 2026Sara Shanian, Xiaoqin Yi, Pavlo Ruban +1Retrieval-Augmented Generation Adaptive RAG
ServiceNow
Does the Selected Object Reach the Reader? Auditing Identity Handoffs in Grounded Language-Model Pipelines
Sep 4, 2026Siddharth Vohra, Runmin Jiang, Xiaomo Li +1Question Answering RAG Evaluation
Carnegie Mellon University, Pittsburgh, PA, USA · Amazon Web Services AI Native, Pittsburgh, PA, USA
Enhancing Financial Question Answering: A Novel Benchmark Dataset of Banks' financial statements
Sep 3, 2026Arianna Miola, Bruno Spaccavento, Lorenzo Silotto +2Financial QA RAG Evaluation
Intesa Sanpaolo Innovation Center, Corso Inghilterra 3, 10138, Torino, Italy. · Universit`a degli Studi di Milano-Bicocca, Milan, Italy · Politecnico di Torino, Corso Duca degli Abruzzi 24, 10129, Turin, Italy. +2
SpecMind: Enabling Spectrum Intelligence via Multi-Agent Hybrid Retrieval-Augmented Generation
Aug 31, 2026Songwei Dong, Bingyan Lu, Makayla Kienlen +2Retrieval-Augmented Generation Wireless Communications
Department of Electrical and Computer Engineering, University of Virginia · Department of Electrical Engineering, University of Notre Dame · Baskin School of Engineering, University of California, Santa Cruz
Faithfulness Is Not Free: Auditing Offline KV-Cache Quantization in Retrieval-Augmented Generation
Aug 31, 2026Atta Ul Asad, Ahsan Bilal, Muhammad Ali +2KV Caching RAG Evaluation
1Lahore University of Management Sciences (LUMS) · University of Oklahoma · 3Air University +1
Towards Expert Financial QA via Self-Improving RAG
Aug 27, 2026Junjie Xiong, Shawheen Ghezavat, Aum HirparaLLM Self-Correction Financial QA
University of California, Berkeley · California Polytechnic State University · Hofstra University
WARP: Wasserstein-Aligned RAG for Population Opinions
Aug 24, 2026Aman Singh Thakur, Aditya Agrawal, Alwarappan Nakkiran +1RAG Evaluation
Query Expansion Should Be Coordinated: Dense Expands, Sparse Anchors
Aug 16, 2026Chunran ZhangRetrieval-Augmented Generation RAG Evaluation
School of Computing and Artificial Intelligence Southwest Jiaotong University Chengdu, China
Who Speaks Matters: Authority-Aware Multi-View RAG over Italian Parliamentary Proceedings
Aug 13, 2026Mirko Tritella, Riccardo Pozzi, Matteo PalmonariRetrieval-Augmented Generation Political Discourse Analysis
University of Milano-Bicocca, Milan, Italy
When Should Multi-Round RAG Stop? Structured Stopping Judgments and Retrieval Reduction in Search-R1
Aug 13, 2026Weimeng LuoAdaptive RAG RAG Evaluation
Unaﬀiliated
A corpus-specific clinical RAG system matches or outperforms newer frontier LLMs on HealthBench
Aug 12, 2026Praveen Reddy, Charuta Mandke, Suvrankar Datta +4Retrieval-Augmented Generation LLM Grounding
LODESTAR: Trustworthy Entropy Is Navigated, Not Merely Measured -- Reinforced Polarizer Keeps a Frozen LLM from Being Confidently Misled by the Wrong Evidence
Aug 12, 2026Po-Jen Ko, Che-Cheng Wu, Hung-Chun Hsu +2Retrieval-Augmented Generation LLM Uncertainty Estimation
1Research Center for Information Technology Innovation, Academia Sinica, Taiwan
EnterpriseRAG: Benchmarking LLM Instruction Adherence and Robustness under Non-Ideal Enterprise Retrieval
Aug 12, 2026Huiqi Miao, Xinbao Sun, Bo Wang +6Retrieval-Augmented Generation Instruction Following
Jiutian Research, China Mobile, Beijing, China
Temporal Misgrounding in Legal RAG: A Versioned-Corpus Benchmark for French Tax Law
Aug 10, 2026Rose Cymbler, Daniel Guez, Laurent FabreTemporal IR Legal QA
Talia, Paris, France · Databricks
Guardian Crawler: Retrieval-First Knowledge Discovery with Bounded LLM Augmentation for Noisy Web Intelligence
Aug 10, 2026Joshua Castillo, Santosh Nukavarapu, Ravi MukkamalaHybrid IR Information Retrieval
Department of Computer Science Old Dominion University
What Would Fix This RAG Failure? Auditing Counterfactual Response with Paired Evidence Interventions
Aug 9, 2026Wenzhang DuRetrieval-Augmented Generation Counterfactual Reasoning in Language Models
Independent Researcher China
DS@GT ARC at Touché: Large Language Models for Retrieval-Augmented Debate
Aug 8, 2026Anthony Miyaguchi, Conor JohnstonLLM Evaluation LLM-as-a-Judge
Georgia Institute of Technology, North Ave NW, Atlanta, GA 30332, USA
Private Anytime Selective-Risk Certification for Federated Retrieval-Augmented Generation: Guarantees and Empirical Limits
Aug 8, 2026Sanjeda Akter, Ibne Farabi Shihab, Anuj SharmaAnytime-Valid Inference RAG Evaluation
Department of Computer Science, Iowa State University · Department of Civil, Construction & Environmental Engineering, Iowa State University
TA-RAG: Tone Awareness as a Design Imperative for Retrieval-Augmented Generation
Aug 7, 2026Yong-Bin Kang, Anthony McCoskerRetrieval-Augmented Generation RAG Evaluation
Department of Media and Communication, Swinburne University of Technology, Melbourne, Victoria, Australia.
HERALD: Counterfactual Audits and Minimal Repairs for Proof-of-Retrieval Rewards
Aug 6, 2026Zhuowen Liu, Bohan Cui, YinShang Guo +2Reward Hacking LLM Auditing
The Chinese University of Hong Kong · Fudan University · Nanjing University +2
Teaching Nemotron Greek: Mining a Corpus, Adapting Retrieval, and Grounding Generation for Modern Greek across Specialist Domains
Aug 5, 2026Ayoub Kirouane, Christos PetrocheilosRetrieval-Augmented Generation LLM Grounding
Sophea AI, KIEFER SA, Athens, Greece
SciRet: A Compute-Aware Empirical Study of Retrieval and Reranking for Scientific RAG
Aug 4, 2026Kaysarul Anas Apurba, Md. Hasibul Hasan, Rofiqul Alam Shehab +1Cross-Encoder Reranking Scientific QA
Laurentian University · North South University
Before Reasoning Can Fail: Pre-Evidence Procedural Failures in Agentic RAG
Aug 3, 2026Daeyoung Roh, Donghee HanRetrieval-Augmented Generation Agentic RAG
Independent Researcher · KAIST
When Retrieval Helps and Distracts: Evaluating Evidence-Generating LLMs for Biomedical Claim Verification
Aug 2, 2026Pritam Deka, Prabhjot SinghExplainable Artificial Intelligence Automated Fact-Checking
Queen’s University Belfast United Kingdom · University of Texas at Austin Austin, Texas, USA
A Triple-Robustness Analysis of Retrieval-Augmented Generation for Multi-Hop Requirements Traceability
Aug 1, 2026Meftun Akarsu, Burak Özdemir, Doğancan Büyükçolak +1LLM-as-a-Judge Graph Retrieval-Augmented Generation
Turkish Aerospace Industries · Technical University of Munich
Verification Without Sufficiency: Per-Chunk Filtering Fails on Multi-Hop RAG, and Decomposition Repairs It
Aug 1, 2026Randhir KumarMulti-Hop QA LLM Answer Verification
Independent Researcher
Towards Query-Agnostic RAG Evaluation via Query Coverage and Claim Verifiability
Jul 31, 2026Jeonghwan Choi, Taewon Yun, Minjeong Ban +3Reference-Free Evaluation RAG Evaluation
1Korea Advanced Institute of Science and Technology (KAIST) · 2Cluvion
Models for minimalist RAG: B1ade 335M Embedding and 1B Parameter Small Language Models
Jul 29, 2026Shreyas Subramanian, Mecit Gungor, Vikram ElangoRetrieval-Augmented Generation LLM Grounding
Amazon, Seattle, USA

Sep 11, 2026, cs.CLJ/K move · Enter open · S save

RAG-Safety-Bench: Reliable Evaluation of Retrieval-Augmented LLM Safety

Adithiyan Rajan Indira Saravanan, Kathleen C. Fraser

Faculty of Engineering University of Ottawa Ottawa, Canada

LLM Safety Benchmarks LLM Safety RAG Security RAG Evaluation

Open paper AskPDF arXiv

Allowing large language models (LLMs) to retrieve information from a set of trusted documents can increase reliability and reduce hallucination. However, recent work has demonstrated that retrieval-augmented generation (RAG) can have unintended side effects on the overall safety of the generated responses, when prompted for harmful or dangerous content. A clearer understanding of the mechanisms leading to this result is needed, as increasing numbers of end users turn to RAG to incorporate corporate documents and knowledge bases into LLM-based systems. We introduce RAG-Safety-Bench, a benchmark to measure the safety impact of RAG on LLM models. By removing the confounding effect of retriever quality, and cleanly separating the problem into four conditions -- non-RAG, RAG with an oracle document containing the answer to the harmful request, RAG with documents related to the harmful request but without the specific answer, and RAG with random, safe documents -- the benchmark isolates the impacts of different factors in the observed safety degradation. We report results across five open-source LLMs, showing an inverse relationship between benign and unsafe capability, strong evidence that baseline safety guardrails do not lead to downstream safety guarantees in the RAG case, and model-specific support for previous findings that even benign documents can lead to unsafe generation in retrieval-enabled systems.

Momentum

Related topics

Latest papers 224