Multimodal RAG

RAG: Retrieval-Augmented Generation

Multimodal RAG studies how to retrieve external evidence across modalities and use it to produce grounded answers, reports, or decisions. Systems combine dense or hybrid search with techniques such as OCR and layout-aware document indexing, image-region or video-event retrieval, and graph-based evidence linking; rerankers and agents can refine what is retrieved and how it is used. This supports knowledge-based visual question answering, long-document and video understanding, and specialized applications where evidence and citations matter. Benchmarks increasingly separate retrieval quality from answer quality, while a central challenge remains ensuring that models use relevant evidence faithfully and efficiently despite noisy, biased, or poisoned context.

Latest papers

158 papers13 papers in the last four weeks, down 7% on the four weeks before. 0.1% of all new papers.

RouteDRAG: Learning to Route and Control Agentic Execution for Multimodal Document RAG
Oct 27, 2026Zekai Chen, Hao Zhu, Yu Liu +5Agentic RAG Multimodal RAG
Beijing Institute of Technology, Beijing, China · Chinese Academy of Sciences, China · Zhejiang University, Hangzhou, China +3
Multimodal Requirements Traceability Link Recovery in Industrial Settings
Oct 25, 2026Haoming Huang, Shinpei Hayashi, Kazuya Kohata +2Requirements Engineering Multimodal RAG
Institute of Science Tokyo, Tokyo, Japan · YSL SOLUTION CO.,LTD., Kanagawa, Japan
GEMS-RAG: Graph-grounded Engineering Multimodal Standards Retrieval-Augmented Generation
Oct 23, 2026Abdul Hannan Azad, Irvin Wang, Jiwan Jiang +6Multimodal RAG Retrieval-Augmented Generation
Texas A&M University, College Station, USA · A&M Consolidated High School College Station, USA · University of Wisconsin, Madison, USA
GARRET: Generative Feedback for Region Retrieval in Visual Documents
Oct 20, 2026Weijian Jian, Siyang Chen, Yuguang Yang +4Visual Document Retrieval Multimodal RAG
AI Research · SAIS, UCAS · CASIA +1
Enhancing RAG-based Medical VQA with Gated Multimodal Representations
Oct 16, 2026Daniela Jimenez-Pano, Hugo Jair Escalante, Delia Irazú Hernández-FaríasGated Multimodal Fusion Medical VQA
LabTL - INAOE, Puebla, Mexico · Institute for AAII, UTEP, El Paso, TX, USA · Department of Computer Science, UTEP, El Paso, TX, USA
Loud and Clear: Making Visual Evidence Count in Multimodal Document RAG
Oct 15, 2026Eunchan Jo, Seounghyun Jang, Sangwoo MoMultimodal RAG Multimodal QA
POSTECH
Locate, Judge, Read: Role-Specialized Tool Agents over a Knowledge-Point Index for Open-Domain Multimodal Document QA
Oct 9, 2026Wenlin Lin, Fanzhe WeiDocument QA Agentic RAG
Metask Lab
Personalization Matters: Long-Horizon Conversation Agent with User-Centric Information in Online Shopping Interactions
Oct 8, 2026Rena Gao, Yue Dai, Hao Guan +4Agentic Commerce Recommender Systems
ByteDance · The University of Melbourne · University of Technology Sydney +1
TopoGraphRAG-Bench: Evaluating Multimodal GraphRAG on Layout-Grounded Evidence Reasoning
Oct 7, 2026Ruochi Li, Jianzhe Lin, Haoxuan Zhang +4Graph Retrieval-Augmented Generation Multimodal RAG
Department of Computer Science, North Carolina State University, USA · Independent Researcher · Department of Information Science, University of North Texas, USA +2
EC-RAG: Event Chain Retrieval-Augmented Generation for Long Video Understanding
Oct 6, 2026Yuhao Qin, Junbo Wang, Yuke Li +1Multimodal RAG Video-Language Models
Northwestern Polytechnical University Xi’an, China
CLIMB: Confidence-Guided Complementary Evidence for Multimodal Retrieval-Augmented Generation
Oct 2, 2026Hang Gao, Wujiang Xu, Zhixing Zhang +3Multimodal RAG Retrieval-Augmented Generation
Rutgers University · Google
Walking the Embedding Space: Datastore Extraction from Multimodal RAG
Oct 1, 2026Maria Carmen Jica, Ali Satvaty, Suzan Verberne +1Multimodal RAG Adversarial Attacks on VLMs
University of Groningen Groningen, The Netherlands
Faithful Chart Generation for Multimodal Deep Research: Frame-Evidence Co-Adaptation
Sep 30, 2026Yuxin Yue, Yingchen Zhang, Ruqing Zhang +3Multimodal RAG Data Visualization
State Key Laboratory of AI Safety Institute of Computing Technology, Chinese Academy of Sciences University of Chinese Academy of Sciences Beijing, China · University of Amsterdam Amsterdam, The Netherlands
TAEC: Trajectory-Aware Evidence Coordination for Multi-Step Visual RAG
Sep 29, 2026Yalun Wu, Bingzhou Wang, Boyang Wang +5Multimodal RAG Visual Question Answering
NExT++ Lab, National University of Singapore
UniK: Universal Knowledge Perception for Digital and Physical AI
Sep 21, 2026Nirmit Desai, Kunal Sawarkar, Aditya Mahakali +3Multimodal RAG Multimodal IR
AIntropy AI
PRISM-RAG: Multimodal Hypergraph Retrieval-Augmented Generation for Tobacco Product and Legislative Policy Reasoning
Sep 20, 2026Manuel Serna-Aguilera, Raegan Anderes, Page Dobbs +1Multimodal RAG Legal IR
Fay W. Boozman College of Public Health, University of Arkansas for Medical Sciences, Little Rock, Arkansas, 72205
Scientific Image Quality Assessment via Multi-modal Retrieval-Augmented Generation
Sep 17, 2026Yinuo Zhang, Bingshuo Liu, Zhiying Tu +6Multimodal RAG Image Quality Assessment
Harbin Institute of Technology · Tencent · Nanyang Technological University
Less Is More: Graph-free Multimodal RAG via Multi-signal Late Fusion
Sep 16, 2026Tithi Rakshit, Hongkuan Zhou, Lavdim Halilaj +1Late Fusion Multimodal RAG
University of Tübingen, Tübingen, Germany · Corporate Research, Robert Bosch GmbH, Renningen, Germany · University of Stuttgart, Stuttgart, Germany
Layers, Sinks, and Scaling: Adaptive Evidence Selection for Multimodal Large Language Models
Sep 15, 2026Zhenbin Wang, Lei Zhang, Lituan Wang +3Multimodal Large Language Models Multimodal QA
Sichuan University
Navigating Sparse Evidence: Agentic Visual RAG via Explicit Context Selection and Consolidation
Sep 14, 2026Yucheng Shen, Lingyong Yan, Jiulong Wu +4Multimodal RAG Agentic RAG
School of Computer Science and Technology, Soochow University · Baidu Inc.
GraMRAG: Orchestrating Multi-Agent Multi-Step Reasoning via Graph Memory with Reinforcement Learning
Sep 12, 2026Zhongyu WangMultimodal RAG Retrieval-Augmented Generation
Beihang University
Beyond the Query: Do Retrieval Signals Improve Adaptive Multimodal RAG Routing?
Sep 11, 2026Qiaomu Li, Qiuyuan Zhang, Nong MingMultimodal RAG Adaptive Model Routing
College of Computing and Software Engineering Kennesaw State University Marietta, GA 30060, USA
Bridging the Semantic-Utility Gap in Multimodal RAG via Generator-in-the-Loop Alignment
Sep 8, 2026Zhan-Lun Chang, Dong-Jun Han, Seyyedali Hosseinalipour +2Multimodal RAG Multimodal Reranking
Elmore Family School of Electrical and Computer Engineering, Purdue University, West Lafayette, IN 47907, USA · Department of Computer Science and Engineering, Yonsei University, Seoul 03722, Republic of Korea · Department of Electrical Engineering, University at Buffalo–SUNY, Buffalo, NY 14260, USA
Capability-Routed Visual Retrieval and Evidence Threading for Long-Context Document Question Answering
Sep 7, 2026Amirul Rahman, Aisha Karim, Kenji Nakamura +1Document QA Long-Document VQA
University of Malaya
WeAgent-MMGenEdit: A Full-Stack Recipe for Multimodal Agentic Image Generation and Editing
Sep 4, 2026Hui Zhang, Zongkai Liu, Liqiang Niu +6Agentic Image Generation Image Editing
Weixin AI, Tencent
EM^2Mem: Event-Centric Multimodal Memory for Large Language Models
Sep 1, 2026Yijun Chen, Yaqi Zheng, Yanya Li +11Multimodal RAG Efficient Multimodal Inference
Zhejiang University · South China University of Technology · Lenovo Group Limited
CrossModalQA: A Cross-modal and Multi-hop Benchmark for Multimodal Retrieval-augmented Generation
Aug 31, 2026Jiacheng Cai, Zijin Hong, Zheng Yuan +3Multimodal RAG Multimodal QA
The Hong Kong Polytechnic University · Jilin University
Doc-REFRAG: Rethinking Multimodal Document Retrieval-Augmented Generation
Aug 31, 2026Ruofan Hu, Shengyang Xu, Minjie Hong +5Multimodal RAG Efficient Multimodal Inference
Zhejiang University
HANIA: Planner-Guided Multimodal Graph Evidence Selection for Grounded Question Answering
Aug 29, 2026Zafar Ali, Asad Khan, Nimbeshaho Thierry +3Multimodal QA Multimodal Reasoning
School of Computer Science and Engineering, Southeast University, Nanjing, China · Portland Institute Nanjing, NJUPT, Nanjing, China · School of Information Science and Engineering, Shandong University, Qingdao, China +1
Beyond Visual Boundaries: Rethinking Scene Segmentation for Movie RAG
Aug 27, 2026Dong-Hee Kim, Seonwoo Choi, Changbeen Kim +6Long-Video Understanding Multimodal RAG
Korea University · KT · KAIST

Oct 27, 2026J/K move · Enter open · S save

RouteDRAG: Learning to Route and Control Agentic Execution for Multimodal Document RAG

Zekai Chen, Hao Zhu, Yu Liu, Xunkai Li+4

Beijing Institute of Technology, Beijing, China · Chinese Academy of Sciences, China · Zhejiang University, Hangzhou, China+3

Agentic RAG Multimodal RAG LLM Agent Workflow Optimization Multimodal QA Graph Retrieval-Augmented Generation

Open paper AskPDF arXiv

Web-based applications increasingly rely on understanding long multimodal documents, where answering questions often requires combining information across pages, tables, and figures. Agentic retrieval-augmented generation addresses these tasks through iterative retrieval and reasoning. However, Agentic execution is not always necessary, introducing additional computation and potentially incorrect answers. Our empirical analysis reveals that Agentic execution provides query-dependent benefits, while additional operations may improve, preserve, or degrade answer accuracy. These findings motivate controlling both Agent invocation and subsequent execution. To address these challenges, we propose RouteDRAG, a learning-based framework comprising four components. Unified Document Index organizes multimodal content into a graph-backed index and derives query representations from initial retrieval. Adaptive Path Routing preserves exact answering and predicts the relative benefit of Agent over shorter execution paths. Dynamic Agent Execution maintains an evolving workspace and refines Planner proposals to control tool use and termination. Sequential Policy Learning first trains operation control from observed outcomes and then trains path routing using the resulting Agent's performance. Across two benchmarks, RouteDRAG achieves an average accuracy improvement of 3.47 percentage points over shorter-path-only execution while assigning only 79.3% of queries to Agent. Notably, it also surpasses uniform Agent execution on both benchmarks while requiring fewer answering calls. Our code is now available at https://github.com/Ereshkigal1/RouteDRAG.