Multimodal RAG

RAG: Retrieval-Augmented Generation

Multimodal RAG studies how to retrieve external evidence across modalities and use it to produce grounded answers, reports, or decisions. Systems combine dense or hybrid search with techniques such as OCR and layout-aware document indexing, image-region or video-event retrieval, and graph-based evidence linking; rerankers and agents can refine what is retrieved and how it is used. This supports knowledge-based visual question answering, long-document and video understanding, and specialized applications where evidence and citations matter. Benchmarks increasingly separate retrieval quality from answer quality, while a central challenge remains ensuring that models use relevant evidence faithfully and efficiently despite noisy, biased, or poisoned context.

Latest papers

158 papers13 papers in the last four weeks, down 7% on the four weeks before. 0.1% of all new papers.

RASR: Retrieval-Augmented Semantic Reasoning for Fake News Video Detection
Apr 8, 2026Hui Li, Peien Ding, Jun Li +5Multimodal RAG Fake News Detection
School of Informatics, Xiamen University, Xiamen, China · School of Computer Science and Information Security, Guilin University of Electronic Technology, Guilin, China · School of Computer and Big Data, Minjiang University, Fuzhou, China +2
WikiSeeker: Rethinking the Role of Vision-Language Models in Knowledge-Based Visual Question Answering
Apr 7, 2026Yingjian Zhu, Xinming Wang, Kun Ding +3Knowledge-Based VQA Multimodal RAG
School of Artificial Intelligence, University of Chinese Academy of Sciences · State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS), Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Science and Technology Beijing
VideoStir: Understanding Long Videos via Spatio-Temporally Structured and Intent-Aware RAG
Apr 7, 2026Honghao Fu, Miao Xu, Yiwei Wang +3Multimodal RAG Long-Video Understanding
University of Queensland · University of California, Merced · Institute of Automation, CAS +1
Region-R1: Reinforcing Query-Side Region Cropping for Multi-Modal Re-Ranking
Apr 7, 2026Chan-Wei Hu, Zhengzhong TuMultimodal Reranking Multimodal RAG
Texas A&M University
MG $^2$ -RAG: Multi-Granularity Graph for Multimodal Retrieval-Augmented Generation
Apr 4, 2026Sijun Dai, Qiang Huang, Xiaoxing You +1Cross-Modal Learning Multimodal RAG
School of Intelligence Science and Engineering, Harbin Institute of Technology (Shenzhen) · School of Computer Science, Hangzhou Dianzi University
Look Twice: Training-Free Evidence Highlighting for Knowledge-based Visual Question Answering
Apr 1, 2026Marco Morini, Sara Sarto, Marcella Cornia +2Knowledge-Based VQA VLM Inference
University of Modena and Reggio Emilia, Italy
Unify-Agent: A Unified Multimodal Agent for World-Grounded Image Synthesis
Mar 31, 2026Shuang Chen, Quanxin Shou, Hangting Chen +16Image Generation Agentic Image Generation
University of California, Los Angeles · Tencent Hunyuan · The Hong Kong University of Science and Technology +1
GUIDE: Resolving Domain Bias in GUI Agents through Real-Time Web Video Retrieval and Plug-and-Play Annotation
Mar 27, 2026Rui Xie, Zhi Gao, Chenrui Shi +3Multimodal RAG Domain Adaptation
Shanghai Jiao Tong University, Shanghai, China · State Key Laboratory for General Artificial Intelligence, BIGAI, Beijing, China · Beijing Institute of Technology, Beijing, China
ForeSea: AI Forensic Search with Multi-modal Queries for Video Surveillance
Mar 24, 2026Hyojin Park, Yi Li, Janghoon Cho +8Video Surveillance Multimodal IR
Qualcomm AI Research, San Diego, CA, USA · Kyunghee University, Gyeonggi, South Korea
Guideline-grounded retrieval-augmented generation for ophthalmic clinical decision support
Mar 23, 2026Shuying Chen, Sen Cui, Zhong CaoMultimodal RAG Retrieval-Augmented Generation
School of Artificial Intelligence and Data Science University of International Business and Economics Beijing, China · Tsinghua University Beijing, China · Heidelberg Institute of Global Health, Faculty of Medicine and University Hospital Heidelberg University Heidelberg, Germany
Grounded Multimodal Retrieval-Augmented Drafting of Radiology Impressions Using Case-Based Similarity Search
Mar 18, 2026Himadri S SamantaRadiology Report Generation Multimodal RAG
Independent AI Researcher, Austin, Texas, USA
V-DyKnow: A Dynamic Benchmark for Time-Sensitive Knowledge in Vision Language Models
Mar 17, 2026Seyed Mahed Mousavi, Christian Moiola, Massimo Rizzoli +2VLM Evaluation Multimodal RAG
Signals and Interactive Systems Lab, University of Trento, Italy
Beyond the Embedding Bottleneck: Adaptive Retrieval-Augmented 3D CT Report Generation
Mar 16, 2026Renjie Liang, Yiling Ma, Yang Xing +6Healthcare Multimodal RAG
University of Florida · Yale University · University of Southern California
Beyond Relevance: On the Relationship Between Retrieval and RAG Information Coverage
Mar 9, 2026Saron Samuel, Alexander Martin, Eugene Yang +5RAG Evaluation IR Evaluation
Johns Hopkins University Baltimore, MD, USA · Johns Hopkins HLTCOE Baltimore, MD, USA · National Institute of Standards and Technology Gaithersburg, MD, USA +1
Retrieval-Augmented Anatomical Guidance for Text-to-CT Generation
Mar 9, 2026Daniele Molino, Camillo Maria Caruso, Paolo Soda +1Medical Image Synthesis Computed Tomography
Unit of Artificial Intelligence and Computer Systems, Department of Engineering, Università Campus Bio-Medico di Roma, Rome, Italy · Department of Diagnostics and Intervention, Biomedical Engineering and Radiation Physics, Umeå University, Umeå, Sweden
Hit-RAG: Learning to Reason with Long Contexts via Preference Alignment
Mar 7, 2026Junming Liu, Yuqi Li, Shiping Wen +2Retrieval-Augmented Generation Multimodal RAG
Tongji University, Shanghai, China · The City University of New York, New York, USA · University of Technology Sydney, Sydney, Australia +2
Beyond Rows to Reasoning: Agentic Retrieval for Multimodal Spreadsheet Understanding and Editing
Mar 6, 2026Anmol Gulati, Sahil Sen, Waqar Sarguroh +1Agentic RAG Spreadsheet Automation
Commercial Technology and Innovation Office, PricewaterhouseCoopers, U.S.
AutothinkRAG: Complexity-Aware Control of Retrieval-Augmented Reasoning for Image-Text Interaction
Mar 5, 2026Jiashu Yang, Chi Zhang, Abudukelimu Wuerkaixi +5Multimodal RAG Multimodal QA
Dalian University of Technology · Meituan LongCat Interaction Team · Tsinghua University
RANGER: Sparsely-Gated Mixture-of-Experts with Adaptive Retrieval Re-ranking for Pathology Report Generation
Mar 4, 2026Yixin Chen, Ziyu Su, Hikmat Khan +1Medical Report Generation Multimodal RAG
Department of Biomedical Engineering, The Ohio State University, Columbus, OH, USA · Department of Pathology, The Ohio State University, Columbus, OH, USA
RADIANT-LLM: an Agentic Retrieval Augmented Generation Framework for Reliable Decision Support in Safety-Critical Nuclear Engineering
Mar 4, 2026Zavier Ndum Ndum, Jian Tao, John Ford +2Agentic RAG Multimodal RAG
Department of Nuclear Engineering, Texas A&M University, College Station, TX, USA · College of Performance, Visualization and Fine Arts, Texas A&M University, College Station, TX, USA
RAGTrack: Language-aware RGBT Tracking with Retrieval-Augmented Generation
Mar 4, 2026Hao Li, Yuhao Wang, Wenning Hao +3Visual Object Tracking Multimodal RAG
College of Command and Control Engineering, Army Engineering University of PLA · School of Future Technology, Dalian University of Technology · School of Information and Communication Engineering, Dalian University of Technology
NICO-RAG: Multimodal Hypergraph Retrieval-Augmented Generation for Understanding the Nicotine Public Health Crisis
Mar 2, 2026Manuel Serna-Aguilera, Raegan Anderes, Page Dobbs +1Hypergraph-Based RAG Multimodal RAG
University of Arkansas, Fayetteville, AR, 72701, USA · University of Arkansas for Medical Sciences, Little Rock, AR, 72205, USA
MC-Search: Evaluating and Enhancing Multimodal Agentic Search with Structured Long Reasoning Chains
Mar 1, 2026Xuying Ning, Dongqi Fu, Tianxin Wei +7Multimodal RAG Multimodal Search Agents
University of Illinois Urbana-Champaign · Meta · IBM Research +1
Multimodal Adaptive Retrieval Augmented Generation through Internal Representation Learning
Feb 28, 2026Ruoshuang Du, Xin Sun, Qiang Liu +4Multimodal RAG Adaptive RAG
Shanghaitech University School of Information Science and Technology · Chinese Academy of Sciences Institute of Automation · Ant Group
Zero-Shot Robotic Manipulation via 3D Gaussian Splatting-Enhanced Multimodal Retrieval-Augmented Generation
Feb 28, 2026Zilong Xie, Jingyu Gong, Xin Tan +2Zero-Shot Robotic Manipulation Generalization in Robotic Manipulation
School of Computer Science and Technology, East China Normal University, Shanghai, China · Chongqing Key Laboratory of Precision Optics, Chongqing Institute of East China Normal University, Chongqing, China · Shanghai Key Laboratory of Computer Software Evaluating and Testing, Shanghai, China
AgenticOCR: Parsing Only What You Need for Efficient Retrieval-Augmented Generation
Feb 27, 2026Zhengren Wang, Dongsheng Ma, Huaping Zhong +4Multimodal RAG Visual Document Retrieval
Shanghai Artificial Intelligence Laboratory · Peking University · SenseTime
Hidden in the Metadata: Stealth Poisoning Attacks on Multimodal Retrieval-Augmented Generation
Feb 26, 2026Kennedy Edemacu, Mohammad Mahdi ShokriRAG Poisoning Attacks Multimodal RAG
The City University of New York, CSI, Staten Island, NY 10314, USA · The City University of New York, Graduate Center, New York, NY 10016, USA
MammoWise: Multi-Model Local RAG Pipeline for Mammography Report Generation
Feb 25, 2026Raiyan Jahangir, Nafiz Imtiaz Khan, Amritanand Sudheerkumar +1Radiology Report Generation Mammography
University of California, Davis, CA, USA
RobustVisRAG: Causality-Aware Vision-Based Retrieval-Augmented Generation under Visual Degradations
Feb 25, 2026I-Hsiang Chen, Yu-Wei Liu, Tse-Yu Wu +3RAG Robustness Multimodal RAG
National Taiwan University · Microsoft
Unlocking Multimodal Document Intelligence: From Current Triumphs to Future Frontiers of Visual Document Retrieval
Feb 23, 2026Yibo Yan, Jiahao Huo, Guanbo Feng +12Multimodal RAG Multimodal IR
Hong Kong University of Science and Technology (Guangzhou) · Alibaba Cloud Computing · Hong Kong University of Science and Technology +1

Apr 8, 2026J/K move · Enter open · S save

RASR: Retrieval-Augmented Semantic Reasoning for Fake News Video Detection

Hui Li, Peien Ding, Jun Li, Guoqi Ma+4

School of Informatics, Xiamen University, Xiamen, China · School of Computer Science and Information Security, Guilin University of Electronic Technology, Guilin, China · School of Computer and Big Data, Minjiang University, Fuzhou, China+2

Multimodal RAG Fake News Detection Multimodal Reasoning Multimodal Fusion

Open paper AskPDF arXiv

Multimodal fake news video detection is a crucial research direction for maintaining the credibility of online information. Existing studies primarily verify content authenticity by constructing multimodal feature fusion representations or utilizing pre-trained language models to analyze video-text consistency. However, these methods still face the following limitations: (1) lacking cross-instance global semantic correlations, making it difficult to effectively utilize historical associative evidence to verify the current video; (2) semantic discrepancies across domains hinder the transfer of general knowledge, lacking the guidance of domain-specific expert knowledge. To this end, we propose a novel Retrieval-Augmented Semantic Reasoning (RASR) framework. First, a Cross-instance Semantic Parser and Retriever (CSPR) deconstructs the video into high-level semantic primitives and retrieves relevant associative evidence from a dynamic memory bank. Subsequently, a Domain-Guided Multimodal Reasoning (DGMP) module incorporates domain priors to drive an expert multimodal large language model in generating domain-aware, in-depth analysis reports. Finally, a Multi-View Feature Decoupling and Fusion (MVDFF) module integrates multi-dimensional features through an adaptive gating mechanism to achieve robust authenticity determination. Extensive experiments on the FakeSV and FakeTT datasets demonstrate that RASR significantly outperforms state-of-the-art baselines, achieves superior cross-domain generalization, and improves the overall detection accuracy by up to 0.93%.