Multimodal RAG

RAG: Retrieval-Augmented Generation

Follow this topic Ask about this topic

Momentum

10 papers in the last four weeks, against 2 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 100

All topics

MARQUIS: A Three-Stage Pipeline for Video Retrieval-Augmented Generation
May 17, 2026Debashish Chakraborty, Dengjia Zhang, Jialiang Jin +7Multimodal RAG Retrieval-Augmented Generation
Johns Hopkins University · Human Language Technology Center of Excellence
Navigating the Emotion Tree: Hierarchical Hyperbolic RAG for Multimodal Emotion Recognition
May 16, 2026Zeheng Wang, Bo Zhao, Yijie Zhu +6Multimodal RAG Hierarchical Classification
Great Bay University Guangdong, Dongguan, China · Tencent Youtu Lab Shanghai, China · CUHK, shenzhen Guangdong, Shenzhen, China
Text Knows What, Tables Know When: Clinical Timeline Reconstruction via Retrieval-Augmented Multimodal Alignment
May 14, 2026Sayantan Kumar, Shahriar Noroozizadeh, Juyong Kim +1Multimodal RAG Healthcare
Division of Intramural Research National Library of Medicine, National Institutes of Health Bethesda, MD, USA
From Scenes to Elements: Multi-Granularity Evidence Retrieval for Verifiable Multimodal RAG
May 14, 2026Guanhua Chen, Chuyue Huang, Yutong Yao +4Multimodal RAG Multimodal Grounding
NLP2CT Lab, Department of Computer and Information Science, University of Macau
Utility-Oriented Visual Evidence Selection for Multimodal Retrieval-Augmented Generation
May 13, 2026Weiqing Luo, Zongye Hu, Xiao Wang +3Multimodal RAG Evidence Selection
♦Arizona State University · ♥Texas A&M University · ♠Morgan Stanley
ViDR: Grounding Multimodal Deep Research Reports in Source Visual Evidence
May 13, 2026Zhuofan Shi, Peilun Jia, Baoqin Sun +4Multimodal RAG Multimodal Grounding
School of Software and Microelectronics, Peking University, Beijing 100871, China · 2National Key Laboratory of Data Space Technology and System, Beijing 100871, China · School of Software Engineering, Beijing Jiaotong University, Beijing, 100044, China
Synthesizing the Expert: A Validated Multimodal Dataset for Trustworthy AI-Assisted Swimming Coaching
May 12, 2026Ahmad Al-Kabbany, Esraa KassemMultimodal RAG Synthetic Benchmark Generation
Multimedia Interaction and Communication Lab · Wearables, Biosensing, and Biosignal Processing Research lab · Arab Academy for Science and Technology +2
Knowledge Poisoning Attacks on Medical Multi-Modal Retrieval-Augmented Generation
May 11, 2026Peiru Yang, Haoran Zheng, Tong Ju +6Multimodal RAG Adversarial Attacks on VLMs
1Tsinghua University · 2Beijing University of Posts and Telecommunications · 3Northwestern Polytechnical University +1
MicroWorld: Empowering Multimodal Large Language Models to Bridge the Microscopic Domain Gap with Multimodal Attribute Graph
May 11, 2026Manyu Li, Ruian He, Chenxi Ma +2Multimodal RAG Visual Question Answering
Shanghai Key Laboratory of Intelligent Information Processing School of Computer Science, Fudan University Shanghai, China
From Clouds to Hallucinations: Atmospheric Retrieval Hijacking in Remote Sensing Vision-Language RAG
May 8, 2026Jiaju Han, Chao Li, Chengyin Hu +8Multimodal RAG Retrieval-Augmented Generation
China University of Petroleum, Beijing at Karamay, Karamay, Xinjiang 834000, China
Retina-RAG: Retrieval-Augmented Vision-Language Modeling for Joint Retinal Diagnosis and Clinical Report Generation
May 7, 2026Abdelrahman Zaian, Sheethal Bhat, Mohamed Abdalkader +1Multimodal RAG Medical Report Generation
Friedrich-Alexander-Universität, Germany
The Cost of Context: Mitigating Textual Bias in Multimodal Retrieval-Augmented Generation
May 7, 2026Hoin Jung, Xiaoqian WangMultimodal RAG Multimodal Grounding
Elmore Family School of Electrical and Computer Engineering Purdue University West Lafayette, IN 47907
Enhancing Visual Question Answering with Multimodal LLMs via Chain-of-Question Guided Retrieval-Augmented Generation
May 5, 2026Quanxing Xu, Ling Zhou, Xian Zhong +3Multimodal RAG Visual Question Answering
School of Computer Science and Engineering, Macau University of Science and Technology, Macao SAR 999078, China · Hubei Key Laboratory of Transportation Internet of Things, School of Computer Science and Artificial Intelligence, Wuhan University of Technology, Wuhan, Hubei 430070, China · State Key Laboratory of Maritime Technology and Safety, Wuhan University of Technology, Wuhan 430063, China +3
From Experimental Limits to Physical Insight: A Retrieval-Augmented Multi-Agent Framework for Interpreting Searches Beyond the Standard Model
May 4, 2026Altan Cakir, Ayca YerlikayaMultimodal RAG Multi-Agent LLM Systems
Istanbul Technical University, Department of Data Science and Analytics, Istanbul, Turkey · Istanbul Technical University, Department of Physics Engineering, Istanbul, Turkey
FT-RAG: A Fine-grained Retrieval-Augmented Generation Framework for Complex Table Reasoning
May 2, 2026Zebin Guo, Weidong Geng, Ruichen MaoMultimodal RAG Table QA
Georgia Institute of Technology Atlanta, Georgia, USA · Zhejiang University Hangzhou, Zhejiang, China · Zhejiang Lab Hangzhou, Zhejiang, China
Iterative Multimodal Retrieval-Augmented Generation for Medical Question Answering
Apr 30, 2026Xupeng Chen, Binbin Shi, Chenqian Le +5Multimodal RAG Retrieval-Augmented Generation
New York University, New York, USA · Tsinghua University, Beijing, China · Independent Researcher +1
GeoSearch: Augmenting Worldwide Geolocalization with Web-Scale Reverse Image Search and Image Matching
Apr 28, 2026Tung-Duong Le-Duc, Hoang-Quoc Nguyen-Son, Minh-Son DaoMultimodal RAG Image Matching
University of Science, VNU-HCM Ho Chi Minh City, Vietnam · National Institute of Information and Communications Technology Tokyo, Japan
M $^3$ -VQA: A Benchmark for Multimodal, Multi-Entity, Multi-Hop Visual Question Answering
Apr 28, 2026Jiatong Ma, Longteng Guo, Yuchen Liu +4Multimodal RAG Visual Question Answering
Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences
Retrieval-Guided Generation for Safer Histopathology Image Captioning
Apr 27, 2026Md. Enamul Hoq, Wataru Uegami, Saghir Alfasly +6Multimodal RAG Computational Pathology
Kimia Lab, Department of Artificial Intelligence & Informatics, Mayo Clinic, Rochester, MN, USA · Lane Department of Computer Science and Electrical Engineering, West Virginia University, Morgantown, WV, USA · Department of Computer Science and Engineering, Princeton University, Princeton, NJ, USA +2
MEG-RAG: Quantifying Multi-modal Evidence Grounding for Evidence Selection in RAG
Apr 27, 2026Xihang Wang, Zihan Wang, Chengkai Huang +2Multimodal RAG Multimodal IR
Peking University Beijing, China · University of New South Wales, Macquarie University Sydney, Australia · University of New South Wales, CSIRO’s Data61 Sydney, Australia
Identity-Decoupled Anonymization for Visual Evidence in Multi-modal Retrieval-Augmented Generation
Apr 26, 2026Zehua Cheng, Wei Dai, Jiahao SunMultimodal RAG Privacy-Preserving ML
Department of Computer Science · University of Oxford,, UK · FLock.io, London, UK
KD-CVG: A Knowledge-Driven Approach for Creative Video Generation
Apr 23, 2026Linkai Liu, Wei Feng, Xi Zhao +9Multimodal RAG Text-to-Video Generation
† Sun Yat-sen University · ⋆ JD.COM
A-MAR: Agent-based Multimodal Art Retrieval for Fine-Grained Artwork Understanding
Apr 21, 2026Shuai Wang, Hongyi Zhu, Jia-Hong Huang +6Multimodal RAG Multimodal Grounding
University of Amsterdam · Part of this work was done during interning at Amazon. · Also with Amazon AGI. +2
AeroRAG: Structured Multimodal Retrieval-Augmented LLM for Fine-Grained Aerial Visual Reasoning
Apr 20, 2026Junxiao Xue, Quan Deng, Tingqi Hu +4Multimodal RAG Visual Question Answering
Research Center for Space Computing System, Zhejiang Lab, Hangzhou 311100, China · Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences, Hangzhou 311000, China · School of Cyber Science and Engineering, Zhengzhou University, Zhengzhou 450002, China +1
KIRA: Knowledge-Intensive Image Retrieval and Reasoning Architecture for Specialized Visual Domains
Apr 18, 2026Parthaw Goswami, Jaynto Goswami DeepMultimodal RAG Visual Reasoning
University of Missouri · SAP Prague
LFRAG: Layout-oriented Fine-grained Retrieval-Augmented Generation on Multimodal Document Understanding
Apr 18, 2026Yifan Zhu, Yu Mi, Yue Lu +2Multimodal RAG Late Interaction Retrieval
Zhejiang University · Hangzhou, China · Institute of Blockchain and Data Security +1
M3DocDep: Multi-modal, Multi-page, Multi-document Dependency Chunking with Large Vision-Language Models
Apr 17, 2026Joongmin Shin, Jeongbae Park, Jaehyung Seo +1Multimodal RAG Document Chunking
Human-inspired AI Research, Korea University · Computer Science and Engineering, Konkuk University · Department of Computer Science and Engineering, Korea University
UniDoc-RL: Coarse-to-Fine Visual RAG with Hierarchical Actions and Dense Rewards
Apr 16, 2026Jun Wang, Shuo Tan, Zelong Sun +5Document Understanding Multimodal RAG
Glint Lab
RASR: Retrieval-Augmented Semantic Reasoning for Fake News Video Detection
Apr 8, 2026Hui Li, Peien Ding, Jun Li +5Multimodal RAG Fake News Detection
School of Informatics, Xiamen University, Xiamen, China · School of Computer Science and Information Security, Guilin University of Electronic Technology, Guilin, China · School of Computer and Big Data, Minjiang University, Fuzhou, China +2
MG $^2$ -RAG: Multi-Granularity Graph for Multimodal Retrieval-Augmented Generation
Apr 4, 2026Sijun Dai, Qiang Huang, Xiaoxing You +1Cross-Modal Learning Multimodal RAG
School of Intelligence Science and Engineering, Harbin Institute of Technology (Shenzhen) · School of Computer Science, Hangzhou Dianzi University

May 17, 2026, cs.IRJ/K move · Enter open · S save

MARQUIS: A Three-Stage Pipeline for Video Retrieval-Augmented Generation

Debashish Chakraborty, Dengjia Zhang, Jialiang Jin, Hanting Liu+6

Johns Hopkins University · Human Language Technology Center of Excellence

Multimodal RAG Retrieval-Augmented Generation Information Retrieval RAG Evaluation Query Expansion Video Retrieval

Open paper AskPDF arXiv

Retrieval-augmented generation from videos requires systems to retrieve relevant audiovisual evidence from large corpora and synthesize it into coherent, attributed text. Current approaches struggle at both ends: retrieval methods fail on complex, multi-faceted queries that cannot be captured by a single embedding, while generation methods lack the high-level reasoning needed to synthesize across multiple videos and face memory constraints over long, multi-video contexts. We present MARQUIS: a three-stage pipeline that addresses these limitations through (1) query expansion, fusion, and reranking, (2) calibrated structured evidence extraction, and (3) article generation from extracted evidence, optionally controlled by an RLM. On the MAGMaR2026 shared task, we improve retrieval performance from 0.195 to 0.759 (nDCG@10). For article generation, ITER-QA-BASE improves average human score from 3.09 to 3.83 over the CAG baseline, while MARQUIS-RLM achieves a human score of 3.30 and the strongest citation recall among non-QA systems.

Momentum

Related topics

Latest papers 100