Multimodal RAG

RAG: Retrieval-Augmented Generation

Follow this topic Ask about this topic

Momentum

10 papers in the last four weeks, against 2 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 100

All topics

GUIDE: Resolving Domain Bias in GUI Agents through Real-Time Web Video Retrieval and Plug-and-Play Annotation
Mar 27, 2026Rui Xie, Zhi Gao, Chenrui Shi +3Multimodal RAG Domain Adaptation
Shanghai Jiao Tong University, Shanghai, China · State Key Laboratory for General Artificial Intelligence, BIGAI, Beijing, China · Beijing Institute of Technology, Beijing, China
ForeSea: AI Forensic Search with Multi-modal Queries for Video Surveillance
Mar 24, 2026Hyojin Park, Yi Li, Janghoon Cho +8Multimodal RAG Multimodal IR
Qualcomm AI Research, San Diego, CA, USA · Kyunghee University, Gyeonggi, South Korea
Guideline-grounded retrieval-augmented generation for ophthalmic clinical decision support
Mar 23, 2026Shuying Chen, Sen Cui, Zhong CaoMultimodal RAG Retrieval-Augmented Generation
School of Artificial Intelligence and Data Science University of International Business and Economics Beijing, China · Tsinghua University Beijing, China · Heidelberg Institute of Global Health, Faculty of Medicine and University Hospital Heidelberg University Heidelberg, Germany
Beyond the Embedding Bottleneck: Adaptive Retrieval-Augmented 3D CT Report Generation
Mar 16, 2026Renjie Liang, Yiling Ma, Yang Xing +6Radiology Report Generation Multimodal RAG
University of Florida · Yale University · University of Southern California
Unlocking Multimodal Document Intelligence: From Current Triumphs to Future Frontiers of Visual Document Retrieval
Feb 23, 2026Yibo Yan, Jiahao Huo, Guanbo Feng +12Multimodal RAG Multimodal IR
Hong Kong University of Science and Technology (Guangzhou) · Alibaba Cloud Computing · Hong Kong University of Science and Technology +1
Event-Grounded Question Answering over Long Audio via Structured Retrieval
Feb 16, 2026Kartik Hegde, Arvind Krishna Sridhar, Naveen Vakada +2Multimodal RAG Audio QA
Qualcomm Technologies, Inc., India · Qualcomm Technologies, Inc., USA
R3G: A Reasoning-Retrieval-Reranking Framework for Vision-Centric Answer Generation
Jan 25, 2026Zhuohong Chen, Zhengxian Wu, Zirui Liao +6Multimodal RAG Visual Question Answering
1The Shenzhen International Graduate School, Tsinghua University, China · School of Computer Science and Information Engineering, Hefei University of Technology, China · 2State Key Laboratory of Nuclear Power Safety Technology and Equipment, China
Seeing Through the MiRAGE: Evaluating Multimodal Retrieval Augmented Generation
Oct 28, 2025Alexander Martin, William Walden, Reno Kriz +5Multimodal RAG RAG Evaluation
Johns Hopkins University · Human Language Technology Center of Excellence
RS-Agent: Automating Remote Sensing Tasks through Intelligent Agent
Jun 11, 2024Wenjia Xu, Zijian Yu, Boyang Mu +3Multimodal RAG Multimodal Large Language Models
State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing 100876, China · City University of HongKong, Hong Kong 999077, China · School of Geographic Sciences, Hunan Normal University, Changsha 410081, China
Retrieved Images as Visual Thought: Training-Free Multimodal In-Context Learning for the Open-vs-Closed Gap
Date pendingBingchen Huang, Zhiling Wang, Yifu Chen +1Multimodal RAG Multimodal ICL
Meituan

Mar 27, 2026, cs.AIJ/K move · Enter open · S save

GUIDE: Resolving Domain Bias in GUI Agents through Real-Time Web Video Retrieval and Plug-and-Play Annotation

Rui Xie, Zhi Gao, Chenrui Shi, Zirui Shang+2

Shanghai Jiao Tong University, Shanghai, China · State Key Laboratory for General Artificial Intelligence, BIGAI, Beijing, China · Beijing Institute of Technology, Beijing, China

Multimodal RAG Domain Adaptation GUI Agents GUI Grounding

Open paper AskPDF arXiv

Large vision-language models have endowed GUI agents with strong general capabilities for interface understanding and interaction. However, due to insufficient exposure to domain-specific software operation data during training, these agents exhibit significant domain bias - they lack familiarity with the specific operation workflows (planning) and UI element layouts (grounding) of particular applications, limiting their real-world task performance. In this paper, we present GUIDE (GUI Unbiasing via Instructional-Video Driven Expertise), a training-free, plug-and-play framework that resolves GUI agent domain bias by autonomously acquiring domain-specific expertise from web tutorial videos through a retrieval-augmented automated annotation pipeline. GUIDE introduces two key innovations. First, a subtitle-driven Video-RAG pipeline unlocks video semantics through subtitle analysis, performing progressive three-stage retrieval - domain classification, topic extraction, and relevance matching - to identify task-relevant tutorial videos. Second, a fully automated annotation pipeline built on an inverse dynamics paradigm feeds consecutive keyframes enhanced with UI element detection into VLMs, inferring the required planning and grounding knowledge that are injected into the agent's corresponding modules to address both manifestations of domain bias. Extensive experiments on OSWorld demonstrate GUIDE's generality as a plug-and-play component for both multi-agent systems and single-model agents. It consistently yields over 5% improvements and reduces execution steps - without modifying any model parameters or architecture - validating GUIDE as an architecture-agnostic enhancement to bridge GUI agent domain bias.

Momentum

Related topics

Latest papers 100