Multimodal QA

QA: Question Answering

Multimodal question answering studies how models interpret visual, audio, and document evidence alongside language to answer questions, often requiring evidence to be combined across time, pages, or sources. Common approaches include vision-language models, retrieval-augmented generation, tool-using agents, and specialized training for tasks such as chart, medical, and video QA. Benchmarks increasingly test long-context and multi-hop reasoning, evidence localization, temporal understanding, and whether systems know when to abstain. A central challenge is making answers reliably grounded and verifiable while controlling errors, inference cost, and sensitivity to prompts and evaluation methods.

Latest papers

195 papers19 papers in the last four weeks, up 12% on the four weeks before. 0.1% of all new papers.

RouteDRAG: Learning to Route and Control Agentic Execution for Multimodal Document RAG
Oct 27, 2026Zekai Chen, Hao Zhu, Yu Liu +5Agentic RAG Multimodal RAG
Beijing Institute of Technology, Beijing, China · Chinese Academy of Sciences, China · Zhejiang University, Hangzhou, China +3
TRIAD: Benchmarking Omni-Modal Ambiguity in Multimodal Large Language Models
Oct 22, 2026Zhaolu Kang, Yidi Wang, Yile Li +4Multimodal Model Evaluation Multimodal QA
Hunyuan, Tencent · Peking University
MMEnterprise-Bench: Evaluating Enterprise Agents over Evolving Multimodal Records
Oct 20, 2026Seohyun Lee, Jongha Kim, Dohwan Ko +4Enterprise AI Agents Question Answering
KAIST, Daejeon, Republic of Korea · Korea University, Seoul, Republic of Korea
EDMA: Entropy-Driven Multimodal Answering
Oct 19, 2026Emanuele Mezzi, Gertjan Burghouts, Fabio Massacci +1Multimodal QA Multimodal Reasoning
Vrije Universiteit Amsterdam · TNO · University of Trento
PACE-Mem: Prefix-Aware Complementary Evidence for Multimodal Agents
Oct 17, 2026Shidu Ren, Rong-cheng Tu, Hang Zhou +1Memory-Augmented Agents Memory-Augmented Video Understanding
Shidu Ren is with the Edward S. Rogers Sr. Department of Electrical and Computer Engineering, University of Toronto, Toronto, ON M5S 3G4, Canada · Rong-cheng Tu is with the College of Computing and Data Science, Nanyang Technological University, Singapore 639798 · Hang Zhou is with the Department of Statistics and Operations Research and the School of Data and Information Science, The University of North Carolina at Chapel Hill, Chapel Hill, NC 27599 USA +1
Variational and Decomposed method for Mitigating Bimodal Bias in Robust Visual Question Answering
Oct 16, 2026Yihan Zhao, Yuncheng JiangVLM Robustness Multimodal QA
School of Computer Science, South China Normal University
Can Visual Generation Expand What Unified Multimodal Models Know?
Oct 15, 2026Wenkai Li, Wenyi Hong, Zhuoyi Yang +1Unified Multimodal Models Transfer Learning
Tsinghua University
DocPilot: Interactive Agent Orchestration and Training for Long Visual Document Understanding
Oct 15, 2026Zhuowen Liang, Xiaotian Lin, Yuyu Luo +2Document QA Multimodal Agents
The Hong Kong University of Science and Technology (Guangzhou) · EvenUp, USA
Loud and Clear: Making Visual Evidence Count in Multimodal Document RAG
Oct 15, 2026Eunchan Jo, Seounghyun Jang, Sangwoo MoMultimodal RAG Multimodal QA
POSTECH
TRACER: A Multimodal Harness for Patient Care in Hospitals
Oct 15, 2026Zane Durante, Paul Tang, Keshigeyan Chandrasegaran +11Video Understanding Temporal Video Understanding
Stanford University
EgoOmniWorld: Unifying Egocentric Understanding via Ego-Action-World Interactions
Oct 13, 2026Songruo Chen, Yuqian Fu, Yu Li +7Egocentric Video Understanding Egocentric Video QA
Fudan University · KAUST · Universiti Malaya +4
MedEvo: Medical Reasoning Evolution through Knowledge Navigation
Oct 13, 2026Xiaoliang Wang, Zhuo Chen, Chuanyang Zheng +11Medical Reasoning Multimodal Reasoning
Zhejiang University · Ant Group · Shanghai Artificial Intelligence Laboratory +1
The Image Still Has a Say: Final-Answer Repair for Multimodal Chain-of-Thought
Oct 13, 2026Yulan Pan, Xiaoyu Xu, Zhihong ShenMultimodal CoT Reasoning Multimodal QA
Fuzhou University
Locate, Judge, Read: Role-Specialized Tool Agents over a Knowledge-Point Index for Open-Domain Multimodal Document QA
Oct 9, 2026Wenlin Lin, Fanzhe WeiDocument QA Agentic RAG
Metask Lab
When History Helps and Hurts: Selective History Use across Multimodal Turns
Oct 8, 2026Shuoyang Sun, Kerui Gu, Hao Fang +2Multi-Turn Dialogue Evaluation Multimodal Model Evaluation
Harbin Institute of Technology, Shenzhen · AgiBot · Tsinghua Shenzhen International Graduate School, Tsinghua University
DSV-Mem: Evaluating Multimodal Memory in Professional Workflows for MLLM Agents
Oct 6, 2026Jike Zhong, Ritwick Chaudhry, Xuanbai Chen +4Multimodal Memory Memory Agent Benchmarks
Amazon AGI · University of Southern California
Decide Before You Look: Learning Which Retrieved Memories Deserve Pixels
Oct 6, 2026Youxing LIMemory-Augmented VLMs Efficient Multimodal Inference
The Chinese University of Hong Kong
ReCast: Contract-Preserving Protection for Fixed-Interface Multimodal Reasoning
Oct 1, 2026Bingchen Pei, Lichong Chen, Bingxi Zhao +8Multimodal QA Multimodal Reasoning
Beijing Jiaotong University · Virginia Polytechnic Institute and State University
InfiMed2: A Generalist Medical Multimodal Foundation Model from Contextual Evidence and Stability-Aware Supervision
Sep 28, 2026Guanghao Zhu, Zeyu Liu, Zhitian Hou +9Multimodal Pretraining Multimodal QA
The Hong Kong Polytechnic University · PolyU-Daya Bay Technology and Innovation Research Institute
WeaveData: A Multimodal Data Analysis System with Self-Critiquing and Self-Evolving LLM Plans
Sep 28, 2026Min Jia, Shihao Zhou, Jun-Peng Zhu +9LLM Self-Refinement LLM Planning
Northwest A&F University · PingCAP · East China Normal University +1
Progressive-View On-Policy Distillation for Regional-to-Global Transfer in Multimodal LLMs
Sep 26, 2026Shanfeng Huang, Zhou Fang, Song Xiao +1Knowledge Distillation On-Policy Distillation
Baidu Inc.
Spatial Action Review: A Visual Analytics Dashboard for Auditing Language-to-Action Hand-offs in Electron Microscopy
Sep 21, 2026Samia Mohinta, Albert CardonaHuman-in-the-Loop Evaluation Multimodal QA
University of Cambridge, UK · MRC Laboratory of Molecular Biology, UK
A $^2$ Safe: Counterfactual Evidence-Aligned Adaptive Agent Collaboration for Safe and Effective Visual Question Answering
Sep 21, 2026Quanxing Xu, Ling Zhou, Xian Zhong +4Visual Question Answering LLM Safety Alignment
School of Computer Science and Engineering, Macau University of Science and Technology, Macao SAR 999078, China · Hubei Key Laboratory of Transportation Internet of Things, School of Artificial Intelligence, Wuhan University of Technology, Wuhan 430070, China · Oulu School, Nanjing Institute of Technology, Nanjing 210096, China +1
PRISM-RAG: Multimodal Hypergraph Retrieval-Augmented Generation for Tobacco Product and Legislative Policy Reasoning
Sep 20, 2026Manuel Serna-Aguilera, Raegan Anderes, Page Dobbs +1Multimodal RAG Legal IR
Fay W. Boozman College of Public Health, University of Arkansas for Medical Sciences, Little Rock, Arkansas, 72205
Beyond Exact Match: Task-Aware GRPO for Cross-Domain PCBA Visual Question Answering
Sep 18, 2026Jia Li, Li Dai, Peng Jia +4Visual Question Answering Multimodal QA
Hefei University of Technology Hefei, China · Hefei University of Technology Intelligent Interconnected Systems Laboratory of Anhui Province Hefei, China · Universiti Malaya Kuala Lumpur, Malaysia
A Multi-Modal Generative Model for Tomato Disease Leaves Understanding
Sep 17, 2026Khang Nguyen Quoc, Minh-Phuoc Tran, Gia-Han Truong +1Plant Disease Classification Multimodal QA
Less Is More: Graph-free Multimodal RAG via Multi-signal Late Fusion
Sep 16, 2026Tithi Rakshit, Hongkuan Zhou, Lavdim Halilaj +1Late Fusion Multimodal RAG
University of Tübingen, Tübingen, Germany · Corporate Research, Robert Bosch GmbH, Renningen, Germany · University of Stuttgart, Stuttgart, Germany
Layers, Sinks, and Scaling: Adaptive Evidence Selection for Multimodal Large Language Models
Sep 15, 2026Zhenbin Wang, Lei Zhang, Lituan Wang +3Multimodal Large Language Models Multimodal QA
Sichuan University
PalmLeaf-VQA: A Multi-Script Visual Question Answering Benchmark for Historical Palm-Leaf Manuscript Understanding Across Diverse Regions
Sep 14, 2026Nimol Thuon, Jun Du, Panhapin TheangHistorical Document Analysis Document Image Understanding
University of Science and Technology of China · Université Paris Cité
Evolution of Multimodal Question Answering: From Modality-Adaptive Extraction to Unified Language Representation
Sep 8, 2026Abdullah Al ShafiUnified Multimodal Models Multimodal QA
Department of CSE, Khulna University of Engineering & Technology, Bangladesh

Oct 27, 2026J/K move · Enter open · S save

RouteDRAG: Learning to Route and Control Agentic Execution for Multimodal Document RAG

Zekai Chen, Hao Zhu, Yu Liu, Xunkai Li+4

Beijing Institute of Technology, Beijing, China · Chinese Academy of Sciences, China · Zhejiang University, Hangzhou, China+3

Agentic RAG Multimodal RAG LLM Agent Workflow Optimization Multimodal QA Graph Retrieval-Augmented Generation

Open paper AskPDF arXiv

Web-based applications increasingly rely on understanding long multimodal documents, where answering questions often requires combining information across pages, tables, and figures. Agentic retrieval-augmented generation addresses these tasks through iterative retrieval and reasoning. However, Agentic execution is not always necessary, introducing additional computation and potentially incorrect answers. Our empirical analysis reveals that Agentic execution provides query-dependent benefits, while additional operations may improve, preserve, or degrade answer accuracy. These findings motivate controlling both Agent invocation and subsequent execution. To address these challenges, we propose RouteDRAG, a learning-based framework comprising four components. Unified Document Index organizes multimodal content into a graph-backed index and derives query representations from initial retrieval. Adaptive Path Routing preserves exact answering and predicts the relative benefit of Agent over shorter execution paths. Dynamic Agent Execution maintains an evolving workspace and refines Planner proposals to control tool use and termination. Sequential Policy Learning first trains operation control from observed outcomes and then trains path routing using the resulting Agent's performance. Across two benchmarks, RouteDRAG achieves an average accuracy improvement of 3.47 percentage points over shorter-path-only execution while assigning only 79.3% of queries to Agent. Notably, it also surpasses uniform Agent execution on both benchmarks while requiring fewer answering calls. Our code is now available at https://github.com/Ereshkigal1/RouteDRAG.