Multimodal QA

QA: Question Answering

Follow this topic Ask about this topic

Momentum

11 papers in the last four weeks, up 22% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 131

All topics

Decide Before You Look: Learning Which Retrieved Memories Deserve Pixels
Oct 6, 2026Youxing LIMemory-Augmented VLMs Efficient Multimodal Inference
The Chinese University of Hong Kong
ReCast: Contract-Preserving Protection for Fixed-Interface Multimodal Reasoning
Oct 1, 2026Bingchen Pei, Lichong Chen, Bingxi Zhao +8Multimodal QA Multimodal Reasoning
Beijing Jiaotong University · Virginia Polytechnic Institute and State University
InfiMed2: A Generalist Medical Multimodal Foundation Model from Contextual Evidence and Stability-Aware Supervision
Sep 28, 2026Guanghao Zhu, Zeyu Liu, Zhitian Hou +9Multimodal Pretraining Multimodal QA
The Hong Kong Polytechnic University · PolyU-Daya Bay Technology and Innovation Research Institute
WeaveData: A Multimodal Data Analysis System with Self-Critiquing and Self-Evolving LLM Plans
Sep 28, 2026Min Jia, Shihao Zhou, Jun-Peng Zhu +9LLM Self-Refinement LLM Planning
Northwest A&F University · PingCAP · East China Normal University +1
Spatial Action Review: A Visual Analytics Dashboard for Auditing Language-to-Action Hand-offs in Electron Microscopy
Sep 21, 2026Samia Mohinta, Albert CardonaHuman-in-the-Loop Evaluation Multimodal QA
University of Cambridge, UK · MRC Laboratory of Molecular Biology, UK
A $^2$ Safe: Counterfactual Evidence-Aligned Adaptive Agent Collaboration for Safe and Effective Visual Question Answering
Sep 21, 2026Quanxing Xu, Ling Zhou, Xian Zhong +4Visual Question Answering LLM Safety Alignment
School of Computer Science and Engineering, Macau University of Science and Technology, Macao SAR 999078, China · Hubei Key Laboratory of Transportation Internet of Things, School of Artificial Intelligence, Wuhan University of Technology, Wuhan 430070, China · Oulu School, Nanjing Institute of Technology, Nanjing 210096, China +1
PRISM-RAG: Multimodal Hypergraph Retrieval-Augmented Generation for Tobacco Product and Legislative Policy Reasoning
Sep 20, 2026Manuel Serna-Aguilera, Raegan Anderes, Page Dobbs +1Multimodal RAG Legal IR
Fay W. Boozman College of Public Health, University of Arkansas for Medical Sciences, Little Rock, Arkansas, 72205
A Multi-Modal Generative Model for Tomato Disease Leaves Understanding
Sep 17, 2026Khang Nguyen Quoc, Minh-Phuoc Tran, Gia-Han Truong +1Plant Disease Classification Multimodal QA
Less Is More: Graph-free Multimodal RAG via Multi-signal Late Fusion
Sep 16, 2026Tithi Rakshit, Hongkuan Zhou, Lavdim Halilaj +1Multimodal RAG Multimodal IR
University of Tübingen, Tübingen, Germany · Corporate Research, Robert Bosch GmbH, Renningen, Germany · University of Stuttgart, Stuttgart, Germany
Evolution of Multimodal Question Answering: From Modality-Adaptive Extraction to Unified Language Representation
Sep 8, 2026Abdullah Al ShafiUnified Multimodal Models Multimodal QA
Department of CSE, Khulna University of Engineering & Technology, Bangladesh
Bridging the Semantic-Utility Gap in Multimodal RAG via Generator-in-the-Loop Alignment
Sep 8, 2026Zhan-Lun Chang, Dong-Jun Han, Seyyedali Hosseinalipour +2Multimodal RAG Multimodal Reranking
Elmore Family School of Electrical and Computer Engineering, Purdue University, West Lafayette, IN 47907, USA · Department of Computer Science and Engineering, Yonsei University, Seoul 03722, Republic of Korea · Department of Electrical Engineering, University at Buffalo–SUNY, Buffalo, NY 14260, USA
Eliciting Self-Verification in Multimodal Reasoning Agents with Reinforcement Learning
Sep 7, 2026Vishwas Sathish, Viresh Ranjan, Xinliang Zhu +2Tool Use in VLMs Multimodal QA
University of Washington · Seattle, WA, USA · Amazon +1
ImageEval 2026: Culturally Grounded Arabic Multimodal Evaluation
Aug 31, 2026Samir Abdaljalil, Hunzalah Hassan Bhatti, Ahlam Bashiti +11Hallucination Detection T2I Generation
Texas A&M University · Qatar Computing Research Institute, Qatar · Birzeit University, Palestine +2
Read the Room, Read the Image: Understanding Indirect Speech Acts in Multimodal Visual Contexts
Aug 31, 2026Jaehee Kim, Ji Hoon Chung, Seoyoon Park +5Multimodal QA Multilingual VLM Evaluation
Department of Korean Language and Literature, Yonsei University · Interdisciplinary Graduate Program of Linguistics and Informatics, Yonsei University · LG AI Research +1
Doc-REFRAG: Rethinking Multimodal Document Retrieval-Augmented Generation
Aug 31, 2026Ruofan Hu, Shengyang Xu, Minjie Hong +5Multimodal RAG Efficient Multimodal Inference
Zhejiang University
ManGo: Manga Active Narrative Grounding Optimization
Aug 30, 2026Hao Qiu, Junyan Wang, Zheyuan Liu +4Visual Question Answering RL for Language Model Reasoning
1Communication University of China · 2Australian Institute for Machine Learning, Adelaide University · University of New South Wales +2
CLAIR-Fin: An Adversarial Multi-Agent Framework for Claim-Level Verification and Adaptive Debate in Cross-Modal Financial QA
Aug 13, 2026Fatema Tuj Johora Faria, Mukaffi Bin Moin, Jubayer Al Mahmud +2LLM Answer Verification LLM Hallucination Mitigation
Ahsanullah University of Science and Technology, Bangladesh · Jashore University of Science and Technology, Bangladesh · American International University - Bangladesh
HounsWorld: A Multimodal World Model for Hidden Patient-State Readout, Reconstruction, and Simulation
Aug 13, 2026Yunhao Bai, Zhongwei Qiu, Guangyu Guo +5Medical Report Generation World Model Learning
1Shanghai Key Laboratory of Multidimensional Information Processing, East China Normal University · 2DAMO Academy, Alibaba Group · 3Hupan Laboratory +1
EgoCITE: Context-Augmented Indexing and Time-Aware Retrieval for Long-Horizon Egocentric Memory
Aug 12, 2026Le Zhang, Ke SunTemporal IR Multimodal QA
University of Michigan, Ann Arbor
Diagram-MMU: A Multi-Modal Benchmark for Scientific Diagrams
Aug 12, 2026Weihao Bo, Shan Zhang, Yanpeng Sun +7Code Generation Multimodal QA
Nanjing University of Science and Technology · Baidu Inc · AIML, Adelaide University +4
Advancing MLLM-based UAV Image Understanding and Reasoning: A Benchmark and a Training-Free Multi-Agent System
Aug 12, 2026Haoyu Zhang, Shuoxun Zhang, Peng Ye +5Visual Question Answering Multi-Agent LLM Systems
Embedded Deep Learning and Visual Analysis Laboratory, College of Future Information Technology, Fudan University, Shanghai 200433, China · Embedded Deep Learning and Visual Analysis Laboratory, College of Future Information Technology, Fudan University, Shanghai 200433, China, and also with Shanghai Artificial Intelligence Laboratory, Shanghai 200232, China, and also with The Chinese University of Hong Kong, Hong Kong · College of Future Information Technology, Fudan University, Shanghai 200433, China +1
When Prompts Become Pixels: Prompt-Region Grounding for Multimodal Reasoning
Aug 5, 2026Yongxin Wang, Ruizhe Zhou, Yueling Tang +4Multimodal QA Multimodal Reasoning
Mohamed bin Zayed University of Artificial Intelligence · Sun Yat-sen University · Transsion +1
LongChart VQA: A Comprehensive Benchmark for MLLMs with Complex Multi-Chart Reasoning
Aug 2, 2026Ziyan Xiao, Yinghao Zhu, Wenting Zhang +2Data Visualization Visual Reasoning
The University of Hong Kong
PMMC: Prospective Multimodal Memory Compilation for Long-Term LVLM Agents
Aug 2, 2026Jingyu Sun, Yan Lin, Yuyang Xue +10Memory-Augmented VLMs Multimodal Memory
The University of Manchester · The University of Newcastle · The University of Edinburgh +5
DualG-MRAG: Decoupling Macro-Reasoning and Micro-Matching for Multimodal Retrieval-Augmented Generation
Jul 30, 2026Jiacheng Tao, Qingyun Sun, Haonan Yuan +2Multimodal RAG Graph Retrieval-Augmented Generation
SKLCCSE, School of Computer Science and Engineering Beihang University Beijing, China
LEDGERMIND: Provenance-Constrained Multimodal Agentic Reasoning with a Structured Evidence Ledger
Jul 30, 2026Enjun Du, Hange Zhou, Chenxu Du +4Multimodal Grounding Multimodal QA
The Hong Kong University of Science and Technology (Guangzhou) · The University of Hong Kong · Tsinghua University +1
Salient Knowledge Pathways: Sparse Cross-Modal Routing for Efficient Knowledge-Intensive Multimodal Question Answering
Jul 28, 2026Noor Islam S. Mohammad, Uluğ BayazıtMultimodal RAG Efficient Multimodal Inference
Department of Computer Science, İTÜ, İstanbul, Türkiye · Faculty of Computer Engineering, İTÜ, İstanbul, Türkiye.
DeCoRAG: Cognitive Decoupling and Semantic-Aware Cropping for Complex Document Understanding
Jul 27, 2026Shuo Wang, Kai Zhang, Wenyuan Huang +5Document Understanding Multimodal RAG
QiYuanLab, Beijing, China · Beijing University of Posts and Telecommunications, Beijing, China
HVM-GraphRAG: Holistic-View Multimodal Graph Retrieval-Augmented Generation on Complex Document
Jul 26, 2026Xin He, Yili Wang, Wenqi Fan +4Multimodal RAG Retrieval-Augmented Generation
Jilin University Changchun, China · The Hong Kong Polytechnic University Hong Kong SAR, China
Visual Saliency Steering Distillation for Multimodal Chain-of-Thought Reasoning
Jul 24, 2026Hao Yang, Jin Wang, Xuejie ZhangVLM Distillation Multimodal QA
School of Information Science and Engineering, Yunnan University, Kunming, China
Multi-Agent Debate and Visual Information Extraction for SeePhys Pro: A 1st-Place Technical Report from ICML 2026 AI4Math Track 3 Challenge
Jul 24, 2026Jiseok Kwak, Suhyeon Jo, Taewoo Kim +3Visual Question Answering Multi-Agent Debate
KAIST
TAP-RAG: Task-Aware Policy Control for Long-Document Multimodal Question Answering
Jul 21, 2026Zhong Ji, Keqi Jin, Yan Zhang +1Multimodal RAG Retrieval-Augmented Generation
School of Electrical and Information Engineering, Tianjin University · The International Joint Institute of Tianjin University, Fuzhou, Tianjin University · Department of Automation, Tsinghua University
Querying Multimodal Scientific Papers with AI: Practices and Preferences Across Blind, Low-Vision, and Sighted Scientists
Jul 20, 2026Arnavi Chheda-Kothary, Lucy Lu Wang, Joseph Chee Chang +1Multimodal Large Language Models Scientific QA
Paul G. Allen School of Computer Science & Engineering University of Washington Seattle, WA, USA · Allen Institute for AI · The Information School University of Washington Allen Institute for AI Seattle, WA, USA +1
Audio-Visual Flamingo: Open Audio-Visual Intelligence for Long and Complex Videos
Jul 17, 2026Sreyan Ghosh, Arushi Goel, Kaousheik Jayakumar +19Audio-Visual Understanding Multimodal Large Language Models
1NVIDIA, USA · University of Maryland, USA
Reinforcing Egocentric Spatial Perception in Multimodal Large Language Models via Ego Scene Augmentation
Jul 16, 2026Chi Kit Wong, Ye Pan, Yuanhuiyi Lyu +6Visual Question Answering Visual Spatial Reasoning
The Hong Kong University of Science and Technology (Guangzhou) · Guangxi Zhuang Autonomous Region Information Center · The Hong Kong University of Science and Technology
EvoGraph-R1: Self-Evolving Multimodal Knowledge Hypergraphs for Agentic Retrieval
Jul 14, 2026Jiashi Lin, Changhong Jiang, Xiangru Lin +12Multimodal RAG Graph Retrieval-Augmented Generation
1Northwestern Polytechnical University · 3The University of Hong Kong · 5The Chinese University of Hong Kong (Shenzhen) +2
LakeQuest: A Three-Domain Benchmark for Grounded Question Answering across Data Lakes
Jul 14, 2026Michael Solodko, Steven Gong, Guangwei Yu +3Multimodal QA Agentic Retrieval
University of Waterloo · Layer 6 AI
Trust Before Fusion: QIMG-7 and Source-Aware Resolution for Polluted Multimodal RAG
Jul 12, 2026Saadeldine Eletter, Owais Aijaz, Preslav NakovMultimodal RAG Retrieval-Augmented Generation
Mohamed bin Zayed University of Artificial Intelligence
UNIBROWSE: A Data-to-Agent Framework for Multimodal BrowseComp
Jul 12, 2026Xiyu Wei, Qingwei Zong, Zhuocheng Yu +1Multimodal QA AI Agent Benchmarks
Key Laboratory of Computational Linguistics, MOE, Peking University · School of Software and Microelectronics, Peking University · School of Computer Science, Peking University
Empowering Long-form Omni-modal Understanding with Robust Audio Perception
Jul 11, 2026Kaiying Yan, Luoyi Sun, Xiao Zhou +1Audio-Visual Understanding Multimodal Pretraining
SAI, Shanghai Jiao Tong University, shanghai, China · Zhejiang University, Hangzhou, Zhejiang, china · Shanghai AI Lab, shanghai, China
What Does Your Short-Answer VQA Score Actually Measure? Evaluator-Dependent Instability in Multimodal Short-Answer Benchmarks
Jul 11, 2026Guanhua Ye, Niu Jingbin, Yan Li +4VLM Evaluation Visual Question Answering
Beijing University of Posts and Telecommunications Beijing, China
Task-Specific Multimodal Question Answering Agents via Confidence Calibration and Incremental Reasoning for QANTA 2026
Jul 10, 2026Nirjhar Das, Md. Al-Mamun ProvathQuestion Answering Multimodal QA
Department of Computer Science and Engineering, Chittagong University of Engineering & Technology, Chattogram, Bangladesh.
Ground3D-LMM: Fine-Grained 3D Point Grounding and Spatial Reasoning with LMM
Jul 6, 2026Amol Harsh, Zongyan Han, Jean Lahoud +53D Spatial Reasoning Multimodal QA
Mohamed bin Zayed University of Artificial Intelligence · The Hong Kong Polytechnic University · Linköping University
Seeing Once is Enough? Online Geometry-Aware Token Pruning for 3D Question Answering
Jul 5, 2026Ruei-Chi Lai, Bolivar Solarte, Chin-Hsuan Wu +2Efficient VLM Inference Multimodal QA
National Tsing Hua University · Industrial Technology Research Institute ITRI · University of Toronto +1
Q-TriM: Question-Guided Tri-Modal Attention for Audio-Visual Question Answering
Jul 4, 2026SungHun Kim, SeungJun BaekMultimodal QA Cross-Modal Attention
Dept. of Computer Science and Engineering Korea University Seoul, South Korea
Modality Relevance is not Modality Utility: Post-hoc Selective Modality Escalation for Cost-Aware Multimodal RAG
Jul 3, 2026Xue Li, Yiming GaiMultimodal RAG Efficient Multimodal Inference
Hangzhou International Innovation Institute, Beihang University Hangzhou, China
EduArt: An educational-level benchmark for evaluating art history knowledge in large language models
Jul 2, 2026Gianmarco Spinaci, Lukas Klic, Giovanni ColavizzaEducational Assessment Multimodal QA
Department of Classical Philology and Italian Studies, University of Bologna, Bologna, Italy · Villa i Tatti – The Harvard University Center for Italian Renaissance Studies, Firenze, Italy · Center for Computational and Digital Humanities, University of Copenhagen, Copenhagen, Denmark
MultAttnAttrib: Training-Free Multimodal Attribution in Long Document Question Answering
Jul 1, 2026Dang Quang Thien Tran, Quang V. Dang, Vinamra Tyagi +7Multimodal QA Document QA
University of Massachusetts, Amherst · 2Adobe Research, San Jose
Towards Developing a Multimodal Chat Assistant for University Stakeholders: RAG-based Approach
Jul 1, 2026Md Abu Hanif Shaikh, Abdullah Al ShafiMultimodal RAG Multimodal QA
Institute of Information and Communication Technology, Khulna University of Engineering & Technology, Khulna, Bangladesh
TACO: Tool-Augmented Credit Optimization for Agentic Tool Use
Jun 29, 2026Mingkuan Feng, Jinyang Wu, Hao Gu +5Tool Use in VLMs Multimodal QA
Department of Automation, BNRist, Tsinghua University · Institute of Automation, Chinese Academy of Sciences
Hybrid Retriever Evolution for Multimodal Document Reasoning Agents
Jun 28, 2026Bohan Yao, Shruthan Radhakrishna, Vikas YadavMultimodal RAG Multimodal QA
1ServiceNow · University of Washington
CORTEX: A Structured Reasoning Benchmark for Trustworthy 3D Chest CT MLLMs
Jun 25, 2026Hashmat Shadab Malik, Anees Ur Rehman Hashmi, Numan Saeed +3Radiology Report Generation 3D Medical Imaging
MBZUAI, Abu Dhabi, UAE · Hasso Plattner Institute, Potsdam, Germany · Khalifa University, Abu Dhabi, UAE
FlameVQA: A Physically-Grounded UAV Wildfire VQA Benchmark with Radiometric Thermal Supervision
Jun 25, 2026Mobin Habibpour, John Spodnik, Niloufar Alipour Talemi +1VLM Evaluation Visual Question Answering
Clemson University · Clemson, SC, USA
ViTexQA: A Multi-Frame Temporal Perception Dataset for Video Text Question Answering
Jun 23, 2026Zhentao Guo, Chen Duan, Tongkun Guan +3Multimodal Large Language Models Video QA
Meituan · Shanghai Jiao Tong University
Ground Then Rank: Revisiting Knowledge-Based VQA with Training-Free Entity Identification
Jun 22, 2026Qian Ma, Qiong Wu, Zhengyi Zhou +1Multimodal RAG Visual Question Answering
Rensselaer Polytechnic Institute · AT&T Chief Data Office
Efficient Multimodal Clinical Question Answering for Pulmonary Embolism Risk Assessment
Jun 21, 2026Xiangyuan Xue, Yang Yu, Yan Gao +5Efficient Multimodal Inference Multimodal QA
University of Auckland, Auckland, New Zealand · University of Cambridge, Cambridge, United Kingdom · University of Adelaide, Adelaide, Australia +1
EvidenceLens: A Claim-Evidence Matrix for Auditing Financial Question Answering
Jun 19, 2026Fengchen Gu, Xiaotian Ren, Zhengyong Jiang +6Data Visualization LLM Answer Verification
School of AI and Advanced Computing, XJTLU Entrepreneur College (Taicang), Xi’an Jiaotong-Liverpool University, Suzhou, Jiangsu, China · ETSI de Telecomunicaci´on, Universidad Polit´ecnica de Madrid, Madrid, Spain
CogniRoute: Learning to Route Social Evidence in Omni-Modal Models
Jun 18, 2026Yifan Shen, Pei Tian, Xinzhuo Li +8RL for Language Model Reasoning Mixture-of-Experts Inference
University of Illinois Urbana-Champaign · 2Columbia University · 3Google
Confidence Calibration for Multimodal LLMs: An Empirical Study through Medical VQA
Jun 18, 2026Yuetian Du, Yucheng Wang, Ming Kong +4Multimodal QA Language Model Calibration
College of Computer Science and Technology, Zhejiang University · School of Computer Science and Technology, Xidian University · Zhihui Medical Technology (Shanghai) Co., Ltd., Shanghai, China +1
VinQA: Visual Elements Interleaved Long-form Answer Generation for Real-World Multimodal Document QA
Jun 15, 2026Young Rok Jang, Hyesoo Kong, Kyunghwan An +3VLM Evaluation Multimodal Large Language Models
LG AI Research

Oct 6, 2026, cs.CVJ/K move · Enter open · S save

Decide Before You Look: Learning Which Retrieved Memories Deserve Pixels

Youxing LI

The Chinese University of Hong Kong

Memory-Augmented VLMs Efficient Multimodal Inference Multimodal Memory Multimodal QA

Open paper AskPDF arXiv

Multimodal assistants answer questions from long-term memories that contain images. After retrieval, each retrieved image reaches the answering model either as pixels, at about a thousand visual tokens per image, or as a stored text proxy that often misses the detail the question asks about. We find that the benefit of pixels usually comes from one or two retrieved memories, and that it can be predicted before the answering model runs, without reading any full-resolution image. In PixelTriage, a plug-in placed after retrieval, a small model that does not generate text reads the dialogue, a short note and a thumbnail of each retrieved memory and predicts how much its pixels would add. It is trained on synthetic memory episodes labeled by a frozen 27B model that answers each question with and without each memory's pixels. With a 7B answering model, PixelTriage lies on the accuracy--cost frontier of M

^3

Exam, DMV and MemEye and uses 11--23% of the visual tokens without a significant loss of accuracy. On DMV it answers 2.9 times faster than opening all images. It outperforms retrieval order and uniform down-sizing at equal budgets and transfers to other memory systems and to a 397B answering model.

Momentum

Related topics

Latest papers 131