Multimodal QA

QA: Question Answering

Follow this topic Ask about this topic

Momentum

11 papers in the last four weeks, up 22% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 131

All topics

Decide Before You Look: Learning Which Retrieved Memories Deserve Pixels
Oct 6, 2026Youxing LIMemory-Augmented VLMs Efficient Multimodal Inference
The Chinese University of Hong Kong
ReCast: Contract-Preserving Protection for Fixed-Interface Multimodal Reasoning
Oct 1, 2026Bingchen Pei, Lichong Chen, Bingxi Zhao +8Multimodal QA Multimodal Reasoning
Beijing Jiaotong University · Virginia Polytechnic Institute and State University
InfiMed2: A Generalist Medical Multimodal Foundation Model from Contextual Evidence and Stability-Aware Supervision
Sep 28, 2026Guanghao Zhu, Zeyu Liu, Zhitian Hou +9Multimodal Pretraining Multimodal QA
The Hong Kong Polytechnic University · PolyU-Daya Bay Technology and Innovation Research Institute
WeaveData: A Multimodal Data Analysis System with Self-Critiquing and Self-Evolving LLM Plans
Sep 28, 2026Min Jia, Shihao Zhou, Jun-Peng Zhu +9LLM Self-Refinement LLM Planning
Northwest A&F University · PingCAP · East China Normal University +1
Spatial Action Review: A Visual Analytics Dashboard for Auditing Language-to-Action Hand-offs in Electron Microscopy
Sep 21, 2026Samia Mohinta, Albert CardonaHuman-in-the-Loop Evaluation Multimodal QA
University of Cambridge, UK · MRC Laboratory of Molecular Biology, UK
A $^2$ Safe: Counterfactual Evidence-Aligned Adaptive Agent Collaboration for Safe and Effective Visual Question Answering
Sep 21, 2026Quanxing Xu, Ling Zhou, Xian Zhong +4Visual Question Answering LLM Safety Alignment
School of Computer Science and Engineering, Macau University of Science and Technology, Macao SAR 999078, China · Hubei Key Laboratory of Transportation Internet of Things, School of Artificial Intelligence, Wuhan University of Technology, Wuhan 430070, China · Oulu School, Nanjing Institute of Technology, Nanjing 210096, China +1
PRISM-RAG: Multimodal Hypergraph Retrieval-Augmented Generation for Tobacco Product and Legislative Policy Reasoning
Sep 20, 2026Manuel Serna-Aguilera, Raegan Anderes, Page Dobbs +1Multimodal RAG Legal IR
Fay W. Boozman College of Public Health, University of Arkansas for Medical Sciences, Little Rock, Arkansas, 72205
A Multi-Modal Generative Model for Tomato Disease Leaves Understanding
Sep 17, 2026Khang Nguyen Quoc, Minh-Phuoc Tran, Gia-Han Truong +1Plant Disease Classification Multimodal QA
Less Is More: Graph-free Multimodal RAG via Multi-signal Late Fusion
Sep 16, 2026Tithi Rakshit, Hongkuan Zhou, Lavdim Halilaj +1Multimodal RAG Multimodal IR
University of Tübingen, Tübingen, Germany · Corporate Research, Robert Bosch GmbH, Renningen, Germany · University of Stuttgart, Stuttgart, Germany
Evolution of Multimodal Question Answering: From Modality-Adaptive Extraction to Unified Language Representation
Sep 8, 2026Abdullah Al ShafiUnified Multimodal Models Multimodal QA
Department of CSE, Khulna University of Engineering & Technology, Bangladesh
Bridging the Semantic-Utility Gap in Multimodal RAG via Generator-in-the-Loop Alignment
Sep 8, 2026Zhan-Lun Chang, Dong-Jun Han, Seyyedali Hosseinalipour +2Multimodal RAG Multimodal Reranking
Elmore Family School of Electrical and Computer Engineering, Purdue University, West Lafayette, IN 47907, USA · Department of Computer Science and Engineering, Yonsei University, Seoul 03722, Republic of Korea · Department of Electrical Engineering, University at Buffalo–SUNY, Buffalo, NY 14260, USA
Eliciting Self-Verification in Multimodal Reasoning Agents with Reinforcement Learning
Sep 7, 2026Vishwas Sathish, Viresh Ranjan, Xinliang Zhu +2Tool Use in VLMs Multimodal QA
University of Washington · Seattle, WA, USA · Amazon +1
ImageEval 2026: Culturally Grounded Arabic Multimodal Evaluation
Aug 31, 2026Samir Abdaljalil, Hunzalah Hassan Bhatti, Ahlam Bashiti +11Hallucination Detection T2I Generation
Texas A&M University · Qatar Computing Research Institute, Qatar · Birzeit University, Palestine +2
Read the Room, Read the Image: Understanding Indirect Speech Acts in Multimodal Visual Contexts
Aug 31, 2026Jaehee Kim, Ji Hoon Chung, Seoyoon Park +5Multimodal QA Multilingual VLM Evaluation
Department of Korean Language and Literature, Yonsei University · Interdisciplinary Graduate Program of Linguistics and Informatics, Yonsei University · LG AI Research +1
Doc-REFRAG: Rethinking Multimodal Document Retrieval-Augmented Generation
Aug 31, 2026Ruofan Hu, Shengyang Xu, Minjie Hong +5Multimodal RAG Efficient Multimodal Inference
Zhejiang University
ManGo: Manga Active Narrative Grounding Optimization
Aug 30, 2026Hao Qiu, Junyan Wang, Zheyuan Liu +4Visual Question Answering RL for Language Model Reasoning
1Communication University of China · 2Australian Institute for Machine Learning, Adelaide University · University of New South Wales +2
CLAIR-Fin: An Adversarial Multi-Agent Framework for Claim-Level Verification and Adaptive Debate in Cross-Modal Financial QA
Aug 13, 2026Fatema Tuj Johora Faria, Mukaffi Bin Moin, Jubayer Al Mahmud +2LLM Answer Verification LLM Hallucination Mitigation
Ahsanullah University of Science and Technology, Bangladesh · Jashore University of Science and Technology, Bangladesh · American International University - Bangladesh
HounsWorld: A Multimodal World Model for Hidden Patient-State Readout, Reconstruction, and Simulation
Aug 13, 2026Yunhao Bai, Zhongwei Qiu, Guangyu Guo +5Medical Report Generation World Model Learning
1Shanghai Key Laboratory of Multidimensional Information Processing, East China Normal University · 2DAMO Academy, Alibaba Group · 3Hupan Laboratory +1
EgoCITE: Context-Augmented Indexing and Time-Aware Retrieval for Long-Horizon Egocentric Memory
Aug 12, 2026Le Zhang, Ke SunTemporal IR Multimodal QA
University of Michigan, Ann Arbor
Diagram-MMU: A Multi-Modal Benchmark for Scientific Diagrams
Aug 12, 2026Weihao Bo, Shan Zhang, Yanpeng Sun +7Code Generation Multimodal QA
Nanjing University of Science and Technology · Baidu Inc · AIML, Adelaide University +4
Advancing MLLM-based UAV Image Understanding and Reasoning: A Benchmark and a Training-Free Multi-Agent System
Aug 12, 2026Haoyu Zhang, Shuoxun Zhang, Peng Ye +5Visual Question Answering Multi-Agent LLM Systems
Embedded Deep Learning and Visual Analysis Laboratory, College of Future Information Technology, Fudan University, Shanghai 200433, China · Embedded Deep Learning and Visual Analysis Laboratory, College of Future Information Technology, Fudan University, Shanghai 200433, China, and also with Shanghai Artificial Intelligence Laboratory, Shanghai 200232, China, and also with The Chinese University of Hong Kong, Hong Kong · College of Future Information Technology, Fudan University, Shanghai 200433, China +1
When Prompts Become Pixels: Prompt-Region Grounding for Multimodal Reasoning
Aug 5, 2026Yongxin Wang, Ruizhe Zhou, Yueling Tang +4Multimodal QA Multimodal Reasoning
Mohamed bin Zayed University of Artificial Intelligence · Sun Yat-sen University · Transsion +1
LongChart VQA: A Comprehensive Benchmark for MLLMs with Complex Multi-Chart Reasoning
Aug 2, 2026Ziyan Xiao, Yinghao Zhu, Wenting Zhang +2Data Visualization Visual Reasoning
The University of Hong Kong
PMMC: Prospective Multimodal Memory Compilation for Long-Term LVLM Agents
Aug 2, 2026Jingyu Sun, Yan Lin, Yuyang Xue +10Memory-Augmented VLMs Multimodal Memory
The University of Manchester · The University of Newcastle · The University of Edinburgh +5
DualG-MRAG: Decoupling Macro-Reasoning and Micro-Matching for Multimodal Retrieval-Augmented Generation
Jul 30, 2026Jiacheng Tao, Qingyun Sun, Haonan Yuan +2Multimodal RAG Graph Retrieval-Augmented Generation
SKLCCSE, School of Computer Science and Engineering Beihang University Beijing, China
LEDGERMIND: Provenance-Constrained Multimodal Agentic Reasoning with a Structured Evidence Ledger
Jul 30, 2026Enjun Du, Hange Zhou, Chenxu Du +4Multimodal Grounding Multimodal QA
The Hong Kong University of Science and Technology (Guangzhou) · The University of Hong Kong · Tsinghua University +1
Salient Knowledge Pathways: Sparse Cross-Modal Routing for Efficient Knowledge-Intensive Multimodal Question Answering
Jul 28, 2026Noor Islam S. Mohammad, Uluğ BayazıtMultimodal RAG Efficient Multimodal Inference
Department of Computer Science, İTÜ, İstanbul, Türkiye · Faculty of Computer Engineering, İTÜ, İstanbul, Türkiye.
DeCoRAG: Cognitive Decoupling and Semantic-Aware Cropping for Complex Document Understanding
Jul 27, 2026Shuo Wang, Kai Zhang, Wenyuan Huang +5Document Understanding Multimodal RAG
QiYuanLab, Beijing, China · Beijing University of Posts and Telecommunications, Beijing, China
HVM-GraphRAG: Holistic-View Multimodal Graph Retrieval-Augmented Generation on Complex Document
Jul 26, 2026Xin He, Yili Wang, Wenqi Fan +4Multimodal RAG Retrieval-Augmented Generation
Jilin University Changchun, China · The Hong Kong Polytechnic University Hong Kong SAR, China
Visual Saliency Steering Distillation for Multimodal Chain-of-Thought Reasoning
Jul 24, 2026Hao Yang, Jin Wang, Xuejie ZhangVLM Distillation Multimodal QA
School of Information Science and Engineering, Yunnan University, Kunming, China

Oct 6, 2026, cs.CVJ/K move · Enter open · S save

Decide Before You Look: Learning Which Retrieved Memories Deserve Pixels

Youxing LI

The Chinese University of Hong Kong

Memory-Augmented VLMs Efficient Multimodal Inference Multimodal Memory Multimodal QA

Open paper AskPDF arXiv

Multimodal assistants answer questions from long-term memories that contain images. After retrieval, each retrieved image reaches the answering model either as pixels, at about a thousand visual tokens per image, or as a stored text proxy that often misses the detail the question asks about. We find that the benefit of pixels usually comes from one or two retrieved memories, and that it can be predicted before the answering model runs, without reading any full-resolution image. In PixelTriage, a plug-in placed after retrieval, a small model that does not generate text reads the dialogue, a short note and a thumbnail of each retrieved memory and predicts how much its pixels would add. It is trained on synthetic memory episodes labeled by a frozen 27B model that answers each question with and without each memory's pixels. With a 7B answering model, PixelTriage lies on the accuracy--cost frontier of M

^3

Exam, DMV and MemEye and uses 11--23% of the visual tokens without a significant loss of accuracy. On DMV it answers 2.9 times faster than opening all images. It outperforms retrieval order and uniform down-sizing at equal budgets and transfers to other memory systems and to a 397B answering model.

Momentum

Related topics

Latest papers 131