Multimodal Query

Follow this topic Ask about this topic

Momentum

5 papers in the last four weeks, up 67% on the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 32

All topics

Seeing and Solving Are Not Enough for Vision-Language Models
Sep 27, 2026Ziheng Wang, Mingxuan Xie, Yilin Liu +3Multimodal Query State-Tracking
Sun Yat-sen University · Zhejiang University · The Hong Kong University of Science and Technology +2
MoGround: Measuring and Mitigating Modality Distraction in Vision-Language Models
Sep 27, 2026Luca Zhou, Bo Zhao, Rose Yu +2Modalities Multimodal Query
Sapienza University of Rome · Harvard University · UC San Diego +2
TEMA: Evidence-Grounded Temporal Question Answering in Multi-Turn Multi-Audio Dialogs
Sep 24, 2026Kaidi Yang, Hualei Wang, Zhaohui Wang +3Temporal Grounding Audio Understanding
Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China. · University of Chinese Academy of Sciences, Beijing, China.
A $^2$ Safe: Counterfactual Evidence-Aligned Adaptive Agent Collaboration for Safe and Effective Visual Question Answering
Sep 21, 2026Quanxing Xu, Ling Zhou, Xian Zhong +4Knowledge-Based Visual Question Answering Multimodal Query
School of Computer Science and Engineering, Macau University of Science and Technology, Macao SAR 999078, China · Hubei Key Laboratory of Transportation Internet of Things, School of Artificial Intelligence, Wuhan University of Technology, Wuhan 430070, China · Oulu School, Nanjing Institute of Technology, Nanjing 210096, China +1
Evolution of Multimodal Question Answering: From Modality-Adaptive Extraction to Unified Language Representation
Sep 8, 2026Abdullah Al ShafiMultimodal Query Multimodal Reasoning
Department of CSE, Khulna University of Engineering & Technology, Bangladesh
EM^2Mem: Event-Centric Multimodal Memory for Large Language Models
Sep 1, 2026Yijun Chen, Yaqi Zheng, Yanya Li +11Multimodal Memory Multimodal Query
Zhejiang University · South China University of Technology · Lenovo Group Limited
Hi-Q: Hierarchical Evidence-guided Query Refinement for Multi-Hop Question Answering
Aug 31, 2026Jueun Kim, Sungho Park, Wook-Shin HanMulti-Hop Reasoning Multimodal Query
Department of Computer Science and Engineering, POSTECH · Graduate School of Artificial Intelligence, POSTECH
XHotpotQA: A Benchmark for Cross-Lingual Knowledge Composition in Multi-Hop Question Answering
Aug 23, 2026Iman Barati, Arash Ghafouri, Behrouz Minaei-BidgoliMultimodal Query Multi-Hop Reasoning
School of Computer Engineering, Iran University of Science and Technology, Tehran 16846-13114, Iran
Witness Evidence Portfolios: Single-Prefill Risk Detection for Closed Multimodal Answers
Jul 30, 2026Fexiang Liu, Shiye Wang, Qiang Qiu +1Reference-Guided Multimodal In-Context Verification Multiple-Choice Visual Question Answering
1State Key Laboratory of AI Safety, Institute of Computing Technology, CAS · University of Chinese Academy of Sciences
Salient Knowledge Pathways: Sparse Cross-Modal Routing for Efficient Knowledge-Intensive Multimodal Question Answering
Jul 28, 2026Noor Islam S. Mohammad, Uluğ BayazıtMultimodal Query Cross-Modal
Department of Computer Science, İTÜ, İstanbul, Türkiye · Faculty of Computer Engineering, İTÜ, İstanbul, Türkiye.
Diverse-Intent Multi-Turn Fashion Image Retrieval
Jul 22, 2026Mingqiang Tang, Haokun Wen, Meng Liu +3Garments Multimodal Query
1Southern University of Science and Technology, Shenzhen, China · 2Harbin Institute of Technology (Shenzhen), Shenzhen, China · 3Shandong University, Jinan, China +1
TAP-RAG: Task-Aware Policy Control for Long-Document Multimodal Question Answering
Jul 21, 2026Zhong Ji, Keqi Jin, Yan Zhang +1Multimodal Query Hievi-Rag
School of Electrical and Information Engineering, Tianjin University · The International Joint Institute of Tianjin University, Fuzhou, Tianjin University · Department of Automation, Tsinghua University
Stop Thinking, Start Looking: Efficient Post-Training for Multimodal Document Question Answering via Reasoning-Free Alignment
Jul 16, 2026Harikrishnan P M, Goutham Vignesh, Ganesh Parab +4Multimodal Query Multimodal Documents
Phi Labs, Quantiphi, India · Phi Labs, Quantiphi, Canada.
LakeQuest: A Three-Domain Benchmark for Grounded Question Answering across Data Lakes
Jul 14, 2026Michael Solodko, Steven Gong, Guangwei Yu +3Data Lakes Multimodal Query
University of Waterloo · Layer 6 AI
Omni-Decision: Evidence-Ledger Planning for Omni-Modal Agents
Jul 13, 2026Ming Ma, Yi Zhu, Yiran Zhong +6Omni-Modal Multimodal Query
Institute of Neuroscience, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Tongyi Lab, Alibaba Group +2
STEC: Evidence Compression for Deep Search in Open-domain Multi-Hop QA
Jul 12, 2026Xinkang Li, Rong Jiang, Xin Song +3Multi-Hop Reasoning Multimodal Query
National University of Defense Technology, China · Harbin Institute of Technology, Shenzhen, China
CLIR-Bench: Benchmarking Multimodal Question Answering over Irregular Clinical Time Series
Jul 10, 2026Frank Nie, Ethan B. Liu, Yuan Zhu +3Time Series Multimodal Query
Shandong University China · University of Auckland New Zealand
Task-Specific Multimodal Question Answering Agents via Confidence Calibration and Incremental Reasoning for QANTA 2026
Jul 10, 2026Nirjhar Das, Md. Al-Mamun ProvathMultimodal Query Reasoning Skills
Department of Computer Science and Engineering, Chittagong University of Engineering & Technology, Chattogram, Bangladesh.
MAGE-RAG: Multigranular Adaptive Graph Evidence for Agentic Multimodal RAG in Long-Document QA
Jun 14, 2026Yilong Zuo, Xunkai Li, Jing Yuan +3Multimodal Retrieval Augmented Generation Hievi-Rag
Beijing Institute of Technology
StepGap: A Hybrid NLI-LLM Checker for Step-Level Evidence-Gap Detectionin Multi-Hop Question Answering
May 23, 2026Yuelyu Ji, Zhuochun Li, Hui Ji +1Multi-Hop Reasoning Evidence Conflict
School of Computing and Information, University of Pittsburgh
$M^3 QuestionIng$ : Multi-modal Multi-span Medical Question Answering
May 19, 2026Anisha Saha, Vaibhav Rathore, Abhisek Tiwari +3Multimodal Clinical Data Multimodal Query
Max Planck Institute for Informatics, Saarland Informatics Campus, Saarbrucken, Germany · Indian Institute of Technology, Bombay, Mumbai, India · Clinical AI Assistance, Gurugram, India +2
QEVA: A Reference-Free Evaluation Metric for Narrative Video Summarization with Multimodal Question Answering
Apr 27, 2026Woojun Jung, Junyeong KimSummarization Multimodal Evaluation
Department of Artificial Intelligence, Chung-Ang University
Multimodal QUD: Inquisitive Questions from Scientific Figures
Apr 26, 2026Yating Wu, William Rudman, Venkata S Govindarajan +2Multimodal Query Multimodal Reasoning
The University of Texas at Austin · Ithaca College · UC Berkeley, BespokeLabs.ai
TEMA: Anchor the Image, Follow the Text for Multi-Modification Composed Image Retrieval
Apr 23, 2026Zixu Li, Yupeng Hu, Zhiheng Fu +3Composed Image Retrieval Image-Text Pairs
School of Software, Shandong University · Department of Computing, Hong Kong Polytechnic University · School of Computer Science and Technology, Harbin Institute of Technology (Shenzhen)
Beyond Single Plots: A Benchmark for Question Answering on Multi-Charts
Apr 23, 2026Azher Ahmed Efat, Seok Hwan Song, Wallapak TavanapongChart Multimodal Query
Department of Computer Science, Iowa State University, Ames, Iowa, USA
INTENT: Invariance and Discrimination-aware Noise Mitigation for Robust Composed Image Retrieval
Apr 20, 2026Zhiwei Chen, Yupeng Hu, Zhiheng Fu +4Composed Image Retrieval Noise-Aware
School of Software, Shandong University
BUZZY: Contrastive Scoring to Mitigate Text-Induced Bias in Multimodal Multiple-Choice QA
Mar 30, 2026Taeyun Roh, Suhyeong Park, Dongyoung Lee +4Multimodal Query Textual Priors
Korea University · AIGEN Sciences · KAIST +1
ForeSea: AI Forensic Search with Multi-modal Queries for Video Surveillance
Mar 24, 2026Hyojin Park, Yi Li, Janghoon Cho +8Long Video Question Answering Video Surveillance
Qualcomm AI Research, San Diego, CA, USA · Kyunghee University, Gyeonggi, South Korea
HiMu: Hierarchical Multimodal Frame Selection for Long Video Question Answering
Mar 19, 2026Dan Ben-Ami, Gabriele Serussi, Kobi Cohen +1Long Video Question Answering Video Multimodal Large Language Models
INSIGHT Lab, Ben-Gurion University of the Negev, Israel · Ben-Gurion University of the Negev, Israel
RA-QA: A Benchmarking System for Respiratory Audio Question Answering Under Real-World Heterogeneity
Feb 4, 2026Gaia A. Bertolino, Yuwei Zhang, Tong Xia +2Large Audio Language Models Multimodal Query
University of Cambridge · Cambridge, United Kingdom · Tsinghua University +3
DeALOG: Decentralized Multi-Agents Log-Mediated Reasoning Framework
Feb 1, 2026Abhijit Chakraborty, Ashish Raj Shekhar, Shiven Agarwal +1Multimodal Query Multimodal Agents
Arizona State University
Generating a Paracosm for Training-Free Zero-Shot Composed Image Retrieval
Jan 31, 2026Tong Wang, Yunhan Zhao, Shu KongZero-Shot Composed Image Retrieval Composed Image Retrieval
University of Macau, Macau 999078, China · Uiversity of California at Irvine, Irvine, CA 92697, USA · Institute of Collaborative Innovation, University of Macau, Macau 999078, China

Sep 27, 2026, cs.CVJ/K move · Enter open · S save

Seeing and Solving Are Not Enough for Vision-Language Models

Ziheng Wang, Mingxuan Xie, Yilin Liu, Dayan Wu+2

Sun Yat-sen University · Zhejiang University · The Hong Kong University of Science and Technology+2

Multimodal Query State-Tracking

Open paper AskPDF arXiv

Vision-language models (VLMs) answer visual questions by combining visual information extraction with downstream problem solving. We investigate a fundamental question: Does an incorrect answer necessarily reflect a failure in visual extraction or problem solving? A model may succeed at both abilities when tested separately yet still fail on the original multimodal question, a distinction that overall answer accuracy cannot reveal. To study this, we perform a question-level empirical analysis across multiple VLMs and visual domains. We define an exactly scorable task state (i.e., the visual information sufficient to solve a question) and use it to test whether the same model can extract the required state, solve the question from the ground-truth state, and answer the original multimodal question. We find that composition failures, where extraction and solving both succeed but direct answering fails, account for 17.7% to 75.6% of direct-answering errors across multiple VLMs and datasets. To address this failure mode, we introduce a simple yet effective method, termed State Realization Tuning (SRT). SRT fine-tunes LoRA adapters attached to the language-model layers while keeping the pretrained VLM weights frozen. It trains the model to output the ground-truth task state before the final answer in a single autoregressive response. SRT improves over standard supervised fine-tuning by 1.7 to 14.1 percentage points and repairs 92.5% to 98.1% of diagnosed composition failures. A single LoRA adapter trained with SRT also improves performance across substantially different task-state structures. Our work shows that having both visual extraction and problem-solving capabilities does not guarantee correct multimodal answering. Requiring the model to first output the visual information needed to solve the question can help bridge this gap.

Momentum

Related topics

Latest papers 32