Multimodal Grounding

Follow this topic Ask about this topic

Momentum

18 papers in the last four weeks, up 200% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 120

All topics

SYNCR: A Cross-Video Reasoning Benchmark with Synthetic Grounding
May 8, 2026Sara Ghazanfari, Siddharth Garg, Prashanth Krishnamurthy +1Multimodal Grounding Video Reasoning
New York University
GeoDial: A Multimodal Conversational Tutoring Dataset for Geometry Problem-Solving with Visual Tutor Turns
May 8, 2026Sankalan Pal Chowdhury, Junling Wang, Donya Rooein +2Multimodal Grounding Intelligent Tutoring Systems
ETH Zurich · ETH Zurich, ETH AI Center · Bocconi University
Uncovering Entity Identity Confusion in Multimodal Knowledge Editing
May 7, 2026Shu Wu, Xiaotian Ye, Xinyu Mou +3Multimodal Grounding Multimodal Large Language Models
New Laboratory of Pattern Recognition (NLPR) State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS) Institute of Automation, Chinese Academy of Sciences · Beijing University of Posts and Telecommunications · School of Artificial Intelligence, University of Chinese Academy of Sciences +3
X-OmniClaw Technical Report: A Unified Mobile Agent for Multimodal Understanding and Interaction
May 7, 2026Xiaoming Ren, Ru Zhen, Chao Li +11Multimodal Grounding GUI Agents
Multi-X Team, OPPO AI Center
The Cost of Context: Mitigating Textual Bias in Multimodal Retrieval-Augmented Generation
May 7, 2026Hoin Jung, Xiaoqian WangMultimodal RAG Multimodal Grounding
Elmore Family School of Electrical and Computer Engineering Purdue University West Lafayette, IN 47907
Mitigating Multimodal LLMs Hallucinations via Relevance Propagation at Inference Time
May 3, 2026Itai Allouche, Joseph KeshetMultimodal Grounding Inference-Time Optimization
Department of Electrical and Computer Engineering, Technion, Haifa, Israel
Ablation Study of Multimodal Perception, Language Grounding, and Control for Human-Robot Interaction in an Object Detection and Grasping Task
May 1, 2026Zi Tian, Guanting ShenMultimodal Grounding Robotic Grasping
Dalian University of Technology, Dalian, China
Affordance Agent Harness: Verification-Gated Skill Orchestration
May 1, 2026Haojian Huang, Jiahao Shi, Yinchuan Li +1Multimodal Grounding LLM Agent Orchestration
HKUST(GZ) · Knowin AI · Harbin Engineering University +1
From Mirage to Grounding: Towards Reliable Multimodal Circuit-to-Verilog Code Generation
Apr 30, 2026Guang Yang, Xing Hu, Xiang Chen +1Multimodal Grounding Code Generation
State Key Laboratory of Blockchain and Data Security, Zhejiang University, Hangzhou, China · Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security, Hangzhou, China · School of Artificial Intelligence and Computer Science, Nantong University, Nantong, China
DualFact+: A Multimodal Fact Verification Framework for Procedural Video Understanding
Apr 28, 2026Cennet Oguz, Yasser Hamidullah, Josef van Genabith +1Multimodal Grounding Video Understanding
German Research Center for Artificial Intelligence (DFKI) · Saarland Informatics Campus
Combating Visual Neglect and Semantic Drift in Large Multimodal Models for Enhanced Cross-Modal Retrieval
Apr 28, 2026Guosheng Zhang, Linkai Liu, Keyao Wang +3Multimodal Grounding Cross-Modal Alignment
Baidu Inc.
MEG-RAG: Quantifying Multi-modal Evidence Grounding for Evidence Selection in RAG
Apr 27, 2026Xihang Wang, Zihan Wang, Chengkai Huang +2Multimodal RAG Multimodal IR
Peking University Beijing, China · University of New South Wales, Macquarie University Sydney, Australia · University of New South Wales, CSIRO’s Data61 Sydney, Australia
Multimodal QUD: Inquisitive Questions from Scientific Figures
Apr 26, 2026Yating Wu, William Rudman, Venkata S Govindarajan +2Multimodal Grounding Scientific QA
The University of Texas at Austin · Ithaca College · UC Berkeley, BespokeLabs.ai
One Identity, Many Roles: Multimodal Entity Coreference for Enhanced Video Situation Recognition
Apr 25, 2026Balaji Darur, Amanmeet Garg, Makarand TapaswiMultimodal Grounding Video Understanding
CVIT, IIIT Hyderabad, India · Amazon Prime Video, Seattle
CGC: Compositional Grounded Contrast for Fine-Grained Multi-Image Understanding
Apr 24, 2026Lihao Zheng, Zhenwei Shao, Yu Zhou +5Multimodal Grounding Multimodal Large Language Models
School of Computer Science and Technology, Hangzhou Dianzi University · Base Model, Li Auto
Using Machine Mental Imagery for Representing Common Ground in Situated Dialogue
Apr 22, 2026Biswesh Mohapatra, Giovanni Duca, Laurent Romary +1Multimodal Grounding Conversational Agents
Inria · University of Trento · Work done during Master’s thesis at Inria. +1
SAKE: Self-aware Knowledge Exploitation-Exploration for Grounded Multimodal Named Entity Recognition
Apr 22, 2026Jielong Tang, Xujie Yuan, Jiayang Liu +6Multimodal Grounding Named Entity Recognition
Sun Yat-sen University Zhuhai, China · Shandong Normal University Jinan, China · University of the Chinese Academy of Sciences Beijing, China +2
A-MAR: Agent-based Multimodal Art Retrieval for Fine-Grained Artwork Understanding
Apr 21, 2026Shuai Wang, Hongyi Zhu, Jia-Hong Huang +6Multimodal RAG Multimodal Grounding
University of Amsterdam · Part of this work was done during interning at Amazon. · Also with Amazon AGI. +2
ZSG-IAD: A Multimodal Framework for Zero-Shot Grounded Industrial Anomaly Detection
Apr 20, 2026Qiuhui Chen, Jiaxiang Song, Shuai Tan +1Multimodal Grounding Interpretable Anomaly Detection
School of Information Science and Engineering, East China University of Science and Technology, Shanghai, China
E2E-GMNER: End-to-End Generative Grounded Multimodal Named Entity Recognition
Apr 19, 2026Meng Zhang, Jinzhong Ning, Xiaolong Wu +2Multimodal Grounding Named Entity Recognition
1Dalian Maritime University · 2Dalian University of Technology
Qwen3.5-Omni Technical Report
Apr 17, 2026Qwen TeamMultimodal Grounding Audio-Visual Understanding
Qwen Team
BOOKAGENT: Orchestrating Safety-Aware Visual Narratives via Multi-Agent Cognitive Calibration
Apr 17, 2026Bo Gao, Chang Liu, Yuyang Miao +2Multimodal Grounding Multi-Agent Collaboration
Carnegie Mellon University · University of Science and Technology of China · Imperial College London +2
GIST: Multimodal Knowledge Extraction and Spatial Grounding via Intelligent Semantic Topology
Apr 16, 2026Shivendra Agrawal, Bradley HayesMultimodal Grounding Embodied Navigation
University of Colorado Boulder · Boulder, Colorado, USA
Rethinking Patient Education as Multi-turn Multi-modal Interaction
Apr 16, 2026Zonghai Yao, Zhipeng Tang, Chengtao Lin +5Multimodal Grounding Healthcare
1VA Bedford Health Care · 2UMass Amherst · 4Yale University +3
SemConFlow: Semantic Grounding of Holistic Co-Speech Gesture Generation with Contrastive Flow-Matching
Mar 27, 2026Lanmiao Liu, Esam Ghaleb, Aslı Özyürek +1Multimodal Grounding Flow Matching
Max Planck Institute for Psycholinguistics · Donders Institute for Brain Cognition and Behaviour · Utrecht University
OmniACBench: A Benchmark for Evaluating Context-Grounded Acoustic Control in Omni-Modal Models
Mar 25, 2026Seunghee Kim, Bumkyu Park, Kyudan Jung +5Multimodal Grounding Unified Multimodal Models
Hanyang University · Seoul National University · KAIST AI +1
SONIC-O1: A Real-World Benchmark for Evaluating Multimodal Large Language Models on Audio-Video Understanding
Jan 29, 2026Ahmed Y. Radwan, Christos Emmanouilidis, Hina Tabassum +2Audio-Language Model Evaluation Multimodal Grounding
Vector Institute for Artificial Intelligence, MaRS Centre, Toronto, ON M5G 1L7, Canada · University of Groningen, Nijenborgh 4, 9747 AG Groningen, Netherlands · York University, 4700 Keele Street, Toronto, ON M3J 1P3, Canada
GUI-AIMA: Aligning Intrinsic Multimodal Attention with a Context Anchor for GUI Grounding
Nov 2, 2025Shijie Zhou, Viet Dac Lai, Hao Tan +4Multimodal Grounding GUI Grounding
University at Buffalo, USA · Adobe Research, USA
Evidence-Grounded Trustworthy Multimodal Reasoning and Evaluation Benchmark in Complex Urban Scenes
Jun 11, 2025Zhaoyang Wei, Bowen Jiang, Xumeng Han +6Multimodal Grounding Multimodal Robustness
University of Chinese Academy of Sciences(UCAS) · Tencent CDG
V-Retrver: Evidence-Driven Agentic Reasoning for Universal Multimodal Retrieval
Date pendingDongyang Chen, Chaoyang Wang, Dezhao Su +6Multimodal IR Multimodal Grounding
Tsinghua University · University of Central Florida · Fudan University +4

May 8, 2026, cs.CVJ/K move · Enter open · S save

SYNCR: A Cross-Video Reasoning Benchmark with Synthetic Grounding

Sara Ghazanfari, Siddharth Garg, Prashanth Krishnamurthy, Farshad Khorrami

New York University

Multimodal Grounding Video Reasoning Multimodal Model Evaluation Physical Reasoning in Video Multimodal Understanding

Open paper AskPDF arXiv

Multimodal Large Language Models (MLLMs) have made rapid progress in single-video understanding, yet their ability to reason across multiple independent video streams remains poorly understood. Existing multi-video benchmarks rely largely on human-annotated real-world footage, limiting the precision of spatial, temporal, and physical ground truth and making it difficult to diagnose model failures. We introduce SYNCR, a controlled synthetic benchmark for cross-video reasoning with programmatically verified grounding. Built using Habitat, Kubric, and CLEVRER simulator engines, SYNCR contains 4,000 multi-video question-answer pairs grounded in 4,827 unique videos. It evaluates MLLMs across eight tasks spanning four diagnostic pillars: Temporal Alignment, Spatial Tracking, Comparative Reasoning, and Holistic Synthesis. Our zero-shot evaluation of leading open- and closed-weight MLLMs reveals a substantial gap between current models and humans: the best model achieves only 64.5% average accuracy, compared to an 89.5% human baseline. Models perform relatively well on temporal ordering but struggle with precise physical and spatial reasoning, with the best model reaching only 29.8% accuracy on Kinematic Comparison. We further find that parameter scaling and reasoning-specialized post-training improve temporal alignment capabilities, but do not reliably address fine-grained physical tracking or global spatial synthesis. Finally, a sim-to-real correlation analysis suggests that SYNCR tracks model-level trends on a real-world multi-video benchmark.

Momentum

Related topics

Latest papers 120