Visually Grounded Reasoning

Follow this topic Ask about this topic

Momentum

17 papers in the last four weeks, up 183% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 285

All topics

Multi-Turn Multi-Agent Dialogue for Collaborative Reconstruction Improves VLM Performance on Spatial Reasoning, But Only Barely
May 29, 2026Chalamalasetti Kranti, Sherzod Hakimov, David SchlangenVisual Spatial Reasoning Multi-Agent Collaboration
Computational Linguistics, Department of Linguistics University of Potsdam, Germany · German Research Center for Artificial Intelligence (DFKI), Berlin, Germany
ERGeoBench:A Comprehensive Benchmark for Embodied Reasoning and Geo-localization in Multimodal Large Language Models
May 29, 2026Kaiwen Xue, Tao Wei, Guoxin Zhang +5Spatial Reasoning Benchmarks Visual Spatial Reasoning
Beijing University of Posts and Telecommunications, Beijing, China · State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing, China · School of Materials Science and Engineering, Shanghai Jiao Tong University, Shanghai, China +2
SpatialAct: Probing Spatial Reasoning-to-Action Capabilities of VLM Agents in 3D Scenes
May 29, 2026Tianhui Liu, Jie Feng, Zhiheng Zheng +6Spatial Reasoning Benchmarks Visual Spatial Reasoning
1The Hong Kong University of Science and Technology (Guangzhou) · 2Zhongguancun Academy · 3Tsinghua University +1
iVGR: Internalizing Visually Grounded Reasoning for MLLMs with Reinforcement Learning
May 29, 2026Chang-Bin Zhang, Yujie Zhong, Qiang Zhang +1Large Vision-Language Models Vision-Language Grounding
Visual AI Lab, The University of Hong Kong · Independent Researcher · University of Science and Technology of China
Attend to Evidence: Evidence-Anchored Spatial Attention Supervision for Multimodal RLVR
May 29, 2026Ruina Hu, Chen Wang, Lai Wei +5Reinforcement Learning with Verifiable Rewards Visually Grounded Reasoning
1Harbin Institute of Technology · 2Zhongguancun Academy · 4Nankai University +4
StemBind: When MLLMs Get Lost Between Rules and Instances in Abstract Visual Reasoning
May 29, 2026Xixiang He, Baiqi Wu, Xingming Li +4VLM Evaluation Visual Reasoning
1National University of Defense Technology · 2Zhejiang University · 3Intelligent Game and Decision Lab
ReGuLaR: Relation-Grounded Latent Reasoning for Large Vision-Language Models
May 28, 2026Zihu Wang, Karthik Somayaji N. S, Peng LiVisual Reasoning Large Vision-Language Models
University of California, Santa Barbara
Seeing Isn't Knowing: Do VLMs Know When Not to Answer Spatial Questions (and Why)?
May 28, 2026Yue Zhang, Zun Wang, Han Lin +3VLM Evaluation Spatial Reasoning Benchmarks
UNC Chapel Hill · Google Research
Grounded 3D-Aware Spatial Vision-Language Modeling
May 28, 2026An-Chieh Cheng, Yang Fu, Yatai Ji +123D Spatial Reasoning Visually Grounded Reasoning
UCSD · NVIDIA · MIT
VisualThink-VLA: Visual Intermediate Reasoning for Effective and Low-Latency Vision-Language-Action Policies
May 28, 2026Mingjian Gao, Wenqiao Zhang, Yuqian Yuan +9Efficient VLM Inference Efficient VLA Model Inference
1Zhejiang University · 2Cornell University · 3National University of Singapore +1
Train the Agent, Not the Expert: Learning to Harness Heterogeneous Experts for Multi-Turn Visual Reasoning
May 28, 2026Yaowu Fan, Tao Han, Dazhao Du +2Tool-Using Vision-Language Agents VLM Reasoning
Sun Yat-sen University · HKUST · Harbin Institute of Technology
SSR3D-LLM: Structured Spatial Reasoning via Latent Steps for Fine-Grained Grounding in Unified 3D-LLMs
May 27, 2026Jiawei Li, Ziyi Liu, Weijie Shi +33D Spatial Reasoning Latent Visual Reasoning
The Hong Kong University of Science and Technology · Soochow University
Look on Demand: A Cognitive Scheduling Framework for Visual Evidence Acquisition in Multimodal Reasoning
May 27, 2026Yang Zhang, Xiaoshuai Sun, Rui Zhao +5Active Perception Multimodal Reasoning
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, 361005, P.R. China. · Sino-Russian ResearchCenter for Digital Economy. · School of Informatics, Xiamen University, China. +2
Mags-RL: Wearing Multimodal LLMs a Magnifying Glass via Agentic Reinforcement Learning For Complex Scene Reasoning
May 27, 2026Xuanzhao Dong, Wenhui Zhu, Peijie Qiu +11Visual Question Answering Tool-Using Vision-Language Agents
Arizona State University, Tempe, AZ, USA · Washington University in St. Louis, St. Louis, MO, USA · Clemson University, Clemson, SC, USA +3
ROVER: Routing Object-Centric Visual Evidence for Grounded Multi-Image Reasoning
May 27, 2026Guannan Lv, Ren Nie, Hongjian Dou +1Multimodal Grounding Efficient Multimodal Inference
Kuaishou Technology
Can Segmentation Models Understand the World? Towards Proactive Affordance Reasoning via Visual Chain-of-Thought
May 26, 2026Yuchen Guo, Junli Gong, Hongmin Cai +2Image Segmentation Text-Guided Image Segmentation
Northwestern University · Northeastern University · South China University of Technology +2
Chartographer: Counterfactual Chart Generation for Evaluating Vision-Language Models
May 26, 2026Yifan Jiang, Dae Yon Hwang, Jesse C. Cresswell +1VLM Evaluation Data Visualization
University of Waterloo · Vector Institute · Layer 6 AI
InterSketch: An Interleaved Reasoning Model with Self-correcting Visual Sketch and Stepwise Reward
May 26, 2026Zhiwei Ning, Wenwen Tong, Xiangli Kong +12Multimodal Reasoning VLM Reasoning
Shanghai Jiao Tong University · SenseTime Research · Shandong Normal University
VisualNeedle: Benchmarking Active Visual Search in Information-Dense Scenes
May 25, 2026Jingru Chen, Yiming Liu, Mingtao Chen +5VLM Evaluation Tool Use in VLMs
Hunyuan, Tencent · Peking University · Zhejiang University
DRScaffold: Boosting Dense-Scene Reasoning in Lightweight Vision Language Models
May 25, 2026Xinrui Shi, Kai Liu, Ziqing Zhang +3Visual Spatial Reasoning Vision-Language Grounding
1Shanghai Jiao Tong University
Diversity Over Frequency: Rethinking Tool Use in Visual Chain-of-Thought Agents
May 25, 2026Dong-Hee Kim, Reuben Tan, Donghyun KimTool Use in VLMs Agentic Reasoning
Department of Artificial Intelligence, Korea University, Seoul, South Korea · 2Microsoft Research, Redmond, USA.
ProSR: Process-Shaped Spatial Reasoning for Reliable Chain-of-Thought in VLMs
May 25, 2026Jiangyang Li, Cong Wan, Changjie Wu +8Visual Spatial Reasoning VLM Reasoning
1Xi’an Jiaotong University · 2Amap, Alibaba Group · 4Shenzhen University +1
Can MLLMs Reason Beyond Language? VisReason: A Comprehensive Benchmark for Vision-Centric Reasoning
May 25, 2026Longteng Guo, Yifan Wang, Pengkang Huo +4Visual Reasoning VLM Reasoning
Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences
DUEL: Adversarial Self-Play for Multimodal Reasoning
May 24, 2026Lin Qiu, Hanqing Zeng, Yao Liu +3VLM Robustness VLM Reasoning
Meta AI
EgoAdapt: A Multi-Scene Egocentric Adaptation Method for CVPR 2026 HD-EPIC VQA Challenge
May 23, 2026Zhiwei Chen, Yupeng Hu, Zixu Li +4Test-Time Adaptation VLM Adaptation
Shandong University · Harbin Institute of Technology (Shenzhen)
Med-R2: An Adversarial Benchmark for Evidence-Grounded Reasoning in Medical VLMs
May 23, 2026Wen Ma, Fucheng Niu, Zhiting Fan +3Adversarial Attacks on VLMs Healthcare
Zhejiang University, Zhejiang, China · Guangdong Institute of Intelligence Science and Technology
OmniEgo-R $^2$ : A Routed Reasoning Framework for the 1st Cross-Domain EgoCross Challenge at CVPR 2026
May 23, 2026Zixu Li, Zhiwei Chen, Zhiheng Fu +4Egocentric Video QA Egocentric Video Understanding
Shandong University · Harbin Institute of Technology (Shenzhen)
ETCHR: Editing To Clarify and Harness Reasoning
May 22, 2026Beichen Zhang, Yuhong Liu, Jinsong Li +3Tool Use in VLMs Multimodal Reasoning
The Chinese University of Hong Kong · Shanghai AI Laboratory · Shanghai Jiao Tong University +2
PGT: Procedurally Generated Tasks for improving visual grounding in MLLMs
May 22, 2026Rim Assouel, Amir Bar, Michal Drozdzal +1Visual Spatial Reasoning Synthetic Data Generation
1Mila - Québec AI Institute · 4McGill University · 3FAIR at Meta Superintelligence Labs +2
Seeing without Looking: Do Vision-Language Benchmarks Really Test Vision?
May 21, 2026Zixuan Lan, Luzhe Sun, Matthew R. Walter +1VLM Evaluation VLM Interpretability
University of Chicago · Toyota Technological Institute at Chicago · Stony Brook University

May 29, 2026, cs.CLJ/K move · Enter open · S save

Multi-Turn Multi-Agent Dialogue for Collaborative Reconstruction Improves VLM Performance on Spatial Reasoning, But Only Barely

Chalamalasetti Kranti, Sherzod Hakimov, David Schlangen

Computational Linguistics, Department of Linguistics University of Potsdam, Germany · German Research Center for Artificial Intelligence (DFKI), Berlin, Germany

Visual Spatial Reasoning Multi-Agent Collaboration Vision-Language Grounding Visually Grounded Reasoning Vision-Language Agents

Open paper AskPDF arXiv

Robots operating in diverse environments rely on visual input to interpret objects and spatial layouts. In human-collaborative tasks, they are expected to communicate this understanding through language. Vision-language models (VLMs) support robotic tasks involving visual interpretation, question answering, and instruction following, but their capabilities in collaborative dialogue tasks requiring spatial reasoning remain underexplored. We study this gap through a collaborative structure-building task that combines visual interpretation, grounding, language-guided interaction, and action generation. We develop a framework in which VLMs use dialogue to reconstruct a target structure from visual and textual inputs. We evaluate open-weight and closed VLMs across interaction settings, input modalities, and image representations. Results show that spatial reasoning over visual representations remains difficult for the evaluated VLMs. Detailed text representations of the target yield higher reconstruction success across modality conditions, while decomposed image representations improve performance. These findings reveal limits in visual spatial grounding and grounded instruction generation for collaborative VLM agents.

Momentum

Related topics

Latest papers 285