cs.CVMar 7, 2026

Perception-Aware Multimodal Spatial Reasoning from Monocular Images

Authors: Yanchun Cheng, Rundong Wang, Xulei Yang, Alok Prakash, Daniela Rus, Marcelo H Ang, ShiJie Li

Organizations: NUS, Singapore · A*STAR, Singapore · NTU, Singapore · MIT, US

Abstract

Spatial reasoning from monocular images is essential for autonomous driving, yet current Vision-Language Models (VLMs) still struggle with fine-grained geometric perception, particularly under large scale variation and ambiguous object appearance. We propose a simple yet effective perception-aware multimodal reasoning framework that equips VLMs with explicit object-centric grounding ability. Instead of relying on textual bounding-box outputs, each referred object is represented using all Visual Reference Tokens (VRTs) within its spatial extent, enabling visual evidence and textual reasoning to be processed jointly in a unified token space. To further strengthen cross-modal interaction, we construct a Multimodal Chain-of-Thought (MM-CoT) dataset that injects aligned visual and textual reasoning signals. A deterministic ordering strategy is introduced to make supervision over inherently unordered VRT sets fully compatible with the VLM's autoregressive next-token prediction. With only standard supervised fine-tuning, our method achieves substantial improvements on the SURDS benchmark, outperforming previous approaches - including those using RL-based post-training - by a large margin across both single-object and multi-object tasks. These results demonstrate that accurate perception and multimodal reasoning are mutually reinforcing, and together form the key to robust spatial understanding in challenging monocular driving scenarios.

Figures & tables

Explore similar work

CardsList
  1. Imaginative Perception Tokens Enhance Spatial Reasoning in Multimodal Language Models

    Jun 2, 2026Mahtab Bigverdi, Linjie Li, Weikai Huang +8Multimodal UnderstandingSpatial Reasoning

  2. DRIVESPATIAL: A Benchmark for Spatiotemporal Intelligence in VLMs for Autonomous Driving

    May 22, 2026Hao Vo, Khoa Vo, Phu Loc Nguyen +10VLM EvaluationAutonomous Driving Datasets

  3. See, Think, Learn: A Self-Taught Multimodal Reasoner

    Dec 2, 2025Sourabh Sharma, Sonam Gupta, SadbhawnaSelf-TrainingVLM Reasoning