cs.CVFeb 16, 2026

Learning Proposes, Geometry Disposes: A Modular Framework for Efficient Spatial Reasoning

Authors: Haichao Zhu, Zhaorui Yang, Qian Zhang

Organizations: Reality Vision · Department of Computer Science and Engineering, University of California, Riverside

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Spatial perception aims to estimate camera motion and scene structure from visual observations, a problem traditionally addressed through geometric modeling and physical consistency constraints. Recent learning-based methods have demonstrated strong representational capacity for geometric perception and are increasingly used to augment classical geometry-centric systems in practice. However, whether learning components should directly replace geometric estimation or instead serve as intermediate modules within such pipelines remains an open question. In this work, we address this gap and investigate an end-to-end modular framework for effective spatial reasoning, where learning proposes geometric hypotheses, while geometric algorithms dispose estimation decisions. In particular, we study this principle in the context of relative camera pose estimation on RGB-D sequences. Using VGGT as a representative learning model, we evaluate learning-based pose and depth proposals under varying motion magnitudes and scene dynamics, followed by a classical point-to-plane RGB-D ICP as the geometric backend. Our experiments on the TUM RGB-D benchmark reveal three consistent findings: (1) learning-based pose proposals alone are unreliable; (2) learning-proposed geometry, when improperly aligned with camera intrinsics, can degrade performance; and (3) when learning-proposed depth is geometrically aligned and followed by a geometric disposal stage, consistent improvements emerge in moderately challenging rigid settings. These results demonstrate that geometry is not merely a refinement component, but an essential arbiter that validates and absorbs learning-based geometric observations. Our study highlights the importance of modular, geometry-aware system design for robust spatial perception.

Figures & tables

Figure 1: Conceptual illustration of the proposed framework. Learning modules propose geometric observations, while pose estimation is validated and decided by an explicit geometric optimization backend. Dashed arrows indicate learning-based proposals without decision authority.
Figure 2: Geometry disposal semantics. A learning-generated proposal is evaluated by geometric consistency. Proposals that satisfy geometric constraints are accepted and integrated, while inconsistent proposals are explicitly rejected.
SequenceStrideIdentity InitLearning Init
  MeanP95MeanP95
fr1_xyz10.6901.8030.6901.803
 52.6537.6142.6047.190
 105.25412.9824.78013.514
 157.61921.1196.08615.687
fr3_walking10.7231.5920.7261.607
Table 1: Final relative rotation error (RPE) after geometric disposal on the TUM RGB-D dataset. Results are reported as mean / 95th percentile (degrees). Identity and learning-based pose initializations yield highly similar final performance across different motion magnitudes and scene dynamics.
Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

SequenceStrideIdentity InitLearning Init
  MeanP95MeanP95
fr1_xyz10.0230.0480.0230.048
 50.1040.2040.1020.204
 100.2030.4290.1930.386
 150.2650.6160.2590.548
fr3_walking10.0300.0640.0300.064
Appendix
Table 2: Final relative translational error (RPE) after geometric disposal on the TUM RGB-D dataset. Results are reported as mean / 95th percentile. Translational errors show minimal variation across different pose initialization strategies.
StrideIdentity InitLearning Init
 MeanP95MeanP95
10.6881.8030.6911.812
52.8057.1382.7477.139
105.39013.4224.89212.504
159.10024.9496.27914.782
Appendix
Table 3: Final relative rotation error (RPE) after geometric disposal using ground-truth depth from the TUM RGB-D dataset on the fr1_xyz sequence. Results are reported as mean / 95th percentile (degrees). This experiment serves as a sanity check to contextualize performance with oracle depth.
More in this topic

Relative Pose Estimation

Relative pose estimation recovers the rotation and translation between cameras, objects, or robots from visual and other sensor measurements.

See all 66 papers

Explore similar work

CardsList
Mar 27, 2026

Make Geometry Matter for Spatial Reasoning

Empowered by large-scale training, vision-language models (VLMs) achieve strong image and video understanding, yet their ability to perform spatial reasoning in both static scenes and dynamic videos remains limited. Recent advances try to handle this limitation by injecting geometry tokens from pretrained 3D foundation models into VLMs. Nevertheless, we observe that naive token fusion followed by standard fine-tuning in this line of work often leaves such geometric cues underutilized for spatial reasoning, as VLMs tend to rely heavily on 2D visual cues. In this paper, we propose GeoSR, a framework designed to make geometry matter by encouraging VLMs to actively reason with geometry tokens. GeoSR introduces two key components: (1) Geometry-Unleashing Masking, which strategically masks portions of 2D vision tokens during training to weaken non-geometric shortcuts and force the model to consult geometry tokens for spatial reasoning; and (2) Geometry-Guided Fusion, a gated routing mechanism that adaptively amplifies geometry token contributions in regions where geometric evidence is critical. Together, these designs unleash the potential of geometry tokens for spatial reasoning tasks. Extensive experiments on both static and dynamic spatial reasoning benchmarks demonstrate that GeoSR consistently outperforms prior methods and establishes new state-of-the-art performance by effectively leveraging geometric information. The project page is available at https://suhzhang.github.io/GeoSR/.

Shihua Zhang, Qiuhong Shen, Shizun Wang +2

National University of Singapore, Singapore

Visual Spatial Reasoning3D Spatial ReasoningVision-Language Models3D VLMs
AskPDF
Mar 15, 2026

Interp3R: Continuous-time 3D Geometry Estimation with Frames and Events

In recent years, 3D visual foundation models, pioneered by pointmap-based approaches such as DUSt3R, have attracted a lot of interest, achieving impressive accuracy and strong generalization across diverse scenes. However, these methods are inherently limited to recovering scene geometry only at the discrete time instants when images are captured, leaving the scene evolution during the blind time between consecutive frames largely unexplored. We introduce Interp3R, to the best of our knowledge, the first method that enhances pointmap-based models to estimate depth and camera poses at arbitrary time instants. It leverages asynchronous event data to interpolate pointmaps produced by frame-based models, enabling temporally continuous geometric representations. Depth and camera poses are then jointly recovered by aligning the interpolated pointmaps together with those predicted by the underlying frame-based models into a consistent spatial framework. We train Interp3R exclusively on a synthetic dataset, yet demonstrate strong generalization across six datasets, both synthetic and real. Compared with the best two-stage baseline, Interp3R reduces absolute relative depth error by 15%-32% on DSEC and absolute trajectory error by up to 51% on EDS.

Shuang Guo, Filbert Febryanto, Lei Sun +2

TU Berlin, Berlin, Germany · Robotics Institute Germany, Berlin, Germany · INSAIT, Sofia University “St. Kliment Ohridski”, Sofia, Bulgaria +1

Vision Foundation Models3D ReconstructionDepth EstimationCamera Pose Estimation
AskPDF
Jun 4, 2026

Learning Geometric Representations from Videos for Spatial Intelligent Multimodal Large Language Models

Multimodal Large Language Models (MLLMs) excel at 2D semantic understanding but lack intrinsic 3D awareness, resulting in representations that fail to maintain geometric and spatial consistency across video frames. Given the scarcity of large-scale 3D data, we present GeoVR, a novel framework that learns geometric representations using purely 2D video sequences. This approach effectively restructures the semantic latent space within MLLMs to unlock spatial intelligence. Rather than employing superficial feature mixing, GeoVR reshapes the internal representations of the MLLM by distilling geometry knowledge from pre-trained 3D foundation models. This is accomplished through a multi-objective learning strategy driven by four complementary geometric targets: (1) estimating inter-frame camera poses to embed varying viewpoint dynamics, (2) regressing dense depth maps to anchor physical distances, (3) predicting a metric scale factor for real-world calibration, and (4) distilling multi-scale 3D features to align the intermediate feature space. Guided by these explicit physical and geometric constraints, the model's internal representations naturally develop strong 3D awareness. Extensive experiments on spatial reasoning benchmarks demonstrate that GeoVR achieves state-of-the-art performance, establishing a new paradigm for endowing foundation models with spatial intelligence.

Haibo Wang, Lifu Huang

University of California, Davis

3D Spatial ReasoningMultimodal Large Language ModelsGeometric Representation LearningVideo Representation Learning3D Representation Learning
AskPDF