Organizations: The Hong Kong University of Science and Technology (Guangzhou) · Beijing Institute for General Artificial Intelligence · Agency for Science, A*STAR · Peking University · National University of Singapore · The Hong Kong University of Science and Technology
Vision-language models (VLMs) struggle with metric and multi-view relationships that remain implicit in RGB observations. Geometry foundation models recover this structure from RGB, but existing methods either fuse the estimated features with visual tokens and learn to use them from question answering alone, or align VLM latents with geometry features. Capturing geometry, however, does not ensure its use: zeroing the aligned latents of a released feature-alignment model leaves its accuracy unchanged. We introduce GeoReader, a framework that learns geometric representations for the VLM that reads them. First, we train lightweight Geometric Readers to produce continuous input tokens, supervising them with geometry descriptions decoded by a frozen VLM, so that the Readers express geometry in a form the target model can already read. We then freeze the Readers to preserve this interface while fine-tuning the VLM on spatial QA, with geometry injection conditioned on the question. Geometry descriptions serve only as training targets: at inference, the VLM answers directly from the continuous geometric tokens without generating a geometry chain of thought. With geometry predicted from RGB images, our approach reaches 94.3 on MindCube-Tiny with Qwen3-VL-4B and, with a multi-source QA mixture, an average of 72.5 across six additional spatial benchmarks with Qwen3.5-9B. These results suggest that a useful geometric representation is one the receiving model can use, not merely one from which geometry can be reconstructed.
Figures & tables
Figure 1: Geo Reader learns geometry in a form the receiving VLM can read. (a) QA-only fusion learns to use geometry through answer supervision. (b) Feature alignment matches VLM-generated representations to external geometry features. (c) Geo Reader first trains Geometric Readers through description prediction by a frozen VLM, then freezes the Readers and trains the VLM for QA. All three routes estimate geometry from RGB, which also directly conditions each VLM. Flames and ice cubes mark trainable and frozen modules. The pictured I1,I2,I3 are input views; the scene and answer are illustrative. Fig. 2 details the architecture.
Figure 2: Overview of Geo Reader . Depth Anything 3 [ Lin et al., 2026 ] predicts camera poses and depth maps from RGB. (a) Stage 1 trains Readers through a frozen VLM’s description loss. (b) With Readers frozen, the injector uses read-pass context hq to map Zm to answer-pass tokens Zm . Flames and ice mark trainable and frozen modules; descriptions train Readers only.
Table 1: MindCube-Tiny [ Wang et al., 2026a ] accuracy (%). Per API or backbone-size block, bold marks the best and gray the runner-up (only with ≥3 models; base rows unshaded); † : evaluated by us; sizes in parentheses are total parameters reported by the provider. Geo Reader uses DA3 geometry and achieves the highest accuracy in four of the five backbone blocks.
School of Arts and Sciences New York University New York City, United States · Dept. of Engineering and Computer Science Oakland University Rochester, United States