Joint estimation of surface normals and depth is essential for holistic 3D scene understanding, yet high-resolution prediction remains difficult due to the trade-off between preserving fine local detail and maintaining global consistency. To address this challenge, we propose the Ultra Resolution Geometry Transformer (URGT), which adapts the Visual Geometry Grounded Transformer (VGGT) into a unified multi-patch transformer for monocular high-resolution depth--normal estimation. A single high-resolution image is partitioned into patches that are augmented with coarse depth and normal priors from pre-trained models, and jointly processed in a single forward pass to predict refined geometric outputs. Global coherence is enforced through cross-patch attention, which enables long-range geometric reasoning and seamless propagation of information across patches within a shared backbone. To further enhance spatial robustness, we introduce a GridMix patch sampling strategy that probabilistically samples grid configurations during training, improving inter-patch consistency and generalization. Our method achieves state-of-the-art results on UnrealStereo4K, jointly improving depth and normal estimation, reducing AbsRel from 0.0582 to 0.0291, RMSE from 2.17 to 1.31, and lowering mean angular error from 23.36 degrees to 18.51 degrees, while producing sharper and more stable geometry. The proposed multi-patch framework also demonstrates strong zero-shot and cross-domain generalization and scales effectively to very high resolutions, offering an efficient and extensible solution for high-quality geometry refinement.
Figures & tables
Figure 2: Multi-patch transformer pipeline for high-resolution geometry refinement. A coarse depth/normal prediction is first obtained and upsampled, after which the image and coarse outputs are patchified and encoded with DINOv2 and global positional embeddings. All patch tokens are jointly processed through transformer blocks with intra- and cross-patch attention, enabling global geometric reasoning. A DPT head predicts offsets that are fused with the coarse input to produce a globally consistent, detail-preserving high-resolution output.
Figure 3: Illustration of the four GridMix patch sampling strategy configurations used in the training strategy: (from left to right) a single randomly sampled patch, a 2×2 grid of patches, a 3×3 grid of patches, and a 4×4 grid covering the entire image. The red grids represent the patch arrangements, while the green regions indicate the valid areas for randomly selecting the top-left corner of the grid, ensuring the entire grid remains fully within the image boundaries.
Figure 4: Qualitative comparison on the UnrealStereo4K [ 23 ] dataset. Red rectangles indicate the bounding boxes of zoomed-in regions. The second and fourth rows show close-up (zoom-in) views corresponding to the first and third rows, respectively. The rightmost column presents the RGB input and ground-truth depth maps. Compared to previous methods, our predictions exhibit smoother depth continuity and sharper geometry boundaries.
Method (Depth)
Infer Time ↓
AbsRel ↓
δ1↑
RMSE ↓
CE ↓
ϵPDBEacc↓
ϵPDBEcompl↓
Depth-Anything v2 [ 28 ]
–
0.0812
0.924
2.86
–
2.46
189.0
PatchRefiner p=16 [ 11 ]
1.02s
0.0633
0.950
2.28
0.0753
1.98
55.66
PatchRefiner p=49 [ 11 ]
4.12s
0.0582
0.956
2.17
0.0715
1.50
169.81
PRO [ 8 ]
1.88
0.0771
0.927
2.73
0.0549
2.74
417.7
Ours Separate
0.94s
0.0295
0.982
1.38
0.0418
1.73
72.8
Ours Joint
0.97s
0.0291
0.983
1.31
0.0415
1.40
64.21
Table 1: Quantitative comparison on the UnrealStereo4K [ 23 ] dataset. Top: depth estimation results (AbsRel, δ1 , RMSE, CE, PDBE). Bottom: surface normal estimation results (Mean/Median/RMSE angular errors and threshold accuracies).
Figure 5: Qualitative comparison on the Zero-Shot depth estimation task. Each row shows samples from different scenes. The rightmost column presents the RGB input. Compared to previous models, our zero-shot predictions demonstrate improved geometric consistency and sharper depth boundaries across diverse domains.
Figure 6: Qualitative comparison of surface normal estimation on the UnrealStereo4K [ 23 ] dataset. Our method produces more accurate and spatially consistent surface normals compared to Metric3D V2.
Methods
Booster
ETH3D
Middle2014
AbsRel ↓
δ1↑
AbsRel ↓
δ1↑
AbsRel ↓
δ1↑
DAV2 [ 28 ]
0.0274
0.996
0.0507
0.978
0.0307
0.994
PR p=16 [ 11 ]
0.0404
0.988
0.0608
0.968
0.0380
0.987
PR p=49 [ 11 ]
0.0382
0.989
0.0577
0.972
0.0376
0.987
PRO [ 8 ]
0.0287
0.997
0.0448
0.981
0.0287
0.996
Ours Separate
0.0269
0.996
0.0446
0.981
0.0291
0.996
Table 2: Zero-shot depth estimation benchmark across multiple real-world datasets. We evaluate depth accuracy on Booster [ 18 ] , ETH3D [ 21 ] , and Middlebury 2014 [ 20 ] . DAV2 and PR are short for DepthAnythingV2 and PatchRefiner, respectively. Best results are shown in bold .
Configurations ( p1,p2,p3,p4 )
AbsRel ↓
δ1↑
RMSE ↓
CE ↓
Depth-Anything v2
0.0812
0.924
2.86
–
(1,0,0,0)
0.0500
0.963
2.01
0.0648
(0.5,0.5,0,0)
0.0473
0.966
1.79
0.0436
(0.1,0.2,0.3,0.4)
0.0295
0.982
1.35
0.0418
(0,0,0,1)
0.0321
0.980
1.42
0.0635
Table 3: Ablation on GridMix Patch Sampling Strategy.
Figure 7: Ablation study on Global Positional Encoding. Comparison of offset predictions under different positional encoding strategies. The Global RoPE enhances cross-patch geometric alignment and yields smoother transitions across patch boundaries compared to the local variant.
Position Embedding
AbsRel ↓
δ1↑
RMSE ↓
CE ↓
Local RoPE
0.0343
0.976
1.60
0.2830
Global RoPE
0.0321
0.980
1.42
0.0635
Table 4: Ablation study on Global Postional Encoding.
Figure 8: Ablation on Cross-Patch Attention. Visual comparison of both depth predictions and offset maps with and without cross-patch attention. Cross-patch attention significantly improves inter-patch geometric coherence, yielding smoother transitions and more stable boundary structures across patch regions.
Attention Setting
AbsRel ↓
δ1↑
RMSE ↓
w/o Cross-Patch Attention
0.0678
0.945
2.51
w/ Cross-Patch Attention
0.0500
0.963
2.01
Table 5: Ablation on Cross-Patch Attention.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Configurations ( p1,p2,p3,p4 )
AbsRel ↓
δ1↑
RMSE ↓
CE ↓
Depth-Anything v2
0.0812
0.924
2.86
–
(1,0,0,0)
0.0500
0.963
2.01
0.0648
(0.5,0.5,0,0)
0.0473
0.966
1.79
0.0436
(0,1,0,0)
0.0405
0.973
1.72
0.0447
(0.5,0,0.5,0)
0.0365
0.975
1.58
0.0440
(0,0,1,0)
0.0350
0.976
1.57
0.0443
Appendix
Table 6: Ablation on GridMix Patch Sampling Strategy.
Figure 9: Failure case on reflective surfaces. Our model struggles to handle strongly reflective objects such as mirrors: the mirror region is incorrectly interpreted as a continuation of the surrounding geometry.
Figure 10: Extension to arbitrary resolutions. Each column shows a different input resolution (2K, 4K, 8K), and each row corresponds to the RGB input, coarse depth prediction by Depth-Anything V2, our refined depth prediction, coarse normal prediction by Metric3D V2, and our refined normal prediction, respectively.
Figure 11: Surface Normal estimation example from ETH3D.
Figure 12: Application: Lens blur.
Figure 13: Out-of-domain 8K manga-style example. We show an in-the-wild 8K manga-style image, which lies far outside the training distribution, together with our depth and normal predictions. Even under this stylized, out-of-domain setting, our model produces geometrically coherent depth and normals, and the full-resolution visualization allows inspection of fine structures and large-scale consistency.
Figure 14: More qualitative results on UnrealStereo4K. We show samples from the UnrealStereo4K [ 23 ] dataset. Each column corresponds to one scene, and rows show the RGB input, coarse depth prediction by Depth-Anything V2 [ 28 ] , depth refinements from PatchRefiner [ 11 ] and PRO [ 8 ] , our refined depth prediction, coarse normal prediction by Metric3D V2 [ 7 ] , and our refined normal prediction. For each column, a consistent zoom-in inset highlights a region of interest across all methods.
Stereo matching and surface normal estimation are fundamental tasks in 3D vision. However, existing feed-forward stereo methods still struggle to produce reliable predictions in challenging regions, mainly due to the lack of strong geometric priors. In this paper, we propose GeoStereo, a unified stereo geometry estimation framework that leverages powerful diffusion priors to jointly predict disparity and surface normals. Specifically, GeoStereo couples a feed-forward stereo matching pipeline with a diffusion-based normal estimation branch. To enable effective interaction between the two tasks, we introduce a disparity to normal initialization strategy and construct a warp to left-view condition for the diffusion process. This coupled design allows the diffusion branch to provide strong structural priors that enhance disparity estimation in ill-posed regions, while the feed-forward branch offers reliable geometric guidance for accurate normal prediction. Extensive experiments show that GeoStereo performs reliably in challenging scenarios, including low-light environments, highly reflective surfaces, and transparent objects. Under zero-shot settings, it achieves Rank-1 disparity estimation on multiple benchmarks, including KITTI and NYUv2, and delivers the best normal estimation accuracy on many real indoor benchmarks, such as iBims-1 and ScanNet. Project page: https://qz-wei.github.io/GeoStereo.github.io/
Qizhe Wei, Xianda Guo, Shaocong Xu +3
Beijing Institute of Technology BAAI Beijing, China · School of Computer Science, Wuhan University Wuhan, China · BAAI Beijing, China +3
This work presents ViGeo, a feed-forward foundation model for recovering spatially dense and temporally consistent geometry from video sequences. Built upon a plain transformer architecture without task-specific architectural modifications, ViGeo supports streaming, full-sequence, and long-video inference within a unified model. The key design is dynamic chunking attention, which exposes the model to both bidirectional and causal temporal contexts during training and allows it to adapt its attention pattern at test time without retraining. To improve supervision quality, we further introduce a completion-based data refinement framework. This framework trains a video depth completion teacher that conditions on sparse and noisy annotations and exploits video/multi-view context to produce dense, temporally coherent, and geometrically reliable training targets. Beyond depth and point maps, ViGeo also predicts surface normals within the same framework. Trained solely on public datasets, ViGeo achieves state-of-the-art performance across online, offline, and long-video depth estimation, surface normal estimation, and video point map estimation.
Zhu Yu, Jingnan Gao, Runmin Zhang +9
Zhejiang University · Shanghai Jiao Tong University · Tongyi Lab, Alibaba Group +1
High-resolution imagery is essential for accurate 3D reconstruction, as many geometric details only emerge at fine spatial scales. Recent feed-forward approaches, such as the Visual Geometry Grounded Transformer (VGGT), have demonstrated the ability to infer scene geometry from large collections of images in a single forward pass. However, scaling these models to high-resolution inputs remains challenging: the number of tokens in transformer architectures grows rapidly with both image resolution and the number of views, leading to prohibitive computational and memory costs. Moreover, we observe that visually ambiguous regions, such as repetitive patterns, weak textures, or specular surfaces, often produce unstable feature tokens that degrade geometric inference, especially at higher resolutions. We introduce HD-VGGT, a dual-branch architecture for efficient and robust high-resolution 3D reconstruction. A low-resolution branch predicts a coarse, globally consistent geometry, while a high-resolution branch refines details via a learned feature upsampling module. To handle unstable tokens, we propose Feature Modulation, which suppresses unreliable features early in the transformer. HD-VGGT leverages high-resolution images and supervision without full-resolution transformer costs, achieving state-of-the-art reconstruction quality.
Tianrun Chen, Yuanqi Hu, Yidong Han +11
KOKONI 3D, Moxin Technology · Zhejiang University · Huzhou University +4