GeoBridge: Evolving Transferable Decoder Topologies for Geometric Vision
Authors: Rui Wang, Haoyu Zhang, Ran Cheng
Organizations: School of Information Science and Technology, Hangzhou Normal University, Hangzhou, P.R. China · Department of Data Science and Artificial Intelligence, The Hong Kong Polytechnic University, Hong Kong SAR, China
Depth estimation and Gaussian reconstruction share a need for multi-scale feature processing, yet use different prediction interfaces and training objectives. Whether a decoder's internal computation can be optimized once and reused across these systems remains a structural design question. Search over primitive configurations alone leaves the organization of transformations fixed. To address this gap, we propose GeoBridge, an evolutionary framework for transferable decoder topologies. First, GeoBridge makes stage-internal topology searchable through residual, sequential, and parallel composition of configurable primitives. Shape-preserving composition retains the stage interfaces of a fixed multi-scale scaffold. Second, a functional-tree encoding supports stage-aware crossover and hierarchical mutation, while a gradient sensitivity-alignment proxy evaluates candidates without training. Multi-objective evolution balances this proxy with computation and parameter cost. Third, selected topologies are instantiated and trained in target systems with task-specific prediction interfaces. A single source search takes approximately 12 GPU hours. GeoBridgeNet-A3 achieves the best depth accuracy among four adapted decoder baselines under a common encoder and training protocol on NYU Depth V2 and KITTI. The same topology reaches 27.11 dB PSNR in Gaussian novel-view synthesis on RealEstate10K, and its refined representations support open-vocabulary segmentation on LERF. These results support decoder topology as a reusable design object across the evaluated geometric vision systems. Code and pretrained weights are available at https://github.com/EMI-Group/GeoBridge-NAS.
Figures & tables
Fig. 1: GeoBridge searches once and instantiates the selected decoder topology in target systems. (a) TartanAir images and cached teacher-derived targets define the source evaluation environment. (f) NSGA-II varies stage compositions and primitive configurations through stage-aware crossover and hierarchical mutation; GSAP, MACs, and parameter count define candidate fitness. (c) The Pareto set supplies (b) the selected GeoBridgeNet-A3 structure. Its topology is retained for (d) depth estimation on real images and (e) the monocular branch of a Gaussian reconstruction system. Each target system trains its own weights and prediction interface.
Fig. 2: Fixed scaffold and searchable stage computation in GeoBridge. DINOv2 ViT-S blocks B2, B5, B8, and B11 supply features resized to 1/4–1/32 of the input resolution. Lateral fusion and upsampling define the connections between four Tree-Mamba stages. Inside each stage, a functional tree determines the composition of Mamba primitives. Search varies this internal topology and its primitive configurations while retaining the scaffold. Selected structures are trained with the prediction interface required by each target system.
Fig. 3: Compositional decoder topology and its tree encoding. Left: RES, SEQ, and PAR define residual paths, serial processing, and parallel aggregation. Top right: the fields of an OP leaf specify its Mamba primitive. Bottom right: an example genotype is decoded into stage computation through recursive composition. Every subtree preserves its stage interface. Table I gives the primitive domains and scale-dependent bounds; Fig. 6 shows the evaluated structures.
Fig. 4: Structure and computation of GSAP. Top: a fixed batch of source images passes through the frozen DINOv2 encoder and a randomly initialized candidate decoder, while the frozen Depth Anything V2 teacher provides a spatially uniform auxiliary target that is computed once and cached. One backward pass yields one raw sensitivity–alignment response without updating any parameter. Bottom: the illustrated sensitivity and alignment constitute one raw response; during search, this computation is repeated for three independent decoder initializations, the three raw products are averaged, and the logarithm is then applied to obtain JGSAP , which forms the fitness together with MACs and parameters.
Fig. 5: Search dynamics of the 100-generation main run using three-repeat GSAP aggregation. Left: distribution of searched architectures in the three-objective space of JGSAP , MACs, and parameters. (a)–(c) Pairwise projections of the same objective space, with the non-dominated boundary highlighted. (d) Normalized hypervolume over generations; generations 29 and 82 are marked to emphasize that improvement continues beyond the early search before the late-stage plateau. This run is separate from the encoding comparison in Table VIII .
Fig. 6: Stage-wise architectures of GeoBridgeNet-A1, A2, and A3, decoded from their genotypes. Each row shows the searched Mamba cells of one variant in decoding order from s32 to s4 ; Up denotes the fixed lateral fusion and upsampling of the scaffold in Fig. 2 . Each card is one OP leaf, whose scan paths, depth-wise kernel, path fusion, projection sharing, and state fusion follow the legend, and a stacked card denotes a two-block primitive. RES, SEQ, and PAR compositions are drawn as in Fig. 3 , and the genotype string of each cell is given beneath it.
Fig. 7: NYU Depth V2 qualitative comparison. White boxes mark boundary differences within 0–3 m.
Fig. 8: KITTI qualitative comparison. White boxes mark boundary differences within 0–60 m.
Fig. 9: RealEstate10K novel-view comparison for GeoBridgeNet-A3. Red boxes mark geometric and detail differences.
Fig. 10: Open-vocabulary 3-D segmentation and object isolation with the GeoBridgeNet-A3-initialized Gaussian model. Red boxes mark queried objects, and labels report IoU (%).
In this work, we show that a single Transformer block, applied recurrently, can match the accuracy of a full-depth vision encoder at comparable inference FLOPs without intermediate feature distillation. reViT restores depth-specific transformations by representing the FFN at each recurrent depth as a convex combination of a small shared expert bank. A continuous normalized-depth coordinate programs this mixture, defining a resampleable trajectory through FFN parameter space. We evaluate this design in two regimes: supervised ImageNet-1k training and distillation from a DINOv2 teacher. Across both regimes, controlled adaptations identify weight-space merging as the strongest tested MoE family at a matching one-FFN budget, ahead of the token-dispatch and output-mixture alternatives. Trained from scratch, reViT-B/16 attains DeiT III accuracy with about 70% fewer stored parameters. An 8-experts model distilled using only the teacher's output features retains nearly all of its DINOv2 teacher's linear-probe accuracy and transfers across classification, segmentation, and depth prediction. Elastic-depth training allows one checkpoint (trained model) to operate at multiple tested depths by resampling the same normalized coordinate interval. For fixed-depth deployment, the recurrent block can be materialized as a conventional dense graph, removing online routing and merging without changing the one-FFN-per-depth compute but expanding deployment storage.
Adrian Bulat, Yassine Ouali, Georgios Tzimiropoulos
Samsung AI Cambridge · Technical University of Iasi · Queen Mary University of London
Recovering detailed geometry from high-resolution images is critical for precise perception of the surroundings and objects. However, existing methods which use latent-space modeling and VAE reconstruction can compromise geometric details. Furthermore, decoding from latent codes introduces substantial inference overhead. To address those issues, we present EagleDepth, an efficient framework for high-resolution monocular depth estimation that combines the geometric priors of latent diffusion with fine-grained pixel-space generation. Our key idea is to retain depth-aware latent representations as guidance while generating the final depth map directly in pixel space. We train the latent and pixel components sequentially: first, we fine-tune a pretrained latent diffusion model using paired RGB--depth supervision; then, we adapt a pretrained pixel diffusion decoder, PiD, to predict depth conditioned on the learned features. Training of the pixel component starts at 1024 resolution and continues across multiple resolutions up to 4K. The latent branch processes resized, lower-resolution RGB images, while the pixel branch generates depth at the target resolution, bypassing the original VAE decoder. This design preserves learned geometric knowledge without requiring the latent backbone to operate at the output resolution. On five commonly used depth estimation datasets and the high-resolution Synth4K dataset, our framework achieves state-of-the-art depth estimation performance, with faster inference and better preservation of fine structures and object boundaries.
Bowen Chai, Tianbao Zhang, Shuyu Wu +3
Shanghai Jiao Tong University · Beihang University
This work presents ViGeo, a feed-forward foundation model for recovering spatially dense and temporally consistent geometry from video sequences. Built upon a plain transformer architecture without task-specific architectural modifications, ViGeo supports streaming, full-sequence, and long-video inference within a unified model. The key design is dynamic chunking attention, which exposes the model to both bidirectional and causal temporal contexts during training and allows it to adapt its attention pattern at test time without retraining. To improve supervision quality, we further introduce a completion-based data refinement framework. This framework trains a video depth completion teacher that conditions on sparse and noisy annotations and exploits video/multi-view context to produce dense, temporally coherent, and geometrically reliable training targets. Beyond depth and point maps, ViGeo also predicts surface normals within the same framework. Trained solely on public datasets, ViGeo achieves state-of-the-art performance across online, offline, and long-video depth estimation, surface normal estimation, and video point map estimation.
Zhu Yu, Jingnan Gao, Runmin Zhang +9
Zhejiang University · Shanghai Jiao Tong University · Tongyi Lab, Alibaba Group +1