cs.CVNov 2, 2026

Image Guided Depth Upsampling using Anisotropic Total Generalized Variation

Authors: David Ferstl, Christian Reinbacher, Rene Ranftl, Matthias Rüther, Horst Bischof

Organizations: Graz University of Technology, Institute for Computer Graphics and Vision, Inffeldgasse 16, 8010 Graz, AUSTRIA

Abstract

In this work we present a novel method for the challenging problem of depth image upsampling. Modern depth cameras such as Kinect or Time of Flight cameras deliver dense, high quality depth measurements but are limited in their lateral resolution. To overcome this limitation we formulate a convex optimization problem using higher order regularization for depth image upsampling. In this optimization an anisotropic diffusion tensor, calculated from a high resolution intensity image, is used to guide the upsampling. We derive a numerical algorithm based on a primaldual formulation that is efficiently parallelized and runs at multiple frames per second. We show that this novel upsampling clearly outperforms state of the art approaches in terms of speed and accuracy on the widely used Middlebury 2007 datasets. Furthermore, we introduce novel datasets with highly accurate groundtruth, which, for the first time, enable to benchmark depth upsampling methods using real sensor data.

Figures & tables

Explore similar work

Feb 18, 2026

Depth from Defocus via Direct Optimization

Though there exists a reasonable forward model for blur based on optical physics, recovering depth from a collection of defocused images remains a computationally challenging optimization problem. In this paper, we show that with contemporary optimization methods and reasonable computing resources, a global optimization approach to depth from defocus is feasible. Our approach rests on alternating minimization. When holding the depth map fixed, the forward model is linear with respect to the all-in-focus image. When holding the all-in-focus image fixed, the depth at each pixel can be computed independently, enabling embarrassingly parallel computation. We show that alternating between convex optimization and parallel grid search can effectively solve the depth-from-defocus problem at higher resolutions than current deep learning methods. We demonstrate our approach on benchmark datasets with synthetic and real defocus blur and show promising results compared to prior approaches. Our code is available at github.com/hollyjackson/dfd.
Feb 10, 2026

Metric Depth Estimation from Arbitrarily Degraded Low-Resolution Depth Prompts

We propose AdaDS, an Adaptive Diffusion Sampling framework for prompted metric depth estimation, which estimates high-resolution metric depth from an image and an arbitrarily degraded low-resolution depth prompt. This setting is commonly studied as depth super-resolution, where existing methods typically regress depth directly without an explicit prior over high-quality depth, and thus often produce artifacts under severe or unseen degradations. In contrast, AdaDS exploits the contraction property of Gaussian smoothing: as noise accumulates in the forward diffusion process, the noisy distributions of degraded prompts and their high-quality counterparts converge toward an isotropic Gaussian prior, while the prompt content is gradually erased. Accordingly, AdaDS refines the prompt and estimates its uncertainty to adaptively select a starting timestep, and injects learned noise to place the intermediate sample in a high-density region of the depth distribution learned by a pre-trained diffusion model. Denoising from this sample yields high-quality depth with fine-grained details and corrects residual errors, even under imperfect prompt refinement. Experiments on real-world and synthetic benchmarks demonstrate its superior zero-shot generalization and robustness to diverse degradations.
Sep 5, 2026

Depth-to-Image Synthesis-Driven Generative Unguided Depth Completion

Guided depth completion methods heavily depend on RGB quality and alignment, while unguided ones often suffer from limited precision due to the absence of explicit visual cues. In this paper, we present Depth-to-Image Synthesis-Driven Generative Unguided Depth Completion (GUDC), a new completion paradigm that innovatively bridges advanced 2D generative models with unguided depth completion, enabling semantics-aware depth inference without real RGB inputs. Our key idea is to exploit ControlNet's powerful depth-conditioned generation capability to synthesize pseudo-images directly from sparse depth, effectively converting the original unguided setting into a semantics-guided one. To address the potential image-depth misalignment caused by depth sparsity, we propose a multi-level dense-to-sparse representation distillation strategy for ControlNet fine-tuning, where dense-depth features act as teacher signals to distill consistent structural representations for sparse-depth inputs. Furthermore, during pseudo-image-guided completion, we propose a pseudo-image semantic attention fusion module to adaptively extract informative semantic cues from pseudo-images while suppressing artifacts (e.g., texture hallucinations). Extensive experiments on KITTI and NYUv2 validate that our GUDC achieves superior accuracy and robustness over existing methods.