cs.CVNov 13, 2026

DIRT: Depth-completed Image-based Rendering using Two-band Blending for Live Novel-view Synthesis

Authors: Mathias Harrer, Laura Fink, Noah Lewis, Paul Himmler, Tim Weyrich

Organizations: Friedrich-Alexander-Universität Erlangen-Nürnberg

Abstract

State-of-the-art novel-view synthesis (NVS) methods commonly require per-scene optimization before rendering, limiting their applicability in live settings. Live or ad-hoc NVS, however, demands low-latency rendering during capture and strong generalization to new scenes. Some previous work tried to address this task by adapting training- and optimization-based methods, trading quality for low latency. We introduce a live NVS approach following traditional image-based rendering that operates on a posed set of depth sensor observations and images. Our method leverages explicit, sparse point-based geometry obtained from active sensors, resulting in a sparsely filled depth buffer that is completed in the (novel) target view's screen-space. Using this proxy, a posed set of images is projected into the target frame before fusing the resulting observations through a frequency-aware two-band blending strategy. The pipeline is entirely non-neural and optimization-free, requiring no training data or scene-specific optimization. As a result, our approach reduces the delay between capture and novel-view rendering and does not rely on favorable network generalization. We validate the method's applicability to heterogeneous sensor setups - including handheld RGB-D cameras, automotive LiDAR, and thermal cameras - as well as challenging scenarios, such as low light captures. Our experiments demonstrate that the approach outperforms prior methods in live reconstruction scenarios, while maintaining competitive performance in conventional NVS benchmarks.

Figures & tables

Explore similar work

CardsList
  1. LagerNVS: Latent Geometry for Fully Neural Real-time Novel View Synthesis

    Mar 20, 2026Stanislaw Szymanowicz, Minghao Chen, Jianyuan Wang +2Novel View Synthesis3D Representation Learning

  2. LiveStre4m: Feed-Forward Live Streaming of Novel Views from Unposed Multi-View Video

    Apr 8, 2026Pedro Quesado, Erkut Akdag, Yasaman Kashefbahrami +2Novel View SynthesisDynamic Novel View Synthesis

  3. DeblurNVS: Geometric Latent Diffusion for Novel View Synthesis from Sparse Motion-Blurred Images

    May 31, 2026Changyue Shi, Wangbo Yu, Chaoran Feng +1Latent Diffusion Models3D Reconstruction