Overview

Novel view synthesis studies how to render a scene from camera positions that were not observed, supporting applications such as 3D reconstruction, virtual reality, robotics, and medical imaging. Common approaches include neural radiance fields, 3D Gaussian Splatting, and feed-forward multi-view transformers; diffusion and video-generation models are increasingly used to fill unseen regions or handle large viewpoint changes. Methods are tested on benchmarks such as LLFF, Mip-NeRF 360, DTU, and RealEstate10K, with growing attention to sparse, unposed, dynamic, and multimodal inputs. A central challenge is balancing photorealistic completion of unseen content with geometrically consistent views, especially when extrapolating beyond the training cameras, while keeping models efficient and robust to real-world conditions.

Latest papers

309 papers25 papers in the last four weeks, down 29% on the four weeks before. 0.2% of all new papers.

Sep 23, 2025

VolSplat: Rethinking Feed-Forward 3D Gaussian Splatting with Voxel-Aligned Prediction

Feed-forward 3D Gaussian Splatting (3DGS) has emerged as a highly effective solution for novel view synthesis. Existing methods predominantly rely on a pixel-aligned Gaussian prediction paradigm, where each 2D pixel is mapped to a 3D Gaussian. We rethink this widely adopted formulation and identify several inherent limitations: it renders the reconstructed 3D models heavily dependent on the number of input views, leads to view-biased density distributions, and introduces alignment errors, particularly when source views contain occlusions or low texture. To address these challenges, we introduce VolSplat, a new multi-view feed-forward paradigm that replaces pixel alignment with voxel-aligned Gaussians. By directly predicting Gaussians from a predicted 3D voxel grid, it overcomes pixel alignment's reliance on error-prone 2D feature matching, ensuring robust multi-view consistency. Furthermore, it enables adaptive control over density based on 3D scene complexity, yielding more faithful Gaussians, improved geometric consistency, and enhanced novel-view rendering quality. Experiments on widely used benchmarks demonstrate that VolSplat achieves state-of-the-art performance, while producing more plausible and view-consistent results. The video results, code and trained models are available on our project page: https://lhmd.top/volsplat.
Sep 21, 2025

SPFSplatV2: Efficient Self-Supervised Pose-Free 3D Gaussian Splatting from Sparse Views

We introduce SPFSplatV2, an efficient feed-forward framework for 3D Gaussian splatting from sparse multi-view images, requiring no ground-truth poses during training or inference. The framework employs a shared feature extraction backbone to jointly predict 3D Gaussian primitives and camera poses in a canonical space from unposed inputs. To enable efficient and accurate pose estimation, we introduce a masked attention mechanism for target-view pose prediction and a reprojection loss that enforces pixel-aligned Gaussian primitives, providing stronger geometric constraints. We further demonstrate the compatibility of our training framework with different reconstruction architectures, resulting in two model variants. Remarkably, despite the absence of pose supervision, our method achieves state-of-the-art performance in both in-domain and out-of-domain novel view synthesis, even under extreme viewpoint changes and limited image overlap. It also surpasses many methods that rely on geometric supervision in relative pose estimation. By eliminating dependence on ground-truth poses, our method offers the scalability to leverage larger and more diverse datasets. Code and pretrained models will be available on our project page: https://ranrhuang.github.io/spfsplatv2/.
Sep 19, 2025

Camera Splatting for Continuous View Optimization

We propose Camera Splatting, a novel view optimization framework for novel view synthesis. Each camera is modeled as a 3D Gaussian, referred to as a camera splat, and virtual cameras, termed point cameras, are placed at 3D points sampled near the surface to observe the distribution of camera splats. View optimization is achieved by continuously and differentiably refining the camera splats so that desirable target distributions are observed from the point cameras, in a manner similar to the original 3D Gaussian splatting. Compared to the Farthest View Sampling (FVS) approach, our optimized views demonstrate superior performance in capturing complex view-dependent phenomena, including intense metallic reflections and intricate textures such as text.
Aug 26, 2025

AgriChrono: A Longitudinal Dataset and Benchmark for Novel-View Synthesis of Field-Grown Crops

Reliable 3D reconstruction of field-grown crops is hindered by thin, overlapping structures, wind-driven foliage motion, and changing illumination. Yet longitudinal field datasets and standardized novel-view synthesis (NVS) benchmarks remain scarce. We present an integrated framework for robotic acquisition, dataset construction, NVS evaluation, and agricultural scene reconstruction. AgriChrobot enables web-based teleoperation and repeated multi-view capture along crop rows. Using this platform, we introduce AgriChrono, a longitudinal canola dataset collected over approximately one month, comprising 160 recording sessions, four RGB streams, two depth maps, and estimated camera poses. We define seven NVS scenarios spanning four times of day and three acquisition dates, and benchmark 15 NeRF- and Gaussian Splatting-based methods. In addition to whole-image metrics, we assess fine-detail reconstruction within manually annotated canola regions using gradient- and edge-based metrics. Our results show that strong whole-image scores do not guarantee faithful reconstruction of thin, overlapping crop structures. To address this gap, we propose AgriSplat, which relaxes visibility filtering during initialization and reallocates model capacity through contribution-based pruning and budgeted reinvestment in sub-threshold candidates ranked by existing gradient statistics. After reallocation, asymmetric directional supervision penalizes insufficient contrast along reference image-gradient directions. Relative to Octree-GS, AgriSplat improves mean crop-region gradient-energy ratio from 0.506 to 0.847 and edge F-score from 0.712 to 0.797 across seven scenarios, while also improving whole-image PSNR, SSIM, and LPIPS. These contributions provide a dataset, benchmark, and method for systematic crop reconstruction under field conditions.
Aug 14, 2025

Novel View Synthesis using DDIM Inversion

Single-image novel view synthesis requires viewpoint-dependent structural changes while retaining source-scene information and inferring unobserved content. We formulate this task as a camera-conditioned initialization problem for a frozen pretrained latent diffusion model. At an intermediate DDIM inversion timestep, we decompose the source representation into a structural mean and a source noise residual. A pose-conditioned translation U-Net (TUNet) learns to predict the target-view mean from the source mean. We then introduce a score-guided fusion strategy: the predicted mean and source noise residual form a proposal that is evaluated by the frozen denoiser, whose noise prediction is combined with the translated mean to initialize DDIM sampling. Only TUNet is trained, while the pretrained model synthesizes the final view without explicit reconstruction or rendering of a shared 3D scene. Experiments on MVImgNet and RealEstate10K demonstrate competitive performance. Beyond image-level metrics, Wasserstein comparisons show that the corrected initialization yields trajectory-curvature distributions closer to those of ground-truth target-initialized sampling than raw fusion. These results support structural translation and source-conditioned initialization as a practical approach to reusing frozen diffusion priors for novel view synthesis.
Oct 2, 2024

MVGS: Multi-view Regulated Gaussian Splatting for Novel View Synthesis

Recent works in volume rendering, e.g. NeRF and 3D Gaussian Splatting (3DGS), significantly advance the rendering quality and efficiency with the help of the learned implicit neural radiance field or 3D Gaussians. Rendering on top of an explicit representation, the vanilla 3DGS and its variants deliver real-time efficiency by optimizing the parametric model with single-view supervision per iteration during training which is adopted from NeRF. Consequently, certain views are overfitted, leading to unsatisfying appearance in novel-view synthesis and imprecise 3D geometries. To solve aforementioned problems, we propose a new 3DGS optimization method embodying four key novel contributions: 1) We transform the conventional single-view training paradigm into a multi-view training strategy. With our proposed multi-view regulation, 3D Gaussian attributes are further optimized without overfitting certain training views. As a general solution, we improve the overall accuracy in a variety of scenarios and different Gaussian variants. 2) Inspired by the benefit introduced by additional views, we further propose a cross-intrinsic guidance scheme, leading to a coarse-to-fine training procedure concerning different resolutions. 3) Built on top of our multi-view regulated training, we further propose a cross-ray densification strategy, densifying more Gaussian kernels in the ray-intersect regions from a selection of views. 4) By further investigating the densification strategy, we found that the effect of densification should be enhanced when certain views are distinct dramatically. As a solution, we propose a novel multi-view augmented densification strategy, where 3D Gaussians are encouraged to get densified to a sufficient number accordingly, resulting in improved reconstruction accuracy.
Aug 30, 2024

2DGH: 2D Gaussian-Hermite Splatting for High-quality Rendering and Better Geometry Features

2D Gaussian Splatting has recently emerged as a significant method in 3D reconstruction, enabling novel view synthesis and geometry reconstruction simultaneously. While the well-known Gaussian kernel is broadly used, its lack of anisotropy and deformation ability leads to dim and vague edges at object silhouettes, limiting the reconstruction quality of current Gaussian splatting methods. To enhance the representation power, we draw inspiration from quantum physics and propose to use the Gaussian-Hermite kernel as the new primitive in Gaussian splatting. The new kernel takes a unified mathematical form and extends the Gaussian function, which serves as the zero-rank special case in the updated general formulation. Our experiments demonstrate that the proposed Gaussian-Hermite kernel achieves improved performance over traditional Gaussian Splatting kernels on both geometry reconstruction and novel-view synthesis tasks. Specifically, on the DTU dataset, our method yields more accurate geometry reconstruction, while on datasets such as MipNeRF360 and our customized Detail dataset, it achieves better results in novel-view synthesis. These results highlight the potential of the Gaussian-Hermite kernel for high-quality 3D reconstruction and rendering.
Jun 1, 2024

MoDGS: Dynamic Gaussian Splatting from Casually-captured Monocular Videos with Depth Priors

In this paper, we propose MoDGS, a new pipeline to render novel views of dy namic scenes from a casually captured monocular video. Previous monocular dynamic NeRF or Gaussian Splatting methods strongly rely on the rapid move ment of input cameras to construct multiview consistency but struggle to recon struct dynamic scenes on casually captured input videos whose cameras are either static or move slowly. To address this challenging task, MoDGS adopts recent single-view depth estimation methods to guide the learning of the dynamic scene. Then, a novel 3D-aware initialization method is proposed to learn a reasonable deformation field and a new robust depth loss is proposed to guide the learning of dynamic scene geometry. Comprehensive experiments demonstrate that MoDGS is able to render high-quality novel view images of dynamic scenes from just a casually captured monocular video, which outperforms state-of-the-art meth ods by a significant margin. The code is publicly available now.
Nov 30, 2023

SparseGS: Sparse View Synthesis using 3D Gaussian Splatting

3D Gaussian Splatting (3DGS) has recently enabled real-time rendering of unbounded 3D scenes for novel view synthesis. However, this technique requires dense training views to accurately reconstruct 3D geometry. A limited number of input views will significantly degrade reconstruction quality, resulting in artifacts such as "floaters" and "background collapse" at unseen viewpoints. In this work, we introduce SparseGS, an efficient training pipeline designed to address the limitations of 3DGS in scenarios with sparse training views. SparseGS incorporates depth priors, novel depth rendering techniques, and a pruning heuristic to mitigate floater artifacts, alongside an Unseen Viewpoint Regularization module to alleviate background collapses. Our extensive evaluations on the Mip-NeRF360, LLFF, and DTU datasets demonstrate that SparseGS achieves high-quality reconstruction in both unbounded and forward-facing scenarios, with as few as 12 and 3 input images, respectively, while maintaining fast training and real-time rendering capabilities.