Sparse-View 4D Reconstruction

Latest papers

13 papers2 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

Oct 13, 2026

Dual-Domain Sparse-View 4D CBCT Reconstruction with Spatial-Temporal Consistency Learning

Four-dimensional cone-beam computed tomography (4D CBCT) provides phase-resolved volumetric images for image-guided radiotherapy. Yet good reconstruction quality usually needs many projections. This increases scan time and radiation dose. We propose STCNet, a sparse-view 4D CBCT reconstruction framework that uses spatial-temporal consistency learning. STCNet reconstructs high-quality images from only 16 projections in each respiratory phase. STCNet integrates physics-driven positional encoding to enforce geometric consistency in the projection domain and introduces temporal encoding in latent space to capture respiratory motion dynamics across phases to capture dynamic respiratory motion across phases. A dual-domain collaborative strategy jointly optimizes both projection and image domains, enhancing structural fidelity and temporal coherence. We tested STCNet on the TCIA-LUNG dataset and a CIRS thorax phantom dataset. STCNet performed better than conventional methods and recent deep learning baselines in image quality, artifact reduction, and structural preservation. On TCIA-LUNG, STCNet reached PSNR values of 33.311, 32.695, and 32.658 dB at phases 2, 5, and 8. The corresponding SSIM values were 0.845, 0.823, and 0.825. On CIRS, STCNet achieved PSNR values up to 36.497 dB and SSIM values up to 0.877. These results support STCNet as a promising framework for low-dose, phase-resolved 4D CBCT reconstruction in image-guided radiotherapy.

Qi Lai, YuTong He

Sparse-View 4D Reconstruction Image Reconstruction Cone-Beam CT Sparse-View CT Reconstruction

Ask PDF

Oct 3, 2026

Sparse-View 4D Gaussian Splatting via Spatiotemporal Priors and Generative Assistance

We present a 4D Gaussian Splatting framework for the Sparse-View Track of the SIGGRAPH Asia 2026 Volumetric Video Challenge, which requires dynamic scene reconstruction from only six cameras with wide baselines. To achieve robust dynamic reconstruction under such sparse views, our framework integrates three components. (1) Region-adaptive spatial priors: We use foreground masks to guide Gaussian initialization and mask voting to control densification separately for the dynamic foreground and static background. Background geometry is regularized using monocular depth aligned to metric scale. (2) Motion-consistent temporal priors: We provide supervision at intermediate times through frame interpolation and constrain projected Gaussian motion with estimated optical flow. (3) Generative assistance: We place virtual cameras in the widest angular gaps and restore their rendered images using a diffusion-based model conditioned on camera poses. The restored images are iteratively incorporated into training as pseudo-supervision. On the validation set, our framework improves full-frame PSNR from 25.60 dB for the baseline to 29.75 dB. On the official test benchmark, it achieves 30.04 dB full-frame PSNR and 27.88 dB foreground PSNR, ranking first overall in the Sparse-View Track.

Shengqi Wang, Zhengxian Yang, Kaiwen Tian +6

Department of Automation, Tsinghua University, Beijing, China · JD.com, Beijing, China · Beijing MEET YUAN Co., Ltd, Beijing, China

Sparse-View 4D Reconstruction Dynamic Scene Reconstruction 3D Gaussian Splatting

Ask PDF

Sep 18, 2026

4DGS-Fixer: Generative Sparse-View 4D Gaussian Splatting with Iterative Refinement Guided by Video Diffusion Priors

This paper addresses the challenges of dynamic scene synthesis from sparse-view videos. Existing methods employ geometric priors, adaptive optimization, or density-control strategies to improve 4D Gaussian modeling under sparse observations. However, they cannot fundamentally resolve the ill-posed problem caused by insufficient observations and missing scene information. Moreover, sparse-view 4D Gaussian Splatting (4DGS) often suffers from poor geometric initialization: with only a few input views, COLMAP typically reconstructs sparse and incomplete point clouds, leaving large scene regions without sufficient Gaussian support and making them difficult to recover through subsequent optimization. To address these limitations, we propose a novel iterative refinement framework based on a video diffusion model to improve the completeness and consistency of dynamic 4D scenes. Specifically, we first estimate multi-view depth maps and fuse them into dense point clouds to provide more complete geometric initialization for a dynamic 4DGS representation. We then employ a pretrained video restoration model to refine sequences rendered along novel camera trajectories at different time steps. The restored sequences serve as pseudo-supervision to regularize and iteratively refine the 4DGS representation. Experiments on a widely used benchmark dataset demonstrate that our method substantially outperforms existing baselines, achieving nearly a 2 dB PSNR improvement over the previous best-performing method.

Haitao Huang, Shenghao Zhao, Boyuan Tian +7

Goertek Alpha Labs Shanghai, China · Singapore Institute of Technology Singapore, Singapore · Goertek Alpha Labs Santa Clara, USA +1

3D Gaussian Splatting Sparse-View 4D Reconstruction Dynamic Scene Reconstruction

Ask PDF

Sep 5, 2026

UniFusion: Sparse-View 4D Reconstruction via Unified Spatio-temporal Depth Alignment

In this paper, we address the challenging problem of 4D reconstruction from sparse-view videos. This setup usually relies on monocular depth estimation to provide priors for the reconstruction model. A key challenge arises from limited cross-view overlap and temporal variation, making monocular depth predictions inconsistent across views and time. Existing methods align spatial and temporal dimensions in separate stages, requiring foreground segmentation masks while failing to leverage temporal cues for cross-view alignment. Contrary to these methods, we propose a unified spatial-temporal depth alignment framework that jointly resolves cross-view and cross-time inconsistencies without distinguishing foreground/background. Our method represents depth maps across views and time as a set of spatio-temporal neural fields. This representation not only yields fast convergence, but also captures spatio-temporal correlation among depth maps implicitly, without dependence on external segmentation/tracking models. We also propose a multi-view depth-order loss while leveraging the classic scale-and-shift-invariant loss to further improve the final depth quality. The aligned depths initialize and supervise Gaussian splatting models for 4D reconstruction. Experiments on Ego-Exo4D and EgoHuman demonstrate that our improved depth alignment substantially benefits dynamic Gaussian-splatting-based reconstruction methods for novel-time/view synthesis and geometry accuracy/consistency.

Yongzhe Lyu, Shaofei Wang, Yixin Chen +1

State Key Laboratory of General Artificial Intelligence, BIGAI, Beijing, China

Sparse-View 4D Reconstruction 4D Reconstruction Dynamic Scene Reconstruction Multi-View Consistency

Ask PDF

Aug 20, 2026

Sparse Light Field Sampling Improves Casual 3D and 4D Reconstruction

Many consumer smartphones, stereo cameras, and light field cameras record multiple synchronized viewpoints in a single exposure event. However, novel view synthesis pipelines commonly use only a monocular stream and rely on camera motion or learned priors to obtain angular coverage. In this paper, we ask: why do we use only one viewpoint? We analyze sensor-limited multi-view, where one sensor trades off spatial and angular resolution, and exposure-limited multi-view, where multiple sensors on one commodity device observe each event simultaneously. We introduce a new dataset incorporating three types of commodity multi-view cameras, and evaluate sparse-view 3DGS and 4DGS baselines measuring reconstruction quality as a function of number of exposures and angle between extreme views. Our results demonstrate that using multiple cameras, even with a low baseline, significantly improves reconstruction quality in single-shot, few-shot, and casual video settings. In addition, under a fixed sensor budget, angular sampling improves reconstruction when exposures are scarce despite lower spatial resolution. The gains are most pronounced for single-shot and dynamic scenes, where a stationary monocular camera lacks the angular diversity to recover scene geometry and motion.

Shamus Li, Ruiming Cao, Laura Waller +2

Cornell University, USA · Adobe, USA · University of California, Berkeley, USA +1

3D Reconstruction Sparse-View 4D Reconstruction Sparse-View 3D Reconstruction 4D Reconstruction Dynamic Scene Reconstruction 3D Gaussian Splatting

Ask PDF

Aug 17, 2026

Shared-Structure 4D Spectral Gaussian Representation for Sparse-View Spectral CT Reconstruction

Sparse-view spectral computed tomography (CT) reconstructs energy-resolved attenuation volumes from limited projection views, requiring simultaneous handling of angular undersampling and spectral coupling. We propose a SharedStructure 4D Spectral Gaussian Representation (4D-SG) that learns shared Gaussian geometry from full spectrum structural projections and uses a Gaussian-wise Spectral Density Curve Network (GSC-Net) to predict Gaussian raw density transformations. This factorization separates shared spatial structure from spectral attenuation variation, avoids independent channel geometry optimization, and establishes a continuous 4D-SG representation from discrete spectral measurements for unobserved spectral channel queries. Experiments on six synthesized, simulated projection, and real projection datasets with 50 views demonstrate the best average performance. Compared with the strongest Gaussian baseline, 4D-SG improves PSNR from 35.56 dB to 36.61 dB, increases SSIM from 0.909 to 0.914, and reduces LPIPS from 0.208 to 0.194, demonstrating its effectiveness for sparse-view spectral CT reconstruction.

Jiancheng Fang, Shaoyu Wang, Wenjun Xia +2

School of Information Engineering, Nanchang University, Nanchang 330031, China · School of Computer Science and Engineering, Southeast University, Nanjing 210096, China · Jiangsu Provincial Joint International Research Laboratory of Medical Information Processing, Laboratory of Image Science and Technology, and Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education

Sparse-View CT Reconstruction Computed Tomography Sparse-View 4D Reconstruction 3D Gaussian Splatting

Ask PDF

Aug 3, 2026

D^2-4DGS: Dual-Depth Guided Sparse-Camera 4D Gaussian Splatting

Dynamic 4D Gaussian Splatting has emerged as an efficient representation for dynamic novel view synthesis through explicit scene modeling and real-time rendering. However, existing methods typically require dense multi-view videos for sufficient geometric constraints, making capture expensive and limiting sparse-camera deployment. Reducing input views lowers acquisition cost but weakens geometry supervision, often causing missing structures and floating Gaussians. Depth priors provide geometric cues, yet no single source offers both dense coverage and reliable geometry. Monocular depth provides dense structure but is scale-ambiguous and locally biased, whereas multi-view geometric depth provides incomplete anchors consistent with the reconstruction coordinate system. To exploit their complementarity, we propose D

^2

-4DGS, a sparse-camera dynamic 4D Gaussian Splatting framework guided by dual-source depth priors. We align monocular estimates with valid multi-view geometric depths and verify their consistency to identify reliable geometric anchors. These verified anchors support consistency-aware pruning and depth supervision, while verified geometric depths and aligned mono-only estimates provide candidate geometry for densification in under-reconstructed regions. Finally, RGB-D joint optimization improves appearance fidelity and geometric consistency under sparse-view supervision. Across all nine dataset--view settings, D

^2

-4DGS achieves the highest PSNR, improving by 1.33 dB on average over the best competing method in each setting.

Jijian Zhao

Huazhong University of Science and Technology

3D Gaussian Splatting Sparse-View 4D Reconstruction Dynamic Scene Reconstruction Dynamic Novel View Synthesis

Ask PDF

Jul 31, 2026

FillGS: Filling Observation Gaps in 4D Gaussian Splatting via Viewpoint-Time Selection and Generative Refinement

4D Gaussian Splatting (4DGS) can render dynamic scenes photorealistically. However, with limited viewpoint coverage, some spatiotemporal regions remain sparsely observed, leading to artifacts, particularly in scenes with large motion. Existing approaches leveraging generative models rely on heuristic virtual-viewpoint selection before refining rendered views. As a result, they cannot actively explore such sparsely observed regions. To address this issue, we propose a pipeline that actively selects spatiotemporal virtual viewpoints to improve 4DGS reconstruction. Our method selects virtual viewpoints for generative enhancement based on the rendering sensitivity and motion-aware observation density of 4D Gaussians, prioritizing views that alleviate observation sparsity. In the refined images, we filter out regions that conflict with captured observations or are likely to contain generative artifacts and then fine-tune 4DGS using only the reliable regions. We evaluate our method on multi-view video benchmarks using new train/test splits designed to induce observation gaps. Results show consistent improvements over prior viewpoint selection strategies and fine-tuning methods in both qualitative and quantitative evaluations, while reducing artifacts.

Takashi Otonari, Toshihiko Yamasaki

The University of Tokyo, Japan

3D Gaussian Splatting Active Perception 4D Reconstruction Sparse-View 4D Reconstruction Dynamic Scene Reconstruction Active View Selection

Ask PDF

Jul 10, 2026

4D Human-Scene Reconstruction from Low-Overlap Captures

Existing volumetric capture of dynamic human performance achieves high fidelity with dense camera arrays. However, in real-world scenarios, only a handful of low-overlap cameras are available, which degrades the output quality and leaves large areas unobserved. Recent 4D reconstruction methods have focused on low-overlap settings, yet they still produce noticeable artifacts in under-observed regions. Video diffusion models have emerged as another option, but they show geometrically inconsistent results for humans. To address these limitations, we propose StudioRecon, a pipeline that reconstructs 4D human scenes from sparse, low-overlap cameras by decoupling background and humans. We densify background supervision by synthesizing hundreds of camera-controlled novel views with a video diffusion model. We also robustly initialize deformable Gaussian humans with cross-view identity association and triangulated multi-view keypoint fitting. Finally, our recursive enhancement module with motion-adaptive consistency injection harmonizes the composed output, thereby further avoiding remaining artifacts. We achieve state-of-the-art novel view synthesis across four real-world datasets and demonstrate applications such as novel trajectory rendering and human replacement.

Minhyuk Hwang, Sangmin Kim, Seunguk Do +2

Seoul National University, Republic of Korea

4D Reconstruction Dynamic Scene Reconstruction Dynamic Novel View Synthesis Sparse-View 3D Reconstruction Sparse-View 4D Reconstruction

Ask PDF

Jun 29, 2026

FFAvatar: Feed-Forward 4D Head Avatar Reconstruction from Sparse Portrait Images

We present FFAvatar, a Transformer-based 3D Gaussian framework for fast construction of high-quality and animatable 4D head avatars from one or more reference portrait images. Unlike existing feed-forward approaches that require a fixed number of input views, FFAvatar supports incremental reconstruction, progressively refining the avatar representation as additional reference images become available. At the core of our method is an alternating attention mechanism that disentangles identity appearance from expression and viewpoint variations, enabling the reconstruction of a canonical 3D appearance that remains consistent across poses and facial expressions. To balance visual fidelity and computational efficiency, we introduce a sparse-to-dense learning paradigm. Coarse appearance features are first learned using sparse primitives anchored to the FLAME vertex level and are subsequently densified in the UV domain to capture fine-grained geometric and texture details. We further propose a plug-and-play motion refinement module that enables subject-specific dynamic personalization by modeling residual motion beyond parametric deformation. Extensive experiments demonstrate that FFAvatar efficiently produces high-fidelity and controllable 4D head avatars, achieving superior flexibility, driving efficiency, and identity-consistent rendering across diverse expressions and viewpoints.

Jianjiang Yao, Ke Xian, Renxiang Dai +1

School of Electronic Information and Communications, Huazhong University of Science and Technology, Wuhan 430074, China

Feed-Forward 3D Reconstruction 4D Reconstruction 3D Gaussian Splatting 3D Avatar Generation Sparse-View 3D Reconstruction Human Avatar Animation+2

Ask PDF

Apr 9, 2026

Stitch4D: Sparse Multi-Location 4D Urban Reconstruction via Spatio-Temporal Interpolation

Dynamic urban environments are often captured by cameras placed at spatially separated locations with little or no view overlap. However, most existing 4D reconstruction methods assume densely overlapping views and struggle under sparse multi-location observations, producing unstable reconstructions in unobserved intermediate regions. To address this practical yet underexplored setting, we propose Stitch4D, a unified 4D reconstruction framework that compensates for missing spatial coverage in sparsely observed urban environments. Stitch4D synthesizes intermediate bridge views between distant camera locations and jointly optimizes real and synthesized observations in a unified coordinate frame with inter-location consistency constraints. By recovering intermediate spatial coverage before optimization, Stitch4D mitigates geometric collapse and improves reconstruction stability in sparse regions. To evaluate this setting, we introduce Urban Sparse 4D (U-S4D), a controlled CARLA-based benchmark for free-viewpoint reconstruction under sparse multi-location configurations. Experiments on U-S4D show that Stitch4D consistently outperforms representative 4D reconstruction baselines in image-quality metrics. These results suggest that recovering intermediate spatial coverage is an effective strategy for stabilizing 4D reconstruction in sparse urban environments. The project page is provided in https://stitch4d-project-page.vercel.app/.

Hina Kogure, Kei Katsumata, Taiki Miyanishi +1

Keio University, Japan · The University of Tokyo, Japan

4D Reconstruction Sparse-View 4D Reconstruction 3D Reconstruction Dynamic Scene Reconstruction Urban Scene Reconstruction

Ask PDF

Apr 5, 2026

4C4D: 4 Camera 4D Gaussian Splatting

This paper tackles the challenge of recovering 4D dynamic scenes from videos captured by as few as four portable cameras. Learning to model scene dynamics for temporally consistent novel-view rendering is a foundational task in computer graphics, where previous works often require dense multi-view captures using camera arrays of dozens or even hundreds of views. We propose 4C4D, a novel framework that enables high-fidelity 4D Gaussian Splatting from video captures of extremely sparse cameras. Our key insight lies that the geometric learning under sparse settings is substantially more difficult than modeling appearance. Driven by this observation, we introduce a Neural Decaying Function on Gaussian opacities for enhancing the geometric modeling capability of 4D Gaussians. This design mitigates the inherent imbalance between geometry and appearance modeling in 4DGS by encouraging the 4DGS gradients to focus more on geometric learning. Extensive experiments across sparse-view datasets with varying camera overlaps show that 4C4D achieves superior performance over prior art. Project page at: https://junshengzhou.github.io/4C4D.

Junsheng Zhou, Zhifan Yang, Liang Han +4

School of Software, Tsinghua University · Kuaishou Technology

Sparse-View 4D Reconstruction 4D Reconstruction Dynamic Scene Reconstruction 3D Gaussian Splatting

Ask PDF

Mar 27, 2026

SparseCam4D: Spatio-Temporally Consistent 4D Reconstruction from Sparse Cameras

High-quality 4D reconstruction enables photorealistic and immersive rendering of the dynamic real world. However, unlike static scenes that can be fully captured with a single camera, high-quality dynamic scenes typically require dense arrays of tens or even hundreds of synchronized cameras. Dependence on such costly lab setups severely limits practical scalability. To this end, we propose a sparse-camera dynamic reconstruction framework that exploits abundant yet inconsistent generative observations. Our key innovation is the Spatio-Temporal Distortion Field, which provides a unified mechanism for modeling inconsistencies in generative observations across both spatial and temporal dimensions. Building on this, we develop a complete pipeline that enables 4D reconstruction from sparse and uncalibrated camera inputs. We evaluate our method on multi-camera dynamic scene benchmarks, achieving spatio-temporally consistent high-fidelity renderings and significantly outperforming existing approaches. Project page available at https://inspatio.github.io/sparse-cam4d/

Weihong Pan, Xiaoyu Zhang, Zhuang Zhang +4

State Key Lab of CAD&CG, Zhejiang University · InSpatio Research

Sparse-View 4D Reconstruction 4D Reconstruction Dynamic Scene Reconstruction Sparse-View 3D Reconstruction

Ask PDF