Time-Archival Camera Virtualization for Sports and Visual Performances

Yunxiao Zhang; William Stone; Suryansh Kumar

cs.CVFeb 16, 2026

Time-Archival Camera Virtualization for Sports and Visual Performances

Authors: Yunxiao Zhang, William Stone, Suryansh Kumar

Abstract

Camera virtualization -- an emerging solution to novel view synthesis -- holds transformative potential for visual entertainment, live performances, and sports broadcasting by enabling the generation of photorealistic images from novel viewpoints using images from a limited set of calibrated multiple static physical cameras. Despite recent advances, achieving spatially and temporally coherent and photorealistic rendering of dynamic scenes with efficient time-archival capabilities, particularly in fast-paced sports and stage performances, remains challenging for existing approaches. Recent methods based on 3D Gaussian Splatting (3DGS) for dynamic scenes could offer real-time view-synthesis results. Yet, they are hindered by their dependence on accurate 3D point clouds from the structure-from-motion method and their inability to handle large, non-rigid, rapid motions of different subjects (e.g., flips, jumps, articulations, sudden player-to-player transitions). Moreover, independent motions of multiple subjects can break the Gaussian-tracking assumptions commonly used in 4DGS, ST-GS, and other dynamic splatting variants. This paper advocates reconsidering a neural volume rendering formulation for camera virtualization and efficient time-archival capabilities, making it useful for sports broadcasting and related applications. By modeling a dynamic scene as rigid transformations across multiple synchronized camera views at a given time, our method performs neural representation learning, providing enhanced visual rendering quality at test time. A key contribution of our approach is its support for time-archival, i.e., users can revisit any past temporal instance of a dynamic scene and can perform novel view synthesis, enabling retrospective rendering for replay, analysis, and archival of live events, a functionality absent in existing neural rendering approaches and novel view synthesis...

Figures & tables

Figure 2: Quantitative comparison of the image rendering quality on the proposed dynamic scene dataset with state.

Figure 3: Overall setup for our application. A typical multiview synchronized camera setup installed in a sports scene for broadcasting (shown in black). The virtual cameras to inspect the scene are shown in red color for time-archival or broadcast from users interest view points. a)-b)Top-row : A typical football sports scene and image-rendering from virtual cameras. c)-d)Bottom-row : A typical tennis sports match scene and respective image-rendering from virtual camera view points.

Figure 4: Visual Performance Qualitative Comparison Results with 4DGS approach [ 3 ] on our synthetic multiview dataset. Left: The four camera frustum highlighted in red shows the virtual cameras that will be used for dynamic scene broadcasting. Right: Our rendered image results from those virtual cameras at a given time as compared to 4D-GS [ 3 ] approach. We also provide the PSNR and LPIPS values for quantitative comparison. Here, VC denotes corresponding virtual camera.

Method $\rightarrow$	D-NeRF [ 72 ]	D-3DGS [ 86 ]	4DGS [ 3 ]	ST-GS [ 4 ]	Ours
Dancing-Walking-Standing
PSNR $\uparrow$	6.44	18.45	28.17	20.03	34.28
LPIPS $\downarrow$	0.572	0.139	0.08	0.112	0.027
Soccer Penalty Kick
PSNR $\uparrow$	10.64	26.45	26.25	25.99	33.81
LPIPS $\downarrow$	0.407	0.071	0.045	0.077	0.028

Table 1: Quantitative comparison of our approach with state-of-the-art approaches on synthetic dataset. The best results are shaded with green and blue for PSNR and LPIPS metric, respectively

Figure 5: Results on CMU Panoptic dataset [ 103 ] . Left: Multiview Camera setup. Actual cameras are shown in black, where as virtual cameras are highlighted with red . Right: Results using our approach on a couple of challenging sports sequence. Here, VC denotes virtual camera.

Method $\rightarrow$	D-NeRF [ 72 ]	D-3DGS [ 86 ]	4DGS [ 3 ]	ST-GS [ 4 ]	Ours
Baseball Bat (CMU Panoptic [ 103 , 86 ] + SAM [ 104 ] )
PSNR $\uparrow$	6.35	$\clubsuit$	$\clubsuit$	$\clubsuit$	29.43
LPIPS $\downarrow$	0.605	$\clubsuit$	$\clubsuit$	$\clubsuit$	0.066
HandGesture (CMU Panoptic [ 103 , 86 ] + SAM [ 104 ] )
PSNR $\uparrow$	12.99	$\clubsuit$	$\clubsuit$	$\clubsuit$	29.19
LPIPS $\downarrow$	0.135	$\clubsuit$	$\clubsuit$	$\clubsuit$	0.050

Table 2: Quantitative comparison of our approach with state-of-the-art approaches on CMU Panoptic dataset.

\clubsuit

symbolizes that the corresponding method fail to give results. Note that D-3DGS [ 86 ] uses active sensor depth data, which is not present in our experimental setting.

Approach/Evaluation Metric	PSNR	LPIPS	Model (per time step)	Input PointCloud (per time step)
3DGS (GT point cloud)	36.47	0.0916	77 MB	$\sim$ 6.2 GB
3DGS (random point cloud)	16.33	0.3761	91 MB	$\sim$ 6.2 GB
Ours (no point cloud)	34.28	0.0255	48.8 MB	0.0

Table 4: Quantitative comparison of our approach with 3DGS for single time instance. Without any ground truth 3d point cloud priors, our approach provides excellent results with significantly low-memory requirement. On the contrary, if we use 3DGS approach here instead of the proposed implicit approach for each time step, we need to maintain a persistent dense folder with 3d data by possibly running COLMAP for each time step (due to dynamic scene setup). For the Dance-Walking-Standing dataset, this folder occupies nearly 6.2 GB to achieve the results mentioned above.

Method	Train Time	Parallelizable (in Time)	PSNR	LPIPS
4DGS [ 3 ]	$\sim 0.30-3.0$ hours	No (sequential)	$\sim$ 26.2 - 28.1	$\sim$ 0.045 - 0.061
ST-GS [ 4 ]	$\sim 0.46-0.71$ hours	No (sequential)	$\sim$ 20.0 - 25.9	$\sim$ 0.078 - 0.112
D-3DGS [ 86 ]	$\sim 1.40$ hours	No (Limited)	$\sim$ 18.5 - 26.5	$\sim$ 0.071 - 0.139
Ours	$\sim$ 5.65 - 8.90 hours	Yes (Fully)	$\sim$ 31.9 - 34.3	$\sim$ 0.027 - 0.039

Table 5: Our per-time-step radiance fields achieve the highest reconstruction quality (PSNR, LPIPS) while offering fully parallelizable training across time, unlike sequential 4DGS, ST-GS, and D-3DGS based approaches. Despite a higher per-sequence training cost on a single GPU, full parallelization makes our method significantly more scalable for long-horizon time-archival in sports and visual-performance applications.

Method $\rightarrow$	D-3DGS [ 86 ]	4DGS [ 3 ]	ST-GS [ 4 ]	D-NeRF [ 72 ]	T-4D [ 63 ]	HP [ 65 ]	KP [ 64 ]	S-RF [ 59 ]	Ours
DWS
PSNR	18.45	28.17	20.03	6.44	16.55	15.82	16.40	18.87	34.28
LPIPS	0.1396	0.0800	0.1120	0.5726	0.2165	0.2470	0.2327	0.2054	0.0275
Memory	2.0MB	21.0MB	3.2MB	512k	280MB	68MB	419MB	2.2GB	3.1GB
Train Time	1.41h	0.83h	0.46h	4.25h	10.02h	0.96h	1.02h	0.65h	5.25h
Iterations	130K	17K	30K	20K	200K	25K	30K	80K	19K

Table 6: Comparison of implicit and explicit dynamic-scene representations on three synthetic datasets. Our per-time-step implicit radiance fields achieve the highest PSNR/LPIPS performance while maintaining a compact memory footprint, outperforming 3DGS- and NeRF-based methods that rely on explicit geometry or deformation tracking. This makes our approach substantially more scalable and robust for long-horizon time-archival in dynamic sports and performance scenes. Note: here we have not included the initial 3d point cloud size that is used by 3DGS based approaches as input . DWS , S-PK , and S-MP stand for Dance-Walking-Standing, Soccer Penalty-Kick, and Soccer Multi-Player dataset.

Figure 7: Qualitative results demonstrating dynamic scene view synthesis from novel viewpoints on our proposed Dancing-Walking-Standing dataset. Left: shows the acquisition setup from top view and side view, while the novel views are shown in red. From Top row to Bottom row : Image-based rendering result with the state-of-the-art neural implicit methods for dynamic scenes. The red numbers in each row indicate the camera-id used by the methods to render the scene. Our approach generates images closer to the ground truth than others.

Figure 8: Additional qualitative results showing results using different approaches on Dancing-Walking-Standing dataset.

Figure 9: Per-frame PSNR variation, illustrating the sensitivity of 3DGS-based approaches to the quality of the reference 3D point cloud (warm start).

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Figure 10: Fibonacci sampling on a hemisphere of radius $R$ for camera placement. The $i$ -th camera center $\mathbf{p}_{i}$ (dark-blue solid marker) has vertical coordinate $z_{i}$ , in-plane radius $r_{i}=\sqrt{R^{2}-z_{i}^{2}}$ , and azimuth $\theta_{i}$ . The next center $\mathbf{p}_{i+1}$ (green solid marker) is rotated by the golden angle $\phi$ about the vertical axis and lies at height $z_{i+1}\!\approx\!z_{i}$ .

Dynamic Novel View Synthesis

Dynamic novel view synthesis creates time-consistent images or videos from unseen camera positions in scenes that change over time.

See all 41 papers

May 12, 2026

3D Gaussian Splatting for Efficient Retrospective Dynamic Scene Novel View Synthesis with a Standardized Benchmark

Retrospective novel view synthesis (NVS) of dynamic scenes is fundamental to applications such as sports. Recent dynamic 3D Gaussian Splatting (3DGS) approaches introduce temporally coupled formulations to enforce motion coherence across time. In this paper, we argue that, in a synchronized multi-view (MV) setting typical of sports, the dynamic scene at each time step is already strongly geometrically constrained. We posit that the availability of calibrated, synchronized viewpoints provides sufficient spatial consistency, and therefore, explicit temporal coupling, or complex multi-body constraints seems unnecessary for retrospective NVS. To this end, we propose an approach tailored for synchronized MV dynamic scene. By initializing the SfM-derived point cloud at the start time and propagating optimized Gaussians over time, we show that efficient retrospective NVS can be achieved without imposing a temporal deformation constraint. Complementing our methodological contribution, we introduce a Dynamic MV dataset framework built on Blender for reproducible NeRF and 3DGS research. The framework generates high-quality, synchronized camera rigs and exports training-ready datasets in standard formats, eliminating inconsistencies in coordinate conventions and data pipelines. Using the framework, we construct a dynamic benchmark suite and evaluate representative NeRF and 3DGS approaches under controlled conditions. Together, we show that, under a synchronized MV setup, efficient retrospective dynamic scene NVS can be achieved using 3DGS. At the same time, the dataset-generation framework enables reproducible and principled benchmarking of dynamic NVS methods.

Yunxiao Zhang, Suryansh Kumar

Visual and Spatial AI Lab1, VCCM Section College of PVFA2, Department of ECEN3, Department of CSCE4, Texas A&M University, College Station, Texas, USA

3D Gaussian Splatting Multi-View 3D Reconstruction Dynamic Novel View Synthesis

Ask PDF

Mar 27, 2026

SparseCam4D: Spatio-Temporally Consistent 4D Reconstruction from Sparse Cameras

High-quality 4D reconstruction enables photorealistic and immersive rendering of the dynamic real world. However, unlike static scenes that can be fully captured with a single camera, high-quality dynamic scenes typically require dense arrays of tens or even hundreds of synchronized cameras. Dependence on such costly lab setups severely limits practical scalability. To this end, we propose a sparse-camera dynamic reconstruction framework that exploits abundant yet inconsistent generative observations. Our key innovation is the Spatio-Temporal Distortion Field, which provides a unified mechanism for modeling inconsistencies in generative observations across both spatial and temporal dimensions. Building on this, we develop a complete pipeline that enables 4D reconstruction from sparse and uncalibrated camera inputs. We evaluate our method on multi-camera dynamic scene benchmarks, achieving spatio-temporally consistent high-fidelity renderings and significantly outperforming existing approaches. Project page available at https://inspatio.github.io/sparse-cam4d/

Weihong Pan, Xiaoyu Zhang, Zhuang Zhang +4

State Key Lab of CAD&CG, Zhejiang University · InSpatio Research

Sparse-View 4D Reconstruction 4D Reconstruction Dynamic Scene Reconstruction Sparse-View 3D Reconstruction

Ask PDF

Sep 14, 2026

Tele360: Real-Time Feed-Forward Human Reconstruction from Sparse Unposed Cameras

Live free-viewpoint visualization of real humans is critical for immersive communication and interactive digital experiences. Existing methods either rely on computationally expensive optimization or require calibrated cameras and low-resolution inputs, making real-time high-resolution deployment impractical. In this work, we present Tele360, the first real-time feed-forward system for dynamic human reconstruction and live free-viewpoint visualization from sparse, unposed RGB streams. Our system jointly estimates camera poses and reconstructs a dynamic 3D Gaussian representation for each time instance in a single forward pass. To achieve this, we start by designing a lightweight sparsity-aware multi-view transformer backbone that tokenizes foreground human regions while preserving global context through a shared scene token. We then employ a fully transformer-based Gaussian decoder to mitigate convolution-induced over-smoothing while keeping decoding sparse and efficient. In addition, we introduce a hybrid feature pyramid that injects multi-scale appearance cues into geometry prediction. We further introduce a lightweight differentiable Levenberg-Marquardt camera refinement layer to enhance multi-view consistency and geometric alignment. Moreover, to stabilize learning under sparse, unposed inputs, we transfer multi-view geometry priors from a large visual-geometry foundation model via teacher-student distillation. Finally, the predicted Gaussian maps are streamed with video codecs to remote devices for interactive free-viewpoint rendering. Extensive experiments show that Tele360 achieves state-of-the-art visual quality on studio benchmarks while supporting real-time 2K input-to-rendering at over 25 FPS on a single consumer GPU. Additional captured sequences illustrate its performance across varied subjects, clothing, and motions under our multi-camera setup.

Hanzhang Tu, Zhanfeng Liao, Wei Min +2

Tsinghua University, Beijing, China · Shadow AI, Beijing, China

3D Gaussian Splatting Camera Pose Estimation Dynamic Scene Reconstruction Sparse-View 3D Reconstruction 3D Human Reconstruction Feed-Forward 3D Reconstruction

Ask PDF

Abstract

Dynamic Novel View Synthesis

Explore similar work

3D Gaussian Splatting for Efficient Retrospective Dynamic Scene Novel View Synthesis with a Standardized Benchmark

SparseCam4D: Spatio-Temporally Consistent 4D Reconstruction from Sparse Cameras

Tele360: Real-Time Feed-Forward Human Reconstruction from Sparse Unposed Cameras