Time-Archival Camera Virtualization for Sports and Visual Performances

Yunxiao Zhang; William Stone; Suryansh Kumar

cs.CVFeb 16, 2026

Time-Archival Camera Virtualization for Sports and Visual Performances

Authors: Yunxiao Zhang, William Stone, Suryansh Kumar

Abstract

Camera virtualization -- an emerging solution to novel view synthesis -- holds transformative potential for visual entertainment, live performances, and sports broadcasting by enabling the generation of photorealistic images from novel viewpoints using images from a limited set of calibrated multiple static physical cameras. Despite recent advances, achieving spatially and temporally coherent and photorealistic rendering of dynamic scenes with efficient time-archival capabilities, particularly in fast-paced sports and stage performances, remains challenging for existing approaches. Recent methods based on 3D Gaussian Splatting (3DGS) for dynamic scenes could offer real-time view-synthesis results. Yet, they are hindered by their dependence on accurate 3D point clouds from the structure-from-motion method and their inability to handle large, non-rigid, rapid motions of different subjects (e.g., flips, jumps, articulations, sudden player-to-player transitions). Moreover, independent motions of multiple subjects can break the Gaussian-tracking assumptions commonly used in 4DGS, ST-GS, and other dynamic splatting variants. This paper advocates reconsidering a neural volume rendering formulation for camera virtualization and efficient time-archival capabilities, making it useful for sports broadcasting and related applications. By modeling a dynamic scene as rigid transformations across multiple synchronized camera views at a given time, our method performs neural representation learning, providing enhanced visual rendering quality at test time. A key contribution of our approach is its support for time-archival, i.e., users can revisit any past temporal instance of a dynamic scene and can perform novel view synthesis, enabling retrospective rendering for replay, analysis, and archival of live events, a functionality absent in existing neural rendering approaches and novel view synthesis...

Figures & tables

Figure 2: Quantitative comparison of the image rendering quality on the proposed dynamic scene dataset with state.

Figure 3: Overall setup for our application. A typical multiview synchronized camera setup installed in a sports scene for broadcasting (shown in black). The virtual cameras to inspect the scene are shown in red color for time-archival or broadcast from users interest view points. a)-b)Top-row : A typical football sports scene and image-rendering from virtual cameras. c)-d)Bottom-row : A typical tennis sports match scene and respective image-rendering from virtual camera view points.

Figure 4: Visual Performance Qualitative Comparison Results with 4DGS approach [ 3 ] on our synthetic multiview dataset. Left: The four camera frustum highlighted in red shows the virtual cameras that will be used for dynamic scene broadcasting. Right: Our rendered image results from those virtual cameras at a given time as compared to 4D-GS [ 3 ] approach. We also provide the PSNR and LPIPS values for quantitative comparison. Here, VC denotes corresponding virtual camera.

Method $\rightarrow$	D-NeRF [ 72 ]	D-3DGS [ 86 ]	4DGS [ 3 ]	ST-GS [ 4 ]	Ours
Dancing-Walking-Standing
PSNR $\uparrow$	6.44	18.45	28.17	20.03	34.28
LPIPS $\downarrow$	0.572	0.139	0.08	0.112	0.027
Soccer Penalty Kick
PSNR $\uparrow$	10.64	26.45	26.25	25.99	33.81
LPIPS $\downarrow$	0.407	0.071	0.045	0.077	0.028

Table 1: Quantitative comparison of our approach with state-of-the-art approaches on synthetic dataset. The best results are shaded with green and blue for PSNR and LPIPS metric, respectively

Figure 5: Results on CMU Panoptic dataset [ 103 ] . Left: Multiview Camera setup. Actual cameras are shown in black, where as virtual cameras are highlighted with red . Right: Results using our approach on a couple of challenging sports sequence. Here, VC denotes virtual camera.

Method $\rightarrow$	D-NeRF [ 72 ]	D-3DGS [ 86 ]	4DGS [ 3 ]	ST-GS [ 4 ]	Ours
Baseball Bat (CMU Panoptic [ 103 , 86 ] + SAM [ 104 ] )
PSNR $\uparrow$	6.35	$\clubsuit$	$\clubsuit$	$\clubsuit$	29.43
LPIPS $\downarrow$	0.605	$\clubsuit$	$\clubsuit$	$\clubsuit$	0.066
HandGesture (CMU Panoptic [ 103 , 86 ] + SAM [ 104 ] )
PSNR $\uparrow$	12.99	$\clubsuit$	$\clubsuit$	$\clubsuit$	29.19
LPIPS $\downarrow$	0.135	$\clubsuit$	$\clubsuit$	$\clubsuit$	0.050

Table 2: Quantitative comparison of our approach with state-of-the-art approaches on CMU Panoptic dataset.

\clubsuit

symbolizes that the corresponding method fail to give results. Note that D-3DGS [ 86 ] uses active sensor depth data, which is not present in our experimental setting.

Approach/Evaluation Metric	PSNR	LPIPS	Model (per time step)	Input PointCloud (per time step)
3DGS (GT point cloud)	36.47	0.0916	77 MB	$\sim$ 6.2 GB
3DGS (random point cloud)	16.33	0.3761	91 MB	$\sim$ 6.2 GB
Ours (no point cloud)	34.28	0.0255	48.8 MB	0.0

Table 4: Quantitative comparison of our approach with 3DGS for single time instance. Without any ground truth 3d point cloud priors, our approach provides excellent results with significantly low-memory requirement. On the contrary, if we use 3DGS approach here instead of the proposed implicit approach for each time step, we need to maintain a persistent dense folder with 3d data by possibly running COLMAP for each time step (due to dynamic scene setup). For the Dance-Walking-Standing dataset, this folder occupies nearly 6.2 GB to achieve the results mentioned above.

Method	Train Time	Parallelizable (in Time)	PSNR	LPIPS
4DGS [ 3 ]	$\sim 0.30-3.0$ hours	No (sequential)	$\sim$ 26.2 - 28.1	$\sim$ 0.045 - 0.061
ST-GS [ 4 ]	$\sim 0.46-0.71$ hours	No (sequential)	$\sim$ 20.0 - 25.9	$\sim$ 0.078 - 0.112
D-3DGS [ 86 ]	$\sim 1.40$ hours	No (Limited)	$\sim$ 18.5 - 26.5	$\sim$ 0.071 - 0.139
Ours	$\sim$ 5.65 - 8.90 hours	Yes (Fully)	$\sim$ 31.9 - 34.3	$\sim$ 0.027 - 0.039

Table 5: Our per-time-step radiance fields achieve the highest reconstruction quality (PSNR, LPIPS) while offering fully parallelizable training across time, unlike sequential 4DGS, ST-GS, and D-3DGS based approaches. Despite a higher per-sequence training cost on a single GPU, full parallelization makes our method significantly more scalable for long-horizon time-archival in sports and visual-performance applications.

Method $\rightarrow$	D-3DGS [ 86 ]	4DGS [ 3 ]	ST-GS [ 4 ]	D-NeRF [ 72 ]	T-4D [ 63 ]	HP [ 65 ]	KP [ 64 ]	S-RF [ 59 ]	Ours
DWS
PSNR	18.45	28.17	20.03	6.44	16.55	15.82	16.40	18.87	34.28
LPIPS	0.1396	0.0800	0.1120	0.5726	0.2165	0.2470	0.2327	0.2054	0.0275
Memory	2.0MB	21.0MB	3.2MB	512k	280MB	68MB	419MB	2.2GB	3.1GB
Train Time	1.41h	0.83h	0.46h	4.25h	10.02h	0.96h	1.02h	0.65h	5.25h
Iterations	130K	17K	30K	20K	200K	25K	30K	80K	19K

Table 6: Comparison of implicit and explicit dynamic-scene representations on three synthetic datasets. Our per-time-step implicit radiance fields achieve the highest PSNR/LPIPS performance while maintaining a compact memory footprint, outperforming 3DGS- and NeRF-based methods that rely on explicit geometry or deformation tracking. This makes our approach substantially more scalable and robust for long-horizon time-archival in dynamic sports and performance scenes. Note: here we have not included the initial 3d point cloud size that is used by 3DGS based approaches as input . DWS , S-PK , and S-MP stand for Dance-Walking-Standing, Soccer Penalty-Kick, and Soccer Multi-Player dataset.

Figure 7: Qualitative results demonstrating dynamic scene view synthesis from novel viewpoints on our proposed Dancing-Walking-Standing dataset. Left: shows the acquisition setup from top view and side view, while the novel views are shown in red. From Top row to Bottom row : Image-based rendering result with the state-of-the-art neural implicit methods for dynamic scenes. The red numbers in each row indicate the camera-id used by the methods to render the scene. Our approach generates images closer to the ground truth than others.

Figure 8: Additional qualitative results showing results using different approaches on Dancing-Walking-Standing dataset.

Figure 9: Per-frame PSNR variation, illustrating the sensitivity of 3DGS-based approaches to the quality of the reference 3D point cloud (warm start).

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Figure 10: Fibonacci sampling on a hemisphere of radius $R$ for camera placement. The $i$ -th camera center $\mathbf{p}_{i}$ (dark-blue solid marker) has vertical coordinate $z_{i}$ , in-plane radius $r_{i}=\sqrt{R^{2}-z_{i}^{2}}$ , and azimuth $\theta_{i}$ . The next center $\mathbf{p}_{i+1}$ (green solid marker) is rotated by the golden angle $\phi$ about the vertical axis and lies at height $z_{i+1}\!\approx\!z_{i}$ .

Dynamic Novel View Synthesis

Dynamic novel view synthesis creates time-consistent images or videos from unseen camera positions in scenes that change over time.

See all 41 papers

3D Gaussian Splatting for Efficient Retrospective Dynamic Scene Novel View Synthesis with a Standardized Benchmark
May 12, 2026Yunxiao Zhang, Suryansh Kumar3D Gaussian Splatting Multi-View 3D Reconstruction
Visual and Spatial AI Lab1, VCCM Section College of PVFA2, Department of ECEN3, Department of CSCE4, Texas A&M University, College Station, Texas, USA
SparseCam4D: Spatio-Temporally Consistent 4D Reconstruction from Sparse Cameras
Mar 27, 2026Weihong Pan, Xiaoyu Zhang, Zhuang Zhang +4Sparse-View 4D Reconstruction 4D Reconstruction
State Key Lab of CAD&CG, Zhejiang University · InSpatio Research
Tele360: Real-Time Feed-Forward Human Reconstruction from Sparse Unposed Cameras
Sep 14, 2026Hanzhang Tu, Zhanfeng Liao, Wei Min +23D Gaussian Splatting Camera Pose Estimation
Tsinghua University, Beijing, China · Shadow AI, Beijing, China

May 12, 2026J/K move · Enter open · S save

3D Gaussian Splatting for Efficient Retrospective Dynamic Scene Novel View Synthesis with a Standardized Benchmark

Yunxiao Zhang, Suryansh Kumar

Visual and Spatial AI Lab1, VCCM Section College of PVFA2, Department of ECEN3, Department of CSCE4, Texas A&M University, College Station, Texas, USA

3D Gaussian Splatting Multi-View 3D Reconstruction Dynamic Novel View Synthesis

Open paper AskPDF arXiv

Retrospective novel view synthesis (NVS) of dynamic scenes is fundamental to applications such as sports. Recent dynamic 3D Gaussian Splatting (3DGS) approaches introduce temporally coupled formulations to enforce motion coherence across time. In this paper, we argue that, in a synchronized multi-view (MV) setting typical of sports, the dynamic scene at each time step is already strongly geometrically constrained. We posit that the availability of calibrated, synchronized viewpoints provides sufficient spatial consistency, and therefore, explicit temporal coupling, or complex multi-body constraints seems unnecessary for retrospective NVS. To this end, we propose an approach tailored for synchronized MV dynamic scene. By initializing the SfM-derived point cloud at the start time and propagating optimized Gaussians over time, we show that efficient retrospective NVS can be achieved without imposing a temporal deformation constraint. Complementing our methodological contribution, we introduce a Dynamic MV dataset framework built on Blender for reproducible NeRF and 3DGS research. The framework generates high-quality, synchronized camera rigs and exports training-ready datasets in standard formats, eliminating inconsistencies in coordinate conventions and data pipelines. Using the framework, we construct a dynamic benchmark suite and evaluate representative NeRF and 3DGS approaches under controlled conditions. Together, we show that, under a synchronized MV setup, efficient retrospective dynamic scene NVS can be achieved using 3DGS. At the same time, the dataset-generation framework enables reproducible and principled benchmarking of dynamic NVS methods.

Abstract

Dynamic Novel View Synthesis

Explore similar work

3D Gaussian Splatting for Efficient Retrospective Dynamic Scene Novel View Synthesis with a Standardized Benchmark

SparseCam4D: Spatio-Temporally Consistent 4D Reconstruction from Sparse Cameras

Tele360: Real-Time Feed-Forward Human Reconstruction from Sparse Unposed Cameras