PackUV: Packed Gaussian UV Maps for 4D Volumetric Video

Aashish Rai; Angela Xing; Anushka Agarwal; Xiaoyan Cong; Zekun Li; Tao Lu; Aayush Prakash; Srinath Sridhar

cs.CVFeb 26, 2026

PackUV: Packed Gaussian UV Maps for 4D Volumetric Video

Authors: Aashish Rai, Angela Xing, Anushka Agarwal, Xiaoyan Cong, Zekun Li, Tao Lu, Aayush Prakash, Srinath Sridhar

Organizations: Brown University · UMass Amherst · Meta

Abstract

Volumetric videos offer immersive 4D experiences, but remain difficult to reconstruct, store, and stream at scale. Existing Gaussian Splatting based methods achieve high-quality reconstruction but break down on long sequences, temporal inconsistency, and fail under large motions and disocclusions. Moreover, their outputs are typically incompatible with conventional video coding pipelines, preventing practical applications. We introduce PackUV, a novel 4D Gaussian representation that maps all Gaussian attributes into a sequence of structured, multi-scale UV atlas, enabling compact, image-native storage. To fit this representation from multi-view videos, we propose PackUV-GS, a temporally consistent fitting method that directly optimizes Gaussian parameters in the UV domain. A flow-guided Gaussian labeling and video keyframing module identifies dynamic Gaussians, stabilizes static regions, and preserves temporal coherence even under large motions and disocclusions. The resulting UV atlas format is the first unified volumetric video representation compatible with standard video codecs (e.g., FFV1) without losing quality, enabling efficient streaming within existing multimedia infrastructure. To evaluate long-duration volumetric capture, we present PackUV-2B, the largest multi-view video dataset to date, featuring more than 50 synchronized cameras, substantial motion, and frequent disocclusions across 100 sequences and 2B (billion) frames. Extensive experiments demonstrate that our method surpasses existing baselines in rendering fidelity while scaling to sequences up to 30 minutes with consistent quality.

Figures & tables

Figure 2: (Top) Three UV-map organization strategies: (a) naïvely stacking UV layers (deep layers become more and more sparse); (b) a geometric-progression UV pyramid (more uniform sparsity with less storage); (c) PackUV, which packs all pyramid layers into a single UV atlas for efficient, codec-friendly processing. (Bottom) We propose PackUV-GS, a new representation based on 3DGS with a discrete spatial distribution constraint via UV fitting. It uses multiple-layer UV images to store the Gaussian attributes during 3DGS fitting. To constrain the 3D Gaussians located on the discrete rays, we propose a UV-based Adaptive Density Control. We also use a stream-based training schema based on keyframes (image with yellow border).

Figure 3: PackUV-GS vs. baselines for large motion and disocclusion handling. The proposed keyframing and Gaussian labeling strategy effectively manages complex scenarios, such as new objects or people entering a room and dispersing. Zoom to view better.

Dataset	Sequence	Frames	Camera	Resolution	Max FPS	Scenario	View Range
D-NeRF [ 61 ]	8	–	–	$800\times 800$	–	Simulator	360
CMU Panoptic [ 30 ]	65	$\sim 15$ M	31 (+ 480 ∗ )	$1920\times 1080$	30	Real-World	360
N3DV [ 40 ]	6	$\sim 38$ K	21	$2028\times 2704$	30	Real-World	face forward
NeRF-DS [ 94 ]	8	$\sim 8$ K	2	$480\times 270$	–	Real-World	face forward
SelfCap [ 92 ]	3	$\sim 1.5$ M	22	$3840\times 2160$	60	Real-World	face forward
PackUV-2B	100	$\sim$ 2B+	55–88	$1920\times 1200$	90	Real-World	360

Table 1: Dataset Comparisons. We compare our newly captured dataset PackUV-2B with existing multi-view datasets across sequence count, total frames, camera setup, resolution, maximum FPS, scenario type, and view range. PackUV-2B contains 100 diverse sequences totaling over 2B (billion) high-quality frames, recorded with more than 50 cameras at

1920\times 1200

resolution. The capture system supports up to 90 FPS, providing high temporal fidelity.

Figure 4: Optical flow. To assess long-term temporal stability, we compute optical flow between consecutive timestamps.

Method	PSNR ↑	SSIM ↑	LPIPS ↓	Method	PSNR ↑	SSIM ↑	LPIPS ↓
Ours	$27.41$	$0.84$	$0.28$	w/o Keyframe	$20.95$	$0.77$	$0.38$
w/o UV Optim	$23.81$	$0.79$	$0.33$	w/o Labeling	$25.42$	$0.82$	$0.31$
No Atlas	$27.43$	$0.84$	$0.28$	w/o Codec	$27.41$	$0.84$	$0.28$
No LPO	$27.52$	$0.85$	$0.27$	$-$	$-$	$-$	$-$

Table 3: We present quantitative ablation study for various components of our method on PSNR, SSIM, and LPIPS.

Dataset	Motion L.	4DGS	Ex4DGS	3DGStream	GIFStream	Ours
N3DV	32.55	32.01	32.11	31.67	28.42	33.06
DeskGames	30.89	30.11	29.48	30.51	29.94	32.74
TechniColor	31.77	28.91	31.33	25.48	27.42	31.87

Table 4: PSNR comparison with the baselines on N3DV ( flame_salmon ), DeskGames, and Technicolor datasets.

Method	LG	Grid4D	Ex4DGS	3DGStream	GIFStream	Ours
Storage (MB)	950	76	108	204	16	10

Table 5: Storage comparison with the baselines (30 frames).

Figure 5: (Left) Compression evaluation via different methods. (Right) PSNR consistency over time.

Figure 6: Post-optimization lossy UV mapping fails to capture details of a real-world scene even with 48 layers and 1K resolution.

Sequence	Num Sequences	Num Cameras	FPS	Setting	Tags	View Range
Baby Dance	3	88	30	Studio	EN, HI, DO	360
Spot	3	88	30, 90	Studio	RI, HI, OI, DO	360
Volleyball	3	88	30, 90	Studio	SP, HI, LM, DO	360
Kitchen	3	55	30	Non-studio	OI	320
Woodwork	4	55	30	Non-studio	OI	320
Pickleball	30	55	30	Non-studio	SP, HI, LM	300

Table 6: PackUV-2B Dataset Layout. We present our newly captured dataset PackUV-2B sequences . We report number of sequences captured, total timestamps (T), total cameras used to capture the sequence, FPS, setting type, and view range. We also report special tags for each dataset describing the type of activity in the captured sequence. PackUV-2B contains 100 diverse sequences totaling over 2B (billion) high-quality frames, recorded with more than 50 cameras at

1920\times 1200

resolution. The capture system supports up to 90 FPS, providing high temporal fidelity. Tags: RI - Robot Interaction, HI - Human-human Interaction, OI - Object Interaction, SP - Sports, LM - Large Motion, DO - Disocclusion, TR - Transparent/Reflective Objects, EN - Entertainment .

Figure 8: Shows gradient explosion in ATGS training.

Figure 9: Baseline comparison on PackUV-2B’s Kitchen sequence.

Figure 10: Shows baseline comparison on SelfCap [ 91 ] dataset.

Figure 11: Shows baselines comparison on PackUV-2B’s Baby Dance sequence.

Figure 12: Baselines comparison on PackUV-2B’s SPOT sequence.

Explore similar work

Aug 31, 2026

ATGS: Anchored Temporal Gaussian Splatting for Long Volumetric Video Representation

Volumetric video enables immersive free viewpoint rendering of dynamic real world scenes, yet existing methods struggle with long sequences and complex motions, often leading to temporal instability and visual artifacts. To address these challenges, we propose \ourname, a Gaussian splatting based framework for volumetric video reconstruction. Our key insight is that explicitly tracking long term complex motion with individual Gaussian primitives is inherently unstable. Instead, we organize Gaussians around time conditioned anchors that localize their spatial and temporal support, thereby reducing long range motion complexity. We further introduce a temporal windowing strategy to activate only anchors relevant to the queried time, which improves scalability and temporal coherence. In addition, to ensure spatial and temporal stability, we design a compact set of multi level anchor features that encode global features, local spatial features, and local temporal features, jointly constraining Gaussian generation. Extensive experiments demonstrate that \ourname \ consistently outperforms prior methods on long sequence volumetric videos with complex motions. Project page: https://github.com/WuJH2001/ATGS.

Jiahao Wu, Jie Liang, Die Hu +6

Guangdong Provincial Key Laboratory of Ultra High Definition Immersive Media Technology, Shenzhen Graduate School, Peking University, Pengcheng Laboratory, China · Pengcheng Laboratory, China · Shenzhen Graduate School, Peking University, Pengcheng Laboratory, China +1

3D Gaussian Splatting 4D Reconstruction Dynamic Scene Reconstruction

Ask PDF

Apr 1, 2026

TRiGS: Temporal Rigid-Body Motion for Scalable 4D Gaussian Splatting

Recent 4D Gaussian Splatting (4DGS) methods achieve impressive dynamic scene reconstruction but often rely on piecewise linear velocity approximations and short temporal windows. This disjointed modeling leads to severe temporal fragmentation, forcing primitives to be repeatedly eliminated and regenerated to track complex nonlinear dynamics. This makeshift approximation eliminates the long-term temporal identity of objects and causes an inevitable proliferation of Gaussians, hindering scalability to extended video sequences. To address this, we propose TRiGS, a novel 4D representation that utilizes unified, continuous geometric transformations. By integrating

SE(3)

transformations, hierarchical Bezier residuals, and learnable local anchors, TRiGS models geometrically consistent rigid motions for individual primitives. This continuous formulation preserves temporal identity and effectively mitigates unbounded memory growth. Extensive experiments demonstrate that TRiGS achieves high fidelity rendering on standard benchmarks while uniquely scaling to extended video sequences (e.g., 600 to 1200 frames) without severe memory bottlenecks, significantly outperforming prior works in temporal stability.

Suwoong Yeom, Joonsik Nam, Seunggyu Choi +7

Sogang University · POSTECH · Seoul National University +2

4D Reconstruction Dynamic Scene Reconstruction 3D Gaussian Splatting Motion Representation Learning

Ask PDF

Oct 7, 2026

DynStream: Online Streaming 4D Gaussian Reconstruction of Dynamic Worlds from Unposed Video

Online reconstruction of dynamic 4D scenes from long, unposed streaming videos requires both continuous processing and photorealistic rendering, which existing methods struggle to achieve simultaneously. Existing feed-forward Gaussian methods are restricted to offline processing, whereas online point-cloud approaches struggle to maintain dense geometry and high-fidelity rendering. We present DynStream, a framework for streaming 4D Gaussian reconstruction from long, unposed videos. Given a continuous video stream, DynStream reconstructs the scene within local temporal windows and incrementally aligns and fuses these local reconstructions into a globally consistent scene, enabling online 4D reconstruction without per-scene optimization. By jointly enforcing cross-window geometric consistency and modeling time-varying scene content, DynStream supports efficient reconstruction and photorealistic rendering over extended video streams. Experiments demonstrate that DynStream enables high-fidelity online dynamic reconstruction and rendering from long video streams, achieving state-of-the-art performance across diverse dynamic indoor and outdoor scenes.

Dingwei Xian, Xiaoyu Zhou, Yajiao Xiong +2

Wangxuan Institute of Computer Technology, Peking University · VGI Labs Co., Ltd. · University of California, Merced

4D Reconstruction Dynamic Scene Reconstruction Streaming 3D Reconstruction 3D Gaussian Splatting

Ask PDF