PocketGS: High-Fidelity On-Device Training for 3D Gaussian Splatting

Wenzhi Guo; Guangchi Fang; Shu Yang; Bing Wang

cs.CVJan 24, 2026

PocketGS: High-Fidelity On-Device Training for 3D Gaussian Splatting

Authors: Wenzhi Guo, Guangchi Fang, Shu Yang, Bing Wang

Abstract

Although 3D Gaussian Splatting (3DGS) supports real-time rendering, training relies on offline geometry, long-horizon densification, and memory-intensive differentiation. We cast post-capture static-scene training as resource-constrained finite-horizon optimization, where terminal quality depends jointly on the initial representation and reference-consistent gradient execution. To the best of our knowledge, PocketGS is the first publicly documented pipeline to complete geometry construction, Gaussian initialization, and 3DGS optimization on a commodity smartphone. Its I, G warm start combines refined poses, a filtered dense prior, and surface-aligned anisotropic Gaussians. Its T operator replays compositing interactions and scatters gradients to canonical parameters, matching a numerical reference at the reported precision. Controlled 500-step experiments on LLFF, Mip-NeRF 360, and MobileScan show high-quality reconstruction at the device horizon. Across MobileScan, every scene remains below 3 GB peak memory and post-capture processing averages 255.2 s, enabling an on-device capture-to-render workflow. Separate 30k experiments characterize native-schedule behavior.

Figures & tables

Figure 2: Overview of the PocketGS framework. PocketGS tackles on-device 3DGS training through three coupled operators: ( $\mathcal{G}$ ) Geometry Prior Construction combines information-gated frame selection, GPU-native Schur-complement BA, and single-reference MVS to construct a dense geometric prior. ( $\mathcal{I}$ ) Prior-Conditioned Parameterization seeds anisotropic Gaussians by estimating local surface statistics (normals $\mathbf{n}_{i}$ and scales) to front-load structure discovery via disc-like covariance seeding. ( $\mathcal{T}$ ) Hardware-Aligned Splatting implements a mobile-native differentiable renderer using unrolled alpha-compositing ( $S={id,C_{in},\alpha}$ ) and index-mapped gradient scattering for reference-consistent gradient execution at the reported precision within the mobile memory bounds of the canonical parameter buffer ( $\mu,\Sigma,c,\alpha$ ) during end-to-end training.

Method	LLFF					Mip-NeRF 360					MobileScan
Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	Time(s)	Count	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	Time(s)	Count	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	Time(s)	Count
3DGS-SFM-WK	23.28	0.789	0.226	116.3	18k	21.05	0.649	0.377	398.7	46k	22.64	0.781	0.319	123.8	23k
3DGS-MVS-WK	21.04	0.730	0.262	320.3	41k	18.26	0.586	0.429	975.6	76k	22.23	0.834	0.226	547.5	165k
Mobile-GS	23.26	0.790	0.227	133.1	18k	21.06	0.650	0.377	405.2	46k	22.61	0.782	0.321	145.1	23k
DashGaussian	23.30	0.790	0.224	115.9	18k	21.54	0.672	0.351	399.9	46k	22.66	0.783	0.317	122.5	23k
Taming-3DGS	23.27	0.788	0.228	117.6	18k	21.09	0.651	0.375	402.6	46k	22.62	0.780	0.322	125.1	23k

Table 1: Reconstruction at a controlled 500-step horizon. Dataset-level averages on LLFF, Mip-NeRF 360, and MobileScan with densification and pruning disabled for every variant. The experiment measures terminal quality at

K=500

from each method’s preprocessing and initialization pipeline, before adaptive density control. Completed native-schedule behavior is evaluated separately. Time reports per-system post-capture processing latency on the stated hardware. Higher is better for PSNR/SSIM, and lower is better for LPIPS. Red , orange , and yellow denote the best, second-best, and third-best results.

Figure 3: MobileScan reconstruction after 500 optimization steps. PocketGS better preserves textures and thin structures; generic MVS remains biased toward textured regions, leaving low-texture and specular areas under-sampled at initialization.

Method	Step	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
3DGS-SFM-WK	30k	25.84	0.847	0.122
	15k	25.71	0.845	0.127
	7k	25.38	0.832	0.146
	3k	24.44	0.800	0.181
	1k	22.83	0.719	0.287
PocketGS	500	22.25	0.702	0.302

Table 2: Early convergence on Mip-NeRF 360. PocketGS’s controlled 500-step endpoint versus the 1k–30k native-schedule checkpoints of 3DGS-SFM-WK; complete native trajectories are reported in the supplement.

Method	On-device Train	FPS $\uparrow$
MobileNeRF ( Chen et al. 2023 )	$\times$	31.40
MetaSapiens ( Lin et al. 2025 )	$\times$	49.24
Mobile-GS ( Du et al. 2026 )	$\times$	62.56
PocketGS (Ours)	✓	52.32

Table 3: Smartphone rendering. Baselines are rendering-only references under their reported setups; PocketGS is measured on the tested smartphone and trains fully on-device.

Variant	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	Time $\downarrow$ (s)
PocketGS (Full)	23.67	0.791	0.225	255.2
w/o Global BA	23.54	0.760	0.227	251.8
w/o MVS	21.15	0.667	0.400	127.1
w/o Init. $\mathcal{I}$	22.49	0.778	0.249	317.0

Table 4: MobileScan ablation. Averages over 16 scenes under the matched 500-step protocol; the w/o MVS runtime reflects its smaller initial primitive count.

3D Gaussian Splatting

3D Gaussian Splatting represents scenes as optimizable 3D Gaussian primitives for fast, photorealistic rendering from new viewpoints.

See all 1,019 papers

TideGS: Scalable Training of Over One Billion 3D Gaussian Splatting Primitives via Out-of-Core Optimization
May 19, 2026Chonghao Zhong, Linfeng Shi, Hua Chen +43D Gaussian Splatting 3D Scene Representation
Hong Kong University of Science and Technology · Great Wall Motor · Tsinghua University +1
PocketSplat: Mobile Gaussian Reconstruction via World-Space Latent Allocatio
Oct 2, 2026Wenzhi Guo, Xianda Chen, Dongxuan Chen +23D Gaussian Splatting 3D Reconstruction
BoundedGS: Memory-Efficient High-Resolution Training for 3D Gaussian Splatting
Oct 25, 2026Xiaobin Deng, Changyu Diao, Min Li +13D Gaussian Splatting 3DGS Training
Zhejiang University

May 19, 2026J/K move · Enter open · S save

TideGS: Scalable Training of Over One Billion 3D Gaussian Splatting Primitives via Out-of-Core Optimization

Chonghao Zhong, Linfeng Shi, Hua Chen, Tiecheng Sun+3

Hong Kong University of Science and Technology · Great Wall Motor · Tsinghua University+1

3D Gaussian Splatting 3D Scene Representation 3DGS Training

Open paper AskPDF arXiv

Training 3D Gaussian Splatting (3DGS) at billion-primitive scale is fundamentally memory-bound: each Gaussian primitive carries a large attribute vector, and the aggregate parameter table quickly exceeds GPU capacity, limiting prior systems to tens of millions of Gaussians on commodity single-GPU hardware. We observe that 3DGS training is inherently sparse and trajectory-conditioned: each iteration activates only the Gaussians visible from the current camera batch, so GPU memory can serve as a working-set cache rather than a persistent parameter store. Building on this insight, we introduce TideGS, an out-of-core training framework that manages parameters across an SSD-CPU-GPU hierarchy via three synergistic techniques: block-virtualized geometry for SSD-aligned spatial locality, a hierarchical asynchronous pipeline to overlap I/O with computation, and trajectory-adaptive differential streaming that transfers only incremental working-set deltas between iterations. Experiments show that TideGS enables training with over one billion Gaussians on a single 24 GB GPU while achieving the best reconstruction quality among evaluated single-GPU baselines on large-scale scenes, scaling beyond prior out-of-core baselines (e.g., approximately 100M Gaussians) and standard in-memory training (e.g., approximately 11M Gaussians).

Abstract

3D Gaussian Splatting

Explore similar work

TideGS: Scalable Training of Over One Billion 3D Gaussian Splatting Primitives via Out-of-Core Optimization

PocketSplat: Mobile Gaussian Reconstruction via World-Space Latent Allocatio

BoundedGS: Memory-Efficient High-Resolution Training for 3D Gaussian Splatting