Improving Continual Learning for Gaussian Splatting based Environments Reconstruction on Commercial Off-the-Shelf Edge Devices

Ivan Zaino; Matteo Risso; Daniele Jahier Pagliari; Miguel de Prado; Toon Van de Maele; Alessio Burrello

cs.CVMar 9, 2026

Improving Continual Learning for Gaussian Splatting based Environments Reconstruction on Commercial Off-the-Shelf Edge Devices

Authors: Ivan Zaino, Matteo Risso, Daniele Jahier Pagliari, Miguel de Prado, Toon Van de Maele, Alessio Burrello

Organizations: Politecnico di Torino · VERSES

Abstract

Novel view synthesis (NVS) is increasingly relevant for edge robotics, where compact and incrementally updatable 3D scene models are needed for SLAM, navigation, and inspection under tight memory and latency budgets. Variational Bayesian Gaussian Splatting (VBGS) enables replay-free continual updates for the 3DGS algorithm by maintaining a probabilistic scene model, but its high-precision computations and large intermediate tensors make on-device training impractical. We present a precision-adaptive optimization framework that enables VBGS training on resource-constrained hardware without altering its variational formulation. We (i) profile VBGS to identify memory/latency hotspots, (ii) fuse memory-dominant kernels to reduce materialized intermediate tensors, and (iii) automatically assign operation-level precisions via a mixed-precision search with bounded relative error. Across the Blender, Habitat, and Replica datasets, our optimised pipeline reduces peak memory from 9.44 GB to 1.11 GB and training time from ~234 min to ~61 min on an A5000 GPU, while preserving (and in some cases improving) reconstruction quality of the state-of-the-art VBGS baseline. We also enable for the first time NVS training on a commercial embedded platform, the Jetson Orin Nano, reducing per-frame latency by 19x compared to 3DGS.

Figures & tables

Figure 1: Novel view synthesis setup: observed training views (in gray) are used for training. Test views (in red) are used to evaluate rendering quality from unseen viewpoints.

Method	Continual	Replay free	Edge training	COTS HW
Voyager [ 11 ]	n.a.	n.a.	n.a.	✓
ECO-GS [ 19 ]	n.a.	n.a.	n.a.	✓
3DGS [ 9 ]	✗	n.a.	✗	✓
Grendel [ 25 ]	✗	n.a.	✗	✓
CLNeRF [ 2 ]	✓	✗	✗	✓
G. S. SLAM [ 12 ]	✓	✗	✗	✓

Table 1: State-of-the-art GS methods.

Panel of Figure 2 — Figure 2: VBGS generative model $\mathcal{M}$ overview. A point $p$ with position $s_{p}$ and color $c_{p}$ is generated by a component $n$ given a latent assignment $z_{p}$ , coming from a categorical distribution with parameter $\pi$ . Gray circles denote observed data.

Panel of Figure 4 — Algorithm 1 VBGS training pipeline

Figure 3: Runtime breakdown of VBGS between the reassignment and fitting steps.

Figure 4: Memory profile during a fit step (top) and a reassign step (bottom) window. The shaded areas represent the function executed during that timestep.

Figure 5: Effect of replacing a materialise-then-reduce implementation with a fused contraction ( Algorithm 3 ).

Panel of Figure 6 — Figure 6: Compilation and runtime behaviour overview of the mixed-precision search algorithm.

	lego	chair	drums	ficus	hotdog	materials	mic	ship	Van Gogh Room	Apartment	Skokloster Castle	Room0	Room1	Office0	Office1
3DGS	19.10	20.59	15.04	19.41	19.81	16.11	23.03	21.15	20.55	19.18	14.30	23.76	26.98	33.03	33.88
(w. replay)	$\pm 1.02$	$\pm 0.85$	$\pm 1.13$	$\pm 0.90$	$\pm 1.86$	$\pm 1.45$	$\pm 0.72$	$\pm 0.82$	$\pm 4.02$	$\pm 5.32$	$\pm 4.36$	$\pm 4.04$	$\pm 2.71$	$\pm 3.13$	$\pm 2.44$
3DGS	8.86	10.73	8.25	13.22	9.72	9.66	16.14	12.27	10.14	10.57	10.74	10.52	11.54	14.55	14.66
(w/o. replay)	$\pm 5.79$	$\pm 2.48$	$\pm 5.09$	$\pm 4.63$	$\pm 6.10$	$\pm 5.11$	$\pm 3.30$	$\pm 5.34$	$\pm 6.44$	$\pm 6.96$	$\pm 9.45$	$\pm 4.73$	$\pm 4.00$	$\pm 11.17$	$\pm 9.59$
VBGS	22.17	22.44	19.50	21.93	24.32	20.67	24.53	22.28	21.54	24.33	17.12	20.74	21.60	27.56	30.04
(baseline)	$\pm 0.80$	$\pm 1.01$	$\pm 0.50$	$\pm 0.76$	$\pm 0.85$	$\pm 1.39$	$\pm 0.61$	$\pm 0.97$	$\pm 3.07$	$\pm 3.22$	$\pm 3.22$	$\pm 2.65$	$\pm 1.52$	$\pm 2.41$	$\pm 3.23$

Table 2: PSNR (dB) on Blender (left block) and Habitat (center block) and Replica (right block). All models are evaluated after 200 training frames on 100 unseen validation frames. Values are reported as

\mu\pm\sigma

. The best performance for each scene is shown in bold .

Implementation	Per-frame average latency	Memory usage
A5000 (3DGS w. replay)	$\sim$ 410 s	4.14 GB
A5000 (VBGS)	70.12 s	9.44 GB
A5000 (Optimized VBGS)	18.33 s	1.11 GB
Jetson (3DGS w. replay)	$\sim$ 3470 s	6.54 GB
Jetson (Baseline VBGS)	n.a. †	n.a. †
Jetson (Optimized VBGS)	$\sim$ 180 s	2.53 GB

Table 3: Performance of 3DGS and VBGS training before/after our optimizations on deployment targets. † OOM on Jetson.

Figure 7: PSNR evolution of VBGS under different homogeneous precision variants on the Habitat Van Gogh Room environment.

Figure 8: Tolerance sweep for the mixed-precision search on Habitat Van Gogh Room .

Figure 9: Performance comparison of the VBGS baseline implementation and our optimized training pipeline on 100 validation frames on the Replica dataset.

Figure 10: Performance comparison of the VBGS baseline implementation and our optimized training pipeline on 100 validation frames on the Habitat dataset.

Figure 11: Performance comparison of the VBGS baseline implementation and our optimized training pipeline on 100 validation frames on the Replica dataset.

Figure 12: Quality comparison of renderings generated from models trained on 200 Blender frames.

Figure 13: Quality comparison of renderings generated from models trained on 200 Habitat and Replica frames.

3DGS Training

See all 30 papers

ImprovedVBGS: Real-time Continual Variational Bayes Gaussian Splatting
Jul 17, 2026Damani Mguni-Coker3D Gaussian Splatting Streaming 3D Reconstruction
Independent Researcher
Gaussians on a Diet: High-Quality Memory-Bounded 3D Gaussian Splatting Training
Apr 21, 2026Yangming Zhang, Jian Xu, Chaojian Li +73D Gaussian Splatting 3D Gaussian Pruning
Dept. of Computer Science University of Texas at Arlington · School of Computer Science Georgia Institute of Technology · School of Computing University of Georgia +2
VEDAL: Variational Error-Driven Asynchronous Learning for 3D Gaussian Splatting Pruning
Jun 1, 2026Aoduo Li, Jiancheng Li, Huan Ye +53DGS Compression 3D Gaussian Splatting
Guangdong University of Technology · Huizhou Boluo Power Supply Bureau, Guangdong Power Grid Co., Ltd. · Shenzhen Polytechnic University +1

Jul 17, 2026J/K move · Enter open · S save

ImprovedVBGS: Real-time Continual Variational Bayes Gaussian Splatting

Damani Mguni-Coker

Independent Researcher

3D Gaussian Splatting Streaming 3D Reconstruction 3D Reconstruction Variational Inference

Open paper AskPDF arXiv

On-the-fly reconstruction is a key requirement for many applications in robotics and autonomous navigation. Variational Bayes Gaussian Splatting (VBGS) enables continual learning without replay buffers using Coordinate Ascent Variational Inference (CAVI), but its per-frame iterations over all observed points make it too slow for real-time use with strict memory and latency requirements. We present ImprovedVBGS, an accelerated framework for on-the-fly continual reconstruction. This is achieved primarily through (i) spatially truncated variational inference, and (ii) improved reassignment that uses forwarding, truncation and eliminates wasteful dynamic recompilation. On the NeRF synthetic dataset, we reduce mean per-frame latency from ~84.0 s to ~0.050 s on an RTX 3070 Ti, a 1680x speed-up while maintaining reconstruction quality. We also improve novel-view synthesis quality using an exact renderer with no added training costs.

Abstract

3DGS Training

Explore similar work

ImprovedVBGS: Real-time Continual Variational Bayes Gaussian Splatting

Gaussians on a Diet: High-Quality Memory-Bounded 3D Gaussian Splatting Training

VEDAL: Variational Error-Driven Asynchronous Learning for 3D Gaussian Splatting Pruning