Novel view synthesis (NVS) is increasingly relevant for edge robotics, where compact and incrementally updatable 3D scene models are needed for SLAM, navigation, and inspection under tight memory and latency budgets. Variational Bayesian Gaussian Splatting (VBGS) enables replay-free continual updates for the 3DGS algorithm by maintaining a probabilistic scene model, but its high-precision computations and large intermediate tensors make on-device training impractical. We present a precision-adaptive optimization framework that enables VBGS training on resource-constrained hardware without altering its variational formulation. We (i) profile VBGS to identify memory/latency hotspots, (ii) fuse memory-dominant kernels to reduce materialized intermediate tensors, and (iii) automatically assign operation-level precisions via a mixed-precision search with bounded relative error. Across the Blender, Habitat, and Replica datasets, our optimised pipeline reduces peak memory from 9.44 GB to 1.11 GB and training time from ~234 min to ~61 min on an A5000 GPU, while preserving (and in some cases improving) reconstruction quality of the state-of-the-art VBGS baseline. We also enable for the first time NVS training on a commercial embedded platform, the Jetson Orin Nano, reducing per-frame latency by 19x compared to 3DGS.
Figures & tables
Figure 1: Novel view synthesis setup: observed training views (in gray) are used for training. Test views (in red) are used to evaluate rendering quality from unseen viewpoints.
Method
Continual
Replay free
Edge training
COTS HW
Voyager [ 11 ]
n.a.
n.a.
n.a.
✓
ECO-GS [ 19 ]
n.a.
n.a.
n.a.
✓
3DGS [ 9 ]
✗
n.a.
✗
✓
Grendel [ 25 ]
✗
n.a.
✗
✓
CLNeRF [ 2 ]
✓
✗
✗
✓
G. S. SLAM [ 12 ]
✓
✗
✗
✓
Table 1: State-of-the-art GS methods.
Figure 2: VBGS generative model M overview. A point p with position sp and color cp is generated by a component n given a latent assignment zp , coming from a categorical distribution with parameter π . Gray circles denote observed data.
Algorithm 1 VBGS training pipeline
Figure 3: Runtime breakdown of VBGS between the reassignment and fitting steps.
Figure 4: Memory profile during a fit step (top) and a reassign step (bottom) window. The shaded areas represent the function executed during that timestep.
Figure 5: Effect of replacing a materialise-then-reduce implementation with a fused contraction ( Algorithm 3 ).
Figure 6: Compilation and runtime behaviour overview of the mixed-precision search algorithm.
lego
chair
drums
ficus
hotdog
materials
mic
ship
Van Gogh Room
Apartment
Skokloster Castle
Room0
Room1
Office0
Office1
3DGS
19.10
20.59
15.04
19.41
19.81
16.11
23.03
21.15
20.55
19.18
14.30
23.76
26.98
33.03
33.88
(w. replay)
±1.02
±0.85
±1.13
±0.90
±1.86
±1.45
±0.72
±0.82
±4.02
±5.32
±4.36
±4.04
±2.71
±3.13
±2.44
3DGS
8.86
10.73
8.25
13.22
9.72
9.66
16.14
12.27
10.14
10.57
10.74
10.52
11.54
14.55
14.66
(w/o. replay)
±5.79
±2.48
±5.09
±4.63
±6.10
±5.11
±3.30
±5.34
±6.44
±6.96
±9.45
±4.73
±4.00
±11.17
±9.59
VBGS
22.17
22.44
19.50
21.93
24.32
20.67
24.53
22.28
21.54
24.33
17.12
20.74
21.60
27.56
30.04
(baseline)
±0.80
±1.01
±0.50
±0.76
±0.85
±1.39
±0.61
±0.97
±3.07
±3.22
±3.22
±2.65
±1.52
±2.41
±3.23
Table 2: PSNR (dB) on Blender (left block) and Habitat (center block) and Replica (right block). All models are evaluated after 200 training frames on 100 unseen validation frames. Values are reported as μ±σ . The best performance for each scene is shown in bold .
Implementation
Per-frame average latency
Memory usage
A5000 (3DGS w. replay)
∼ 410 s
4.14 GB
A5000 (VBGS)
70.12 s
9.44 GB
A5000 (Optimized VBGS)
18.33 s
1.11 GB
Jetson (3DGS w. replay)
∼ 3470 s
6.54 GB
Jetson (Baseline VBGS)
n.a. †
n.a. †
Jetson (Optimized VBGS)
∼ 180 s
2.53 GB
Table 3: Performance of 3DGS and VBGS training before/after our optimizations on deployment targets. † OOM on Jetson.
Figure 7: PSNR evolution of VBGS under different homogeneous precision variants on the Habitat Van Gogh Room environment.
Figure 8: Tolerance sweep for the mixed-precision search on Habitat Van Gogh Room .
Figure 9: Performance comparison of the VBGS baseline implementation and our optimized training pipeline on 100 validation frames on the Replica dataset.
Figure 10: Performance comparison of the VBGS baseline implementation and our optimized training pipeline on 100 validation frames on the Habitat dataset.
Figure 11: Performance comparison of the VBGS baseline implementation and our optimized training pipeline on 100 validation frames on the Replica dataset.
Figure 12: Quality comparison of renderings generated from models trained on 200 Blender frames.
Figure 13: Quality comparison of renderings generated from models trained on 200 Habitat and Replica frames.
On-the-fly reconstruction is a key requirement for many applications in robotics and autonomous navigation. Variational Bayes Gaussian Splatting (VBGS) enables continual learning without replay buffers using Coordinate Ascent Variational Inference (CAVI), but its per-frame iterations over all observed points make it too slow for real-time use with strict memory and latency requirements. We present ImprovedVBGS, an accelerated framework for on-the-fly continual reconstruction. This is achieved primarily through (i) spatially truncated variational inference, and (ii) improved reassignment that uses forwarding, truncation and eliminates wasteful dynamic recompilation. On the NeRF synthetic dataset, we reduce mean per-frame latency from ~84.0 s to ~0.050 s on an RTX 3070 Ti, a 1680x speed-up while maintaining reconstruction quality. We also improve novel-view synthesis quality using an exact renderer with no added training costs.
3D Gaussian Splatting (3DGS) has revolutionized novel view synthesis with high-quality rendering through continuous aggregations of millions of 3D Gaussian primitives. However, it suffers from a substantial memory footprint, particularly during training due to uncontrolled densification, posing a critical bottleneck for deployment on memory-constrained edge devices. While existing methods prune redundant Gaussians post-training, they fail to address the peak memory spikes caused by the abrupt growth of Gaussians early in the training process. To solve the training memory consumption problem, we propose a systematic memory-bounded training framework that dynamically optimizes Gaussians through iterative growth and pruning. In other words, the proposed framework alternates between incremental pruning of low-impact Gaussians and strategic growing of new primitives with an adaptive Gaussian compensation, maintaining a near-constant low memory usage while progressively refining rendering fidelity. We comprehensively evaluate the proposed training framework on various real-world datasets under strict memory constraints, showing significant improvements over existing state-of-the-art methods. Particularly, our proposed method practically enables memory-efficient 3DGS training on NVIDIA Jetson AGX Xavier, achieving similar visual quality with up to 80% lower peak training memory consumption than the original 3DGS.
Yangming Zhang, Jian Xu, Chaojian Li +7
Dept. of Computer Science University of Texas at Arlington · School of Computer Science Georgia Institute of Technology · School of Computing University of Georgia +2
3D Gaussian Splatting (3DGS) achieves remarkable novel view synthesis quality with real-time rendering, yet suffers from excessive memory consumption due to millions of Gaussian primitives. Existing pruning methods rely on heuristic importance scores or synchronous batch updates, leading to suboptimal compression and training instability. We propose VEDAL, a principled framework that formulates Gaussian pruning as variational free energy minimization. Our approach introduces (1) a prediction-error gating mechanism that asynchronously activates pruning based on per-Gaussian reconstruction uncertainty, and (2) a variational uncertainty head that models pruning decisions as latent variables with learnable priors. The free energy objective naturally balances reconstruction fidelity against model complexity through an information-theoretic lens. Extensive experiments on Mip-NeRF 360, Tanks&Temples, and Deep Blending demonstrate that VEDAL achieves 5.2x compression with only 0.31 dB PSNR drop, outperforming PUP 3D-GS by +0.05 dB at a higher compression ratio and LightGaussian by +0.35 dB at comparable quality, while maintaining real-time rendering at 185 FPS.
Aoduo Li, Jiancheng Li, Huan Ye +5
Guangdong University of Technology · Huizhou Boluo Power Supply Bureau, Guangdong Power Grid Co., Ltd. · Shenzhen Polytechnic University +1