Ray Tracing enables rendering capabilities that are difficult to achieve with rasterization-based approaches such as 3D Gaussian Splatting (3DGS). However, the training cost of Gaussian-based ray tracing (GRT) is much higher than that of 3DGS. In this paper, we study how ray tracing using Gaussians executes on GPUs and show that a considerable fraction of the training cost arises from how the computation is organized. We develop FastGRT, which decouples ray traversal from blending and reuses work done in the forward pass for the backward pass. FastGRT further incorporates optimizations such as fast gradient accumulation and a hybrid BVH update strategy to tackle other major bottlenecks in the training pipeline. We implement FastGRT in Vulkan. To the best of our knowledge, it is the first Vulkan-based GRT implementation that supports both training and rendering. Compared to 3DGRT, FastGRT improves training time by 1.9× to 4.97×, while preserving the original rendering formulation. Our code is open source.
Figures & tables
Figure 1: Ray tracing using 3D Gaussians. A ray is cast from each pixel and the hit Gaussians’ contributions are blended together to produce the pixel color.
Figure 2: Breakdown of 3DGRT training time for the Bonsai and Bicycle scenes across training stages and image resolutions.
Resolution/Scene
Bonsai
Room
Counter
Kitchen
Bicycle
Garden
Stump
Standard
1.5
1.2
1.2
1.4
6.8
4.5
8.5
Full
1.4
1.0
1.0
1.2
2.2
3.2
1.9
Table 1: Gaussian population (in millions) at the final trained checkpoint for scenes in the Mip-NeRF 360 dataset Barron et al. (2022) . Standard denotes the commonly used downsampled resolution (1/2 for indoor and 1/4 for outdoor scenes); Full denotes the original resolution.
Figure 3: Breakdown of 3DGRT backward-pass time into retraversal, blending, and gradient computation versus gradient atomics, for Bonsai and Bicycle at full resolution.
Figure 4: Overview of our Vulkan Training and Ray Tracing pipeline.
Figure 5: Adaptive round budget over training iterations for the Bonsai scene at full resolution.
Figure 6: Forward-pass and Backward-pass time of FastGRT and 3DGRT over training iterations.
Figure 7: Per-lane Gaussian gradients are staged in shared memory, and the accumulation work is divided among lanes.
Figure 8: BVH update time per step of FastGRT and 3DGRT over training iterations.
Figure 9: End-to-end training performance at Standard resolution.
Figure 10: End-to-end training time for MipNeRF360 scenes at full resolution
Figure 11: Performance benefit of each of the three proposed optimizations at selected training checkpoints.
Figure 12: Comparable Inference Time Rendering Speed
Mip-NeRF 360
Deep Blending
Tanks and Temples
Zip-NeRF
Method/Metric
PSNR ↑
LPIPS ↓
SSIM ↑
PSNR ↑
LPIPS ↓
SSIM ↑
PSNR ↑
LPIPS ↓
SSIM ↑
PSNR ↑
LPIPS ↓
SSIM ↑
3DGRT
27.88
0.241
0.851
29.45
0.332
0.896
22.54
0.216
0.829
24.33
0.257
0.848
FastGRT (Ours)
28.08
0.241
0.852
29.45
0.330
0.896
22.54
0.214
0.831
24.34
0.261
0.847
Table 2: Rendering quality of the final converged checkpoints at standard resolution (mean over all scenes in each dataset).
Scene/Method
FastGRT (GiB)
3DGRT (GiB)
Bonsai
13.1
7.5
Counter
10.9
5.3
Kitchen
13.0
5.8
Room
10.0
5.4
Bicycle
49.0
32.6
Garden
30.7
19.5
Table 3: Peak GPU memory during training at standard resolution
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 13: Per-phase time per training iteration, Mip-NeRF 360 at standard resolution.
Figure 14: Per-phase time per training iteration, Mip-NeRF 360 at full resolution.
Figure 15: Per-phase time per training iteration, Tanks&Temples and Deep Blending.
3D Gaussian Splatting has made Gaussian primitives a highly efficient representation for real-time novel view synthesis, but its rasterisation-based formulation relies on screen-space approximations that limit accurate view-dependent ordering and the integration of secondary ray effects such as reflections, refractions, and shadows. Gaussian ray tracing addresses these limitations by evaluating explicit ray-primitive intersections, yet it remains costly to train. We observe that the main bottleneck is not ray traversal alone, but the pixel-centric backward propagation, where many threads concurrently accumulate gradients into the same primitive parameters, causing severe atomic contention and thread serialisation. We present 3DGART, a practical training framework for ray-traced Gaussian rendering. Our key idea is to reorganise backward propagation around primitives rather than pixels. Using conservative perspective-correct screen-space bounds, we build a compact intermediate buffer and a tile-primitive mapping that allows each thread to accumulate the contribution of one primitive over its covered pixels within a tile. This transforms gradient computation from a contention-heavy scatter operation into a structured gather-like process. On Mip-NeRF 360, 3DGART achieves an ≈3−3.5× raw training speedup over per-pixel baseline and ≈4× over 3DGRT on Mip-NeRF 360 while improving quality. More importantly, 3DGART makes fully ray-traced Gaussian training practical, reaching runtimes competitive with rasterisation-based pipelines while preserving benefits of ray tracing.
High-resolution 3D Gaussian Splatting training increases pixel and Gaussian-tile intersection counts, expanding rasterization caches that coexist with projection and gradient state at peak memory. A complete-view update aggregates gradients across the view; each region's forward and backward caches can be released after its contribution is collected. BoundedGS separates rasterization and projection backward passes and bounds the active local rasterization working set according to its measured load. Candidate-indexed projection and compact gradient storage address the view-level peaks exposed by this change, while CPU-assisted structure rebuilding and bounded image supply manage resources outside rendering. In a resolution sweep at a fixed count of about 6M Gaussians, the measured rendering working set stays within 1.073-1.077 GiB from 1K to 16K; at 11K, a warm pass takes 19.79 seconds versus 21.76 for full-frame execution. Across five native 5K outdoor scenes, full training reduces the mean rendering working set from 9.97 to 1.76 GiB (82.3%), retains similar reconstruction quality, and shortens training time from 36.7 to 33.9 minutes. Project page: https://xiaobin2001.github.io/BoundedGS-Web/
We present VkSplat, a high-performance, cross-vendor 3D Gaussian Splatting (3DGS) training pipeline implemented fully in Vulkan compute, addressing performance and compatibility limitation of existing training pipelines. With various optimizations, we achieve 3.3× speed and 33% VRAM reduction over CUDA+PyTorch baseline, maintaining quality, and demonstrating compatibility across GPU vendors. To the best of our knowledge, this is the first fully-Vulkan-based 3DGS training pipeline that achieves state-of-the-art performance. Code: https://github.com/harry7557558/vksplat