3D Gaussian Accelerated Ray Tracing: Fast training through particle-based backward propagation
Organizations: University of Canterbury, New Zealand
Abstract
3D Gaussian Splatting has made Gaussian primitives a highly efficient representation for real-time novel view synthesis, but its rasterisation-based formulation relies on screen-space approximations that limit accurate view-dependent ordering and the integration of secondary ray effects such as reflections, refractions, and shadows. Gaussian ray tracing addresses these limitations by evaluating explicit ray-primitive intersections, yet it remains costly to train. We observe that the main bottleneck is not ray traversal alone, but the pixel-centric backward propagation, where many threads concurrently accumulate gradients into the same primitive parameters, causing severe atomic contention and thread serialisation. We present 3DGART, a practical training framework for ray-traced Gaussian rendering. Our key idea is to reorganise backward propagation around primitives rather than pixels. Using conservative perspective-correct screen-space bounds, we build a compact intermediate buffer and a tile-primitive mapping that allows each thread to accumulate the contribution of one primitive over its covered pixels within a tile. This transforms gradient computation from a contention-heavy scatter operation into a structured gather-like process. On Mip-NeRF 360, 3DGART achieves an raw training speedup over per-pixel baseline and over 3DGRT on Mip-NeRF 360 while improving quality. More importantly, 3DGART makes fully ray-traced Gaussian training practical, reaching runtimes competitive with rasterisation-based pipelines while preserving benefits of ray tracing.
Figures & tables
| Mip-NeRF 360 | ||||
| #G | Pixel | Hybrid | Primitive | Gain |
| 1.00M | 44m08s | 19m55s | 12m35s | 3.51 |
| 2.00M | 51m13s | 24m56s | 16m32s | 3.10 |
| Mip-NeRF 360 | Tanks&Temples | Deep Blending | |||||||||||||||
| Model | Type | Iter | PSNR | SSIM | LPIPS | Train | #G | PSNR | SSIM | LPIPS | Train | #G | PSNR | SSIM | LPIPS | Train | #G |
| ZipNeRF | – | – | 28.54 | 0.828 | 0.219 | 5h | – | – | – | – | – | – | – | – | – | – | – |
| 3DGS | Rast. | 30K | 27.35 | 0.814 | 0.216 | 20m19s | 3.31M | 23.52 | 0.842 | 0.183 | 11m53s | 1.84M | 29.77 | 0.906 | 0.263 | 20m55s | 2.81M |
| 3DGUT | Rast. | 30K | 27.47 | 0.812 | 0.218 | 16m09s | 3.23M | 23.52 | 0.848 | 0.173 | 11m24s | 2.15M | 29.60 | 0.902 | 0.256 | 8m35s | 1.37M |
| HTGS | Rast. | 30K | 27.04 | 0.818 | 0.196 | 10m09s | 2.20M | 23.09 | 0.846 | 0.172 | 6m32s | 0.77M | 29.93 | 0.907 | 0.234 | 8m25s | 1.01M |
| 3DGRT | RT | 30K | 27.11 | 0.809 | 0.214 | 62m21s | 4.19M | 22.81 | 0.844 | 0.165 | 36m47s | 4.05M | 29.62 | 0.904 | 0.247 | 50m51s | 1.74M |