Real-time novel view synthesis is dominated by rasterized explicit primitives. These projection-based pipelines provide high throughput but require specialized extensions for non-pinhole effects such as distortion, rolling shutter, and depth of field. Ray-based rendering expresses these effects naturally but is generally assumed too slow for competitive real-time rendering. We analyze the factors governing throughput in differentiable Voronoi ray tracing and identify traversal length, per-cell work, and memory locality as principal determinants. Guided by this, we introduce VoroTracing, which co-designs the scene representation, optimization, and GPU execution to reduce these costs. Compact octahedral appearance textures reduce memory traffic, while surface-concentrated opacity promotes early termination. The fixed-budget representation is optimized without pruning or densification and rendered with a GPU implementation designed for coherent traversal. On Mip-NeRF 360, VoroTracing renders at 623 FPS on an RTX 5090, providing 3.2× the throughput of the fastest prior ray-based method and 2.8× that of 3D Gaussian Splatting, while maintaining competitive reconstruction quality. Our renderer supports fisheye, rolling-shutter, motion-blur, and depth-of-field effects through ray generation and sampling, requiring no specialized rasterization. These results show that real-time throughput can be achieved with the flexibility of ray-based rendering. We release our source code, see https://research.zenseact.com/publications/vorotracing
Figures & tables
Figure 1. Ray-based rendering with VoroTracing. The main image is a shallow-depth-of-field rendering of the Garden scene. The highlighted path visualizes an actual camera ray traversing the scene’s Voronoi diagram from cell to cell. The light blue regions show the projected footprints of the 3D cells traversed by the ray. The lower-right inset shows a fisheye rendering of the same scene from an overhead viewpoint. Both effects are produced by the same trained representation through changes to ray generation and sampling. Teaser showing a shallow-depth-of-field rendering of a garden scene. A highlighted camera ray passes through a sequence of Voronoi cells, whose projected footprints are shown as light blue regions. A lower-right inset shows an overhead fisheye view of the same scene.
Figure 2. Speed–quality trade-off on Mip-NeRF 360. Rendering speed (RTX 5090) versus reconstruction quality, averaged over the seven scenes. VoroTracing renders 2.1× faster than the fastest prior baseline while matching the best rasterized quality to within 0.2 dB. Marker shape indicates the renderer used for the reported timing. Scatter plot of average PSNR against rendering speed for ray-based and rasterized novel-view-synthesis methods, with our method in the high-speed, high-quality region.
Figure 3. Adjacency-based ray traversal. A camera ray r(t)=o+tω walks a 2D Voronoi diagram cell by cell. Starting from the cell containing o (found by a single nearest-neighbor query), each step intersects the ray with the perpendicular bisector between the current site pi and each of its Delaunay neighbors, and advances across the nearest forward crossing—the cell’s exit face—into the adjacent cell pj . The exit is the bisector the ray reaches first, while the other faces are crossed farther along. The traversal thus follows edges of the Delaunay adjacency graph (dashed), visiting a short sequence of cells whose ray-segment lengths feed the piecewise volume rendering. Diagram of a ray crossing a two-dimensional Voronoi diagram cell by cell, with the traversed cells, Delaunay adjacency graph, and nearest exit-face test highlighted.
Figure 4. Appearance values loaded per cell. A spherical-harmonic cell accesses all of its RGB coefficients to evaluate color in any direction, a count that grows quadratically with degree ( 3(L+1)2 ); at degree 3 , the setting used by Radiant Foam and 3DGS, this is 48 values. Our deployed representation performs two bilinear lookups, one in each appearance map. Each lookup requires four RGB texels, giving a fixed total of 24 appearance values independent of map resolution, halving the appearance traffic. Bar chart comparing the number of appearance values loaded per cell for spherical harmonics of increasing degree and for the proposed octahedral texture lookup.
Figure 5. Octahedral appearance lookup. (a) A camera ray crosses a Voronoi cell; the unit direction d from the site pi to the hit point (violet) selects the appearance. (b) d is ℓ1 -normalized onto the unit octahedron, landing on one of its eight faces (here the red octant); the eight faces are color-coded by octant. (c) The octahedron is unfolded into the square texture domain, where the upper hemisphere ( dz≥0 ) fills the central diamond and the lower hemisphere ( dz<0 ) the corners. Three-panel diagram. Left: a camera ray strikes a convex Voronoi cell at a point, and the unit direction from the cell's site to that point is highlighted. Middle: a 3D octahedron whose eight faces are color-coded by octant, with the highlighted direction landing on the red face. Right: the octahedron unfolded into a square, its eight faces shown as colored triangles, with the looked-up texel marked in the matching red wedge.
Figure 6. View-dependent contribution. We render a held-out view with the full model and again with the view-dependent map Tvd zeroed (a pure-diffuse render); density is unchanged, so the difference isolates the view-dependent term. Diffuse regions remain similar, while glossy and reflective surfaces show clear changes. The right panel shows that the residual is localized on reflective tabletop objects and foliage, exactly the surfaces whose appearance changes with viewpoint. Wide figure comparing the full render, the diffuse-only render with the view-dependent effects disabled, and a difference visualization that highlights reflective objects and foliage.
Figure 7. Cell-opacity distribution on the Mip-NeRF 360 scenes. Characteristic per-cell opacity α=1−e−σδ , where δ is the mean distance to a cell’s Voronoi neighbors. Our exponential density parameterization and distortion loss drives opacity to a strongly bimodal distribution — cells are either near-transparent or near-opaque — with 24% of cells above α=0.9 . Radiant Foam’s softplus model instead leaves most cells semi-transparent ( 44% in [0.1,0.9] , only 4% above 0.9 ), so rays must composite many partially-occupied cells before terminating. Our method reaches this with roughly half the cells ( 2.0 M vs. 4.1 M). Histogram comparing per-cell opacity distributions; our method has peaks near transparent and opaque cells, while Radiant Foam has more semi-transparent cells.
Method
Outdoor
Indoor
All
PSNR ↑
SSIM ↑
LPIPS ↓
FPS ↑
PSNR ↑
SSIM ↑
LPIPS ↓
FPS ↑
PSNR ↑
SSIM ↑
LPIPS ↓
FPS ↑
Zip-NeRF
27.13
0.808
0.197
0.29
32.29
0.929
0.197
0.19
30.08
0.877
0.197
0.23
iNGP (base)
25.16
0.636
0.397
30
28.59
0.841
0.340
25
27.12
0.753
0.365
27
iNGP (big)
26.01
0.697
0.342
28
29.45
0.863
0.297
24
27.98
0.792
0.316
26
3DGRT
25.90
0.785
0.219
103
29.93
0.914
0.240
77
28.20
0.859
0.231
88
RadFoam
25.40
0.731
0.298
156
30.72
0.902
0.261
222
28.44
0.829
0.277
194
Table 1. Mip-NeRF 360, ray-based methods. Per metric column, best , second and third . VoroTracing renders 2 – 3× faster than every other ray-based method while improving on its predecessor Radiant Foam across all quality metrics.
Method
Outdoor
Indoor
All
PSNR ↑
SSIM ↑
LPIPS ↓
FPS ↑
PSNR ↑
SSIM ↑
LPIPS ↓
FPS ↑
PSNR ↑
SSIM ↑
LPIPS ↓
FPS ↑
3DGS
26.45
0.801
0.204
143
31.11
0.924
0.237
279
29.11
0.872
0.223
220
3DGUT
26.21
0.796
0.211
183
31.07
0.925
0.235
307
28.98
0.870
0.225
254
PowerFoam (rast.)
25.39
0.730
0.278
222
31.31
0.913
0.241
350
28.78
0.835
0.257
295
Radiance Meshes
26.11
0.783
0.249
59
30.16
0.919
0.240
50
28.43
0.861
0.244
53
Radiance Meshes (Vulkan)
23.98
0.649
0.318
317
28.07
0.847
0.274
345
26.32
0.762
0.293
333
Table 2. Mip-NeRF 360, rasterized methods vs. ours. Per metric column, best , second and third . VoroTracing is the only ray-traced method here; it trails the strongest rasterizers slightly on appearance metrics yet renders roughly 2× faster than all of them.
Figure 8. Cropped qualitative comparison on held-out Mip-NeRF 360 views. We show challenging regions that are observed by few training images and therefore stress fine-scale reconstruction. VoroTracing uses a fixed budget of 2 M cells, half the Radiant Foam budget on outdoor scenes. Unlike methods that densify during training, it cannot add cells later to recover underrepresented regions; this limitation is visible in some crops, where VoroTracing preserves more structure than Radiant Foam but remains less detailed than 3DGS. Red circles mark representative artifacts or missing details in the compared reconstructions. Grid of cropped held-out Mip-NeRF 360 views comparing Radiant Foam, PowerFoam, 3DGS, our method, and ground truth across garden, bicycle, counter, and bonsai scenes.
Configuration
PSNR ↑
LPIPS ↓
FPS ↑
Speedup
Base (packed fp16)
28.98
0.235
230
1.00 ×
+ Morton cell ordering
28.98
0.235
378
1.64 ×
+ warp-coherent tiling
28.98
0.235
536
2.33 ×
+ cell skip (10−3)
28.98
0.235
623
2.71 ×
Table 3. Inference-stack ablation. The representation is fixed. Starting from the base inference renderer, each subsequent row adds one optimization. Averages are over the seven Mip-NeRF 360 scenes.
Figure 9. Cells traversed per ray , ours vs. Radiant Foam on held-out views of the Mip-NeRF 360 scenes. Unlike rasterization or BVH ray tracing, the cost of Voronoi traversal is set by this quantity rather than the total cell count N . Our surface-concentrated opacity ( Fig. 7 ) shifts the whole distribution left. Rays visit 46.1 cells on average versus 66.9 for Radiant Foam, a 31% reduction in mean traversal count, while we also reach higher reconstruction quality ( Table 1 ). Distribution plot showing that our method shifts the cells-per-ray histogram left relative to Radiant Foam, with a lower mean traversal count.
Figure 10. Cells traversed per ray, in image space. For a held-out view of the garden scene we map the per-pixel number of cells each ray visits before terminating. Cost concentrates on geometrically complex regions (the foliage, object silhouettes) and is low on smooth surfaces. On the same view and color scale, our model shows lower traversal counts than Radiant Foam across most image regions, reducing the mean from 73 to 49 cells per ray. The speedup therefore comes from a broad reduction in traversal depth, not only from a few isolated easy pixels. Three-panel figure with a garden rendering, our per-pixel traversal-count heat map, and Radiant Foam's traversal-count heat map on the same color scale.
Method
points
PSNR ↑
LPIPS ↓
cells/ray ↓
FPS ↑
RadFoam original (SH)
4.2/2.1M
28.44
0.277
70.2
194
+ our inference stack
4.2/2.1M
28.37
0.279
70.2
384
+ fixed dense init.
2M
27.90
0.290
59.6
430
+ exp. density
2M
28.18
0.284
51.7
496
+ distortion loss
2M
28.23
0.280
52.1
475
Octahedral texture formulation:
Table 4. Cumulative ablation from RadFoam to VoroTracing on Mip-NeRF 360. Each row adds one component to the previous row. From the second row onward, all variants use the same deployed inference stack, so later speed changes reflect the representation and training choices. Points are approximate for Radiant Foam and fixed for our variants.
Figure 11. Early-training appearance, exp vs. softplus. The same held-out view of the stump scene rendered after only 1000 training steps. With softplus the surface is still hazy and washed out; the scale-invariant exponential parameterization has already recovered sharp geometry and color much closer to the ground truth. Three rendered crops of the stump scene after early training: a hazy softplus-density result, a sharper exponential-density result, and the ground-truth image.
Figure 12. Distortion loss trades quality for speed. As the distortion weight λdist grows, opacity concentrates onto thinner, more opaque surfaces ( Fig. 7 ), so rays terminate after fewer cells and rendering speeds up. We operate at the marked point, which keeps near-peak quality while already capturing most of the speedup. Line plot sweeping the distortion-loss weight, showing PSNR on one axis and rendering speed on the other, with the chosen operating point marked.
Resolution
PSNR ↑
SSIM ↑
LPIPS ↓
FPS ↑
4×4
28.85
0.842
0.254
619
6×6
28.94
0.846
0.243
615
8×8 (ours)
28.98
0.848
0.235
623
Table 5. Texture resolution ablation on Mip-NeRF 360. Mean reconstruction quality over the 7 scenes and rendering speed on an RTX 5090. Increasing the octahedral diffuse-map resolution mainly improves LPIPS, while rendering speed remains nearly unchanged.
Figure 13. Ray-based camera effects. All panels render the same trained representation while changing only the rays sent to the renderer. The examples show a standard pinhole camera, a wide-angle fisheye camera, row-dependent poses for rolling shutter, and temporal averaging for camera motion blur. Four-panel figure showing the same scene rendered with pinhole, fisheye, rolling-shutter, and motion-blur camera effects.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Setting
Outdoor
Indoor
Image downsampling
4×
2×
Site-position learning rate
2×10−4→5×10−6
same
Density learning rate
10−1→10−2
same
Texture learning rate
2×10−2→5×10−4
same
View-dependent ramp length
4,000
0
λdist
2×10−3
same
Appendix
Table 6. Training settings used for the Mip-NeRF 360 experiments. Learning-rate entries give the schedule endpoints.
Figure 14. Mobile viewer demonstration. The Garden representation runs interactively on an iPhone 16. It uses the same Garden model with 2 M sites as the desktop evaluation, without compression, a lower cell count, or retraining. Photograph of an iPhone running an interactive viewer that displays the Garden scene, with an on-screen frame-rate counter.
Indoor
Outdoor
Method
Room
Counter
Bonsai
Kitchen
Bicycle
Garden
Stump
Mean
Zip-NeRF
32.97
29.06
34.73
32.42
25.87
28.19
27.32
30.08
iNGP (base)
30.37
27.85
31.27
24.85
24.00
25.58
25.89
27.12
iNGP (big)
31.97
28.54
32.14
25.17
24.52
26.73
26.78
27.98
3DGRT
30.08
28.37
31.59
29.68
24.73
26.73
26.23
28.20
RadFoam
30.82
28.56
32.26
31.23
24.21
26.54
25.45
28.44
Appendix
Table 7. Per-scene PSNR ↑ on Mip-NeRF 360. Per column, best , second , and third .
Indoor
Outdoor
Method
Room
Counter
Bonsai
Kitchen
Bicycle
Garden
Stump
Mean
Zip-NeRF
0.929
0.906
0.952
0.930
0.773
0.863
0.788
0.877
iNGP (base)
0.869
0.829
0.902
0.762
0.564
0.690
0.654
0.753
iNGP (big)
0.894
0.853
0.921
0.785
0.617
0.765
0.709
0.792
3DGRT
0.903
0.901
0.937
0.914
0.744
0.847
0.765
0.859
RadFoam
0.901
0.875
0.925
0.906
0.674
0.810
0.709
0.829
Appendix
Table 8. Per-scene SSIM ↑ on Mip-NeRF 360. Per column, best , second , and third .
Indoor
Outdoor
Method
Room
Counter
Bonsai
Kitchen
Bicycle
Garden
Stump
Mean
Zip-NeRF
0.238
0.223
0.195
0.133
0.227
0.127
0.237
0.197
iNGP (base)
0.375
0.363
0.297
0.326
0.464
0.320
0.407
0.365
iNGP (big)
0.319
0.321
0.259
0.290
0.411
0.256
0.360
0.316
3DGRT
0.298
0.256
0.241
0.166
0.256
0.145
0.256
0.231
RadFoam
0.300
0.286
0.268
0.192
0.370
0.190
0.334
0.277
Appendix
Table 9. Per-scene LPIPS ↓ on Mip-NeRF 360. Per column, best , second , and third . Lower is better.
Indoor
Outdoor
Method
Room
Counter
Bonsai
Kitchen
Bicycle
Garden
Stump
Mean
Zip-NeRF
0.19
0.19
0.19
0.19
0.30
0.28
0.29
0.23
iNGP (base)
25
18
22
32
30
38
21
27
iNGP (big)
26
17
22
32
29
37
19
26
3DGRT
111
75
77
45
96
103
112
88
RadFoam
230
221
228
210
51
233
185
194
Appendix
Table 10. Per-scene FPS ↑ on Mip-NeRF 360. Per column, best , second , and third . Frames per second are measured on an RTX 5090.
Threshold
PSNR ↑
Δ PSNR
LPIPS ↓
FPS ↑
0
28.98
+0.00
0.235
538
10−5
28.98
+0.00
0.235
597
10−4
28.98
+0.00
0.235
609
3×10−4
28.98
+0.00
0.235
613
10−3
28.98
-0.00
0.235
623
3×10−3
28.93
-0.04
0.234
631
Appendix
Table 11. Cell-skip threshold sweep. Morton ordering and warp-coherent tiling are enabled for all rows. Increasing the threshold skips more low-contribution texture evaluations, trading quality for speed.
We introduce a differentiable 3D representation that unifies the ray tracing capabilities of foam-based ray tracing with the efficiency of modern rasterization pipelines. While prior foam representations enable constant-time ray traversal through an explicit volumetric partition of space, their potentially unbounded cells hinder efficient tile-based rasterization. We address this limitation by generalizing Voronoi foams to bounded power diagrams with controllable cell extents, enabling spatially bounded primitives without requiring expensive Delaunay triangulations during training. We further introduce an oriented surface formulation that explicitly models interfaces between interior and exterior regions, and decouple geometry from appearance by embedding differentiable texture directly on these surfaces. Together, these contributions yield a representation that preserves state-of-the-art ray tracing efficiency while achieving rasterization performance competitive with current generation 3DGS, providing a practical path toward unified real-time differentiable rendering.
Shrisudhan Govindarajan, Daniel Rebain, Dor Verbin +3
Simon Fraser University · Google · Google Deepmind +2
Radiance field reconstruction aims to recover high-quality 3D representations from multi-view RGB images. Recent advances, such as 3D Gaussian splatting, enable real-time rendering with high visual fidelity on sufficiently powerful graphics hardware. However, efficient online transmission and rendering across diverse platforms requires drastic model simplification, reducing the number of primitives by several orders of magnitude. We introduce DiffSoup, a radiance field representation that employs a soup (i.e., a highly unstructured set) of a small number of triangles with neural textures and binary opacity. We show that this binary opacity representation is directly differentiable via stochastic opacity masking, enabling stable training without a mollifier (i.e., smooth rasterization). DiffSoup can be rasterized using standard depth testing, enabling seamless integration into traditional graphics pipelines and interactive rendering on consumer-grade laptops and mobile devices. Code is available at https://github.com/kenji-tojo/diffsoup.
3D Gaussian Splatting has made Gaussian primitives a highly efficient representation for real-time novel view synthesis, but its rasterisation-based formulation relies on screen-space approximations that limit accurate view-dependent ordering and the integration of secondary ray effects such as reflections, refractions, and shadows. Gaussian ray tracing addresses these limitations by evaluating explicit ray-primitive intersections, yet it remains costly to train. We observe that the main bottleneck is not ray traversal alone, but the pixel-centric backward propagation, where many threads concurrently accumulate gradients into the same primitive parameters, causing severe atomic contention and thread serialisation. We present 3DGART, a practical training framework for ray-traced Gaussian rendering. Our key idea is to reorganise backward propagation around primitives rather than pixels. Using conservative perspective-correct screen-space bounds, we build a compact intermediate buffer and a tile-primitive mapping that allows each thread to accumulate the contribution of one primitive over its covered pixels within a tile. This transforms gradient computation from a contention-heavy scatter operation into a structured gather-like process. On Mip-NeRF 360, 3DGART achieves an ≈3−3.5× raw training speedup over per-pixel baseline and ≈4× over 3DGRT on Mip-NeRF 360 while improving quality. More importantly, 3DGART makes fully ray-traced Gaussian training practical, reaching runtimes competitive with rasterisation-based pipelines while preserving benefits of ray tracing.