Real-Time Volumetric Light Transport Inference without a Path
Organizations: Technical University of Munich, Germany
Abstract
We present a real-time method for reconstructing fully integrated pixel radiance in rendered volumetric media under direct and ambient environmental illumination, using compact, low-cost scene features and statistics. Unlike Monte Carlo denoising pipelines for volumetric rendering, which depend on computationally expensive path-traced radiance estimates, our approach directly infers the final light-transport solution from inexpensive primary-ray and environment sample evaluations. We show that these features form a near-sufficient representation of the scene, enabling accurate reconstruction at substantially lower computational cost. Using an attention-augmented image-to-image network, we demonstrate that complete pixel radiance can be inferred directly from these auxiliary features, without requiring the original radiance signal. Operating in image space also provides a more stable computational cost as the camera zooms into or out of the volume, with substantially less dependence on volume density than approaches based on explicit multiple-scattering simulation. We demonstrate real-time performance and high reconstruction accuracy in challenging volumetric cloud scenarios.
Figures & tables
| Architecture | ||||
|---|---|---|---|---|
| HaloRT | 1.411 | 1.563 | 2.914 | 10.528 |
| HaloRT without attention | 1.233 | 1.378 | 2.646 | 10.007 |
| Full attn at same stages | 1.446 | 1.618 | 4.700 | 31.659 |
| CNN-based Autoencoder | 1.029 | 1.132 | 1.835 | 6.830 |
| Method | MAE | PSNR (dB) | SSIM | LPIPS | RMS-SD | Time (ms) |
|---|---|---|---|---|---|---|
| Denoiser | 0.00391 0.00179 | 41.62 3.05 | 0.9687 0.0250 | 0.0901 0.0652 | 0.00628 0.00330 | 23.84 9.61 |
| DenoiserX | 0.00338 0.00151 | 44.23 2.80 | 0.9827 0.0151 | 0.0517 0.0420 | 0.00323 0.00123 | 24.76 9.98 |
| HaloRT | 0.00324 0.00265 | 42.79 3.91 | 0.9874 0.0112 | 0.0420 0.0418 | 0.00070 0.00035 | 10.66 0.52 |
Physically Based Rendering
Physically based rendering models how light interacts with geometry and materials to produce images for synthesis, relighting, and reconstruction.
See all 46 papers