ReflFlow: Learning Geometry-Guided Ray Tracing for Dynamic Specular Reconstruction
Organizations: University of Illinois Chicago · Harvard Medical School · University of Michigan
Abstract
We present ReflFlow, a novel framework for high-fidelity rendering of dynamic specular scenes by addressing two key challenges: precise reflection direction estimation and physically accurate modeling. To achieve this, we propose a Residual Material-Augmented 2D Gaussian Splatting representation that models dynamic geometry and material properties, allowing accurate reflection ray computation. Furthermore, we introduce a Dynamic Environment Gaussian and a hybrid rendering pipeline that decomposes rendering into diffuse and specular components, enabling physically informed specular synthesis via rasterization and ray tracing. Finally, we devise a coarse-to-fine training strategy to improve optimization stability and promote physically meaningful decomposition. Extensive experiments on dynamic scene benchmarks demonstrate that ReflFlow outperforms prior methods quantitatively and qualitatively, producing sharper and more realistic specular reflections in complex dynamic environments.
Figures & tables
| As | Basin | Bell | Cup | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Method | PSNR↑ | SSIM↑ | LPIPS↓ | PSNR↑ | SSIM↑ | LPIPS↓ | PSNR↑ | SSIM↑ | LPIPS↓ | PSNR↑ | SSIM↑ | LPIPS↓ | |||
| Deformable 3DGS ( Yang et al., 2024b ) | 26.04 | 0.8805 | 0.1850 | 19.53 | 0.7855 | 0.1924 | 23.96 | 0.7945 | 0.2767 | 24.49 | 0.8822 | 0.1658 | |||
| 4DGS ( Yang et al., 2024c ) | 24.85 | 0.8632 | 0.2038 | 19.26 | 0.7670 | 0.2196 | 22.86 | 0.8015 | 0.2061 | 23.82 | 0.8695 | 0.1792 | |||
| GaussianShader ( Jiang et al., 2024 ) | 21.89 | 0.7739 | 0.3620 | 17.79 | 0.6670 | 0.4187 | 20.69 | 0.8169 | 0.3024 | 20.40 | 0.7437 | 0.3385 | |||
| GS-IR ( Liang et al., 2024 ) | 21.58 | 0.8033 | 0.3033 | 18.06 | 0.7248 | 0.3135 | 20.66 | 0.7829 | 0.2603 | 20.34 | 0.8193 | 0.2719 | |||
| NeRF-DS ( Yan et al., 2023 ) | 25.34 | 0.8803 | 0.2150 | 20.23 | 0.8053 | 0.2508 | 22.57 | 0.7811 | 0.2921 | 24.51 | 0.8802 | 0.1707 | |||
| Method | PSNR↑ | SSIM↑ | LPIPS↓ |
|---|---|---|---|
| General dynamic reconstruction methods | |||
| Deformable 3DGS ( Yang et al., 2024b ) | 22.78 | 0.6201 | 0.3380 |
| 4DGS ( Yang et al., 2024c ) | 24.89 | 0.6781 | 0.3422 |
| HyperNeRF ( Park et al., 2021b ) | 23.11 | 0.6387 | 0.3968 |
| Specular reconstruction methods | |||
| NeRF-DS ( Yan et al., 2023 ) | 23.65 | 0.6405 | 0.3972 |
| PSNR↑ | SSIM↑ | LPIPS↓ | ||||
|---|---|---|---|---|---|---|
| 16.29 | 0.6655 | 0.4126 | ||||
| ✓ | 18.97 | 0.7793 | 0.2389 | |||
| ✓ | ✓ | 20.25 | 0.8201 | 0.2313 | ||
| ✓ | ✓ | ✓ | 22.12 | 0.8441 | 0.2149 | |
| ✓ | ✓ | ✓ | ✓ | 25.23 | 0.8855 | 0.1536 |
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
| Method | GPU | Iterations | Training Time | FPS↑ |
|---|---|---|---|---|
| SpectroMotion ( Fan et al., 2024 ) | RTX 6000 Ada | 40,000 | 1.1 hours | 33 |
| Ours | RTX 6000 Ada | 60,000 | 2.8 hours | 32 |
| w/ coarse-to-fine | PSNR↑ | SSIM↑ | LPIPS↓ |
|---|---|---|---|
| 15.28 | 0.5782 | 0.4471 | |
| ✓ | 21.10 | 0.8415 | 0.1821 |
Dynamic Scene Reconstruction
Dynamic scene reconstruction builds time-varying 3D models of moving objects and environments from video or multi-view images.
See all 310 papers