Factorized Multi-Resolution HashGrid for Efficient Neural Radiance Fields: Execution on Edge-Devices
Organizations: Dept. of E.E., POSTECH · Dept. of CS, The Univ. of British Columbia · Grad. School of AI, POSTECH · School of Computing Engineering, KAIST ·
Abstract
We introduce Fact-Hash, a novel parameter-encoding method for training on-device neural radiance fields. Neural Radiance Fields (NeRF) have proven pivotal in 3D representations, but their applications are limited due to large computational resources. On-device training can open large application fields, providing strength in communication limitations, privacy concerns, and fast adaptation to a frequently changing scene. However, challenges such as limited resources (GPU memory, storage, and power) impede their deployment. To handle this, we introduce Fact-Hash, a novel parameter-encoding merging Tensor Factorization and Hash-encoding techniques. This integration offers two benefits: the use of rich high-resolution features and the few-shot robustness. In Fact-Hash, we project 3D coordinates into multiple lower-dimensional forms (2D or 1D) before applying the hash function and then aggregate them into a single feature. Comparative evaluations against state-of-the-art methods demonstrate Fact-Hash's superior memory efficiency, preserving quality and rendering speed. Fact-Hash saves memory usage by over one-third while maintaining the PSNR values compared to previous encoding methods. The on-device experiment validates the superiority of Fact-Hash compared to alternative positional encoding methods in computational efficiency and energy consumption. These findings highlight Fact-Hash as a promising solution to improve feature grid representation, address memory constraints, and improve quality in various applications. Project page: https://facthash.github.io/
Figures & tables
| Case | Model | Avg. PSNR | Avg. SSIM | Avg. LPIPS | ||
| Name | #Params (M) | Size (Mb) | ||||
| Full views | iNGP | 11.46 | 45.24 | 31.59 | 0.953 | 0.061 |
| TensoRF | 17.38 | 68.69 | 32.88 | 0.961 | 0.028 | |
| K-planes | 33.04 | 393.08 | 32.36 | 0.962 | 0.049 | |
| TensoRF | 6.51 | 25.73 | 31.69 | 0.951 | 0.064 | |
| K-planes | 16.53 | 64.82 | 32.06 | 0.956 | 0.057 | |
| Model | 10 Watt | 20 Watt Avg. | ||||||||
| chair | drums | ficus | hotdog | lego | materials | mic | ship | Avg. | ||
| iNGP | 71.8 | 142.3 | 507.0 | 167.4 | 126.3 | 251.3 | 96.9 | 566.6 | 241.2 | 191.4 |
| TensoRF | 109.5 | 199.8 | 174.3 | 199.2 | 188.4 | 395.6 | 149.0 | 641.1 | 257.1 | 219.6 |
| K-Planes | 97.8 | 195.8 | 159.1 | 206.4 | 167.8 | 339.3 | 122.1 | 504.7 | 224.1 | 185.9 |
| Ours | 93.7 | 164.5 | 145.1 | 130.7 | 161.8 | 319.1 | 89.6 | 229.8 | 166.8 | 133.2 |
| Case | Model | Avg. PSNR | Avg. SSIM | Avg. LPIPS | ||
| Name | #Params (M) | Size (Mb) | ||||
| Full views | iNGP | 11.46 | 44.18 | 28.05 | 0.918 | 0.128 |
| TensoRF | 4.89 | 18.92 | 27.93 | 0.912 | 0.139 | |
| K-planes | 16.53 | 63.30 | 28.40 | 0.923 | 0.120 | |
| Ours | 3.40 | 13.22 | 28.42 | 0.919 | 0.121 | |
| 10% views | iNGP | 11.46 | 44.18 | 19.73 | 0.869 | 0.186 |
Neural Radiance Fields
Neural radiance fields represent 3D scenes as continuous functions of position and viewing direction to synthesize images from new camera views.
See all 82 papers