UltraVoxelGS: Voxel-First Feed-Forward Gaussian Splatting for 3D Ultrasound Reconstruction
Organizations: Department of Computer Science, Hong Kong Baptist University, Hong Kong SAR, China
Abstract
We present UltraVoxelGS, a 3D ultrasound reconstruction method that achieves both fast reconstruction and physically faithful rendering for the first time, bringing live spatial feedback during 2D ultrasound scanning within reach. This is made possible by a feed-forward design that predicts a complete Gaussian scene representation from posed ultrasound images directly, entirely bypassing the per-scene optimization bottleneck that has confined prior methods to offline use. Realising such feed-forward prediction in this setting is non-trivial: unlike natural images, ultrasound slices share at most one-dimensional intersections and exhibit view-dependent appearance due to acoustic propagation and attenuation, violating the dense overlap and photometric consistency assumed by existing feed-forward Gaussian methods. Hence, to achieve fast reconstruction, we introduce a feed-forward voxel-to-Gaussian pass that decouples representation size from input count by lifting all posed slices into a fixed-size voxel support and predicting Gaussian parameters directly. For faithful rendering, we propose ultrasound-aware appearance rendering strategy factorize reflection from cumulative attenuation during training and exploit appearance continuity among spatially adjacent slices for efficient Render-time Appearance Adaptation. On both real-world and simulated datasets, UltraVoxelGS achieves best PSNR/SSIM compared with per-scene optimization baselines while reducing inference from 10-20 minutes to approximately 10 seconds (over 35 times faster) and scales to sequences exceeding 1,000 frames. Code is available at https://github.com/YuebyYue/UltraVoxelGS-camera-ready.
Figures & tables
| Method | Type | Dataset A: Real-world | Dataset B: Simulated | ||||
|---|---|---|---|---|---|---|---|
| PSNR | SSIM | LPIPS | PSNR | SSIM | LPIPS | ||
| Kernel Interpolation [ 6 ] | Interpolation | 13.52 | 0.3952 | 0.5419 | 17.47 | 0.2391 | 0.6259 |
| ImplicitVol [ 8 ] | Per-scene optimization | 19.13 | 0.6128 | 0.5218 | 18.97 | 0.2837 | 0.7192 |
| UltraNeRF [ 10 ] | Per-scene optimization | 18.49 | 0.4056 | 0.5761 | 17.75 | 0.1975 | 0.5985 |
| 3DGS [ 12 ] | Per-scene optimization | 22.55 | 0.6695 | 0.3884 | 18.04 | 0.2532 | 0.6702 |
| UltraGauss [ 13 ] | Per-scene optimization | 22.06 | 0.6220 | 0.4750 | 15.01 | 0.1838 | 0.7057 |
Appendix figures & tables17 assets
Supplementary material from the paper’s appendix.
Appendix
| Item | Setting |
|---|---|
| Input image resolution | |
| Voxel grid spacing | |
| Voxel patch size | |
| Top- neighbors | 3 |
| SH order | 2 |
| Gaussian parameters |
| Method | Type | Input Setting | Optimization / Inference Setting |
|---|---|---|---|
| Kernel Interpolation | Interpolation | Same sparse input | No optimization |
| ImplicitVol | Per-scene optimization | Same sparse input | 50k steps |
| UltraNeRF | Per-scene optimization | Same sparse input | 10k steps |
| 3DGS | Per-scene optimization | Same sparse input | 10k steps |
| UltraGauss | Per-scene optimization | Same sparse input | 10k steps |
| UltraVoxelGS w/o RAA | Feed-forward | Same sparse input | Feed-forward only |
| Stage | Time (s) |
|---|---|
| Voxel support construction | 1.12 |
| View-interacted block extraction | 1.96 |
| Network inference | 4.83 |
| Gaussian fusion | 2.13 |
| Representation construction | 10.04 |
| Rendering per target view w/o RAA | 0.00362 |
| Variant | Steps | PSNR | SSIM | LPIPS | Render / View (s) |
|---|---|---|---|---|---|
| w/o RAA | 0 | 19.76 | 0.5770 | 0.5130 | 0.00362 |
| RAA | 10 | 23.00 | 0.7484 | 0.3513 | 0.140 |
| RAA | 20 | 23.95 | 0.8003 | 0.2872 | 0.259 |
| RAA | 50 | 24.29 | 0.8329 | 0.2380 | 0.623 |
| RAA | 100 | 24.19 | 0.8350 | 0.2224 | 1.212 |
| Strategy | Anchor | Translation distance (mm) | Rotation angle ( ∘ ) | PSNR | SSIM | LPIPS |
| UltraVoxelGS | Top-1 | 20.87 | 0.3138 | 0.6190 | ||
| Translation | Top-3 | 19.89 | 0.2889 | 0.6919 | ||
| Top-5 | 19.39 | 0.2813 | 0.7018 | |||
| Top-10 | 18.76 | 0.2719 | 0.7136 | |||
| Rotation | Top-3 | 19.91 | 0.2889 | 0.6922 | ||
| Top-5 | 19.41 | 0.2812 | 0.7020 |
| Per-scene optimization | RAA | |
|---|---|---|
| Target | Complete scene | Current queried view |
| Updated variables | Geometry and appearance | Transient |
| Global representation | Optimized per scene | Feed-forward and fixed |
| Scope | Entire scene | Query-relevant local support |
| Persistence | Stored in the scene | Discarded after rendering |
| Runtime | Increases with scene scale and input views | Independent of source views (623 ms / 50 steps) |
| Rendering Variant | Decoupled | SH Reflection | PSNR | SSIM |
|---|---|---|---|---|
| UltraGauss | ✗ | ✗ | 22.06 | 0.6220 |
| UltraVoxelGS-opt | ✓ | ✓ | 22.55 | 0.6245 |
| Variant | Consistency Loss | RAA | PSNR | SSIM |
|---|---|---|---|---|
| Block-wise only | ✗ | ✗ | 19.85 | 0.5742 |
| Block-wise + Consistency | ✓ | ✗ | 19.76 | 0.5770 |
| Support resolution | Method | PSNR | SSIM | LPIPS |
|---|---|---|---|---|
| mm | UltraVoxelGS w/o RAA | 19.76 | 0.5770 | 0.5130 |
| mm | UltraVoxelGS | 24.29 | 0.8329 | 0.2384 |
| mm | UltraVoxelGS w/o RAA | 20.14 | 0.5909 | 0.4719 |
| mm | UltraVoxelGS | 24.22 | 0.8200 | 0.2131 |
| Method | Regime | PSNR | SSIM | LPIPS |
|---|---|---|---|---|
| VolSplat | Feed-forward | 15.70 | 0.4522 | 0.5884 |
| UltraVoxelGS w/o RAA | FF + DRA | 19.76 | 0.5770 | 0.5130 |
| UltraVoxelGS | FF + DRA + RAA | 24.29 | 0.8329 | 0.2384 |
| (mm) | ( ∘ ) | Condition | PSNR | SSIM | LPIPS |
|---|---|---|---|---|---|
| 0 | 0 | A | |||
| 8 | 4 | B | |||
| C | |||||
| D | |||||
| 12 | 6 | B | |||
| C |
| Setting | PSNR | SSIM | LPIPS |
|---|---|---|---|
| UltraGauss (Handheld acquisition) | 20.05 | 0.5858 | 0.5580 |
| UltraVoxelGS (Robotic acquisition) | 24.29 | 0.8329 | 0.2384 |
| UltraVoxelGS (Handheld acquisition) | 23.96 | 0.7703 | 0.3356 |
| Setting | PSNR | SSIM | LPIPS |
|---|---|---|---|
| Progressive streaming | 24.08 | 0.8240 | 0.2555 |
| Final streaming | 24.11 | 0.8280 | 0.2476 |
| Direct reconstruction | 24.29 | 0.8329 | 0.2384 |
3D Gaussian Splatting
3D Gaussian Splatting represents scenes as optimizable 3D Gaussian primitives for fast, photorealistic rendering from new viewpoints.
See all 1,080 papers