GrapeSplat: Geometry-Grounded Reconstruction via Amalgamated Pose-Free Encoding for Feed-Forward 3D Gaussian Splatting
Organizations: National Taiwan University · National Taiwan University of Science and Technology · NTU AI Center of Research Excellence (NTU AI-CoRE) · VinUniversity
Abstract
Feed-forward 3D Gaussian Splatting now reconstructs renderable scenes from unposed, uncalibrated images. Yet, most models supervise only photometric consistency and predict Gaussians pixel by pixel, which leaves global structure fragile and ties primitive count to image resolution and view count. To this end, GrapeSplat amalgamates multi-view cues into a voxel-aligned scene representation and decodes Gaussians directly from the learned grid, requiring no per-scene optimization or post-processing. An Atlas Encoder lifts all views into pixel-wise geometry-and-appearance features anchored at predicted 3D points. PEACH-Vox compands the unbounded scene into a bounded sparse grid through a smooth per-axis map with an exact closed-form inverse. The Sparse Decoder then consolidates the grid with sparse convolutions and decodes the full scene as multiple Gaussians per occupied cell. This amalgamated representation exploits sparse voxel occupancy, where the Gaussian count follows the occupied cells and saturates as views cover the scene, while grid resolution sets its ceiling. GrapeSplat turns unposed images into a renderable Gaussian scene in a single forward pass. Trained with 2D and 3D supervision on 8-view sequences, it generalizes zero-shot from 4 to 64 views across indoor and unbounded scenes. Code and trained weights are available at https://github.com/VAISR/GrapeSplat
Figures & tables
| NRGBD | ScanNet++ | 7-Scenes | DL3DV | |||||||||||||
| Method | 4 | 8 | 16 | 32 | 4 | 8 | 16 | 32 | 4 | 8 | 16 | 32 | 4 | 8 | 16 | 32 |
| AnySplat | 0.580 | 0.501 | 0.465 | 0.443 | 0.675 | 0.585 | 0.499 | 0.451 | 0.663 | 0.600 | 0.564 | 0.550 | 0.623 | 0.560 | 0.522 | 0.505 |
| 2Xplat | 0.570 | 0.463 | 0.368 | OOM | 0.657 | 0.560 | 0.472 | OOM | 0.665 | 0.581 | 0.530 | OOM | 0.656 | 0.593 | 0.503 | OOM |
| SplatWeaver | 0.579 | 0.487 | 0.443 | 0.417 | 0.652 | 0.564 | 0.495 | 0.451 | 0.652 | 0.588 | 0.550 | 0.539 | 0.614 | 0.554 | 0.518 | OOM |
| StructSplat | 0.523 | 0.473 | OOM | OOM | 0.604 | 0.557 | OOM | OOM | 0.628 | 0.578 | OOM | OOM | 0.617 | 0.596 | OOM | OOM |
| NRGBD | ScanNet++ | 7-Scenes | DL3DV | |||||||||||||
| Method | 4 | 8 | 16 | 32 | 4 | 8 | 16 | 32 | 4 | 8 | 16 | 32 | 4 | 8 | 16 | 32 |
| Voxel | 0.520 | 0.363 | 0.313 | 0.286 | 0.617 | 0.481 | 0.379 | 0.327 | 0.638 | 0.577 | 0.551 | 0.542 | 0.634 | 0.584 | 0.558 | 0.552 |
| +PEACH | 0.534 | 0.366 | 0.288 | 0.280 | 0.619 | 0.488 | 0.381 | 0.331 | 0.645 | 0.580 | 0.547 | 0.542 | 0.621 | 0.568 | 0.541 | 0.540 |
| +Cluster | 0.524 | 0.366 | 0.290 | 0.262 | 0.624 | 0.488 | 0.380 | 0.328 | 0.641 | 0.584 | 0.552 | 0.545 | 0.607 | 0.544 | 0.513 | 0.505 |
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
| Setting | Value |
|---|---|
| Decoding heads, one MLP per attribute | |
| Hidden widths | 128 then 512 |
| Cluster embedding init | normal, |
| Gaussian attributes | |
| Opacity | sigmoid, bias (0.12 at init) |
| Color | 16 harmonic coefficients per channel |
| DTU | ETH3D | NRGBD | |||||||
| Method | 4 | 8 | 16 | 4 | 4 | 8 | 16 | 32 | 64 |
| AnySplat | 7.86 | 9.59 | 12.00 | 6.83 | 9.87 | 13.64 | 15.92 | 16.68 | 16.93 |
| DA3 | 14.84 | 16.13 | 16.83 | 12.60 | 17.23 | 21.33 | 23.65 | 24.89 | 24.43 |
| 2Xplat | 9.76 | 11.98 | 13.84 | 8.59 | 12.30 | 16.97 | 19.22 | OOM | OOM |
| SplatWeaver | 9.24 | 11.59 | 13.57 | 6.87 | 9.64 | 14.03 | 16.70 | 17.63 | OOM |
| ScanNet++ | 7-Scenes | DL3DV | ||||||||||||
| Method | 4 | 8 | 16 | 32 | 4 | 8 | 16 | 32 | 64 | 4 | 8 | 16 | 32 | 64 |
| AnySplat | 7.79 | 10.24 | 13.20 | 15.26 | 9.28 | 11.87 | 13.45 | 13.82 | 13.87 | 10.50 | 12.72 | 13.98 | 14.54 | 14.76 |
| DA3 | 12.71 | 16.07 | 18.29 | 19.81 | 12.69 | 14.94 | 15.40 | 15.38 | 15.38 | 13.37 | 15.16 | 16.25 | 16.96 | 17.25 |
| 2Xplat | 9.28 | 13.01 | 15.70 | OOM | 10.24 | 13.95 | 15.16 | OOM | OOM | 12.37 | 14.27 | 15.77 | OOM | OOM |
| SplatWeaver | 8.44 | 11.27 | 14.18 | 16.16 | 9.44 | 12.18 | 13.96 | 14.28 | OOM | 11.04 | 13.15 | 14.38 | OOM | OOM |
| DTU | ETH3D | NRGBD | |||||||
| Method | 4 | 8 | 16 | 4 | 4 | 8 | 16 | 32 | 64 |
| Voxel | 8.61 | 9.32 | 10.12 | 6.48 | 9.27 | 14.62 | 18.51 | 21.22 | 21.49 |
| +PEACH | 8.32 | 9.09 | 9.91 | 6.77 | 9.31 | 14.62 | 19.06 | 21.43 | 22.13 |
| +Cluster | 8.25 | 8.82 | 9.61 | 6.73 | 9.04 | 14.22 | 18.53 | 21.42 | 21.89 |
| ScanNet++ | 7-Scenes | DL3DV | ||||||||||||
| Method | 4 | 8 | 16 | 32 | 4 | 8 | 16 | 32 | 64 | 4 | 8 | 16 | 32 | 64 |
| Voxel | 8.46 | 11.99 | 16.18 | 20.01 | 9.56 | 12.02 | 13.68 | 14.29 | 14.48 | 10.76 | 13.01 | 14.59 | 15.26 | 15.40 |
| +PEACH | 8.30 | 11.74 | 16.07 | 19.91 | 9.27 | 11.87 | 13.78 | 14.33 | 14.61 | 10.47 | 12.77 | 14.44 | 15.10 | 15.20 |
| +Cluster | 8.22 | 11.62 | 15.98 | 19.68 | 9.36 | 11.70 | 13.69 | 14.34 | 14.55 | 10.47 | 12.94 | 14.72 | 15.55 | 15.72 |
| DTU | ETH3D | NRGBD | |||||||
| Method | 4 | 8 | 16 | 4 | 4 | 8 | 16 | 32 | 64 |
| +PEACH | 8.32 | 9.09 | 9.91 | 6.77 | 9.31 | 14.62 | 19.06 | 21.43 | 22.13 |
| +Sem | 8.81 | 9.85 | 10.55 | 6.36 | 9.18 | 14.32 | 18.64 | 21.37 | 21.91 |
| +PEACH | 0.331 | 0.354 | 0.379 | 0.407 | 0.542 | 0.684 | 0.759 | 0.767 | 0.767 |
| ScanNet++ | 7-Scenes | DL3DV | ||||||||||||
| Method | 4 | 8 | 16 | 32 | 4 | 8 | 16 | 32 | 64 | 4 | 8 | 16 | 32 | 64 |
| +PEACH | 8.30 | 11.74 | 16.07 | 19.91 | 9.27 | 11.87 | 13.78 | 14.33 | 14.61 | 10.47 | 12.77 | 14.44 | 15.10 | 15.20 |
| +Sem | 8.29 | 11.75 | 15.98 | 19.77 | 9.32 | 11.84 | 13.76 | 14.41 | 14.57 | 10.33 | 12.68 | 14.25 | 14.87 | 14.96 |
| +PEACH | 0.538 | 0.611 | 0.695 | 0.749 | 0.454 | 0.498 | 0.534 | 0.549 | 0.562 | 0.322 | 0.360 | 0.390 | 0.406 | 0.413 |
| DTU | ETH3D | NRGBD | |||||||
| Method | 4 | 8 | 16 | 4 | 4 | 8 | 16 | 32 | 64 |
| Ours (white) | 8.25 | 8.82 | 9.61 | 6.73 | 9.04 | 14.22 | 18.53 | 21.42 | 21.89 |
| Ours (black) | 9.97 | 11.23 | 11.34 | 8.33 | 12.12 | 16.78 | 19.81 | 21.32 | 21.61 |
| Ours (white) | 0.336 | 0.348 | 0.370 | 0.404 | 0.552 | 0.672 | 0.737 | 0.767 | 0.760 |
| ScanNet++ | 7-Scenes | DL3DV | ||||||||||||
| Method | 4 | 8 | 16 | 32 | 4 | 8 | 16 | 32 | 64 | 4 | 8 | 16 | 32 | 64 |
| Ours (white) | 8.22 | 11.62 | 15.98 | 19.68 | 9.36 | 11.70 | 13.69 | 14.34 | 14.55 | 10.47 | 12.94 | 14.72 | 15.55 | 15.72 |
| Ours (black) | 8.25 | 11.77 | 16.31 | 19.80 | 9.80 | 11.99 | 13.40 | 14.17 | 14.43 | 10.31 | 12.65 | 14.40 | 15.27 | 15.44 |
| Ours (white) | 0.532 | 0.604 | 0.690 | 0.741 | 0.453 | 0.499 | 0.533 | 0.550 | 0.560 | 0.326 | 0.370 | 0.402 | 0.422 | 0.429 |
| DTU | ETH3D | NRGBD | |||||||
| Method | 4 | 8 | 16 | 4 | 4 | 8 | 16 | 32 | 64 |
| AnySplat | 0.536 | 0.514 | 0.477 | 0.696 | 0.580 | 0.501 | 0.465 | 0.443 | 0.432 |
| DA3 | 0.450 | 0.429 | 0.392 | 0.645 | 0.470 | 0.307 | 0.226 | 0.192 | 0.195 |
| 2Xplat | 0.607 | 0.558 | 0.489 | 0.671 | 0.570 | 0.463 | 0.368 | OOM | OOM |
| SplatWeaver | 0.549 | 0.514 | 0.486 | 0.704 | 0.579 | 0.487 | 0.443 | 0.417 | OOM |
| ScanNet++ | 7-Scenes | DL3DV | ||||||||||||
| Method | 4 | 8 | 16 | 32 | 4 | 8 | 16 | 32 | 64 | 4 | 8 | 16 | 32 | 64 |
| AnySplat | 0.675 | 0.585 | 0.499 | 0.451 | 0.663 | 0.600 | 0.564 | 0.550 | 0.546 | 0.623 | 0.560 | 0.522 | 0.505 | 0.499 |
| DA3 | 0.585 | 0.478 | 0.393 | 0.343 | 0.627 | 0.541 | 0.506 | 0.496 | 0.493 | 0.560 | 0.470 | 0.411 | 0.377 | 0.367 |
| 2Xplat | 0.657 | 0.560 | 0.472 | OOM | 0.665 | 0.581 | 0.530 | OOM | OOM | 0.656 | 0.593 | 0.503 | OOM | OOM |
| SplatWeaver | 0.652 | 0.564 | 0.495 | 0.451 | 0.652 | 0.588 | 0.550 | 0.539 | OOM | 0.614 | 0.554 | 0.518 | OOM | OOM |