MaRO-GS: Mask-Robust Object-Centric Gaussian Splatting from Inconsistent Multi-view Masks
Organizations: School of Computer Science and Engineering Kyungpook National University, Daegu, Republic of Korea
Abstract
We address the challenge of accurate 3D object reconstruction from multi-view images in Gaussian Splatting. Existing object-level 3DGS methods reconstruct the entire scene rather than directly optimizing the target object, even when only the target object is needed, which incurs substantial computational overhead. They also rely on 2D segmentation masks to associate Gaussians with objects, but these masks are often inconsistent across views. Such inconsistencies corrupt Gaussian optimization and produce incorrectly supervised Gaussians that degrade object reconstruction fidelity. To overcome these limitations, we propose MaRO-GS, a 3DGS framework that directly optimizes target-object Gaussians from object-masked multi-view images and remains robust to inconsistent supervision. For reliable supervision, mask-reliability view filtering excludes unreliable views. Object-supported Gaussian density control suppresses Gaussians irrelevant to the target object and prevents background densification, while Silhouette-aligned Object Loss maintains object-focused optimization. Extensive experiments across diverse datasets demonstrate that MaRO-GS improves PSNR, segmentation accuracy, and computational efficiency, with the largest PSNR gain of 2.05 dB on the small-object LERF-Mask dataset.
Figures & tables
| Category | Method | PSNR | SSIM | LPIPS | FPS |
| LERF-Mask (2.01%) | |||||
| Baseline | 3DGS [ 20 ] | 40.67 | 0.990 | 0.010 | 639 |
| Efficient GS | PUP 3D-GS [ 14 ] | 39.50 | 0.989 | 0.012 | 1,934 |
| Object-centric | ObjectGS [ 48 ] | 37.65 | 0.988 | 0.013 | 350 |
| Trace3D [ 35 ] | 34.74 | 0.986 | 0.019 | 772 | |
| High-Fidelity | FDS-GS [ 44 ] | 40.84 | 0.990 | 0.010 | 449 |
| Method | Preprocessing | Training | Rendering | Total | Max VRAM |
|---|---|---|---|---|---|
| Time (s) | Time (s) | Time (s) | Time (s) | (MB) | |
| 3DGS [ 20 ] | N/A | 363 | 4.84 | 367.84 | 4,305 |
| PUP 3D-GS [ 14 ] | 363 | 103 | 4.69 | 470.69 | 3,551 |
| ObjectGS [ 48 ] | 338 | 2,196 | 8.07 | 2,542.07 | 24,300 |
| Ours | N/A | 345 | 3.53 | 348.53 | 4,187 |
Appendix figures & tables35 assets
Supplementary material from the paper’s appendix.
Appendix
| Example Prompt (Mip-NeRF 360) |
|---|
| A bicycle parked outdoors on the ground. |
| A road bike standing alone in an outdoor scene. |
| A two-wheeled bicycle with handlebars and a seat. |
| A bicycle with thin wheels leaning or standing in a park. |
| A classic bicycle frame with wheels visible in the scene. |
| A bike parked on a dirt or gravel path surrounded by trees. |
| Scene | Object | Object Ratio (%) | Resolution |
| figurines (15) | camera | 1.67 | 986 728 |
| figurines (33) | hand figurine | 0.33 | |
| figurines (36) | green apple | 1.05 | |
| figurines (40) | cat figurine | 0.30 | |
| figurines (43) | rabbit figurine | 0.10 | |
| figurines (49) | elephant figurine | 0.81 |
| Scene | Object | Object Ratio (%) | Error Ratio (%) | Resolution |
| scan2 | rabbit statue | 36.19 | 0.0 | 1,588 1,190 |
| scan4 | bird doll | 17.86 | 0.0 | 1,587 1,190 |
| scan11 | power supply | 57.27 | 6.0 | 1,583 1,186 |
| scan31 | pumpkin | 22.08 | 10.0 | 1,586 1,189 |
| scan32 | cacao bag | 29.61 | 6.0 | 1,587 1,190 |
| scan36 | brick | 20.51 | 8.0 | 1,588 1,190 |
| Setting | Method | PSNR | SSIM | LPIPS |
|---|---|---|---|---|
| Full scene | ObjectGS [ 48 ] | 23.45 | 0.860 | 0.235 |
| Trace3D [ 35 ] | 22.64 | 0.821 | 0.263 | |
| Extracted from full scene | ObjectGS [ 48 ] | 39.90 | 0.995 | 0.003 |
| Trace3D [ 35 ] | 36.36 | 0.994 | 0.005 | |
| Masked input (Main Table 1) | ObjectGS [ 48 ] | 37.65 | 0.988 | 0.013 |
| Trace3D [ 35 ] | 34.74 | 0.986 | 0.019 |
| P | M | G | R | PSNR | FPS | #Gaussians |
|---|---|---|---|---|---|---|
| – | – | – | – | 41.98 | 1,458 | 42,846 |
| ✓ | – | – | – | 42.55 | 1,630 | 25,805 |
| – | ✓ | – | – | 42.38 | 1,586 | 29,966 |
| – | – | ✓ | – | 41.86 | 1,401 | 43,576 |
| – | – | – | ✓ | 37.11 | 1,281 | 60,735 |
| ✓ | ✓ | – | – | 42.52 | 1,784 | 21,157 |
| Geo. | Reg. | Easy | Medium | Hard |
|---|---|---|---|---|
| 31.84 | 23.23 | 23.13 | ||
| ✓ | 31.66 | 25.68 | 23.07 | |
| ✓ | 33.34 | 25.98 | 23.03 | |
| ✓ | ✓ | 33.26 | 26.50 | 22.88 |
| Difficulty | # Scenes | Avg. # of input views | Avg. # of error views | Avg. # of filtered views | Recall (%) | Precision (%) |
|---|---|---|---|---|---|---|
| Easy | 21 | 53.3 | 2.33 | 0.8 | 24.5 | 75.0 |
| Medium | 3 | 49.0 | 21.7 | 5.0 | 23.1 | 100.0 |
| Hard | 4 | 49.0 | 38.0 | 1.8 | 4.6 | 100.0 |
| Geo. | Reg. | PSNR | SSIM | LPIPS |
|---|---|---|---|---|
| 18.82 | 0.809 | 0.192 | ||
| ✓ | 20.84 | 0.846 | 0.173 | |
| ✓ | 18.89 | 0.810 | 0.193 | |
| ✓ | ✓ | 20.71 | 0.846 | 0.174 |
| Filtering Schedule | LERF-Mask | DTU (Medium) | |||||
|---|---|---|---|---|---|---|---|
| Geo. Iter. | Reg. Iter. | PSNR | SSIM | LPIPS | PSNR | SSIM | LPIPS |
| 1K/3K/5K (default) | 1.8K | 42.98 | 0.992 | 0.008 | 26.50 | 0.939 | 0.085 |
| 1K/3K/5K | 5.8K | 42.86 | 0.992 | 0.008 | 26.57 | 0.939 | 0.084 |
| 5K/7K/9K | 1.8K | 42.89 | 0.992 | 0.008 | 25.95 | 0.934 | 0.089 |
| 5K/7K/9K | 5.8K | 42.87 | 0.992 | 0.009 | 26.58 | 0.939 | 0.085 |
| Parameter | Value | LERF-Mask | Mip-NeRF 360 | ||
|---|---|---|---|---|---|
| PSNR | LPIPS | PSNR | LPIPS | ||
| 0.00 | 44.12 | 0.0085 | 34.58 | 0.0243 | |
| 0.25 | 44.18 | 0.0085 | 34.61 | 0.0244 | |
| 0.50 | 44.17 | 0.0082 | 34.45 | 0.0244 | |
| 0.75 | 44.24 | 0.0085 | 34.54 | 0.0244 | |
| 1.00 | 44.13 | 0.0086 | 34.50 | 0.0244 | |
| Tanks and Temples | ||||
|---|---|---|---|---|
| Scene | Barn | Caterpillar | Ignatius | Truck |
| Error Rate | 30.97 | 62.14 | 25.70 | 52.58 |
| Mip-NeRF 360 | ||||
| Scene | bicycle | bonsai | counter | garden |
| Error Rate | 80.51 | 72.26 | 52.50 | 61.08 |
| ObjectGS [ 48 ] | MaRO-GS (Ours) | |
|---|---|---|
| Main Task | Object-aware scene reconstruction and understanding | Robust object-centric reconstruction under inconsistent mask supervision |
| Input Data | 2D multi-view images + 2D instance masks | Object-masked multi-view images + 2D object masks |
| Problem Formulation | Semantic ambiguity in scene-level rendering | Rendering instability under inconsistent mask supervision |
| Role of Masks | Semantic supervision for object-level classification | Object supervision for Gaussian optimization |
| Dataset | Method | PSNR | SSIM | LPIPS |
|---|---|---|---|---|
| LERF-Mask | FDS-GS | 40.76 0.38 | 0.991 0.0001 | 0.009 0.0001 |
| Ours | 42.90 0.14 | 0.992 0.0000 | 0.008 0.0000 | |
| Mip-NeRF 360 | FDS-GS | 33.19 0.29 | 0.977 0.0012 | 0.024 0.0008 |
| Ours | 34.64 0.11 | 0.979 0.0001 | 0.022 0.0000 | |
| Tanks & Temples | 3DGS | 30.68 0.13 | 0.964 0.0002 | 0.049 0.0001 |
| PUP 3D-GS | 30.07 0.08 | 0.955 0.0002 | 0.070 0.0001 |
| Split | Method | PSNR | SSIM | LPIPS |
|---|---|---|---|---|
| Easy | ObjectGS | 31.55 0.11 | 0.967 0.0008 | 0.056 0.0004 |
| Ours | 33.26 0.39 | 0.963 0.0001 | 0.055 0.0003 | |
| Medium | ObjectGS | 23.27 0.59 | 0.903 0.0038 | 0.116 0.0043 |
| Ours | 26.50 0.05 | 0.939 0.0012 | 0.085 0.0006 | |
| Hard | ObjectGS | 21.15 0.43 | 0.913 0.0034 | 0.106 0.0018 |
| Ours | 22.95 0.10 | 0.923 0.0003 | 0.085 0.0005 |