Recent advances in 3D Gaussian Splatting (3DGS) have achieved state-of-the-art results for novel view synthesis. However, efficiently capturing high-fidelity reconstructions of specific objects within complex scenes remains a significant challenge. A key limitation of existing active reconstruction methods is their reliance on scene-level uncertainty metrics, which are often biased by irrelevant background clutter and lead to inefficient view selection for object-centric tasks. We present OUGS, a novel framework that addresses this challenge with a more principled, physically-grounded uncertainty formulation for 3DGS. Our core innovation is to derive uncertainty directly from the explicit physical parameters of the 3D Gaussian primitives (e.g., position, scale, rotation). By propagating the covariance of these parameters through the rendering Jacobian, we establish a highly interpretable uncertainty model. This foundation allows us to then seamlessly integrate semantic segmentation masks to produce a targeted, object-aware uncertainty score that effectively disentangles the object from its environment. This allows for a more effective active view selection strategy that prioritizes views critical to improving object fidelity. Experimental evaluations on public datasets demonstrate that our approach significantly improves the efficiency of the 3DGS reconstruction process and achieves higher quality for targeted objects compared to existing state-of-the-art methods, while also serving as a robust uncertainty estimator for the global scene.
Figures & tables
Figure 1: A complex background can inflate image-level uncertainty. This can mislead active view selection away from the object.
Figure 2: Object - aware uncertainty guides 3DGS view planning for precise object reconstruction. Our physically-grounded uncertainty model, derived from the explicit parameters of the 3D Gaussians, is combined with a semantic mask to generate an object-level uncertainty score. This score effectively guides the active view selection to improve object fidelity, as shown for 5, 10, and 20 selected views.
Figure 3: The parameterization of a 3D Gaussian primitive. Each Gaussian, the fundamental building block of our scene representation, is defined by a set of explicit physical parameters.
Figure 4: Object - aware approach speeds up convergence. Curves are recorded on the statue scene of the LF dataset as new views are added. Top: panoramic PSNR/SSIM/LPIPS; middle: the same metrics evaluated only inside the object. Bottom: visual progression at 5, 10, 15 and 20 views; red circles mark regions where the competing method keeps struggling while our reconstruction sharpens steadily.
Table 1: Active View Selection on Mip-NeRF360, Tanks&Temples, and LF datasets . "Panoramic" evaluates the full image; "Object-aware" evaluates only inside the object mask. Rows denote selection policies.
Figure 5: Qualitative result on Mip - NeRF360. From left to right are ground truth, Ours (OUGS), FisheRF, and Random. Each row corresponds to a different scene. 20 views were selected to train a model and render the result on the test set. The blue box circles the object of our interest, while the red box circles some of the background.
africa
basket
statue
torch
TNT-Train
TNT-Truck
Δ MSE ↓
Δ MAE ↓
Δ MSE ↓
Δ MAE ↓
Δ MSE ↓
Δ MAE ↓
Δ MSE ↓
Δ MAE ↓
MSE ↓
MAE ↓
MSE ↓
MAE ↓
ActiveNeRF
1.123
0.958
0.642
0.546
0.818
0.732
1.513
1.246
1.279
1.076
0.994
0.438
Bayes’ Rays
0.445
0.271
0.326
0.284
0.192
0.182
0.342
0.224
0.822
0.689
0.865
0.529
FisherRF
0.181
0.186
0.212
0.225
0.191
0.178
0.247
0.254
0.892
0.632
0.843
0.589
OUGS
0.192
0.187
0.122
0.131
0.181
0.181
0.248
0.217
0.787
0.589
0.651
0.487
Table 2: Validation of our parameter-centric FIM as a standalone uncertainty estimator. To isolate the quality of our core uncertainty model, we evaluate it on the full scene without any object-aware masks. The table reports the Area Under the Sparsification Error (AUSE), a metric for uncertainty quality (lower is better).
Figure 6: Qualitative validation of our parameter-centric uncertainty. These results on the TNT-Train and LF-Basket scenes showcase the remarkable accuracy of our uncertainty estimation. The uncertainty heatmap (right, yellow indicates high uncertainty) precisely localizes the regions where the final rendering (middle) deviates from the ground truth (left), such as blurry structures and ghosting artifacts. This strong visual correlation demonstrates the effectiveness of our physically-grounded model in predicting and explaining rendering errors.
Figure 7: Sensitivity to mask quality. We analyze object-level AUSE (lower is better) by varying the mask binarization threshold. The clear basin of stability indicates that OUGS is robust to a wide range of segmentation hyperparameters and effectively utilizes the mask to filter background clutter without requiring pixel-perfect thresholds.
Figure 8: NBV robustness under segmentation noise (TNT–Truck). Selected-view object PSNR (lower is better) versus noise intensity for Drop (left), Ghost (center), and Boundary Drift (right).
Method
PSNR ↑
PSNR Drop ↓
SSIM ↑
LPIPS ↓
I. Clean masks (ideal)
FisherRF
23.48
—
0.921
0.140
OUGS (ours)
26.25
—
0.933
0.117
II. “Ghost” noise (false positives)
FisherRF
20.15
3.33
0.845
0.210
OUGS (ours)
24.05
2.20
0.895
0.152
Table 3: Robustness of object-aware reconstruction on TNT under degraded object masks during NBV planning.
africa
basket
statue
torch
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
Online FisherRF
24.80
0.975
0.018
24.95
0.958
0.048
27.60
0.972
0.032
22.60
0.958
0.029
Offline FisherRF
32.45
0.991
0.011
29.35
0.973
0.038
31.85
0.983
0.024
29.66
0.978
0.020
Online OUGS
27.94
0.988
0.014
27.35
0.968
0.041
30.02
0.981
0.023
25.63
0.969
0.025
Offline OUGS
33.82
0.996
0.010
30.65
0.982
0.036
33.25
0.989
0.022
31.02
0.988
0.019
Table 4: Quantitative comparison across four scenes of the LF dataset. We compare our Warp-based mask generation against ground-truth SAM2 masks (GT). Metrics are evaluated on object regions after training with 20 views for 30k iterations.
EMA Schedule Strategy
Object-Aware PSNR (dB) ↑
No EMA (Instantaneous)
29.52
Low Momentum ( αt=0.90 )
31.63
Medium Momentum ( αt=0.95 )
31.91
High Momentum ( αt=0.99 )
32.04
Ours (Decaying Momentum)
32.18
Table 5: Ablation on the EMA update schedule, evaluated on the LF-Statue scene. Our decaying momentum strategy outperforms all constant momentum alternatives.
Active 3D Gaussian reconstruction fundamentally relies on selecting informative next-best views under limited sensing budgets. Existing active 3DGS methods primarily plan viewpoints according to geometric information gain, treating object-induced hidden regions in the same manner as general unexplored space. Under tight frame budgets, such geometry-driven strategies may prioritize global scene coverage while leaving partially observed objects incompletely reconstructed. To address this limitation, we propose OccamView, an object-conditioned view-selection framework for frame-budgeted active 3D Gaussian reconstruction. Rather than predicting unseen object geometry or performing shape completion, OccamView maintains an online object memory from open-vocabulary detections grounded in measured RGB-D observations and represents unresolved local occupancy around detected objects as conservative hidden-region proxies. Candidate viewpoints are then evaluated using an occlusion-aware proxy-coverage score. Furthermore, we introduce a Geo-Floor mechanism that restricts object-conditioned re-ranking to geometrically competitive candidates, allowing object-conditioned cues to guide complementary observations while preserving the geometry-driven exploration behavior of the underlying planner. Experiments on Replica and Matterport3D under a unified frame-budgeted protocol show that OccamView consistently reduces Completion and improves Completion Ratio across five frame budgets, with particularly pronounced gains under limited frame budgets. These results demonstrate that lightweight object-conditioned cues effectively complement geometry-driven active view planning.
Hongbo Gao, Wei Zhang, Zeyu Ni +4
Harbin Institute of Technology · The University of Sydney
We propose Self-Augmented Residual 3D Gaussian Splatting (SA-ResGS), a novel framework to stabilize uncertainty quantification and enhancing uncertainty-aware supervision in next-best-view (NBV) selection for active scene reconstruction. SA-ResGS improves both the reliability of uncertainty estimates and their effectiveness for supervision by generating Self-Augmented point clouds (SA-Points) via triangulation between a training view and a rasterized extrapolated view, enabling efficient scene coverage estimation. While improving scene coverage through physically guided view selection, SA-ResGS also addresses the challenge of under-supervised Gaussians, exacerbated by sparse and wide-baseline views, by introducing the first residual learning strategy tailored for 3D Gaussian Splatting. This targeted supervision enhances gradient flow in high-uncertainty Gaussians by combining uncertainty-driven filtering with dropout- and hard-negative-mining-inspired sampling. Our contributions are threefold: (1) a physically grounded view selection strategy that promotes efficient and uniform scene coverage; (2) an uncertainty-aware residual supervision scheme that amplifies learning signals for weakly contributing Gaussians, improving training stability and uncertainty estimation across scenes with diverse camera distributions; (3) an implicit unbiasing of uncertainty quantification as a consequence of constrained view selection and residual supervision, which together mitigate conflicting effects of wide-baseline exploration and sparse-view ambiguity in NBV planning. Experiments on active view selection demonstrate that SA-ResGS outperforms state-of-the-art baselines in both reconstruction quality and view selection robustness.
Kim Jun-Seong, Tae-Hyun Oh, Eduardo Pérez-Pellitero +1
We address the challenge of accurate 3D object reconstruction from multi-view images in Gaussian Splatting. Existing object-level 3DGS methods reconstruct the entire scene rather than directly optimizing the target object, even when only the target object is needed, which incurs substantial computational overhead. They also rely on 2D segmentation masks to associate Gaussians with objects, but these masks are often inconsistent across views. Such inconsistencies corrupt Gaussian optimization and produce incorrectly supervised Gaussians that degrade object reconstruction fidelity. To overcome these limitations, we propose MaRO-GS, a 3DGS framework that directly optimizes target-object Gaussians from object-masked multi-view images and remains robust to inconsistent supervision. For reliable supervision, mask-reliability view filtering excludes unreliable views. Object-supported Gaussian density control suppresses Gaussians irrelevant to the target object and prevents background densification, while Silhouette-aligned Object Loss maintains object-focused optimization. Extensive experiments across diverse datasets demonstrate that MaRO-GS improves PSNR, segmentation accuracy, and computational efficiency, with the largest PSNR gain of 2.05 dB on the small-object LERF-Mask dataset.
Eunji Kim, Gahyeon Kim, Gianella Cravioto +4
School of Computer Science and Engineering Kyungpook National University, Daegu, Republic of Korea