AGILE-GS: Anchor-Guided Fast Next-Best-View Selection for Active 3D Gaussian Splatting

Amirhossein Mollaei Khass; Nader Motee

cs.CVSep 28, 2026

AGILE-GS: Anchor-Guided Fast Next-Best-View Selection for Active 3D Gaussian Splatting

Authors: Amirhossein Mollaei Khass, Nader Motee

Organizations: Department of Mechanical Engineering and Mechanics, Lehigh University, Bethlehem, PA, 18015, USA

Abstract

Radiance fields need hundreds of views, and their placement matters as much as their number. Next-best-view (NBV) selection for 3D Gaussian Splatting (3DGS) usually scores every candidate in the pool and keeps one. Searching for information and choosing a camera, however, are separable problems. We present AGILE-GS, an anchor-guided NBV method that separates the two. A virtual anchor pose is optimized on SE(3) by Riemannian gradient ascent on expected information gain. It need not be reachable or in the pool; it marks where the model is most uncertain. Candidates are scored against the anchor's viewing geometry, and a greedy ridge-leverage step distills the pool into a small, non-redundant shortlist without rendering any candidate. The shortlist can be used in two ways. AGILE-GS takes the first view on it as the next view, so no Fisher information is computed for any candidate. AGILE-GS+ computes the Fisher information gain of each shortlisted view and picks the best, so the expensive evaluation runs on a handful of views rather than the whole pool. On standard benchmarks and in closed-loop embodied acquisition, both match or exceed existing baselines while cutting selection latency by one to two orders of magnitude.

Figures & tables

Figure 1 : A simulated robot manipulator autonomously selects informative viewpoints for 3D Gaussian Splatting reconstruction.

Figure 2 : System overview. Given a candidate view pool $\mathcal{R}_{t}$ (red) around the active views $\mathcal{A}_{t}$ (blue), a virtual anchor is optimized to locate a maximally informative region of pose space (uncertainty heatmap). Candidates are scored against the anchor’s frustum sample geometry to form a score matrix, from which fast greedy ridge-leverage selection extracts a compact, non-redundant subset views.

T_{t}^{\star}\leftarrow\arg\max_{T\in\mathrm{SE}(3)}\psi_{t}(T)

Algorithm 1 AGILE-GS : Anchor-Guided Fast Next-Best-View Selection for Active 3D Gaussian Splatting

Figure 3 : Qualitative evaluation renders of the 3DGS comparison on representative views from different view selection methods.

Figure 4 : Accuracy–efficiency trade-off relative to COVER, normalized to $1\times$ selection speed and $0$ dB PSNR difference.

Embodied Method	Mip-NeRF 360			Custom
Embodied Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
Random	24.786	0.6712	0.2071	20.186	0.5441	0.2633
FisherRF	24.672	0.6712	0.2083	20.400	0.5550	0.2512
COVER	26.571	0.8158	0.1838	22.629	0.6912	0.2186
AGILE-GS	26.134	0.7960	0.1882	22.373	0.6514	0.2385
AGILE-GS+	26.431	0.8073	0.1895	23.835	0.7657	0.1962

Table 1 : Embodied view selection restricted to the K=5 nearest candidate frames.

Figure 5 : Selected views and reconstruction quality. The colored camera trajectories visualize the ordering of acquired views.

Method	Mip-NeRF 360			Tanks & Temples			Deep Blending
Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
Random	26.977	0.8306	0.1165	20.820	0.7693	0.1932	24.264	0.6245	0.2834
FisherRF	26.515	0.8225	0.1312	21.214	0.7816	0.1795	24.578	0.6856	0.2246
COVER	27.231	0.8283	0.1145	22.161	0.8028	0.1587	26.154	0.7196	0.1977
AGILE-GS	26.996	0.8310	0.1159	21.823	0.7872	0.1743	26.412	0.8344	0.1704
AGILE-GS+	27.523	0.8467	0.1112	22.003	0.7986	0.1683	27.328	0.8450	0.1632

Table 2 : Reconstruction quality and next-view selection efficiency on Mip-NeRF 360, Tanks & Temples, and Deep Blending. Reconstruction metrics are averaged over the evaluated scenes in each benchmark; selection time denotes the average latency per acquisition event. Best and second-best reconstruction results are shown in bold and underlined , respectively.

Dataset	Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	Speed up
Mip-NeRF 360 and Custom	$K$ -nearest Views	26.545	0.8603	0.2081	1.52 $\times$
	Online Row Sampling	27.202	0.8840	0.1600	1.48 $\times$
	Top independent norm	26.883	0.8845	0.1639	1.12 $\times$
	AGILE-GS (Ours)	27.692	0.9155	0.1413	1 $\times$

Table 3 : Batch view selection ablation averaged over Mip-NeRF 360 and the custom dataset.

AGILE-GS+ Budget $N$	Avg Mip-NeRF 360–Custom			Speed up selection $\uparrow$
AGILE-GS+ Budget $N$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	Speed up selection $\uparrow$
5	27.518	0.8339	0.1110	1.1 $\times$
10	27.523	0.8467	0.1112	1 $\times$
20	27.538	0.8471	0.1109	0.56 $\times$
50	27.520	0.8463	0.1110	0.34 $\times$

Table 4: Effect of the AGILE-GS+ shortlist candidate view set

N

Figure 6 : Embodied selection in Isaac Sim. Left: candidate views (green) and the NBV by Agile-GS (yellow). Right: the NBV $T_{t}^{\star}$ recovered by Riemannian ascent (yellow), concentrating on the least-constrained region of the scene.

Environment	Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	Views / Time (s)
Warehouse	Random	17.07	0.622	0.537
Warehouse	AGILE-GS	18.34	0.724	0.457	9 / 144.5 (s)
Kitchen	Random	19.70	0.782	0.481
Kitchen	AGILE-GS	20.44	0.823	0.472	11 / 621.8 (s)
Airlab	Random	21.94	0.811	0.324
Airlab	AGILE-GS	26.36	0.900	0.245	100 / 998.6 (s)

Table 5 : Embodied reconstruction in Isaac Sim environments.

Explore similar work

SA-ResGS: Self-Augmented Residual 3D Gaussian Splatting for Next Best View Selection
Jan 6, 2026Kim Jun-Seong, Tae-Hyun Oh, Eduardo Pérez-Pellitero +1Sparse-View Residual Learning
1POSTECH · 2KAIST · 3Huawei Noah’s Ark Lab +1
Rendering-Aware Bayesian 3D Gaussian Splatting with Native Uncertainty and Adaptive Complexity Control
Jul 6, 2026Gaoxiang Jia, Vikram Appia, Junzhou Huang +13D Gaussian Novel View Synthesis
Advanced Micro Devices, Inc. · Dept. of Computer Science and Engineering University of Texas at Arlington · Dept. of Mathematics & Division of Data Science University of Texas at Arlington
TRACE: Privacy-Preserving Next-Best-View Selection over Distributed 3D Gaussian-Splat Maps
Sep 30, 2026Amirhossein Mollaei Khass, Athanasios Cosse, Qiyu Sun +1Gaussian Splatting Simultaneous Localization And Mapping Systems 3D Mapping
Department of Mechanical Engineering and Mechanics, Lehigh University, Bethlehem, PA, 18015, USA

Jan 6, 2026, cs.CVJ/K move · Enter open · S save

SA-ResGS: Self-Augmented Residual 3D Gaussian Splatting for Next Best View Selection

Kim Jun-Seong, Tae-Hyun Oh, Eduardo Pérez-Pellitero, Youngkyoon Jang

1POSTECH · 2KAIST · 3Huawei Noah’s Ark Lab+1

Sparse-View Residual Learning

Open paper AskPDF arXiv

We propose Self-Augmented Residual 3D Gaussian Splatting (SA-ResGS), a novel framework to stabilize uncertainty quantification and enhancing uncertainty-aware supervision in next-best-view (NBV) selection for active scene reconstruction. SA-ResGS improves both the reliability of uncertainty estimates and their effectiveness for supervision by generating Self-Augmented point clouds (SA-Points) via triangulation between a training view and a rasterized extrapolated view, enabling efficient scene coverage estimation. While improving scene coverage through physically guided view selection, SA-ResGS also addresses the challenge of under-supervised Gaussians, exacerbated by sparse and wide-baseline views, by introducing the first residual learning strategy tailored for 3D Gaussian Splatting. This targeted supervision enhances gradient flow in high-uncertainty Gaussians by combining uncertainty-driven filtering with dropout- and hard-negative-mining-inspired sampling. Our contributions are threefold: (1) a physically grounded view selection strategy that promotes efficient and uniform scene coverage; (2) an uncertainty-aware residual supervision scheme that amplifies learning signals for weakly contributing Gaussians, improving training stability and uncertainty estimation across scenes with diverse camera distributions; (3) an implicit unbiasing of uncertainty quantification as a consequence of constrained view selection and residual supervision, which together mitigate conflicting effects of wide-baseline exploration and sparse-view ambiguity in NBV planning. Experiments on active view selection demonstrate that SA-ResGS outperforms state-of-the-art baselines in both reconstruction quality and view selection robustness.

Abstract

Explore similar work

SA-ResGS: Self-Augmented Residual 3D Gaussian Splatting for Next Best View Selection

Rendering-Aware Bayesian 3D Gaussian Splatting with Native Uncertainty and Adaptive Complexity Control

TRACE: Privacy-Preserving Next-Best-View Selection over Distributed 3D Gaussian-Splat Maps