MAGICIAN: Efficient Long-Term Planning with Imagined Gaussians for Active Mapping
Authors: Shiyao Li, Antoine Guédon, Shizhe Chen, Vincent Lepetit
Organizations: LIGM, École Nationale des Ponts et Chaussées, IP Paris, Univ Gustave Eiffel, CNRS, France · École Polytechnique, France · Inria, École normale supérieure, CNRS, PSL Research University, France
Active mapping aims to determine how an agent should move to efficiently reconstruct unknown environments. Most existing approaches rely on greedy next-best-view prediction, resulting in inefficient exploration and incomplete reconstruction. To address this, we introduce MAGICIAN, a novel long-term planning framework that maximizes accumulated surface coverage gain through Imagined Gaussians, a scene representation based on 3D Gaussian Splatting, derived from a pre-trained occupancy network with strong structural priors. This representation enables efficient coverage gain computation for any novel viewpoint via fast volumetric rendering, allowing its integration into a tree-search algorithm for long-horizon planning. We update Imagined Gaussians and refine the trajectory in a closed loop. Our method achieves state-of-the-art performance across indoor and outdoor benchmarks with varying action spaces, highlighting the advantage of long-term planning in active mapping.
Figures & tables
Figure 2: Overview of the proposed MAGICIAN framework. At time t , we first predict the occupancy field using the occupancy model and update the Imagined Gaussians. We can then efficiently estimate the coverage gain and apply beam search to plan Nb candidate trajectories, selecting the one with the highest expected gain. The agent then executes the first Nf actions of the best trajectory τk of length Nd before repeating this process in the next planning loop. In this figure, lighter colors in the Imagined Gaussians indicate higher novelty, while darker colors correspond to previously observed areas. The first trajectory darkens the novelty field the most, representing the optimal path at time t .
Figure 3: Computing coverage gain with Imagined Gaussians. During beam search, we evaluate candidate poses by rendering novelty maps from the Imagined Gaussians to compute the coverage gain. The corresponding depth maps are then used to update the novelty γ^ of Gaussians within a depth tolerance ϵd .
Figure 4: Evolution of Imagined Gaussians Compared with Ground Truth Mesh. The brighter the Gaussians, the higher their predicted occupancy. As exploration progresses (from left to right), our Imagined Gaussians increasingly align with the ground truth mesh, demonstrating improved environmental modeling.
Figure 5: 3D reconstructions obtained with our trajectories. We show Gaussian splatting renderings (top row) and normal maps of the reconstructed meshes (bottom row) after applying Mesh-In-the-Loop Gaussian Splatting [ 17 ] on 100 RGB images collected along our trajectories. The trajectories output by our method cover the entire scene surfaces, resulting in complete and accurate surface meshes.
MP3D
Macarons++
Wheeled Robot
Drone
Drone
Action space
Forward (6.5 cm), Turn ( ±10∘ )
6 DoF
6 DoF
Camera Height
1.25 m above floor
N/A (Free 3D)
N/A (Free 3D)
Image Resolution
256×256
1200×680
456×256
Field of View
90∘×90∘
60∘×90∘
60∘×91.6∘
Total Steps
1,000 or 2,000
5,000
100
Table 1: Configuration details on MP3D and Macarons++ benchmarks for active mapping evaluation.
Figure 6: Qualitative comparison of novel view synthesis (top row) and surface reconstruction (bottom row) in outdoor and indoor scenes. For each method, we show RGB Gaussian splatting renderings and normal maps of reconstructed meshes after applying Mesh-In-the-Loop Gaussian Splatting [ 17 ] on 100 images collected along the trajectory. The trajectories computed with our method produce more accurate and complete reconstructions, resulting in better rendering quality and preventing holes in reconstructed surfaces.
AUC ↑
Final coverage ↑
Random Walk
0.241
0.324
SCONE [ 19 ]
0.534
0.670
MACARONS [ 20 ]
0.647
0.819
FisherRF [ 26 ]
0.546
0.786
MAGICIAN (Ours)
0.721
0.919
Table 2: Evaluation results on the Macarons++ dataset.
Method
SSIM ↑
PSNR ↑
LPIPS ↓
Acc. (%) ↑
FisherRF [ 26 ]
0.55
13.95
0.38
79.15
MACARONS [ 20 ]
0.61
15.68
0.34
86.42
MAGICIAN (Ours)
0.64
17.12
0.30
94.20
Table 3: Evaluation of novel-view rendering and mesh reconstruction on large-scale real-world scanned scenes. Our method achieves the best performance across all metrics.
Setting
Method
Comp. (%) ↑
Comp. (cm) ↓
Wheeled Robot
Random
45.67
26.53
FBE [ 44 ]
71.18
9.78
UPEN [ 16 ]
69.06
10.60
OccAnt [ 38 ]
71.72
9.40
ANM [ 45 ]
73.15
9.11
NBP [ 32 ]
79.38
6.78
Table 4: Evaluation results on the MP3D dataset. Our method consistently outperforms existing approaches under various robot and action-space settings.
Figure 7: Ablation study on the beam search parameters. The horizontal axis denotes the beam width Nb , and the vertical axis represents the look-ahead steps Nd . Five steps correspond roughly to half the size of the scene.
Figure 8: Ablation study on replanning frequency. The horizontal axis indicates the number Nf of movement steps executed after each planning phase before replanning. Replanning every 6 steps already provides state-of-the-art results.
Occupancy Model
AUC ↑
Cov. ↑
Pretrained
0.652
0.888
Fine-tuned
0.646
0.893
Table 5: Ablation study on comparing models with and without fine-tuning on indoor environments. The fine-tuned version shows a minor 0.5% improvement in final coverage, while the original model retains higher exploration efficiency.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Scene
Rand. Walk
SCONE [ 19 ]
MACARONS [ 20 ]
FisherRF [ 26 ]
MAGICIAN (Ours)
Dunnottar Castle
0.149
0.366
0.618
0.500
0.745
Colosseum
0.219
0.589
0.656
0.551
0.704
Bannerman Castle
0.192
0.559
0.575
0.595
0.761
Pantheon
0.198
0.465
0.601
0.270
0.644
Christ the Redeemer
0.439
0.772
0.859
0.727
0.793
Statue of Liberty
0.323
0.632
0.711
0.553
0.797
Appendix
Table 6: AUCs of full scenes on the Macarons++ dataset.
Scene
Rand. Walk
SCONE [ 19 ]
MACARONS [ 20 ]
FisherRF [ 26 ]
MAGICIAN (Ours)
Dunnottar Castle
0.225
0.527
0.820
0.809
0.975
Colosseum
0.272
0.755
0.794
0.757
0.872
Bannerman Castle
0.240
0.722
0.834
0.801
0.917
Pantheon
0.309
0.610
0.796
0.444
0.842
Christ the Redeemer
0.581
0.924
0.967
0.876
0.973
Statue of Liberty
0.443
0.819
0.909
0.819
0.947
Appendix
Table 7: Final Coverages of full scenes on the Macarons++ dataset.
Figure 9: Standard deviation of the final coverage across different methods and scenes. Our method achieves consistently low values for this metric, indicating strong robustness to random starting poses, whereas other methods exhibit much larger variability.
Figure 10: Visualization of exploration trajectories and qualitative comparisons of novel view synthesis and surface reconstruction in outdoor scenes. From top to bottom, the scenes are Pisa Cathedral and Neuschwanstein Castle. In the same scene, all methods start from the same initial camera pose, and for each trajectory visualization, we additionally show the final camera pose at the end of the trajectory. Our trajectory planning method yields more accurate and complete reconstructions, resulting in higher-quality renderings and effectively preventing holes or noise in the reconstructed surfaces.
Figure 11: Visualization of exploration trajectories and qualitative comparisons of novel view synthesis and surface reconstruction in indoor scenes. From top to bottom, the scenes are St. Sofia Church and Barts. In the same scene, all methods start from the same initial camera pose, and for each trajectory visualization, we additionally show the final camera pose at the end of the trajectory. Our trajectory planning method yields more accurate and complete reconstructions, resulting in higher-quality renderings and effectively preventing holes or noise in the reconstructed surfaces.
Nd
10
15
20
25
30
50
AUC
0.652
0.673
0.664
0.662
0.658
0.652
Cov.
0.888
0.887
0.892
0.879
0.881
0.878
Appendix
Table 8: Ablation study on look-ahead steps Nd .
Density
0.5×
1×
2×
4×
AUC
0.640
0.652
0.672
0.685
Cov.
0.848
0.888
0.895
0.905
Appendix
Table 9: Ablation study on proxy point sampling density ( 1× indicates the original density).
Existing active reconstruction systems with Gaussian-splatting maps select observations greedily, optimizing a single next-best-view (NBV) at each step and connecting the chosen views by short-horizon path planning. This greedy decoupling disregards the global structure of scene information, producing inefficient trajectories that waste sensing capacity in transit between selected views. In this work, we study active reconstruction as an ergodic coverage problem: the time-averaged spatial statistics of the sensor trajectory should match a target information distribution induced by the current map. Our approach derives this target distribution online from uncertainty and visibility, and calculates ergodic trajectories via a kernel-ergodic horizon planner with gradient flow and footprint depletion, closing the loop between mapping and trajectory optimization. We thoroughly evaluate TRACE on the Replica dataset against the Next-Best-View (NBV) baselines, improving PSNR by 1.5 dB. Code: https://github.com/spikelab-jhu/trace-active-reconstruction.
Ziyue Zheng, Linli Shi, Bingkun He +2
1Johns Hopkins University, USA · University of Pennsylvania, USA
To advance the development of embodied intelligence, Open-Environment Active 3D Mapping has attracted increasing attention, aiming to perform a long-horizon and shortest trajectory exploration for reconstructing unseen scenarios. Since only limited information about unseen environments is available, methods built on the closed-set assumption, i.e., assuming that the test environments are similar to those seen during training, cannot generalize satisfactorily. In existing active mapping methods, long-horizon exploration is often guided by predicting a coarse long-range goal and then converting it into an executable path. However, this stage is usually formulated as single-point prediction. Under partial observability, the same local observation may correspond to multiple plausible exploration directions, making such deterministic prediction prone to brittle decisions and degraded performance in unseen scenarios. Our experiments further verify that this is a key factor underlying their weak generalization. To address this issue, we reformulate long-horizon target prediction as conditional multimodal anchor generation using Conditional Flow Matching. Instead of predicting a single goal, our method learns a conditional distribution over coarse exploration anchors from the current mapping state. These anchors are first converted into executable candidate paths through obstacle-aware planning. We then apply exploration-mode clustering to compress geometrically similar trajectories and reduce candidate redundancy. Finally, a hierarchical selection module selects the most promising mode and reranks paths within it to produce the final executable trajectory. Experiments show that our method improves generalization and reconstruction efficiency in open environments.
Yang Li, Aming Wu, Zihao Zhang +3
School of Artificial Intelligence, Tianjin University, China · School of Computer Science and Information Engineering, Hefei University of Technology, China
Current 3D mapping pipelines generally assume static environments, which limits their ability to accurately capture and reconstruct moving objects. To address this limitation, we introduce the novel task of active mapping of moving objects, in which a mapping agent must plan its trajectory while compensating for the object's motion. Our approach, Paparazzo, provides a learning-free solution that robustly predicts the target's trajectory and identifies the most informative viewpoints from which to observe it, to plan its own path. We also contribute a comprehensive benchmark designed for this new task. Through extensive experiments, we show that Paparazzo significantly improves 3D reconstruction completeness and accuracy compared to several strong baselines, marking an important step toward dynamic scene understanding. Project page: https://davidea97.github.io/paparazzo-page/
Davide Allegro, Shiyao Li, Stefano Ghidoni +1
University of Padova · 2LIGM, ´Ecole Nationale des Ponts et Chauss´ees, IP Paris, Univ Gustave Eiffel, CNRS