Learning-based 3D Scanning plays a crucial role in enabling efficient and accurate scanning of target objects. However, recent reinforcement learning-based methods often require large-scale training data and still struggle to generalize to unseen object categories.In this work, we propose a data-efficient 3D scanning framework that uses Diffusion Policy to imitate human-like scanning strategies. To enhance robustness and generalization, we adopt the Occupancy Grid Mapping instead of direct point cloud processing, offering improved noise resilience and handling of diverse object geometries. We also introduce a hybrid approach combining a sphere-based space representation with a path optimization procedure that ensures path safety and scanning efficiency. This approach addresses limitations in conventional imitation learning, such as redundant or unpredictable behavior. We evaluate our method on diverse unseen objects in both shape and scale. Ours achieves higher coverage and shorter paths than baselines, while remaining robust to sensor noise. We further confirm practical feasibility and stable operation in real-world execution.
Figures & tables
Fig. 1 : Generalizable 3D Scanning Policy. Our proposed framework ScanDP can generalize to unseen objects with a small amount of training data (Right). Prior works lack generalization capability and are not robust to changes in conditions (left).
Input
Camera
Encoder
Pose
Image
Point Cloud
Count
Mobile
Diffusion Policy [ 11 ]
✓
✓
✗
2
✗
ResNet
3D Diffusion Policy [ 15 ]
✓
✗
✓
1
✗
MLP
GenDP [ 17 ]
✓
✓
✓
4
✗
PointNet++ [ 18 ]
ScanDP (ours)
✓
✗
✓
1
✓
SparseConv [ 19 ]
TABLE I : Comparison of prior works.
Fig. 2 : Method Overview. ScanDP consists of two main components: Path Generation (Upper part) and Path Optimization (Lower part). In the Path Generation phase, actions are generated from a diffusion policy conditioned on an occupancy grid map (OGM) and the current camera pose. In the Path Optimization phase, the generated actions are refined through a bubble-based collision filter and viewpoint extraction to optimize the scanning trajectory.
TABLE II : Evaluation results of Scanning policies. All policies are trained only on trajectories scanning the Stanford Bunny . Coverage is shown in mean ± std. We find that ScanDP consistently achieves the highest coverage while maintaining low variance.
Fig. 3 : Path length comparison (Scale × 1.0)
Fig. 4 : Path length comparison (Scale × 1.5)
Fig. 5 : Visualization of movement paths from DP3 and ScanDP. ScanDP with path optimization attains the shortest path length and smoothest movement. We find that DP3 tends to get stuck in a particular location when scanning objects not seen during training.
TABLE III: Coverage [%] under noisy inputs.
Camera
L515 ( 70∘×43∘ )
D435 ( 87∘×58∘ )
D415 ( 65∘×40∘ )
DP3 [ 15 ]
58.36
60.89
61.97
ScanDP
89.44
83.13
97.40
TABLE IV: Coverage [%] under different FoV.
Variant
Coverage [%] ↑
Path length [m] ↓
ScanDP
97.84
19.56
Thresholded OGM
42.38
51.72
Grid size = 0.01 m
N/A
N/A
Grid size = 0.04 m
65.49
21.58
TABLE V: Ablation results. The default configuration (probabilistic OGM and 0.02 m grid size) gives the best overall trade-off.
Fig. 6 : Real-world experiment setup. We use a 6-DoF manipulator with an RGB-D sensor, combined with a turntable to provide an additional rotational degree of freedom.
Fig. 7 : Real-world Evaluation Results. ScanDP demonstrates feasible and stable scanning behavior in real-world conditions.