While single-view 3D reconstruction has seen significant progress, extrapolating complex 3D structures from inherently ambiguous 2D observations remains fundamentally ill-posed, particularly in the critically underexplored data-scarce regime. To address this challenge, we propose Point Diffusion Mamba (PDM), a method that integrates the generative power of diffusion models with the efficiency of state-space model for single-view 3D reconstruction under data-scarce conditions. Specifically, PDM employs a lightweight reconstruction module tailored to handle unordered point-cloud inputs effectively. By combining a Local Geometric Aggregation module with Mamba blocks, our approach jointly models global geometric structures and local details. In 3D reconstruction, each point in the initial noisy input requires a precise prediction, yet the high-level features extracted by the Mamba module capture only abstract semantic information from sparse points. To bridge this gap, we introduce the Hierarchical Feature Integration Network, which fuses high-level semantic and local geometric features for each point, overcoming the limitations of token-based point-cloud reconstruction. Furthermore, we propose a Dynamic Weighted Sampling strategy that adaptively unifies 3D generation with single-view reconstruction by leveraging generative priors to enhance reconstruction quality. Experimental results on the ShapeNet and Pix3D benchmarks demonstrate that PDM outperforms state-of-the-art methods, providing an effective solution for 3D reconstruction under data-scarce settings. Code is available at: https://github.com/NWUzhouwei/PDM.
Figures & tables
Figure 1: Schematic Illustration of Our PDM. PDM reconstructs original point clouds from noisy inputs. An Encoder extracts image-conditioned features from a single-view image. At each Denoising step, a Dynamic Weighted Sampling (DWS) strategy fuses two representations: the reconstruction prediction and generative prior. Guided by these projected features and DWS, the network learns richer hierarchical geometric priors to progressively generate the final prediction.
Figure 2: Different Architectures. (a) The PC 2 architecture firstly voxelizes the point cloud, then extracts features using 3D convolution, and finally decodes using the channel attention mechanism SE. (b) The DiT architecture also voxelizes the point cloud and organizes it into ordered tokens. It extracts features through 3D window attention and finally decodes them using a linear layer. (c) Our PDM avoids the expensive memory overhead of voxelization by directly using KNN and FPS, while feature extraction is performed using Mamba-diffusion. Finally, HFINet is used for decoding.
Figure 3: Our PDM Architecture. (a) The pipeline of our PDM. We firstly extract features from the input image and back-project them onto the point cloud. The point cloud is then divided into point patches and embedded for encoding. Local features of the point cloud are extracted through the LGA module. Next, we learn global features using the Mamba block which encodes temporal conditions through the DiT structure. Finally, we use HFINet to recover the original point cloud from the randomly perturbed point cloud. (b) LGA propagates point features to fuse the features of neighboring points. (c) Detailed structure of the HFINet, which fuse high-level features with local features through point feature propagation.
Figure 4: Qualitative Comparisons on The Synthetic ShapeNet Dataset. Columns 2-4 present the results with 10% of the data, while columns 5-7 display the results with 50% of the data. Column 8 provides the corresponding ground truth.
method
Chair
Airplane
Car
10%
50%
100%
10%
50%
100%
10%
50%
100%
CD↓
F1↑
CD↓
F1↑
CD↓
F1↑
CD↓
F1↑
CD↓
F1↑
CD↓
F1↑
CD↓
F1↑
CD↓
F1↑
CD↓
F1↑
PC 2 (CVPR 23) [ 24 ]
97.25
0.393
73.58
0.437
65.57
0.464
88.00
0.605
76.39
0.628
65.97
0.655
64.99
0.524
62.59
0.542
64.36
0.547
CCD-3DR (arXiv 23) [ 5 ]
89.79
0.418
63.13
0.474
58.47
0.498
81.29
0.612
72.46
0.635
62.77
0.651
63.13
0.531
62.25
0.550
61.88
0.562
BDM-M (CVPR 24) [ 42 ]
94.94
0.395
71.56
0.446
64.48
0.468
87.75
0.604
73.19
0.629
65.16
0.653
63.53
0.524
60.71
0.549
64.16
0.554
BDM-B (CVPR 24) [ 42 ]
94.67
0.410
69.99
0.463
64.21
0.485
83.62
0.612
68.66
0.641
59.04
0.660
60.48
0.539
62.58
0.554
65.85
0.559
Table 1: Performance Evaluation on ShapeNet-R2N2. This table compares our PDM with PC 2 , CCD-3DR, BDM - M / BDM - B + PC 2 and MESC-3D, across three training data scales (10%, 50%, and 100%) using the reconstructed diffusion model. Bold values indicate the best results, while underlined values indicate the second-best results.
Method
Chair
Sofa
Table
CD↓
F1↑
CD↓
F1↑
CD↓
F1↑
PC 2 (CVPR 23) [ 24 ]
115.94
0.443
47.17
0.445
202.77
0.397
CCD-3DR (arXiv 23) [ 5 ]
111.42
0.456
44.91
0.450
196.28
0.418
BDM-M (CVPR 24) [ 42 ]
113.40
0.449
44.50
0.451
202.08
0.413
BDM-B (CVPR 24) [ 42 ]
110.60
0.455
45.05
0.455
186.46
0.429
MESC-3D (CVPR 25) [ 17 ]
91.36
0.370
41.98
0.284
206.16
0.306
Table 2: Performance Evaluation on Pix3D. This table compares the proposed PDM with PC 2 , CCD-3DR, BDM-M / BDM - B + PC 2 and MESC-3D, using the reconstructed diffusion model and complete training data.
Figure 5: Qualitative Comparisons on The Real-world Pix3D Dataset under Three Distinct Categories.
Method
Parameters (M)
Runtime (s)
GPU Memory (GB)
PC 2 /CCD-3DR (CVPR 23)
47.41
48.46
1.73
BDM-M (CVPR 24)
74.82
52.47
2.01
BDM-B (CVPR 24)
73.78
50.81
1.93
MESC-3D (CVPR 25)
74.84
34.05
0.86
PDM
56.34
20.73
0.39
Table 3: Comparison of Model Efficiency. We compared the model parameters, run time, and GPU memory usage.
Method
F1↑
CD↓
Input Size
F1↑
CD↓
W/O LGA
0.478
91.14
32
0.485
84.31
W/O HFINet
0.435
126.57
64
0.474
91.15
Self-Attention
0.481
85.46
128
0.499
79.28
One-SSM
0.474
88.72
256
0.491
80.57
FULL
0.499
79.28
384
0.487
82.29
Table 4: Ablation Study on Architectural Components and Input Patch Sequence Length. Left: Examines the effects of various modules. Right: Investigates the impact of input sequence lengths (ranging from 32 to 384).
Model
F1↑
CD↓
Strategy
F1↑
CD↓
PDM-S
0.479
90.87
Direct Sampling
0.483
85.27
PDM-B
0.499
79.28
BDM Sampling
0.492
80.23
PDM-L
0.492
80.23
Our Sampling
0.499
79.28
Table 5: Ablation Study on Model Size and Sampling Strategy. Left: Evaluates the influence of varying model sizes. Right: Assesses the impact of different sampling strategies.
Department of Computer Science, the University of Hong Kong · Department of Computer Science, City University of Hong Kong · Faculty of Humanities and Arts, Macau University of Science and Technology +2