While single-view 3D reconstruction has seen significant progress, extrapolating complex 3D structures from inherently ambiguous 2D observations remains fundamentally ill-posed, particularly in the critically underexplored data-scarce regime. To address this challenge, we propose Point Diffusion Mamba (PDM), a method that integrates the generative power of diffusion models with the efficiency of state-space model for single-view 3D reconstruction under data-scarce conditions. Specifically, PDM employs a lightweight reconstruction module tailored to handle unordered point-cloud inputs effectively. By combining a Local Geometric Aggregation module with Mamba blocks, our approach jointly models global geometric structures and local details. In 3D reconstruction, each point in the initial noisy input requires a precise prediction, yet the high-level features extracted by the Mamba module capture only abstract semantic information from sparse points. To bridge this gap, we introduce the Hierarchical Feature Integration Network, which fuses high-level semantic and local geometric features for each point, overcoming the limitations of token-based point-cloud reconstruction. Furthermore, we propose a Dynamic Weighted Sampling strategy that adaptively unifies 3D generation with single-view reconstruction by leveraging generative priors to enhance reconstruction quality. Experimental results on the ShapeNet and Pix3D benchmarks demonstrate that PDM outperforms state-of-the-art methods, providing an effective solution for 3D reconstruction under data-scarce settings. Code is available at: https://github.com/NWUzhouwei/PDM.
Figures & tables
Figure 1: Schematic Illustration of Our PDM. PDM reconstructs original point clouds from noisy inputs. An Encoder extracts image-conditioned features from a single-view image. At each Denoising step, a Dynamic Weighted Sampling (DWS) strategy fuses two representations: the reconstruction prediction and generative prior. Guided by these projected features and DWS, the network learns richer hierarchical geometric priors to progressively generate the final prediction.
Figure 2: Different Architectures. (a) The PC 2 architecture firstly voxelizes the point cloud, then extracts features using 3D convolution, and finally decodes using the channel attention mechanism SE. (b) The DiT architecture also voxelizes the point cloud and organizes it into ordered tokens. It extracts features through 3D window attention and finally decodes them using a linear layer. (c) Our PDM avoids the expensive memory overhead of voxelization by directly using KNN and FPS, while feature extraction is performed using Mamba-diffusion. Finally, HFINet is used for decoding.
Figure 3: Our PDM Architecture. (a) The pipeline of our PDM. We firstly extract features from the input image and back-project them onto the point cloud. The point cloud is then divided into point patches and embedded for encoding. Local features of the point cloud are extracted through the LGA module. Next, we learn global features using the Mamba block which encodes temporal conditions through the DiT structure. Finally, we use HFINet to recover the original point cloud from the randomly perturbed point cloud. (b) LGA propagates point features to fuse the features of neighboring points. (c) Detailed structure of the HFINet, which fuse high-level features with local features through point feature propagation.
Figure 4: Qualitative Comparisons on The Synthetic ShapeNet Dataset. Columns 2-4 present the results with 10% of the data, while columns 5-7 display the results with 50% of the data. Column 8 provides the corresponding ground truth.
method
Chair
Airplane
Car
10%
50%
100%
10%
50%
100%
10%
50%
100%
CD↓
F1↑
CD↓
F1↑
CD↓
F1↑
CD↓
F1↑
CD↓
F1↑
CD↓
F1↑
CD↓
F1↑
CD↓
F1↑
CD↓
F1↑
PC 2 (CVPR 23) [ 24 ]
97.25
0.393
73.58
0.437
65.57
0.464
88.00
0.605
76.39
0.628
65.97
0.655
64.99
0.524
62.59
0.542
64.36
0.547
CCD-3DR (arXiv 23) [ 5 ]
89.79
0.418
63.13
0.474
58.47
0.498
81.29
0.612
72.46
0.635
62.77
0.651
63.13
0.531
62.25
0.550
61.88
0.562
BDM-M (CVPR 24) [ 42 ]
94.94
0.395
71.56
0.446
64.48
0.468
87.75
0.604
73.19
0.629
65.16
0.653
63.53
0.524
60.71
0.549
64.16
0.554
BDM-B (CVPR 24) [ 42 ]
94.67
0.410
69.99
0.463
64.21
0.485
83.62
0.612
68.66
0.641
59.04
0.660
60.48
0.539
62.58
0.554
65.85
0.559
Table 1: Performance Evaluation on ShapeNet-R2N2. This table compares our PDM with PC 2 , CCD-3DR, BDM - M / BDM - B + PC 2 and MESC-3D, across three training data scales (10%, 50%, and 100%) using the reconstructed diffusion model. Bold values indicate the best results, while underlined values indicate the second-best results.
Method
Chair
Sofa
Table
CD↓
F1↑
CD↓
F1↑
CD↓
F1↑
PC 2 (CVPR 23) [ 24 ]
115.94
0.443
47.17
0.445
202.77
0.397
CCD-3DR (arXiv 23) [ 5 ]
111.42
0.456
44.91
0.450
196.28
0.418
BDM-M (CVPR 24) [ 42 ]
113.40
0.449
44.50
0.451
202.08
0.413
BDM-B (CVPR 24) [ 42 ]
110.60
0.455
45.05
0.455
186.46
0.429
MESC-3D (CVPR 25) [ 17 ]
91.36
0.370
41.98
0.284
206.16
0.306
Table 2: Performance Evaluation on Pix3D. This table compares the proposed PDM with PC 2 , CCD-3DR, BDM-M / BDM - B + PC 2 and MESC-3D, using the reconstructed diffusion model and complete training data.
Figure 5: Qualitative Comparisons on The Real-world Pix3D Dataset under Three Distinct Categories.
Method
Parameters (M)
Runtime (s)
GPU Memory (GB)
PC 2 /CCD-3DR (CVPR 23)
47.41
48.46
1.73
BDM-M (CVPR 24)
74.82
52.47
2.01
BDM-B (CVPR 24)
73.78
50.81
1.93
MESC-3D (CVPR 25)
74.84
34.05
0.86
PDM
56.34
20.73
0.39
Table 3: Comparison of Model Efficiency. We compared the model parameters, run time, and GPU memory usage.
Method
F1↑
CD↓
Input Size
F1↑
CD↓
W/O LGA
0.478
91.14
32
0.485
84.31
W/O HFINet
0.435
126.57
64
0.474
91.15
Self-Attention
0.481
85.46
128
0.499
79.28
One-SSM
0.474
88.72
256
0.491
80.57
FULL
0.499
79.28
384
0.487
82.29
Table 4: Ablation Study on Architectural Components and Input Patch Sequence Length. Left: Examines the effects of various modules. Right: Investigates the impact of input sequence lengths (ranging from 32 to 384).
Model
F1↑
CD↓
Strategy
F1↑
CD↓
PDM-S
0.479
90.87
Direct Sampling
0.483
85.27
PDM-B
0.499
79.28
BDM Sampling
0.492
80.23
PDM-L
0.492
80.23
Our Sampling
0.499
79.28
Table 5: Ablation Study on Model Size and Sampling Strategy. Left: Evaluates the influence of varying model sizes. Right: Assesses the impact of different sampling strategies.
Single-view 3D reconstruction is a challenging task in computer vision due to information missing from the single input image. Generative model-based approaches can produce plausible 3D objects from a single image, thanks to data-driven priors learnt from rich and large-scale datasets. However, plausible generation does not guarantee fidelity to the geometry and appearance of the particular input object. Inspired by object perception in human vision, we propose OPERA, a framework that guides multi-view diffusion sampling with pretrained perception models through lightweight alignment modules. These modules are trained independently while both the generative and perception models remain frozen, allowing multiple signals to be combined at inference without joint fusion training. We evaluate OPERA on two single-view 3D reconstruction baselines using subsets of Google Scanned Objects and OmniObject3D. On the primary baseline, combined guidance reduces mean Chamfer Distance by 30.4% and 24.8%, respectively, relative to unguided reconstruction. We also compare our method with recent image-to-3D models. We provide in-depth analyses of the design choices and their effects across datasets and backbones. Our project page is at https://opera-3d.github.io/.
Y Huynh, Duc Thanh Nguyen, Mohamed Abdelrazek
Applied Artificial Intelligence Initiative Deakin University · School of Information Technology Deakin University
Sparse-view 3D reconstruction is essential for modeling scenes from casual captures, but remain challenging for non-generative reconstruction. Existing diffusion-based approaches mitigates this issues by synthesizing novel views, but they often condition on only one or two capture frames, which restricts geometric consistency and limits scalability to large or diverse scenes. We propose AnyRecon, a scalable framework for reconstruction from arbitrary and unordered sparse inputs that preserves explicit geometric control while supporting flexible conditioning cardinality. To support long-range conditioning, our method constructs a persistent global scene memory via a prepended capture view cache, and removes temporal compression to maintain frame-level correspondence under large viewpoint changes. Beyond better generative model, we also find that the interplay between generation and reconstruction is crucial for large-scale 3D scenes. Thus, we introduce a geometry-aware conditioning strategy that couples generation and reconstruction through an explicit 3D geometric memory and geometry-driven capture-view retrieval. To ensure efficiency, we combine 4-step diffusion distillation with context-window sparse attention to reduce quadratic complexity. Extensive experiments demonstrate robust and scalable reconstruction across irregular inputs, large viewpoint gaps, and long trajectories.
Yutian Chen, Shi Guo, Renbiao Jin +7
Shanghai AI Lab · Shanghai Jiao Tong University · The University of Hong Kong +2
In this paper, we introduce DecoRec, a novel system designed to elevate single-view 2D images to a decomposed 3D scene mesh. Current methods for single-view scene reconstruction typically rely on object retrieval or the regression of coarse 3D voxels or surfaces, leading to inaccuracies in capturing the appearance and geometry of the input image. The lack of high-quality large-scale scene-level datasets further complicates direct 3D scene generation from single-view images. To achieve high-quality 3D scene generation from a single-view image, DecoRec takes advantage of recent diffusion-based single-view object reconstruction methods to reconstruct individual objects separately. Subsequently, a refinement pipeline is proposed to effectively merge these reconstructed objects, enhancing appearance and geometry through a differentiable rendering technique and diffusion-guided refinement. Our results demonstrate that DecoRec facilitates high-quality single-view scene reconstruction in both geometry and novel synthesis, offering significant benefits for downstream applications like room interior design.
Yuhan Ping, Yuan Liu, Xiaoxiao Long +6
Department of Computer Science, the University of Hong Kong · Department of Computer Science, City University of Hong Kong · Faculty of Humanities and Arts, Macau University of Science and Technology +2