Generating complete 3D assets from sparse, unposed images requires preserving observed geometry while completing unseen regions. Joint reconstruction and generation offers a way to address both goals: reconstruction recovers the geometry observed in the input images, while generation contributes learned shape priors. However, combining camera-relative reconstruction with canonical-space generation introduces a coordinate mismatch that makes direct geometric interaction more difficult. Moreover, conditioning each 3D location on all views without accounting for their relevance may mix useful geometric evidence with unrelated image content, hindering local detail recovery. We present UniForge3D, which jointly models reconstruction and generation in a shared anchor-guided space, resolving their coordinate mismatch. This space follows the anchor view's azimuth while keeping the object upright. The two branches exchange features bidirectionally to predict per-view point maps and a complete sparse voxel structure. We further condition point-map prediction on the generated structure to establish pixel-to-voxel correspondences and select the most informative input view for each voxel. Features from the selected views then guide detailed shape generation, helping preserve observed geometric details. Our method achieves state-of-the-art appearance and geometry results on Toys4K and GSO, generating complete 3D assets with improved fidelity. Project webpage: https://xishuxishu.github.io/uniforge3d-project/
Figures & tables
Figure 1: UniForge3D generates faithful 3D assets from sparse, unposed multi-view images.
Figure 2: Different views provide different evidence for the same 3D region. The 3D box marks three blueberries, which are clearly visible, seen at an oblique angle, or fully occluded across the three views, motivating region-specific view selection.
Figure 3: Overview of UniForge3D. Left: The reconstruction and generation branches interact bidirectionally to predict multi-view point maps and a complete sparse structure in a shared anchor-guided space. Right: Structure-conditioned point maps establish voxel-to-pixel correspondences, enabling the model to select informative views and use their image features to guide detailed shape generation.
Figure 4: Qualitative comparisons on Toys4K and GSO. Each row shows four input views, ground-truth renderings, and results from different methods. To enable direct comparison, each generated mesh is geometrically aligned with the ground-truth asset and rendered using the corresponding input-view cameras. Our method better preserves the observed appearance and local structures while producing complete and consistent geometry.
Figure 5: Qualitative comparisons on real-world sparse, unposed images. All results are aligned and rendered from common viewpoints for visual comparison.
Dataset
Toys4K ( Stojanov et al., 2021 )
GSO ( Downs et al., 2022 )
Metric
PSNR ↑
SSIM ↑
LPIPS ↓
CD ( ×10−2 ) ↓
F-score ↑
PSNR ↑
SSIM ↑
LPIPS ↓
CD ( ×10−2 ) ↓
F-score ↑
Stream3D + TRELLIS.2 ( Zhou et al., 2026 )
20.955
0.9351
0.0789
6.3608
0.5003
19.576
0.9119
0.0990
6.2441
0.4700
Hunyuan3D-2MV ( Team, 2025 )
22.726
0.9470
0.0576
2.9175
0.6723
20.526
0.9206
0.0780
3.2713
0.6139
MV-SAM3D ( Li et al., 2026a )
23.071
0.9469
0.0590
2.7147
0.6830
20.653
0.9222
0.0791
2.7447
0.6517
ReconViaGen v0.2 ( Chang et al., 2026 )
22.713
0.9451
0.0548
2.6155
0.6786
20.498
0.9196
0.0738
2.5905
0.6697
ReconViaGen v0.5 ( Chang et al., 2026 )
25.020
0.9553
0.0519
2.7007
0.6840
22.774
0.9313
0.0731
2.7002
0.6489
Table 1: Quantitative comparison on the Toys4K ( Stojanov et al., 2021 ) and GSO ( Downs et al., 2022 ) datasets. Best results are shown in bold and second-best results are underlined. PSNR, SSIM, and LPIPS are computed on RGB renderings.
Method
VGGT Space
DiT Space
IoU ↑
CD ( ×10−2 ) ↓
DINO Conditioning
N/A
Anchor-Guided Space
0.4261
2.0019
DINO–VGGT Conditioning
Anchor-Guided Space
Anchor-Guided Space
0.5533
1.1710
Canonical-Space Joint Training
Canonical Space
Canonical Space
0.4601
1.5825
Cross-Space Joint Training
First-Camera Space
Canonical Space
0.4563
1.6291
Anchor-Guided Joint Training (Ours)
Anchor-Guided Space
Anchor-Guided Space
0.5827
1.0563
Table 2: Ablation of conditioning schemes and coordinate spaces for sparse-structure generation on GSO ( Downs et al., 2022 ) .
Method
Acc. ↓
Comp. ↓
Overall ↓
Image-Only VGGT
5.025
4.667
4.846
Jointly Trained VGGT
4.053
4.250
4.152
Structure-Conditioned VGGT
3.539
3.086
3.313
Table 3: Point-map reconstruction results on the GSO dataset ( Downs et al., 2022 ) . Distances are reported in normalized object coordinates ( ×10−3 ).
Method
PSNR ↑
SSIM ↑
LPIPS ↓
CD ( ×10−2 ) ↓
F-score ↑
Full Multi-View Conditioning
26.892
0.9649
0.0401
1.4739
0.8758
Voxel-Wise View Selection and Conditioning (Ours)
27.207
0.9657
0.0388
1.4047
0.8897
Table 4: Ablation of view-conditioning strategies for detailed shape generation on GSO ( Downs et al., 2022 ) . Both variants take the same predicted 643 sparse structure as input. PSNR, SSIM, and LPIPS are computed on rendered normal maps.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Qualitative comparison of point-map reconstruction. Each row shows one object. From left to right: four input views, point maps predicted by image-only VGGT, jointly trained VGGT, and structure-conditioned VGGT (ours), followed by ground-truth visible point maps.
Figure 7: Qualitative ablation of voxel-wise view selection and conditioning. Each row shows four input views followed by rendered normal maps of shapes generated with full multi-view conditioning and our method, respectively. Both variants use the same predicted sparse structure. Red boxes highlight reference details in the input images, and black boxes mark the corresponding regions in the generated shapes.
Figure 8: Additional qualitative results on rendered 3D assets. Each group shows four input views on a white background and the corresponding renderings of our generated mesh on a light-blue background. For direct visual comparison, we geometrically align the generated mesh with the ground-truth asset and render it using the same cameras as the input views.
Figure 9: Visualization of voxel-wise view selection from our inference results. Each row shows a generated sparse structure with a query voxel highlighted in magenta, followed by four input views. Colored pixels and enlarged insets show the predicted pixel-to-voxel correspondences, with numbers indicating highlighted pixels within each inset. Green checkmarks identify the views selected by our model.
Figure 10: Shared-space consistency of Structure-Conditioned VGGT . Each row shows the reconstructed colored point cloud, the generated mesh, and their direct overlay from left to right. No object-specific alignment or registration is applied.
Figure 11: Shared-space consistency of Jointly Trained VGGT . Each row shows the reconstructed colored point cloud, the generated mesh, and their direct overlay from left to right. No object-specific alignment or registration is applied.