VA-Assembly: View-Aligned Score Guidance for Training-Free 3D Part Assembly
Organizations: School of Mechanical Engineering, Yonsei University, Seoul, Republic of Korea.
Abstract
Image-to-3D foundation models have improved 3D reconstruction, but typically produce monolithic representations without explicit part-level structure. Part-aware generation and learning-based assembly methods often require part-level supervision, category-specific priors, or paired part-layout training data. We instead consider a practical but underexplored setting in which independently prepared or generated 3D parts have no predefined inter-part relationships, shared coordinate frames, or consistent scales, and must be assembled into a reference-consistent object from a single image. We present VA-Assembly, a training-free framework for 3D part assembly through part-level transformation estimation using view-aligned diffusion guidance. Given a reference image and fixed 3D part geometries, VA-Assembly optimizes each part's translation, rotation, and scale. We condition a frozen view-conditioned diffusion model on same-view renderings of a reference-derived monolithic 3D representation, producing score gradients geometrically aligned with the part configuration. This guidance provides multi-view structural cues under large initial misalignment. By repurposing pretrained visual-structural priors as transformation-level supervision, VA-Assembly requires no annotated part layouts, predefined part taxonomies, category-specific assembly training, or bounding-box-based layout initialization. Experiments compare VA-Assembly with geometric registration and compositional layout baselines using each baseline's supported transformation components, showing improved part-wise transformation estimation. Ablation studies show that view-aligned conditioning, rather than diffusion guidance alone, is critical for reliable transformation gradients. Updating only part transformations preserves explicit component separability and supports component-level editing, replacement, and reconfiguration.
Figures & tables
| Method | Task Type | Category Dependency | Translation | Rotation | Scale |
|---|---|---|---|---|---|
| CAP-Net ( Huang et al., 2025 ) | Object Pose/shape Estimation | Category-specific | |||
| CRISP ( Shi et al., 2025 ) | Object Pose/shape Estimation | Category-agnostic | |||
| TEASER++ ( Yang et al., 2020 ) | 3D Registration | N/A | |||
| Go-ICP ( Yang et al., 2015 ) | 3D Registration | N/A | |||
| CFPA ( Zhang et al., 2026 ) | 3D Arrangement | Category-specific | |||
| Assembler ( Zhao et al., 2025 ) | 3D Arrangement | Category-agnostic |
| Optimization parameters | Method | Chamf. [NCS] | Trans. [NCS] | Rot. [deg] | Scale | ||||
|---|---|---|---|---|---|---|---|---|---|
| Mean | SD | Mean | SD | Mean | SD | Mean | SD | ||
| Translation, Rotation | Go-ICP | 0.181 | 0.179 | 0.269 | 0.280 | 44.61 | 28.324 | – | – |
| Assembler | 0.037 | 0.004 | 0.049 | 0.001 | 61.038 | 25.156 | – | – | |
| Ours | 0.019 | 0.007 | 0.051 | 0.013 | 14.061 | 37.415 | – | – | |
| Translation Scale | REPARO | 0.257 | 0.077 | 0.406 | 0.093 | – | – | 0.150 | 0.034 |
| Ours | 0.017 | 0.009 | 0.039 | 0.021 | – | – | 0.005 | 0.001 | |
| Method | CLIP | PSNR | SSIM |
|---|---|---|---|
| OmniPart | 0.9021 | 23.2988 | 0.9172 |
| Part123 | 0.7460 | 18.5328 | 0.9113 |
| Ours | 0.9310 | 23.0371 | 0.9198 |
| Init. | Config. | Trans. [NCS] | Rot. [deg] | Scale |
|---|---|---|---|---|
| Local | 0.039 | 4.557 | 0.072 | |
| 0.035 | 3.471 | 0.023 | ||
| Global | 0.441 | 44.543 | 0.124 | |
| 0.053 | 13.763 | 0.061 |
| SDS Config | Chamf. [NCS] | Trans. [NCS] | Rot. [deg] | Scale |
|---|---|---|---|---|
| Single view Zero-1-to-3 | 0.102 | 0.183 | 45.426 | 0.112 |
| Multi view Zero-1-to-3 | 0.111 | 0.234 | 57.881 | 0.078 |
| Ours: View-aligned SDS | 0.038 | 0.057 | 12.795 | 0.018 |