SARe: Structure-Aware Generative 3D Fragment Reassembly
Authors: Hanze Jia, Chunshi Wang, Yuxiao Yang, Zhonghua Jiang, Yawei Luo, Shuainan Ye, Tan Tang
Organizations: State Key Lab of CAD&CG, Zhejiang University · School of Software Technology, Zhejiang University · Laboratory of Art and Archaeology Image, Zhejiang University
3D fragment reassembly estimates the rigid pose of each fragment to recover a complete object from unordered point clouds or meshes. The task becomes increasingly challenging as the fragment count grows, since irregular fragments provide weak semantic cues and admit rapidly increasing numbers of plausible contact relations and global configurations. We propose Structure-Aware Reassembly (SARe), a generative framework that integrates query-aligned local geometry and task-native structural supervision into point-flow assembly. SARe-Gen conditions each transported surface query on a local latent and jointly supervises intermediate flow tokens with query-level fracture-region and fragment-level contact targets. Because these heads are optimized together with flow matching, structural supervision directly shapes the representations that drive coordinate transport, without requiring additional reassembly-specific pretraining or a separate teacher-alignment stage. At inference time, SARe-Refine geometrically verifies predicted relations and uses reliable local subassemblies to guide a second sampling pass, reinforcing consistent regions while resampling uncertain fragments. We evaluate SARe across three settings, including synthetic fractures, simulated fractures from scanned real objects, and scans of physically fractured objects. The results demonstrate state-of-the-art performance, with higher part accuracy and more graceful degradation in challenging many-fragment settings.
Figures & tables
Figure 1: Representative oracle partial-structure diagnostic on RPF. Preserving a local subassembly specified by ground-truth contact edges while regenerating the remaining fragments improves PA from 75% to 82% in this example.
Figure 2: Overview of SARe. SARe-Gen transports query-aligned points into the assembled frame while jointly predicting query-level fracture-region probabilities and a fragment-level contact graph. SARe-Refine geometrically verifies the predicted relations and uses the retained local subassemblies to guide second-pass resampling.
Figure 3: SARe-Refine pipeline. Predicted contact edges are filtered to identify reliable substructures, after which uncertain regions are resampled in a RePaint-style process conditioned on a stable mask.
Everyday, K=2 – 20
Everyday, K=21 – 50
Artifact, K=2 – 20
Artifact, K=21 – 50
Method
PA ↑ (%)
CD ↓×10−3
RE ↓ (deg.)
TE ↓×10−2
PA ↑ (%)
CD ↓×10−3
RE ↓ (deg.)
TE ↓×10−2
PA ↑ (%)
CD ↓×10−3
RE ↓ (deg.)
TE ↓×10−2
PA ↑ (%)
CD ↓×10−3
RE ↓ (deg.)
TE ↓×10−2
Jigsaw
57.30
13.30
42.30
10.70
–
–
–
–
45.60
14.30
52.40
22.20
–
–
–
–
PF++
72.40
3.65
39.20
8.46
–
–
–
–
53.53
7.63
51.26
14.37
–
–
–
–
GARF
93.22
1.10
20.01
4.17
51.80
5.99
75.52
31.97
89.22
2.79
29.87
4.09
37.23
6.30
68.84
29.41
RPF
93.39
0.78
20.60
3.90
58.51
4.70
75.18
21.56
92.76
1.60
25.60
4.06
57.16
5.92
75.64
22.28
TORA-CKA
94.92
0.58
15.42
3.84
71.74
1.58
65.33
15.23
94.13
0.86
21.45
5.46
68.39
3.36
64.73
15.82
Table 1: Quantitative comparison on the non-volume-constrained Breaking Bad benchmark. Results are reported on the Everyday and Artifact subsets under different fragment-count ranges.
Method
PA ↑
CD ↓
RE ↓
TE ↓
GARF
80.12
1.32
22.54
5.37
RPF
82.32
1.35
20.06
4.33
TORA-CKA
93.75
0.47
14.78
1.94
SARe-Gen
97.68
0.40
9.92
1.06
Table 2: Quantitative results on scan-derived simulated fractures. OmniObject3D-Fracture is generated by applying simulated fracture processes to real-world scanned objects.
FRACTURA
Fantastic Breaks
Method
PA ↑
CD ↓
RE ↓
TE ↓
PA ↑
CD ↓
RE ↓
TE ↓
GARF
44.26
6.01
41.07
26.91
88.31
2.12
8.20
2.10
RPF
68.11
3.64
51.15
11.23
96.90
2.53
6.32
2.18
TORA-CKA
76.35
1.89
36.40
7.72
97.28
0.26
3.03
0.80
SARe-Gen
81.44
0.59
19.23
4.14
98.53
0.23
7.78
0.35
Table 3: Zero-shot transfer to unseen fracture datasets. All methods are evaluated without fine-tuning.
Method
PA ↑
CD ↓
RE ↓
TE ↓
SARe-Gen
81.34
1.85
34.35
6.83
SARe-Refine
84.89
1.72
32.51
6.19
Δ
+3.55
-0.13
-1.84
-0.64
Table 4: Effect of SARe-Refine on difficult Breaking Bad cases. Results are averaged over the N=2411 pooled Breaking Bad test samples whose first-pass PA is below 95%.
Figure 4: Qualitative comparisons of SARe-Gen and SARe-Refine. Red circles highlight regions corrected by the second pass.
Figure 5: Ablations of model components and head attachment layers, with a fracture-localization probe.
Setting
Time/obj. (s)
PA all ↑ (%)
PA K=21–50↑ (%)
M=2560
0.79
93.06
69.35
M=5120
1.03
94.81
83.05
M=10240
3.13
95.19
84.89
Uniform ( M=5120 )
1.02
79.69
44.96
Table 5: Test-time query-budget scaling and query-allocation ablation using the same checkpoint trained with M=5120 .
Most existing 3D assembly methods treat the problem as pure pose estimation, rearranging observed parts via rigid transformations. In contrast, human assembly naturally couples structural reasoning with holistic shape inference. Inspired by this intuition, we reformulate 3D assembly as a joint problem of assembly and generation. We show that these two processes are mutually reinforcing: assembly provides part-level structural priors for generation, while generation injects holistic shape context that resolves ambiguities in assembly. Unlike prior methods that cannot synthesize missing geometry, we propose CRAG, which simultaneously generates plausible complete shapes and predicts poses for input parts. Extensive experiments demonstrate state-of-the-art performance across in-the-wild objects with diverse geometries, varying part counts, and missing pieces. Project Page: https://ai4ce.github.io/CRAG/
3D part assembly aims to reconstruct a complete shape from a set of unassembled parts by predicting a rigid SE(3) transformation for each part. Because the placement of a part is rarely determined by its own geometry, the task demands reasoning over the entire part set, and existing methods, which rely on pairwise relations or iterative generation and verification, scale poorly to objects with many parts. We present LLM2R, which casts part assembly as sequence modeling with a pretrained decoder-only large language model (LLM): a Point-BERT encoder and a Q-Former compress each part into a block of geometric tokens, the blocks of all parts are concatenated, and a causal LLM adapted with Low-Rank Adaptation (LoRA) processes the sequence in one forward pass. Causal attention, however, lets the hidden states of a part observe only the parts that precede it. LLM2R therefore appends a block of learnable summary tokens after the last part: these tokens follow every part, so their states aggregate the whole assembly at a constant token cost, and a part-aware pose head fuses each part's states with them before predicting its rotation and translation. One architecture thus handles semantic and geometric assembly and predicts all poses in a single non-autoregressive pass. LLM2R achieves the highest part accuracy and success rate on all three PartNet categories, the best rotation error, translation error, and part accuracy on PartNeXt, and the lowest translation error and fastest inference on Breaking Bad, where only a fracture-specialized method is more accurate. Its advantage grows with the number of parts: at 100 parts it exceeds GARF by 33.5 points in part accuracy on Breaking Bad and Assembler by 29.1 points on PartNet, and it transfers zero-shot to unseen categories with more than 16 points higher part accuracy than the strongest of three baselines.
Zhouhan Zhong, Qiuhong Shen, Xingyi Yang +2
National University of Singapore · The Hong Kong Polytechnic University
Generating 3D assets with explicit parts requires two properties at once: geometric detail inside each part and structural coherence across the whole object. Recent image-conditioned 3D generation models adopt a structured 3D representation that anchors tokens to explicit spatial positions, capturing both properties, but their output remains a single fused mesh. We ask how a part-decomposed generation model can inherit both properties of these holistic priors, and answer with STRUCTPARTS, a framework whose core technical contribution is a dual-frame coordinate interleaving mechanism: each shape latent is interpreted under both a part-local and an object-global reference frame, and processed through a global-local inter-leaved transformer that simultaneously reuses the prior's geometric detail at the part scale and its structural coherence at the whole-object scale. As its structural input, STRUCTPARTS uses a segmented mesh scaffold, a coarse mesh paired with a face-level segmentation mask kept entirely within the prior's native 3D representation. With only lightweight finetuning, both intra-part detail and inter-part coherence emerge from the same pretrained weights. On PartObjaverse-Tiny, STRUCTPARTS matches or surpasses prior part-decomposed models at a fraction of their training cost, while naturally supporting part-mesh refinement and part-level editing.
Yiqun Zhao, Binbin Huang, Haobin Duan +2
The University of Hong Kong · Transengram · Tencent +1