PartRAG: Retrieval-Augmented Part-Level 3D Generation and Editing

Peize Li; Zeyu Zhang; Hao Tang

cs.CVFeb 19, 2026

PartRAG: Retrieval-Augmented Part-Level 3D Generation and Editing

Authors: Peize Li, Zeyu Zhang, Hao Tang

Organizations: King’s College London · Peking University

Abstract

Single-image 3D generation with part-level structure remains challenging: learned priors struggle to cover the long tail of part geometries and maintain multi-view consistency, and existing systems provide limited support for precise, localized edits. We present PartRAG, a retrieval-augmented framework that integrates an external part database with a diffusion transformer to couple generation with an editable representation. To overcome the first challenge, we introduce a Hierarchical Contrastive Retrieval module that aligns dense image patches with 3D part latents at both part and object granularity, retrieving from a curated bank of 1,236 part-annotated assets to inject diverse, physically plausible exemplars into denoising. To overcome the second challenge, we add a masked, part-level editor that operates in a shared canonical space, enabling swaps, attribute refinements, and compositional updates without regenerating the whole object while preserving non-target parts and multi-view consistency. PartRAG achieves competitive results on Objaverse, ShapeNet, and ABO-reducing Chamfer Distance from 0.1726 to 0.1528 and raising F-Score from 0.7472 to 0.844 on Objaverse-with inference of 38s and interactive edits in 5-8s. Qualitatively, PartRAG produces sharper part boundaries, better thin-structure fidelity, and robust behavior on articulated objects. Code: https://github.com/AIGeeksGroup/PartRAG. Website: https://aigeeksgroup.github.io/PartRAG.

Figures & tables

Figure 2: PartRAG architecture. Top : Retrieval-augmented pipeline generating structured part meshes. Bottom : PartCrafter DiTModel Transformer, Retrieval Module, and Contrastive Learning.

Method	Objaverse			ShapeNet			ABO			Time
	CD $\downarrow$	F $\uparrow$	IoU $\downarrow$	CD $\downarrow$	F $\uparrow$	IoU $\downarrow$	CD $\downarrow$	F $\uparrow$	IoU $\downarrow$
Dataset	–	–	0.0796	–	–	0.1827	–	–	0.0137	–
TripoSG [ 17 ]	0.3104	0.5940	–	0.3751	0.5050	–	0.2017	0.7096	–	–
TripoSG*	0.1821	0.7115	–	0.3301	0.5589	–	0.1503	0.7723	–	–
HoloPart [ 37 ]	0.1916	0.6916	0.0443	0.3511	0.5498	0.1107	0.1338	0.8093	0.0449	18min
PartCrafter [ 20 ]	0.1726	0.7472	0.0359	0.3205	0.5668	0.0293	0.1047	0.8617	0.0243	34s

Table 1: Main results comparing PartRAG with baselines. CD (Chamfer Distance): lower is better. F-Score: higher is better. IoU: mean part-overlap IoU (lower indicates better part separation).

Parameter	Value
Learning rate	$3\times 10^{-5}$
Batch size	48
Number of DiT blocks	21
Hidden dimension	1024
Latent dimension $d$	1024
Temperature $\tau$	0.07

Table 2: Key hyperparameters and model configurations.

Figure 3: Qualitative comparison. Each row shows the input photograph (left), the HoloPart baseline and PartCrafter(middle), and our PartRAG result (right). Different colors indicate different parts of the object. Our method preserves crisper part boundaries and cleaner normals across diverse categories.

Configuration	Preserve $\uparrow$	Boundary $\uparrow$	MV-Cons $\downarrow$	Time (s)
	IoU	Quality (%)	(CD var)
Full regeneration	0.000	–	0.0031	38.0
W/o boundary smoothing	0.979	67	0.0089	6.2
W/o canonical alignment	0.941	73	0.0124	5.8
W/o masked flow (naive)	0.856	61	0.0156	7.1
PartRAG (Full)	0.982	94	0.0067	6.5

Table 3: Editing quality ablation on 150 editing operations across three types. While full regeneration achieves better multi-view consistency (lower CD variance), editing provides 5.8

\times

speedup while preserving 98.2% of non-target geometry, making it preferable for interactive workflows.

Configuration	CD $\downarrow$	F-Score $\uparrow$	IoU $\downarrow$	Training Time
Baseline (PartCrafter) [ 20 ]	0.1726	0.7472	0.0359	24h
+ RAG (top- $k$ =3)	0.1598	0.779	0.0312	27h
+ Object Contrastive	0.1569	0.796	0.0284	31h
+ Part Contrastive	0.1547	0.809	0.0267	33h
+ Both (Full PartRAG)	0.1528	0.844	0.025	36h

Table 4: Component ablation study on Objaverse validation split.

Setting	Config	CD $\downarrow$	F-Score $\uparrow$	IoU $\downarrow$
Top- $k$	$k=1$	0.1567	0.816	0.0281
	$\mathbf{k=3}$	0.1528	0.844	0.025
	$k=5$	0.1541	0.835	0.0263
	$k=10$	0.1556	0.827	0.0276
Encoder	CLIP [ 30 ]	0.1582	0.820	0.0275
	DINOv2 [ 27 ]	0.1549	0.831	0.0261

Table 5: Retrieval configuration analysis on Objaverse validation split.

$\lambda_{\text{obj}}$	$\lambda_{\text{part}}$	CD $\downarrow$	F-Score $\uparrow$	IoU $\downarrow$	Stability
0.01	0.01	0.1562	0.827	0.0265	Stable (0% NaN)
0.02	0.02	0.1544	0.835	0.0258	Stable (0% NaN)
0.03	0.03	0.1528	0.844	0.025	Stable (0% NaN)
0.05	0.05	0.1538	0.836	0.0256	Unstable (0.8% NaN)
0.10	0.10	0.1565	0.825	0.0273	Unstable (12.3% NaN)

Table 6: Contrastive loss weight ablation on Objaverse validation split.

Figure 4: Part-level editing across six examples. Our method enables localized, structure-aware edits that preserve non-target parts, honor learned attachment transforms $T_{i}$ , coordinate multi-part changes, and maintain multi-view consistency, achieved in 5–8 s per edit.

3D Shape Retrieval

See all 8 papers

Seg3DParts: Segmentation-Grounded Controllable Part-Level 3D Generation
Sep 29, 2026Jiantao Lin, Meixi Chen, Yingjie Xu +53D Mesh Generation Part-Level 3D Generation
The Hong Kong University of Science and Technology (Guangzhou) · Knowin AI · The Hong Kong University of Science and Technology
Seeing Both the Forest and the Trees: Reusing Holistic 3D Priors for Part-Decomposed Generation
Oct 15, 2026Yiqun Zhao, Binbin Huang, Haobin Duan +23D Shape Generation 3D Mesh Generation
The University of Hong Kong · Transengram · Tencent +1
SAM3D-Part: Interactive Part Selection and Generation from 3D Objects
Sep 14, 2026Jiahao Chang, Dong Du, Wanhu Sun +63D Mesh Generation Part-Level 3D Generation
SSE, CUHKSZ and FNii-Shenzhen and Meshy AI, China · Meshy AI, China · Nanjing University of Science and Technology, China +5

Sep 29, 2026J/K move · Enter open · S save

Seg3DParts: Segmentation-Grounded Controllable Part-Level 3D Generation

Jiantao Lin, Meixi Chen, Yingjie Xu, Chenbo Fu+4

The Hong Kong University of Science and Technology (Guangzhou) · Knowin AI · The Hong Kong University of Science and Technology

3D Mesh Generation Part-Level 3D Generation 3D Asset Generation 3D Shape Generation

Open paper AskPDF arXiv

Part-level 3D assets are essential for editing, reassembly, and interaction, yet recovering such structure from a single image remains challenging due to occlusion, ambiguous boundaries, and the need for coherent multi-part reasoning. Existing approaches struggle to achieve both controllable part-level generation and coherent multi-part structure, as part identity and spatial allocation are typically inferred implicitly. We present Seg3DParts, a segmentation-grounded framework for controllable part-level 3D generation from a single image. By treating segmentation as an explicit grounding signal, our method defines part identity during generation, enabling each component to be anchored to a corresponding image region. To ensure coherent assemblies, we introduce structured cross-part interaction that allows components to exchange global context throughout the generative process. As a result, Seg3DParts directly generates well-aligned part meshes in a shared canonical space without post-hoc alignment, supporting flexible and controllable decomposition. We further introduce PartObjectNet, a large-scale dataset with over 200K objects and 1M annotated parts. Experiments demonstrate that Seg3DParts achieves superior geometry quality, cross-part coherence, and part-level controllability over existing methods.

Abstract

3D Shape Retrieval

Explore similar work

Seg3DParts: Segmentation-Grounded Controllable Part-Level 3D Generation

Seeing Both the Forest and the Trees: Reusing Holistic 3D Priors for Part-Decomposed Generation

SAM3D-Part: Interactive Part Selection and Generation from 3D Objects