Organizations: School of Electronics, Information and Electrical Engineering, Shanghai Jiao Tong University, Dongchuan Road 800, Shanghai, 200240,China.
Existing 3D editing methods often produce unrealistic and unrefined results due to the deeply integrated nature of their reconstruction networks. To address the challenge, this paper introduces CEI-3D, an editing-oriented reconstruction pipeline designed to facilitate realistic and fine-grained editing. Specifically, we propose a collaborative explicit-implicit reconstruction approach, which represents the target object using an implicit SDF network and a differentially sampled, locally controllable set of handler points. The implicit network provides a smooth and continuous geometry prior, while the explicit handler points offer localized control, enabling mutual guidance between the global 3D structure and user-specified local editing regions. To independently control each attribute of the handler points, we design a physical properties disentangling module to decouple the color of the handler points into separate physical properties. We also propose a dual-diffuse-albedo network in this module to process the edited and non-edited regions through separate branches, thereby preventing undesired interference from editing operations. Building on the reconstructed collaborative explicit-implicit representation with disentangled properties, we introduce a spatial-aware editing module that enables part-wise adjustment of relevant handler points. This module employs a cross-view propagation-based 3D segmentation strategy, which helps users to edit the specified physical attributes of a target part efficiently. Extensive experiments on both real and synthetic datasets demonstrate that our approach achieves more realistic and fine-grained editing results than the state-of-the-art (SOTA) methods while requiring less editing time. Our code is available on https://github.com/shiyue001/CEI-3D.
Figures & tables
Figure 1: Comparison with popular methods. Our approach shows better editing results. It maintains the photo-realism and fidelity to the user scribble when synthesized from different viewpoints. Not only is our PSNR (dB) improvement encouraging, but the reduction in edit time is impressive. Concretely, editing times on this example of EditNeRF ( Liu et al., 2021 ) , NeuMesh ( Yang et al., 2022 ) , and Ours are 45s, 1h, and 14s.
Figure 2: The main ideas and the capabilities comparison. (a) Unlike traditional explicit or implicit methods, our approach facilitates editing by introducing flexible handler points and disentangling physical properties. (b) In comparison, our approach satisfies properties that are desirable for editing.
Figure 3: Pipeline of CEI-3D. Training Phase. Given images and camera parameters, we first train an implicit surface network using an SDF MLP, shown on the top left of the diagram. To allow for flexible user control, we construct a handler point set H by sampling points on the implicit surface in a differential manner. Then, the combination of implicit surface and explicit handler points constitutes the collaborative 3D representation. To achieve controllability at the attribute level, we disentangle physical properties by optimizing learnable material parameters, lighting parameters, and diffuse branch Fat for non-edited regions in the DDA network. Editing Phase. Given the user scribble, we find points to be modified and update the edited branch Fae of the DDA network using the target attribute values of specified points. After network parameter optimization, the edited results of any view can be rendered.
Figure 4: Appearance editing comparisons. Qualitative comparison of our method with EditNeRF and NeuMesh for pattern painting edits on synthetic and real datasets. From left to right, we show the source image, user scribble, and novel-view rendering of edited results for each method. It can be observed that our approach provides consistent pattern edits on all of the examples.
Dataset
Methods
EditNeRF
NeuMesh
Seal3D
GaussianEditor
Ours
Phy. Syn.
PSNR↑
19.42
30.28
26.63
27.42
33.65
SSIM↑
0.731
0.942
0.935
0.965
0.983
LPIPS↓
0.324
0.045
0.029
0.057
0.022
NeRF Syn.
PSNR↑
15.31
30.94
28.32
26.89
31.02
SSIM↑
0.684
0.951
0.926
0.913
0.955
LPIPS↓
0.552
0.043
0.075
0.102
0.042
Table 1: Quantitative comparison on the non-edited area of novel-view rendering. We report average PSNR (dB)(higher is better)/ SSIM (higher is better)/ LPIPS (lower is better) on each dataset. All the methods are evaluated on unseen test views. Our method achieves the best results on the three metrics, which proves that the local modification performed by our method can better preserve the non-modified area. The best and second-best results are bold and underlined.
Dataset
EditNeRF
NeuMesh
Seal3D
GaussianEditor
Ours
Phy. Syn.
270
178
206
235
124
NeRF Syn.
196
130
152
145
121
DTU
216
198
186
201
175
PhotoShape
139
131
124
128
102
Table 2: Quantitative comparison of appearance editing. To quantitatively assess the quality of editing, we use the Frechet Inception Distance (FID) metric between image sets before and after editing, where a lower FID indicates better quality. Our method shows the best result across all datasets. The best and second-best results are bold and underlined.
Dataset
EditNeRF
NeuMesh
Seal3D
GaussianEditor
Ours
Phy. Syn.
37s
1.2h
29s
44s
14s
NeRF Syn.
28s
1.3h
32s
38s
18s
DTU
76s
1.6h
34s
42s
20s
PhotoShape
19s
1.1h
28s
36s
12s
Mean
48s
1.4h
32s
40s
16s
Table 3: Efficiency Comparison. We compare the editing time between SOTA methods and our method, demonstrating the advantage of our method in editing efficiency. The best and second-best results are bold and underlined.
Figure 5: Boxplot illustration of user study. Our method demonstrates superior performance (high means) and greater stability across test objects (narrow interquartile range) in both editing accuracy and realism.
Figure 6: More editing applications. In addition to the appearance editing, our framework supports part-aware geometry deformation, part removal, environment lighting changing and material roughness editing.
Figure 7: Effect of handler points. The artifacts annotated in the red box on the left side are eliminated with the handler points.
Figure 8: Effect of PPD. Properties disentangling ensures that the scribbles blend seamlessly into the environment and the editing results are realistic.
Figure 9: Robustness Analysis. (a) Editing results of inputs with different resolutions.(b) Editing results of different input image numbers. Our approach shows stronger robustness than the comparison method on both rendering fidelity and editing quality. We use the same scribble as Fig. 1 . PSNR is reported in dB.
Figure 10: Effect of CPS. The CPS strategy enables part-aware editing with sparse and single-view scribbles, making the process more efficient and intuitive.
Local editing of 3D objects remains a long-standing challenge. When interacting with 3D content, humans naturally tend to specify a coarse region of interest for modification rather than defining precise editing boundaries. However, previous methods rely on fully edited 2D images, precise 3D masks, or redundant pipelines, which present a gap. To bridge this gap, we propose EditVerse3D, a novel 3D editing framework that enables high-quality object editing under such coarse guidance. Our approach takes as input a 3D object to be edited, a coarse 3D bounding box indicating the target region, and a reference 2D image describing the desired modification. It produces a coherent, high-fidelity edited 3D object. To facilitate this editing, we introduce a novel region-aware adaptive loss that emphasizes hard-to-learn regions and balances the objective between target and preserved areas. Complementing our loss function, we enhance model robustness and generalization through targeted data augmentations, such as training with scaled 3D masks and filtering out unrealistic editing pairs. We construct a large-scale 3D editing dataset derived from parts information. Extensive experiments demonstrate that EditVerse3D achieves superior visual quality and quantitative performance compared to existing 3D editing approaches. Please visit our project page at https://editverse3d.github.io.
Youtan Yin, Yanning Zhou, Jiacheng Wei +6
College of Computing and Data Science, Nanyang Technological University · Tencent AIPD
Existing 3D editing methods have made notable progress in controllability, yet they remain limited in several important ways. Most approaches rely on text-driven editing, which struggles to express fine-grained visual changes intended by the user. Moreover, many methods require manually supplied 3D masks or introduce unintended changes to regions that should remain untouched. These limitations largely arise from the absence of fine-grained semantic understanding, making it difficult for existing models to retrieve or modify specific 3D components. We introduce ES3D, a framework that embeds semantics directly into 3D space, enabling component-aware retrieval and editing of a 3D asset conditioned on multiple local reference images and optional text queries. We first construct a 3D semantic embedding by projecting multi-view semantic features into the voxelized space of the asset. We then perform 3D component retrieval by computing feature similarity between the 3D semantic embedding and the semantic embeddings of image or text queries. For editing, we employ a pretrained 3D generative model with an inpainting mechanism to modify the retrieved components guided by user-provided images while preserving the rest of the asset. Overall, ES3D is a 3D editing framework that retrieves editable regions based on semantic cues and uses multiple images as conditions. Extensive experiments demonstrate that ES3D produces geometrically consistent and semantically coherent edits, enabling robust image-based and text-assisted control for 3D editing.
Xuancheng Jin, Rengan Xie, Jiayuan Lu +5
State Key Laboratory of CAD&CG, Zhejiang University · NetEase Fuxi AI Lab
Text-guided 3D editing aims to modify existing 3D assets using natural-language instructions. Current methods struggle to jointly understand complex prompts, automatically localize edits in 3D, and preserve unedited content. We introduce Vinedresser3D, an agentic framework for high-quality text-guided 3D editing that operates directly in the latent space of a native 3D generative model. Given a 3D asset and an editing prompt, Vinedresser3D uses a multimodal large language model to infer rich descriptions of the original asset, identify the edit region and edit type (addition, modification, deletion), and generate decomposed structural and appearance-level text guidance. The agent then selects an informative view and applies an image editing model to obtain visual guidance. Finally, an inversion-based rectified-flow inpainting pipeline with an interleaved sampling module performs editing in the 3D latent space, enforcing prompt alignment while maintaining 3D coherence and unedited regions. Experiments on diverse 3D edits demonstrate that Vinedresser3D outperforms prior baselines in both automatic metrics and human preference studies, while enabling precise, coherent, and mask-free 3D editing.
Yankuan Chi, Xiang Li, Zixuan Huang +1
The Hong Kong University of Science and Technology · University of Illinois Urbana-Champaign