Organizations: School of Computer Science and Technology, Zhejiang Gongshang University, Hangzhou, China · Zhejiang Key Laboratory of Big Data and Future E-Commerce Technology, Hangzhou, China · Jianpei Technology Co., Ltd., Hangzhou, China · School of Artificial Intelligence and Computer Science, Nantong University, Nantong, China · Department of Computer Science, College of Computer Engineering and Sciences, Prince Sattam Bin Abdulaziz University, Al Kharj, Saudi Arabia · Department of Computer Science, Faculty of Computers and Information, Qena University, Qena, Egypt · Department of Computing Sciences, Faculty of Information Technology and Communication Sciences, Tampere University, Tampere, Finland · Division of Oral and Maxillofacial Surgery, Faculty of Dentistry, The University of Hong Kong, Hong Kong, China · Systems Research Institute of the Polish Academy of Sciences, Warsaw, Poland · AGH University of Krakow, Kraków, Poland · SAN University, Łódź, Poland
Text-driven 3D scene editing has attracted considerable interest due to its convenience and user-friendliness. However, methods that rely on implicit 3D representations, such as Neural Radiance Fields (NeRF), while effective in rendering complex scenes, are hindered by slow processing speeds and limited control over specific regions of the scene. Moreover, existing approaches, including Instruct-NeRF2NeRF and GaussianEditor, which utilize multi-view editing strategies, frequently produce inconsistent results across different views when executing text instructions. This inconsistency can adversely affect the overall performance of the model, complicating the task of balancing the consistency of editing results with editing efficiency. To address these challenges, we propose a novel method termed Single-View to 3D Object Editing via Gaussian Splatting (SVGS), which is a single-view text-driven editing technique based on 3D Gaussian Splatting (3DGS). Specifically, in response to text instructions, we introduce a single-view editing strategy grounded in multi-view diffusion models, which reconstructs 3D scenes by leveraging only those views that yield consistent editing results. Additionally, we employ sparse 3D Gaussian Splatting as the 3D representation, which significantly enhances editing efficiency. We conducted a comparative analysis of SVGS against existing baseline methods across various scene settings, and the results indicate that SVGS outperforms its counterparts in both editing capability and processing speed, representing a significant advancement in 3D editing technology. For further details, please visit our project page at: https://amateurc.github.io/svgs.github.io.
Figures & tables
Figure 1. Results of 3D Object Editing with SVGS . This study demonstrates that text-driven 3D object editing can be accomplished through the modification of a single image, allowing for the generation of consistent multi-view images based on the edited outcomes. Notably, our algorithm facilitates precise alterations, such as changing a light blue hue to a dark blue hue, without impacting other regions of the object. Furthermore, the edited model can be exported as a high-quality mesh.
Figure 2. The Framework of SVGS. 1) Initially, a single-view image is modified using a correlation editing strategy based on IP2P. 2) Following the editing process, a multi-view diffusion model is employed to generate multi-view images consistent with the modifications. 3) For sparse reconstruction using 3DGS, a visual hull is constructed from camera parameters and masked images to initialize the 3D Gaussians, which are subsequently refined using the loss function Lgs . 4) Within the depth regularization module, depth maps are rendered for the input views, and the loss is computed against pre-generated monocular depth maps. The resulting Gaussian field facilitates efficient, high-quality novel-view synthesis.
Figure 3. Visualization of the Relevance-Aware Editing Mechanism. We visualize the intermediate steps of our algorithm. The Noise Difference Heatmap (b) , derived from Eq. ( 7 ), effectively acts as a semantic attention mechanism. It highlights the target region (e.g., the eyes for “put glasses on”) where the text instruction exerts significant influence, while suppressing the background. This is thresholded into a binary mask (c) to ensure the final output strictly preserves the unedited content.
Figure 4. Consistency Experiments. The top row presents six images sourced from the original dataset, whereas the middle row illustrates the outcomes of independently editing each view utilizing the IP2P method. The results of these edits exhibit significant inconsistency and do not correspond effectively with the provided textual instructions. In contrast, our proposed method produces consistent and well-edited multi-view images, as demonstrated in the bottom row.
Parameter
Value
Hardware
3× RTX 4090, PyTorch
Render Resolution
512×512
Stage 1: Sparse Recon.
(10k Iterations)
Loss Params ( γ,τ )
0.1,0.95
Stage 2: Text Editing
(1k-3k Iterations)
Guidance Scales ( ST,SI )
7.5,1.5
Table 1. Hyperparameter settings for SVGS. Detailed parameters are listed to facilitate reproducibility.
Figure 5. Qualitative comparison of text-driven 3D object editing. The visual results are organized by methods (columns) and editing tasks (rows). The first column displays the original reference image, while the remaining columns present multi-view renderings from Instruct-N2N ( Haque et al., 2023 ) , GaussianEditor ( Wang et al., 2024 ) , ViCA-NeRF ( Dong and Wang, 2023 ) , and our proposed SVGS. The seven rows correspond to distinct objects and editing instructions: (Row 1) color modification; (Row 2) accessory addition; (Row 3) style transfer; (Row 4) structural deformation (elongating rabbit ears); (Row 5) material transfer (wood to glossy marble); (Row 6) texture pattern addition (gold leaf); and (Row 7) localized part recoloring. Two views are rendered for each method to illustrate the appearance across different viewpoints.
Methods
CLIP-DS ↑
Time ↓
User Study ↑
Instruct-N2N [10]
0.05
51 min
15.24%
GaussianEditor [2]
0.09
40 min
17.72%
ViCA-NeRF [6]
0.04
40 min
6.72%
Ours-SVGS
0.13
20 min
60.32%
Table 2. Quantitative Evaluation . SVGS outperforms in both user study evaluations and CLIP Directional Similarity ( Gal et al., 2022 ) metrics.
Figure 6. Ablation Study of Relevance Editing. Comparison of our image editing results with those of IP2P. For both models, ST and SI were set to 7.5 and 1.5, respectively. IP2P fails to isolate the specified region and tends to over-edit the input content. In contrast, our model accurately predicts the editing scope and confines the edits to the designated area.
Method
LPIPS * ↓
PSNR ↑
SSIM ↑
Ours w/o Visual Hull
0.18
20.08
0.7693
Ours w/o Regularization
0.17
21.42
0.8134
SVGS (Ours)
0.13
23.36
0.8497
Table 3. Ablation study on key components. The table shows the performance of our method (SVGS) with and without the visual hull and regularization components. Results highlight that both components contribute to improved LPIPS, PSNR, and SSIM scores, with the full method outperforming the ablated versions.
Figure 7. Ablation Study on Various Components. The term “VH” refers to the visual hull. It is evident that the inclusion of the visual hull enhances the completeness of the reconstructed object’s structure. Moreover, the integration of depth regularization contributes to a richer and more refined representation of the object’s details.
Figure 8. Illustration of mutli-view diffusion model limitations. For complex objects, particularly those composed of multiple components, the processing by UNet often results in multi-view images that struggle to maintain the original object’s structure. This inconsistency in the generated structure can lead to a reconstructed model that fails to achieve the desired editing outcomes, leaving users dissatisfied.
Recent advancements in text-guided 3D editing, particularly with 3D Gaussian Splatting (3DGS), have shown promise but still lag behind the flexibility of 2D editing tools. Existing methods face a critical trade-off between preserving multi-view consistency and allowing structural modifications. Image-based editors can also struggle to apply the intended edit to objects viewed from unusual angles. To address these limitations, we introduce VidEdit3D, a framework that reformulates 3D scene editing as a trajectory-based video editing task. By drawing a parallel between camera movement in static 3D scenes and temporal dynamics in video, we leverage the robust spatiotemporal priors of pre-trained Video Diffusion Models (VDMs) to encourage semantic and structural coherence across views. Specifically, we propose Trajectory-Guided Rendering (TGR) to select informative keyframes and construct smooth video clips aligned with VDM input requirements. To propagate edits from an anchor view to these clips, we develop Flow-Aligned Video Editing (FAVE), a training-free framework incorporating an Editing Consistency Guidance (ECG) mechanism. ECG contrasts source and edited image conditions at a shared latent state, using scheduled source-image amplification to strengthen adherence to the edited reference while maintaining original scene context. Finally, we introduce Region-Selective 3DGS Refinement (RS-GS) to strictly confine geometric and appearance updates to target primitives, reducing background attribute leakage. Experiments demonstrate stronger instruction alignment and better multi-view consistency than the baselines.
Yanming Yang, Chenxi Song, Tong Zhao +5
Westlake University · University of Leeds · Nanyang Technological University +1
Recent advancements in diffusion and flow models have greatly improved text-based image editing, yet methods that edit images independently often produce geometrically and photometrically inconsistent results across different views of the same scene. Such inconsistencies are particularly problematic for editing of 3D representations such as NeRFs or Gaussian splat models. We propose a training-free guidance framework that enforces multi-view consistency during the image editing process. The key idea is that corresponding points should look similar after editing. To achieve this, we introduce a consistency loss that guides the denoising process toward coherent edits. The framework is flexible and can be combined with widely varying image editing methods, supporting both dense and sparse multi-view editing setups. Experimental results show that our approach significantly improves 3D consistency compared to existing multi-view editing methods. We also show that this increased consistency enables high-quality Gaussian splat editing with sharp details and strong fidelity to user-specified text prompts. Please refer to our project page for video results: https://3d-consistent-editing.github.io/
Josef Bengtson, David Nilsson, Dong In Lee +2
Chalmers University of Technology · Korea University
High-quality 3D scene reconstruction has recently advanced toward generalizable feed-forward architectures, enabling the generation of complex environments in a single forward pass. However, despite their strong performance in static scene perception, these models remain limited in responding to dynamic human instructions, which restricts their use in interactive applications. Existing editing methods typically rely on a 2D-lifting strategy, where individual views are edited independently and then lifted back into 3D space. This indirect pipeline often leads to blurry textures and inconsistent geometry, as 2D editors lack the spatial awareness required to preserve structure across viewpoints. To address these limitations, we propose VGGT-Edit, a feed-forward framework for text-conditioned native 3D scene editing. VGGT-Edit introduces depth-synchronized text injection to align semantic guidance with the backbone's spatial poses, ensuring stable instruction grounding. This semantic signal is then processed by a residual transformation head, which directly predicts 3D geometric displacements to deform the scene while preserving background stability. To ensure high-fidelity results, we supervise the framework with a multi-term objective function that enforces geometric accuracy and cross-view consistency. We also construct the DeltaScene Dataset, a large-scale dataset generated through an automated pipeline with 3D agreement filtering to ensure ground-truth quality. Experiments show that VGGT-Edit substantially outperforms 2D-lifting baselines, producing sharper object details, stronger multi-view consistency, and near-instant inference speed. The project page is https://chriszkxxx.github.io/VGGT-Edit/.
Kaixin Zhu, Yiwen Tang, Yifan Yang +9
1Peking University · 2Tencent · 3The Chinese University of Hong Kong +4