TRACE: High-Fidelity 3D Scene Editing via Tangible Reconstruction and Geometry-Aligned Contextual Video Masking

Jiyuan Hu; Zechuan Zhang; Zongxin Yang; Yi Yang

cs.CVApr 1, 2026

TRACE: High-Fidelity 3D Scene Editing via Tangible Reconstruction and Geometry-Aligned Contextual Video Masking

Authors: Jiyuan Hu, Zechuan Zhang, Zongxin Yang, Yi Yang

Organizations: ReLER Lab, CCAI, Zhejiang University · DBMI, HMS, Harvard University

Abstract

Existing 3D Gaussian Splatting (3DGS) editing methods primarily focus on appearance modification and often struggle to support flexible geometry editing while preserving structural integrity and scene-consistent appearance. To address this limitation, we present TRACE, a mesh-guided 3DGS editing framework that automatically aligns explicit 3D geometry with Gaussian scenes and decouples Geometric Anchoring from Appearance Harmonization. First, Multi-view 3D-Anchor Synthesis, trained on our MV-TRACE dataset for scene-coherent object addition and modification, generates geometrically aligned editing anchors, while Tangible Geometry Alignment (TGA) performs coarse-to-fine mesh-scene registration. Then, Contextual Video Masking (CVM) integrates projected 3D anchors into an autoregressive video diffusion pipeline, harmonizing their appearance with the surrounding scene while maintaining multi-view consistency. We evaluate TRACE on eight held-out scenes across six editing categories. TRACE completes each edit in approximately 10 minutes on a single NVIDIA RTX Pro 6000 GPU. Extensive experiments demonstrate consistent improvements over existing methods in editing versatility, structural integrity, semantic alignment, multi-view consistency, and visual quality.

Figures & tables

Figure 1: Limitations of existing 3D editing approaches. (a) Weak 3D grounding leads to inconsistent or appearance-only edits. (b) Explicit geometry suffers from visual mismatch and inflexible alignment.

Figure 2: Representative TRACE edits. Top: sequential interactive edits on the “Stone Horse” scene, demonstrating part-level geometric manipulation. Bottom left: normalized performance across six edit types. Bottom right: multi-view examples of object addition, texture synthesis, style transfer, and replacement. Scenes come from IN2N, BlendedMVS, and Mip-NeRF 360.

Method	Consistency	Shape Edit	Global Edit	Auto Align
DGE	Extrap. Attn.	✗	✓	—
GaussCtrl	Depth Const.	✗	✓	—
TIP-Editor	SDS-based	✓	✗	✗
EditSplat	Iter. Update	✗	✓	—
GaussianEditor	Iter. Update	✓	✓	✗
Vip3dedit	Video/3D Prior	✗	✓	—

Table 1: Comparison of editing capabilities. Shape Edit means explicit 3D geometry modification; ✓ / ✗ denote claimed native support/unsupported, and "—" means Auto Align is inapplicable because no explicit asset is introduced.

Figure 3: Overview of TRACE. Given a 3DGS scene and an editing instruction, Stage I generates sparse edited anchors and aligns a generated or retrieved mesh to the scene with the TGA module. Stage II projects the aligned mesh anchor along rendered camera trajectories and uses CVM to repaint the object and nearby context before reconstructing the edited 3DGS.

Figure 4: Construction of MV-TRACE. The 100k pairs come from 96-view rendering of 25 training scenes, automatic filtering, and human-assisted asset alignment and color/contact refinement; all construction scenes are disjoint from evaluation.

Figure 5: Two-stage TGA alignment. Phase 1 initializes orientation and a coarse similarity transform; Phase 2 first performs 7-DoF coarse docking and then fixes rotation for fine scale/translation anchoring. The bottom row is a representative case of adding sunglasses to the man’s face.

Figure 6: Selected CVM and trajectory examples. Left: frames at three trajectory densities $\rho$ (smaller is denser). Right: boundary crops before/after refinement.

Figure 7: Qualitative comparisons. Red boxes highlight baseline artifacts, while green boxes show TRACE’s more consistent geometry and cleaner local details. For clarity, we enlarge the second-view results of TRACE and selected competitive baselines.

		Global/Local Style Editing				Partial/Whole Object Geometry Editing						Time
Method	Pub.	CLIP ${}_{dir}\uparrow$	CLIP ${}_{sim}\uparrow$	DINO $\uparrow$	Aes. $\uparrow$	CLIP ${}_{dir}\uparrow$	CLIP ${}_{sim}\uparrow$	DINO $\uparrow$	Aes. $\uparrow$	MEt3R $\downarrow$	Uni3D-T $\uparrow$	min $\downarrow$
DGE	ECCV’24	0.1082	0.2337	0.8693	5.5453	0.0228	0.2405	0.9005	6.0495	0.2095	0.0574	10 min
GaussCtrl	ECCV’24	0.0349	0.1955	0.8962	5.3021	0.0543	0.2297	0.8962	5.7307	0.1820	0.0042	20 min
TIP-Editor	SIG’24	0.1095	0.2398	0.8693	5.2792	0.0931	0.2234	0.8375	5.4916	0.2318	0.1531	45 min
GaussianEditor	CVPR’24	0.0737	0.2136	0.8774	5.6241	0.0623	0.2372	0.8568	5.2373	0.1847	0.1690	16 min
EditSplat	CVPR’25	0.0790	0.2328	0.8658	5.7483	0.0734	0.2270	0.9010	5.8659	0.1645	0.0283	18 min

Table 2: Unified evaluation on style and geometry editing. The evaluation covers local style change, global style change, insertion, removal, replacement, and deformation. MEt3R and Uni3D-T are reported for final reconstructed geometry-editing regions.

Panel of Figure 8 — Figure 8: Pairwise user preference across six editing types. Results aggregate 540 blind comparison units as normalized weighted preference strengths. Tasks 1–6 denote local/global style, insertion, removal, replacement, and deformation.

(a) Ablation on 3D-Anchor Synthesis
Method	$L_{\text{LPIPS}}\downarrow$	$\text{PSNR}_{\text{bg}}\uparrow$	$IoU_{\text{align}}\uparrow$
No LoRA	0.224	26.8	0.190
Multi-Angle LoRA	0.205	28.5	0.588
Triplet	0.198	24.2	0.240
Ours (3D-LoRA)	0.144	33.1	0.815

Table 3: Ablation studies of TRACE’s key components: 3D-Anchor synthesis, TGA alignment, and CVM refinement.

Figure 9: Comparison with direct video editing. Yellow marks subject synthesis and red marks background stability; TRACE better preserves 3D placement and scene details.

Figure 10: Multi-view editing ablation. View 1/2/3 are marked in red , yellow , and purple . No LoRA lacks 3D awareness, Multi-Angle LoRA suffers from inconsistent placement and background collapse, while 3D-LoRA preserves both object placement and scene background.

3D Scene Editing

3D scene editing studies how to make controlled changes to objects, appearance, layout, and motion while keeping a scene coherent across views.

See all 90 papers

VidEdit3D: Flexible 3D Gaussian Editing with Video Diffusion
Oct 26, 2026Yanming Yang, Chenxi Song, Tong Zhao +5Text-Guided 3D Scene Editing 3D Gaussian Splatting
Westlake University · University of Leeds · Nanyang Technological University +1
Look Before You Edit: Attention-Guided Camera Placement and Multi-View Alignment for 3D Gaussian Splatting Editing
Jul 22, 2026Jaeyeon Park, Taeho Kang, Youngki LeeMulti-View Consistency 3D Gaussian Splatting
Seoul National University, South Korea
TransSplat: Unbalanced Semantic Transport for Language-Driven 3DGS Editing
Apr 21, 2026Yanhui Chen, Jiahong Li, Jingchao Wang +33D Gaussian Splatting 3D Scene Editing
Guangdong University of Technology · Peking University

Oct 26, 2026J/K move · Enter open · S save

VidEdit3D: Flexible 3D Gaussian Editing with Video Diffusion

Yanming Yang, Chenxi Song, Tong Zhao, Juntao Xu+4

Westlake University · University of Leeds · Nanyang Technological University+1

Text-Guided 3D Scene Editing 3D Gaussian Splatting Multi-View Consistency

Open paper AskPDF arXiv

Recent advancements in text-guided 3D editing, particularly with 3D Gaussian Splatting (3DGS), have shown promise but still lag behind the flexibility of 2D editing tools. Existing methods face a critical trade-off between preserving multi-view consistency and allowing structural modifications. Image-based editors can also struggle to apply the intended edit to objects viewed from unusual angles. To address these limitations, we introduce VidEdit3D, a framework that reformulates 3D scene editing as a trajectory-based video editing task. By drawing a parallel between camera movement in static 3D scenes and temporal dynamics in video, we leverage the robust spatiotemporal priors of pre-trained Video Diffusion Models (VDMs) to encourage semantic and structural coherence across views. Specifically, we propose Trajectory-Guided Rendering (TGR) to select informative keyframes and construct smooth video clips aligned with VDM input requirements. To propagate edits from an anchor view to these clips, we develop Flow-Aligned Video Editing (FAVE), a training-free framework incorporating an Editing Consistency Guidance (ECG) mechanism. ECG contrasts source and edited image conditions at a shared latent state, using scheduled source-image amplification to strengthen adherence to the edited reference while maintaining original scene context. Finally, we introduce Region-Selective 3DGS Refinement (RS-GS) to strictly confine geometric and appearance updates to target primitives, reducing background attribute leakage. Experiments demonstrate stronger instruction alignment and better multi-view consistency than the baselines.

Abstract

3D Scene Editing

Explore similar work

VidEdit3D: Flexible 3D Gaussian Editing with Video Diffusion

Look Before You Edit: Attention-Guided Camera Placement and Multi-View Alignment for 3D Gaussian Splatting Editing

TransSplat: Unbalanced Semantic Transport for Language-Driven 3DGS Editing