Recent advances in unified visual editing have begun to consolidate image and video editing within a single model, while 3D scene editing remains largely separate and constrained by scarce paired training data. Given the rich image and video editing supervision now available and the common editing semantics shared across modalities, a natural question is whether this knowledge can be transferred to data-scarce 3D editing. We present VitronEdit, a universal framework for image, video, and 3D scene editing. VitronEdit adopts a shared sequence representation and progressive multimodal training to transfer editing knowledge across modalities. We further introduce geometry-aware post-training, which leverages a pretrained geometry reconstruction model to provide depth and camera-pose supervision. Surprisingly, applying geometric supervision only to 3D samples yields the best overall performance, while image and video editing also improve despite receiving no direct geometric supervision, suggesting that geometric regularization transfers through the shared backbone. Our analysis further shows that image and video training mainly transfers editing semantics to 3D, whereas 3D supervision primarily improves multi-view consistency. The project page: https://vitron-llm.github.io/VitronEdit/.
Figures & tables
Figure 1: Image and video editing data are abundant, while paired multiview data are scarce, and semantically correct edits can still be geometrically inconsistent.
Figure 2: Overview of VitronEdit. Geometry-aware post-training uses a frozen 3D decoder to estimate depth and camera poses from predicted and target latents.
Method
Consistency
CLIP Semantics
Struct. Dist. ↓
PSNR ↑
MSE ↓
LPIPS ↓
SSIM ↑
Whole ↑
Edited ↑
Ref. ↑
×102
×103
×102
×102
×102
×102
×102
Instruction-based editing
ICEdit
3.54
27.88
6.01
5.76
89.39
24.24
21.94
/
VAREdit
5.75
23.52
11.02
8.19
81.75
25.47
22.74
/
ChordEdit
2.88
22.60
8.04
12.35
76.52
24.59
22.06
/
Table 1: Quantitative comparison on Image Editing . Struct. Dist.: Structure Distance. Ref.: Reference. Best in bold , second-best underlined .
Figure 3: Comparison with image-editing baselines: (a) instruction-based, (b) reference-based.
Table 3: Quantitative comparison on 3D Editing . Best in bold , second-best underlined .
Figure 5: Qualitative comparison against all 3D-editing baselines.
Figure 6: Cross-modal transfer to 3D editing. Stage-1 training on image and video data substantially improves editing semantics on multi-view inputs, while incorporating 3D data yields further gains primarily in cross-view consistency.
Method
Image Editing
Video Editing
3D Editing
LPIPS ↓
Whole ↑
LPIPS ↓
Whole ↑
MFS ↑
CLIPDir ↑
CLIPSim ↑
3D Consist. ↑
- w/o Geo
14.88 (-0.02)
23.48 (-0.07)
34.20 (-0.42)
26.90 (-0.05)
76.05 (-0.75)
12.47 (-0.68)
27.53 (-0.90)
73.98 (-0.36)
- w/o Camera
15.02 (-0.16)
23.48 (-0.07)
34.01 (-0.23)
26.92 (-0.03)
76.38 (-0.42)
11.76 (-1.39)
27.49 (-0.94)
72.17 (-2.17)
- w/o Depth
15.19 (-0.33)
23.57 (+0.02)
33.72 (+0.06)
26.92 (-0.03)
77.05 (+0.25)
12.10 (-1.05)
27.54 (-0.89)
72.45 (-1.89)
Geo-All
14.96 (-0.10)
23.50 (-0.05)
34.35 (-0.57)
26.89 (-0.06)
76.07 (-0.73)
11.62 (-1.53)
27.33 (-1.10)
72.82 (-1.52)
Ours (Geo-3D)
14.86
23.55
33.78
26.95
76.80
13.15
28.43
74.34
Table 4: Ablation of geometry-aware post-training. Values in parentheses show changes from Ours (Geo-3D), with ↑ and ↓ directions considered. Red indicates degradation after ablation, while gray indicates improvement. 3D Consist. denotes 3D Consistency.
Figure 10
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Setting
Stage 1: progressive training
Stage 2
Ph. 1 (I)
Ph. 2 (I+V)
Ph. 3 (I+V+3D)
Geometry-aware
Training steps
10K
7K
3K
500
Learning rate (LoRA)
5×10−5
5×10−5
2×10−5
1×10−5
Learning rate (modality tokens)
1×10−5
1×10−5
5×10−6
Frozen
Effective batch size (image)
200
480
480
128
Effective batch size (video)
/
120
120
32
Appendix
Table 5: Training schedule and hyperparameters. I, V, and 3D denote image, video, and 3D multi-view editing, respectively. A dash indicates an inactive modality.
Modality
Data Source
Task Type
Samples
Image
AnyEdit ( Yu et al., 2025 )
Instruct-based Editing
∼ 2M
NHR-Edit ( Kuprashevich et al., 2026 )
Instruct-based Editing
AnyInsertion ( Song et al., 2026 )
Reference-based Editing
Video
OpenVE ( He et al., 2026 )
Instruct-based Editing
∼ 500K
Kiwi-Edit ( Lin et al., 2026 )
Reference-based Editing
3D
Self-built scene editing dataset
Instruct-based Editing
∼ 8K
Appendix
Table 6: Training data used for image, video, and 3D multi-view editing.
Quantity
Same-direction accumulation
Alternating errors
Position error ei
iδ
21−(−1)iδ
Displacement error
δ
(−1)i+1δ
Mean local loss
ϕ(δ)
ϕ(δ)
Mean anchor loss
M1∑i=1Mϕ(iδ)
21ϕ(δ)
Final position error
Mδ
0
Appendix
Table 7: Analytical comparison of translation error patterns. Here, ϕ denotes SmoothL1 applied to each coordinate and averaged over the three coordinates. Equal local losses can correspond to different accumulated position errors.
Category
Parameter
Image
Video
3D
SmoothL1 thresholds
Depth βd
0.02
0.02
0.02
Translation βt
–
0.001
0.001
Translation weights
Local λrel
–
1.00
1.00
Anchor λabs
–
1.30
1.30
Effective loss weights
Depth λD
0.47
0.13
0.45
Translation λT
–
0.73
2.62
Appendix
Table 8: SmoothL1 thresholds and effective geometry loss weights. Image and video weights apply only to ablation settings. A dash indicates an inapplicable setting.
Editing operation
Count
Share
Material or texture change
2,368
29.3%
Color change
1,433
17.7%
Global style or tone change
1,005
12.4%
Object replacement
829
10.3%
Object removal
809
10.0%
Object addition
745
9.2%
Appendix
Table 9: Distribution of editing operations in the final dataset. Percentages may not sum to 100 due to rounding.
Existing 3D scene editing methods typically rely on per-scene optimization over explicit 3D representations or cascaded edit-and-reconstruct pipelines, resulting in high test-time cost, limited 3D awareness, and structural inconsistencies. To couple appearance synthesis with geometry prediction, we adapt a pretrained unified RGB-geometry latent space to feed-forward scene editing. Given a source video and an edited reference image, JointEdit3D performs asymmetric latent inpainting: it observes only the edited RGB reference latent and jointly generates the remaining RGB latents and the entire geometry latent along the source trajectory. JointEdit3D introduces a dedicated SceneAnchor Branch to inject source-scene structure without forcing direct copying, and adopts edit/background-aware losses to balance edited-region fidelity with unedited-content preservation. To address the lack of paired resources for standardized 3D scene editing evaluation, we introduce SceneEdit3D-15K, a dataset with 15K paired editing samples and renderer-provided 3D annotations, together with SceneEdit3D-Bench, a curated 100-sample benchmark. Experiments show that JointEdit3D improves edited-region quality and 3D structural completeness over prior baselines while maintaining competitive background preservation.
Xinnan Zhu, Ruijie Xu, Jiayu Ying +4
East China Normal University · Shanghai Artificial Intelligence Laboratory · Fudan University +1
Leveraging the priors of 2D diffusion models for 3D editing has emerged as a promising paradigm. However, multi-view consistency remains challenging in edited results, and the extreme scarcity of paired 3D-consistent editing data makes supervised fine-tuning (SFT) impractical, despite its effectiveness for editing tasks. In this paper, we observe that, while generating multi-view consistent 3D content is highly challenging, verifying 3D consistency is tractable, naturally positioning reinforcement learning (RL) as a feasible solution. Motivated by this, we propose RL3DEdit, a single-pass framework driven by RL optimization with novel rewards derived from the 3D foundation model, VGGT. Specifically, we leverage VGGT's robust priors learned from massive real-world data, feed the edited images into it, and utilize the output confidence maps and pose estimation errors as reward signals, effectively anchoring the 2D editing priors onto a 3D-consistent manifold via RL. Extensive experiments demonstrate that RL3DEdit achieves stable multi-view consistency and outperforms state-of-the-art methods in editing quality with high efficiency. To promote the development of 3D editing, we will release the code and model.
Recent advancements in text-guided 3D editing, particularly with 3D Gaussian Splatting (3DGS), have shown promise but still lag behind the flexibility of 2D editing tools. Existing methods face a critical trade-off between preserving multi-view consistency and allowing structural modifications. Image-based editors can also struggle to apply the intended edit to objects viewed from unusual angles. To address these limitations, we introduce VidEdit3D, a framework that reformulates 3D scene editing as a trajectory-based video editing task. By drawing a parallel between camera movement in static 3D scenes and temporal dynamics in video, we leverage the robust spatiotemporal priors of pre-trained Video Diffusion Models (VDMs) to encourage semantic and structural coherence across views. Specifically, we propose Trajectory-Guided Rendering (TGR) to select informative keyframes and construct smooth video clips aligned with VDM input requirements. To propagate edits from an anchor view to these clips, we develop Flow-Aligned Video Editing (FAVE), a training-free framework incorporating an Editing Consistency Guidance (ECG) mechanism. ECG contrasts source and edited image conditions at a shared latent state, using scheduled source-image amplification to strengthen adherence to the edited reference while maintaining original scene context. Finally, we introduce Region-Selective 3DGS Refinement (RS-GS) to strictly confine geometric and appearance updates to target primitives, reducing background attribute leakage. Experiments demonstrate stronger instruction alignment and better multi-view consistency than the baselines.
Yanming Yang, Chenxi Song, Tong Zhao +5
Westlake University · University of Leeds · Nanyang Technological University +1