VitronEdit: Universal Visual Editing across Images, Videos, and 3D Scenes

Jing Wang; Yanlin Li; Mong-Li Lee; Wynne Hsu; Shengqiong Wu; Hao Fei

cs.CVOct 19, 2026

VitronEdit: Universal Visual Editing across Images, Videos, and 3D Scenes

Authors: Jing Wang, Yanlin Li, Mong-Li Lee, Wynne Hsu, Shengqiong Wu, Hao Fei

Organizations: National University of Singapore · University of Oxford

Abstract

Recent advances in unified visual editing have begun to consolidate image and video editing within a single model, while 3D scene editing remains largely separate and constrained by scarce paired training data. Given the rich image and video editing supervision now available and the common editing semantics shared across modalities, a natural question is whether this knowledge can be transferred to data-scarce 3D editing. We present VitronEdit, a universal framework for image, video, and 3D scene editing. VitronEdit adopts a shared sequence representation and progressive multimodal training to transfer editing knowledge across modalities. We further introduce geometry-aware post-training, which leverages a pretrained geometry reconstruction model to provide depth and camera-pose supervision. Surprisingly, applying geometric supervision only to 3D samples yields the best overall performance, while image and video editing also improve despite receiving no direct geometric supervision, suggesting that geometric regularization transfers through the shared backbone. Our analysis further shows that image and video training mainly transfers editing semantics to 3D, whereas 3D supervision primarily improves multi-view consistency. The project page: https://vitron-llm.github.io/VitronEdit/.

Figures & tables

Figure 1: Image and video editing data are abundant, while paired multiview data are scarce, and semantically correct edits can still be geometrically inconsistent.

Figure 2: Overview of VitronEdit. Geometry-aware post-training uses a frozen 3D decoder to estimate depth and camera poses from predicted and target latents.

Method	Consistency					CLIP Semantics
	Struct. Dist. $\downarrow$	PSNR $\uparrow$	MSE $\downarrow$	LPIPS $\downarrow$	SSIM $\uparrow$	Whole $\uparrow$	Edited $\uparrow$	Ref. $\uparrow$
	$\times 10^{2}$		$\times 10^{3}$	$\times 10^{2}$	$\times 10^{2}$	$\times 10^{2}$	$\times 10^{2}$	$\times 10^{2}$
Instruction-based editing
ICEdit	3.54	27.88	6.01	5.76	89.39	24.24	21.94	/
VAREdit	5.75	23.52	11.02	8.19	81.75	25.47	22.74	/
ChordEdit	2.88	22.60	8.04	12.35	76.52	24.59	22.06	/

Table 1: Quantitative comparison on Image Editing . Struct. Dist.: Structure Distance. Ref.: Reference. Best in bold , second-best underlined .

Figure 3: Comparison with image-editing baselines: (a) instruction-based, (b) reference-based.

Figure 4: Comparison with unified-model baselines (UniVideo, VINO, OmniWeaving): (a) instruction-based, (b) reference-based.

Method	CLIPDir $\uparrow$ $\times 10^{2}$	CLIPSim $\uparrow$ $\times 10^{2}$	3D Consistency $\uparrow$ $\times 10^{2}$	Ph-Loss $\downarrow$ $\times 10^{2}$
GaussianEditor	9.43	26.84	64.14	20.93
DGE	11.27	28.12	64.79	19.86
EditSplat	13.76	28.60	66.30	19.69
VitronEdit (Ours)	13.15	28.43	74.34	18.88

Table 3: Quantitative comparison on 3D Editing . Best in bold , second-best underlined .

Figure 5: Qualitative comparison against all 3D-editing baselines.

Figure 6: Cross-modal transfer to 3D editing. Stage-1 training on image and video data substantially improves editing semantics on multi-view inputs, while incorporating 3D data yields further gains primarily in cross-view consistency.

Method	Image Editing		Video Editing			3D Editing
Method	LPIPS $\downarrow$	Whole $\uparrow$	LPIPS $\downarrow$	Whole $\uparrow$	MFS $\uparrow$	CLIPDir $\uparrow$	CLIPSim $\uparrow$	3D Consist. $\uparrow$
- w/o Geo	14.88 (-0.02)	23.48 (-0.07)	34.20 (-0.42)	26.90 (-0.05)	76.05 (-0.75)	12.47 (-0.68)	27.53 (-0.90)	73.98 (-0.36)
- w/o Camera	15.02 (-0.16)	23.48 (-0.07)	34.01 (-0.23)	26.92 (-0.03)	76.38 (-0.42)	11.76 (-1.39)	27.49 (-0.94)	72.17 (-2.17)
- w/o Depth	15.19 (-0.33)	23.57 (+0.02)	33.72 (+0.06)	26.92 (-0.03)	77.05 (+0.25)	12.10 (-1.05)	27.54 (-0.89)	72.45 (-1.89)
Geo-All	14.96 (-0.10)	23.50 (-0.05)	34.35 (-0.57)	26.89 (-0.06)	76.07 (-0.73)	11.62 (-1.53)	27.33 (-1.10)	72.82 (-1.52)
Ours (Geo-3D)	14.86	23.55	33.78	26.95	76.80	13.15	28.43	74.34

Table 4: Ablation of geometry-aware post-training. Values in parentheses show changes from Ours (Geo-3D), with

\uparrow

and

\downarrow

directions considered. Red indicates degradation after ablation, while gray indicates improvement. 3D Consist. denotes 3D Consistency.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Setting	Stage 1: progressive training			Stage 2
Setting	Ph. 1 (I)	Ph. 2 (I+V)	Ph. 3 (I+V+3D)	Geometry-aware
Training steps	10K	7K	3K	500
Learning rate (LoRA)	$5\times 10^{-5}$	$5\times 10^{-5}$	$2\times 10^{-5}$	$1\times 10^{-5}$
Learning rate (modality tokens)	$1\times 10^{-5}$	$1\times 10^{-5}$	$5\times 10^{-6}$	Frozen
Effective batch size (image)	200	480	480	128
Effective batch size (video)	/	120	120	32

Appendix

Table 5: Training schedule and hyperparameters. I, V, and 3D denote image, video, and 3D multi-view editing, respectively. A dash indicates an inactive modality.

Modality	Data Source	Task Type	Samples
Image	AnyEdit ( Yu et al., 2025 )	Instruct-based Editing	$\sim$ 2M
	NHR-Edit ( Kuprashevich et al., 2026 )	Instruct-based Editing
	AnyInsertion ( Song et al., 2026 )	Reference-based Editing
Video	OpenVE ( He et al., 2026 )	Instruct-based Editing	$\sim$ 500K
Video	Kiwi-Edit ( Lin et al., 2026 )	Reference-based Editing	$\sim$ 500K
3D	Self-built scene editing dataset	Instruct-based Editing	$\sim$ 8K

Appendix

Table 6: Training data used for image, video, and 3D multi-view editing.

Quantity	Same-direction accumulation	Alternating errors
Position error $e_{i}$	$i\delta$	$\frac{1-(-1)^{i}}{2}\delta$
Displacement error	$\delta$	$(-1)^{i+1}\delta$
Mean local loss	$\phi(\delta)$	$\phi(\delta)$
Mean anchor loss	$\frac{1}{M}\sum_{i=1}^{M}\phi(i\delta)$	$\frac{1}{2}\phi(\delta)$
Final position error	$M\delta$	$0$

Appendix

Table 7: Analytical comparison of translation error patterns. Here,

\phi

denotes SmoothL1 applied to each coordinate and averaged over the three coordinates. Equal local losses can correspond to different accumulated position errors.

Category	Parameter	Image	Video	3D
SmoothL1 thresholds	Depth $\beta_{d}$	0.02	0.02	0.02
SmoothL1 thresholds	Translation $\beta_{t}$	–	0.001	0.001
Translation weights	Local $\lambda_{\mathrm{rel}}$	–	1.00	1.00
Translation weights	Anchor $\lambda_{\mathrm{abs}}$	–	1.30	1.30
Effective loss weights	Depth $\lambda_{D}$	0.47	0.13	0.45
	Translation $\lambda_{T}$	–	0.73	2.62

Appendix

Table 8: SmoothL1 thresholds and effective geometry loss weights. Image and video weights apply only to ablation settings. A dash indicates an inapplicable setting.

Editing operation	Count	Share
Material or texture change	2,368	29.3%
Color change	1,433	17.7%
Global style or tone change	1,005	12.4%
Object replacement	829	10.3%
Object removal	809	10.0%
Object addition	745	9.2%

Appendix

Table 9: Distribution of editing operations in the final dataset. Percentages may not sum to 100 due to rounding.

3D Scene Editing

3D scene editing studies how to make controlled changes to objects, appearance, layout, and motion while keeping a scene coherent across views.

See all 89 papers

JointEdit3D: Feed-Forward 3D Scene Editing in a Unified Latent Space
Jun 11, 2026Xinnan Zhu, Ruijie Xu, Jiayu Ying +43D Scene Editing
East China Normal University · Shanghai Artificial Intelligence Laboratory · Fudan University +1
Edit in 2D, Verify in 3D: Reinforcement Learning for Multi-view Consistent Scene Editing
Mar 3, 2026Jiyuan Wang, Chunyu Lin, Lei Sun +8Multi-View Consistency Reinforcement Learning
1BJTU, 2AMap Alibaba Group, 3NTU, 4CQUPT
VidEdit3D: Flexible 3D Gaussian Editing with Video Diffusion
Oct 26, 2026Yanming Yang, Chenxi Song, Tong Zhao +5Text-Guided 3D Scene Editing 3D Gaussian Splatting
Westlake University · University of Leeds · Nanyang Technological University +1

Jun 11, 2026J/K move · Enter open · S save

JointEdit3D: Feed-Forward 3D Scene Editing in a Unified Latent Space

Xinnan Zhu, Ruijie Xu, Jiayu Ying, Daoguo Dong+3

East China Normal University · Shanghai Artificial Intelligence Laboratory · Fudan University+1

3D Scene Editing

Open paper AskPDF arXiv

Existing 3D scene editing methods typically rely on per-scene optimization over explicit 3D representations or cascaded edit-and-reconstruct pipelines, resulting in high test-time cost, limited 3D awareness, and structural inconsistencies. To couple appearance synthesis with geometry prediction, we adapt a pretrained unified RGB-geometry latent space to feed-forward scene editing. Given a source video and an edited reference image, JointEdit3D performs asymmetric latent inpainting: it observes only the edited RGB reference latent and jointly generates the remaining RGB latents and the entire geometry latent along the source trajectory. JointEdit3D introduces a dedicated SceneAnchor Branch to inject source-scene structure without forcing direct copying, and adopts edit/background-aware losses to balance edited-region fidelity with unedited-content preservation. To address the lack of paired resources for standardized 3D scene editing evaluation, we introduce SceneEdit3D-15K, a dataset with 15K paired editing samples and renderer-provided 3D annotations, together with SceneEdit3D-Bench, a curated 100-sample benchmark. Experiments show that JointEdit3D improves edited-region quality and 3D structural completeness over prior baselines while maintaining competitive background preservation.

Abstract

3D Scene Editing

Explore similar work

JointEdit3D: Feed-Forward 3D Scene Editing in a Unified Latent Space

Edit in 2D, Verify in 3D: Reinforcement Learning for Multi-view Consistent Scene Editing

VidEdit3D: Flexible 3D Gaussian Editing with Video Diffusion