From Ideal to Real: Stable Video Object Removal under Imperfect Conditions

Jiagao Hu; Yuxuan Chen; Fuhao Li; Zepeng Wang; Fei Wang; Daiguo Zhou; Jian Luan

cs.CVMar 10, 2026

From Ideal to Real: Stable Video Object Removal under Imperfect Conditions

Authors: Jiagao Hu, Yuxuan Chen, Fuhao Li, Zepeng Wang, Fei Wang, Daiguo Zhou, Jian Luan

Organizations: MiLM Plus, Xiaomi Inc.

Abstract

Removing objects from videos remains difficult in the presence of real-world imperfections such as shadows, abrupt motion, and defective masks. Existing diffusion-based video inpainting models often struggle to maintain temporal stability and visual consistency under these challenges. We propose Stable Video Object Removal (SVOR), a robust framework that achieves shadow-free, flicker-free, and mask-defect-tolerant removal through three key designs: (1) Mask Union for Stable Erasure (MUSE), a windowed union strategy applied during temporal mask downsampling to preserve all target regions observed within each window, effectively handling abrupt motion and reducing missed removals; (2) Denoising-Aware Segmentation (DA-Seg), a lightweight segmentation head on a decoupled side branch equipped with Denoising-Aware AdaLN and trained with mask degradation to provide an internal diffusion-aware localization prior without affecting content generation; and (3) Curriculum Two-Stage Training: where Stage I performs self-supervised pretraining on unpaired real-background videos with online random masks to learn realistic background and temporal priors, and Stage II refines on synthetic pairs using mask degradation and side-effect-weighted losses, jointly removing objects and their associated shadows/reflections while improving cross-domain robustness. Extensive experiments show that SVOR attains new state-of-the-art results across multiple datasets and degraded-mask benchmarks, advancing video object removal from ideal settings toward real-world applications. Project page: https://xiaomi-research.github.io/svor/.

Figures & tables

Figure 1: Results of our Stable Video Object Removal compared with MiniMax-Remover [ 46 ] and ROSE [ 23 ] in three common real-world challenges. The proposed SVOR achieves stable and artifact-free removal.

Figure 2: The framework of SVOR. Stage I: pretrain on unpaired real-world background videos using Random Mask Strategy to simulate object motion. Stage II: refine on paired synthetic data with Mask Degradation to mimic imperfect masks, where DA-Seg complements defective guidance. MUSE performs windowed union retention during mask temporal downsampling, preventing loss of dynamic location information.

	DAVIS					ROSE Bench					RORD-50
	bgPSNR↑	bgSSIM↑	TF↓	ReMove↑	GPT↑	PSNR↑	SSIM↑	TF↓	ReMove↑	GPT↑	PSNR↑	SSIM↑	TF↓	ReMove↑	GPT↑
FuseFormer [ 22 ]	29.51	0.8600	0.9921	0.8763	9.379	25.75	0.8847	0.9921	0.9058	9.776	27.04	0.8576	0.9961	0.9167	10.79
FGT [ 40 ]	30.74	0.9025	0.9537	0.8731	9.465	26.35	0.9059	0.9916	0.8927	10.89	27.51	0.8804	0.9961	0.9117	10.73
Propainter [ 45 ]	36.12	0.9753	0.9529	0.8607	9.446	25.14	0.9241	0.9903	0.8204	7.780	29.54	0.9367	0.9958	0.9040	10.16
DiffuEraser [ 17 ]	33.76	0.9467	0.9522	0.8626	10.25	26.83	0.9040	0.9917	0.8837	10.72	29.84	0.9345	0.9958	0.9126	11.53
VACE [ 10 ]	27.06	0.8656	0.9471	0.7081	5.263	22.71	0.8802	0.9913	0.7154	7.617	19.21	0.8622	0.9856	0.6842	3.962

Table 1: Quantitative comparison of different methods. The best performance is highlighted in bold , while the second-best is underlined . All results are reproduced using their official implementations to ensure fairness. For gen-omni, we use the recommended CogVideoX-Fun-V1.5-5b-InP version.

	Propainter [ 45 ]	gen-omni [ 16 ]	minimax [ 46 ]	ROSE [ 23 ]	Ours
Erasure	0.897	0 .904	0.926	0.934	0.978
Completion	0.228	0.588	0.412	0.353	0 .551
Average	0.563	0 .746	0.669	0.644	0.765

Table 2: User study results on DAVIS. Our SVOR achieves the highest overall perceptual performance.

Figure 3: Qualitative comparison between our SVOR and several state-of-the-art methods on real-world and synthetic samples. Previous methods facing issues like Undesired object , Artifacts , Blur , Undesired remove , Unremoved shadow , Unremoved effects . Our SVOR achieves consistently cleaner removal, fewer artifacts, and better shadow handling.

Figure 4: Effect of MUSE under abrupt-motion frames. MUSE improves removal even without additional training. “T”/“I” denote Training/Inference, “ $\times$ ”/“ $\checkmark$ ” indicate without/with MUSE.

Figure 5: Robust removal under SAM2 failures. Existing methods miss unsegmented objects when SAM2 drops, while our SVOR still achieves temporally consistent removal.

Panel of Figure 6 — Figure 6: ReMOVE performance under mask drop. Our SVOR remains stable while existing methods collapse.

Stage I	Stage II					ROSE Bench
bg_train	Base	MUSE	MaskD	Seg	w-loss	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	Remove $\uparrow$
$\checkmark$						25.94	0.909	0.077	0.864
	$\checkmark$					27.33	0.918	0.066	0.901
	$\checkmark$	$\checkmark$				27.87	0.921	0.061	0.902
	$\checkmark$	$\checkmark$	$\checkmark$			28.12	0.922	0.064	0.906
	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$		28.35	0.919	0.060	0.904

Table 3: Ablation results of our strategies

	DAVIS			ROSE Bench						RORD-50
	bgLPIPS $\downarrow$	TC $\uparrow$	CFD $\downarrow$	LPIPS $\downarrow$	bgPSNR $\uparrow$	bgSSIM $\uparrow$	bgLPIPS $\downarrow$	TC $\uparrow$	CFD $\downarrow$	LPIPS $\downarrow$	bgPSNR $\uparrow$	bgSSIM $\uparrow$	bgLPIPS $\downarrow$	TC $\uparrow$	CFD $\downarrow$
DiffuEraser [ 17 ]	0.0211	0.9676	0.4453	0.0885	28.5813	0.9214	0.0635	0.9870	0.3043	0.0469	31.2622	0.9506	0.0289	0.9878	0.3369
gen-omni [ 16 ]	0.1048	0.9746	0.4608	0.1013	28.4689	0.9000	0.0791	0.9901	0.3073	0.0442	31.7696	0.9308	0.0305	0.9973	0.3133
minimax [ 46 ]	0.0628	0.9739	0.4484	0.0917	28.0605	0.9144	0.0678	0.9903	0.3134	0.0507	30.3690	0.9433	0.0318	0.9912	0.3117
ROSE [ 23 ]	0.0631	0.9683	0.4805	0.0770	32.5895	0.9292	0.0417	0.9894	0.3486	0.0440	32.2127	0.9361	0.0272	0.9915	0.3150
Ours	0.0576	0.9746	0.4464	0.0451	33.2219	0.9443	0.0341	0.9902	0.3336	0.0440	32.4764	0.9511	0.0273	0.9931	0.2855

Table 4: Extended quantitative evaluation on DAVIS, ROSE Bench, and RORD-50. The best performance is highlighted in bold , while the second-best is underlined . All results are reproduced using the official implementations to ensure fairness.

Category	Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	ReMove $\uparrow$
Common	ROSE	36.60	0.952	0.041	0.859
Common	Ours	36.37	0.952	0.022	0.853
Shadow	ROSE	33.79	0.923	0.063	0.937
Shadow	Ours	33.61	0.939	0.038	0.938
Light	ROSE	30.07	0.921	0.086	0.900
Light	Ours	31.16	0.932	0.041	0.896

Table 5: Category-wise comparison on ROSE Bench. Best results are highlighted in bold . Overall, our method outperforms ROSE in the majority of categories.

	ROSE Bench				RORD-50
	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	Remove $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	Remove $\uparrow$
raw VACE	22.71	0.8802	0.1175	0.7154	19.21	0.8622	0.1395	0.6842
copy-paste data	25.78	0.9014	0.0770	0.8547	27.39	0.9310	0.0665	0.8420
Background data	25.94	0.9086	0.0773	0.8640	28.99	0.9350	0.0519	0.9135
Only Stage II	28.68	0.9268	0.0522	0.9071	30.02	0.9357	0.0479	0.9163
Stage I + II	31.47	0.9335	0.0451	0.9082	31.26	0.9378	0.0440	0.9179

Table 6: Ablation study of our two-stage training scheme on ROSE Bench and RORD-50.

Figure 7: Effectiveness of Stage I pre-training. Training with background videos significantly improves removal quality and success rate.

Figure 8: Comparison between Stage II–only training and the full two-stage training scheme. The complete two-stage training substantially improves background completion and shadow removal in real-world scenarios.

Figure 9: Effectiveness of MUSE . All methods suffer from missed removals or artifacts under abrupt motion, which is notably reduced after applying MUSE preprocessing.

	$k=0$				$k=2$				$k=4$
	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	Remove $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	Remove $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	Remove $\uparrow$
gen-omni	27.08	0.8831	0.1013	0.8998	26.78	0.8761	0.1151	0.9011	26.46	0.8698	0.1251	0.9014
gen-omni w/ MUSE	27.08	0.8838	0.1014	0.9003	26.80	0.8761	0.1152	0.9020	26.47	0.8697	0.1250	0.9027
minimax	26.30	0.8950	0.0917	0.8960	26.00	0.8884	0.0988	0.8968	25.89	0.8831	0.1042	0.9001
minimax w/ MUSE	26.33	0.8958	0.0918	0.8955	26.05	0.8885	0.0990	0.8987	25.97	0.8833	0.1042	0.9032
ROSE	31.12	0.9170	0.0770	0.9081	29.81	0.9019	0.0671	0.9060	29.08	0.8920	0.0745	0.9016

Table 7: Quantitative comparison with and without MUSE under different temporal compression ratios

k

in ROSE Bench.

Figure 10: Effectiveness of DA-Seg . Accurate mask predictions lead to better removals, while broken masks cause degraded results.

	Mask Drop 0				Mask Drop 50%
	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	Remove $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	Remove $\uparrow$
Vanilla Seg	29.50	0.9360	0.0478	0.9170	25.07	0.9141	0.0835	0.8286
DA-Seg	30.02	0.9357	0.0479	0.9179	27.42	0.9220	0.0708	0.8599

Table 8: Ablation study of our DA-Seg design on RORD-50.

Figure 11: Ablation of segmentation head design. DA-Seg produces more accurate localization, indicating that the context block extracts more reliable control context for DiT. This enables the model to suppress the target object in latent features and achieve more stable removal under degraded mask guidance.

Figure 12: Results under single mask condition. In some cases, our SVOR can remove the target object even with only single mask.

Panel of Figure 13 — Figure 13: Prompt for GPT-based evaluation

Video Editing

Video editing uses AI to alter, compose, relight, or reorganize moving footage while controlling how changes persist across frames.

See all 171 papers

From Understanding to Erasing: Towards Complete and Stable Video Object Removal
Apr 2, 2026Dingming Liu, Wenjing Wang, Chen Li +2Object Removal Video Editing
Peking University · WeChat Vision, Tencent Inc.
EffectErase: Joint Video Object Removal and Insertion for High-Quality Effect Erasing
Mar 19, 2026Yang Fu, Yike Zheng, Ziyun Dai +1Object Removal Video Editing
Institute of Big Data, College of Computer Science and Artificial Intelligence, Fudan University, China
Learning Stochastic Bridges for Video Object Removal via Video-to-Video Translation
Jan 17, 2026Zijie Lou, Xiangwei Feng, Jiaxin Wang +7Video Editing Object Removal
MT Lab, Meitu Inc., Beijing 100083, China · Beijing Jiaotong University, Beijing 100044, China

Apr 2, 2026J/K move · Enter open · S save

From Understanding to Erasing: Towards Complete and Stable Video Object Removal

Dingming Liu, Wenjing Wang, Chen Li, Jing Lyu+1

Peking University · WeChat Vision, Tencent Inc.

Object Removal Video Editing Video Diffusion Models Temporal Consistency Image Inpainting

Open paper AskPDF arXiv

Video object removal aims to erase target objects while reconstructing visually plausible and temporally coherent content. However, target objects often induce shadows, reflections, illumination changes and other effects that extend beyond the provided mask, making conventional mask-conditioned completion prone to visible residuals. We therefore formulate side-effect-aware object removal as an understanding-guided process that integrates object--effect relations, affected-region localization, and context-aware reconstruction. Specifically, we introduce Object-Induced Relation Distillation to transfer token-level object--effect relations from a pretrained vision foundation model to the video diffusion model. We then design Object-aware Framewise Context Cross-Attention to combine target-object semantics with per-frame background context for removal and reconstruction, and propose Attention-guided Region Localization to derive a soft spatial prior over the target object and its affected regions. Extensive experiments across multiple benchmarks demonstrate that our method achieves more complete object-and-effect removal and outperforms existing approaches in removal quality, side-effect suppression, and temporal consistency.

Abstract

Video Editing

Explore similar work

From Understanding to Erasing: Towards Complete and Stable Video Object Removal

EffectErase: Joint Video Object Removal and Insertion for High-Quality Effect Erasing

Learning Stochastic Bridges for Video Object Removal via Video-to-Video Translation