From Understanding to Erasing: Towards Complete and Stable Video Object Removal

Dingming Liu; Wenjing Wang; Chen Li; Jing Lyu; Haotian Dong

cs.CVApr 2, 2026

From Understanding to Erasing: Towards Complete and Stable Video Object Removal

Authors: Dingming Liu, Wenjing Wang, Chen Li, Jing Lyu, Haotian Dong

Organizations: Peking University · WeChat Vision, Tencent Inc.

Abstract

Video object removal aims to erase target objects while reconstructing visually plausible and temporally coherent content. However, target objects often induce shadows, reflections, illumination changes and other effects that extend beyond the provided mask, making conventional mask-conditioned completion prone to visible residuals. We therefore formulate side-effect-aware object removal as an understanding-guided process that integrates object--effect relations, affected-region localization, and context-aware reconstruction. Specifically, we introduce Object-Induced Relation Distillation to transfer token-level object--effect relations from a pretrained vision foundation model to the video diffusion model. We then design Object-aware Framewise Context Cross-Attention to combine target-object semantics with per-frame background context for removal and reconstruction, and propose Attention-guided Region Localization to derive a soft spatial prior over the target object and its affected regions. Extensive experiments across multiple benchmarks demonstrate that our method achieves more complete object-and-effect removal and outperforms existing approaches in removal quality, side-effect suppression, and temporal consistency.

Figures & tables

Figure 2: Overview of our video object removal framework. Given the input video, object mask, and object-removed ground truth, we construct the diffusion input by concatenating the noisy latent, encoded input video, and resized mask, and derive a side-effect mask from the difference between the input and ground-truth videos. Object-Induced Relation Distillation transfers token-level object–effect relations from DINOv2 to the video diffusion transformer. Object-aware Framewise Context Cross-Attention combines target-object semantics with per-frame background context to guide removal and reconstruction. Attention-guided Region Localization fuses self- and cross-attention responses to predict a soft spatial prior over the target object and its affected regions, which is aligned with a difference-based affected-region cue.

Figure 3: Comparison of the attention weights from all query tokens to selected key tokens within target object regions in Wan2.1 and DINOv2 for the same video frames. Wan2.1 exhibits incomplete and noisy responses to the target object and its induced side effects, while DINOv2 produces more accurate and localized token-wise responses, indicating stronger object–effect relational cues.

Method	ROSE-Bench			CAMERA-Bench			VOR-Eval			VOR-Wild			Time (s)
Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	Rem. $\uparrow$	Mot. $\uparrow$	User $\uparrow$	Time (s)
FuseFormer [ 31 ]	25.9234	0.8857	0.2185	25.0572	0.9155	0.2020	21.0045	0.7641	0.3659	3.1860	3.1833	2.12 $\pm$ 0.25	9.998
FGT [ 52 ]	26.2947	0.8999	0.1888	25.1073	0.9095	0.1905	21.0661	0.7734	0.3309	2.8942	2.8942	1.86 $\pm$ 0.18	96.241
ProPainter [ 57 ]	25.3245	0.9105	0.1281	25.4201	0.9203	0.1268	20.8422	0.7767	0.2630	2.7855	2.7855	1.84 $\pm$ 0.33	180.231
VACE [ 17 ]	20.8162	0.7181	0.2238	16.9868	0.6928	0.2182	17.2086	0.5746	0.3488	3.7949	3.7897	1.18 $\pm$ 0.12	1191.391
DiffuEraser [ 28 ]	25.0410	0.8983	0.1198	26.7892	0.9409	0.0899	21.0845	0.7842	0.2503	1.3808	1.3808	1.35 $\pm$ 0.28	499.873

Table 1: Quantitative comparison with state-of-the-art methods. For VOR-Wild, Rem. and Mot. denote the Gemini-3 scores for object-removal quality and temporal coherence, respectively, while User denotes the mean user-study score for overall removal quality with 95% confidence intervals based on Student’s

t

-distribution. Additional results evaluated exclusively on the corresponding effect regions are provided in the supplementary material.

Figure 4: Qualitative comparison on diverse object-induced effects. Please refer to the supplementary material for more results.

Method	ROSE-Bench	CAMERA-Bench	VOR-Eval
FuseFormer [ 31 ]	0.004234	0.002088	0.006063
FGT [ 52 ]	0.004912	0.001831	0.007903
ProPainter [ 57 ]	0.004444	0.003563	0.007552
VACE [ 17 ]	0.004515	0.006747	0.009817
DiffuEraser [ 28 ]	0.004415	0.002339	0.007184
MiniMax-Remover [ 58 ]	0.004032	0.002432	0.006304

Table 2: Temporal consistency comparison with state-of-the-art methods using flow-warping error. Lower is better.

OIRD	OFCCA	ARL	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	No.
			22.3897	0.7866	0.2439	①
DINOv2			22.7473	0.7873	0.2337	②
	✓		22.4818	0.7869	0.2363	③
	✓	✓	22.8686	0.7890	0.2256	④
DINOv2	✓		22.9477	0.7904	0.2235	⑤
SAM2	✓	✓	23.3398	0.7911	0.2208

Table 3: Ablation study of different settings.

Figure 5: Qualitative results of different settings.

Figure 6: Visualization of attention weights from all query tokens to selected key tokens in different VFMs. Compared with SAM2 and V-JEPA2, DINOv2 yields more localized and semantically consistent responses over the target object and its associated effect regions.

Figure 7: Qualitative comparison using different teacher VFMs in OIRD. DINOv2 yields more accurate and localized removal, whereas SAM2 may mistakenly remove other objects of the same category and V-JEPA2 often fails to completely eliminate the target object and its induced effects.

Appendix figures & tables21 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 8: Samples from CAMERA-Bench for video object removal.

Method	ROSE-Bench			CAMERA-Bench			VOR-Eval
Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
FuseFormer [ 31 ]	17.1780	0.6637	0.0813	19.7633	0.7855	0.0773	17.6084	0.6329	0.2650
FGT [ 52 ]	17.1951	0.6728	0.0805	19.6013	0.7947	0.0768	17.5656	0.6433	0.2603
ProPainter [ 57 ]	16.9533	0.6691	0.0777	19.7189	0.8212	0.0702	17.3333	0.6480	0.2393
VACE [ 17 ]	15.8542	0.5914	0.0903	16.6771	0.6206	0.0980	16.3969	0.5219	0.2847
DiffuEraser [ 28 ]	17.0905	0.6582	0.0774	20.0279	0.8295	0.0692	17.3647	0.6479	0.2369

Appendix

Table 4: Quantitative evaluation on object-induced effect regions outside the input object mask on ROSE-Bench, CAMERA-Bench, and VOR-Eval. PSNR and SSIM are computed exclusively over the effect-region pixels, while LPIPS denotes the effect-focused perceptual distance.

$\lambda_{\mathrm{OIRD}}$	$\lambda_{\mathrm{ARL}}$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
0	0.1	22.8686	0.7890	0.2256
0.05	0.1	23.4215	0.7922	0.2178
0.1	0.1	23.8686	0.7944	0.2104
0.2	0.1	23.6127	0.7931	0.2149
0.1	0	22.9477	0.7904	0.2235
0.1	0.05	23.5068	0.7928	0.2162

Appendix

Table 5: Sensitivity analysis of the loss weights

\lambda_{\mathrm{OIRD}}

and

\lambda_{\mathrm{ARL}}

on VOR-Eval. When varying one weight, the other is fixed at 0.1.

Figure 9: Visualization of mask augmentation variants.

Figure 10: Limitation in scenes with strong physical interactions. After removing the person, the basketball may still follow its original trajectory, resulting in physically implausible motion.

Figure 11: Illustration of the user-study interface. Due to space limitations, only three anonymized results are displayed, whereas the actual study includes all evaluated methods.

Figure 12: Prompt used for Gemini-3-based evaluation on VOR-Wild.

Figure 13: More comparison results on VOR-Wild.(1/2)

Figure 14: More comparison results on VOR-Wild.(2/2)

Figure 15: More comparison results on VOR-Wild.(1/2)

Figure 16: More comparison results on VOR-Wild.(2/2)

Figure 17: More comparison results on CAMERA-Bench.(1/2)

Figure 18: More comparison results on CAMERA-Bench.(2/2)

Figure 19: More comparison results on VOR-Wild.(1/2)

Figure 20: More comparison results on VOR-Wild.(2/2)

Figure 21: More comparison results on VOR-Eval.(1/2)

Figure 22: More comparison results on VOR-Eval.(2/2)

Figure 23: More comparison results on VOR-Wild.(1/2)

Figure 24: More comparison results on VOR-Wild.(2/2)

Figure 25: More comparison results on ROSE-Bench.(1/2)

Figure 26: More comparison results on ROSE-Bench.(2/2)

Object Removal

See all 27 papers

EffectLearner: World-Aware Object-Effect Reasoning for Real-World Video Object Removal
Aug 6, 2026Feier Wu, Wanke Xia, Xu He +8Video Diffusion Models Video Editing
Tsinghua University · China Agricultural University · Jiangnan University +1
EffectErase: Joint Video Object Removal and Insertion for High-Quality Effect Erasing
Mar 19, 2026Yang Fu, Yike Zheng, Ziyun Dai +1Object Removal Video Editing
Institute of Big Data, College of Computer Science and Artificial Intelligence, Fudan University, China
From Ideal to Real: Stable Video Object Removal under Imperfect Conditions
Mar 10, 2026Jiagao Hu, Yuxuan Chen, Fuhao Li +4Video Editing Video Diffusion Models
MiLM Plus, Xiaomi Inc.

Aug 6, 2026J/K move · Enter open · S save

EffectLearner: World-Aware Object-Effect Reasoning for Real-World Video Object Removal

Feier Wu, Wanke Xia, Xu He, Zilang Zhou+7

Tsinghua University · China Agricultural University · Jiangnan University+1

Video Diffusion Models Video Editing Physical Reasoning in Video Video Restoration Object Removal

Open paper AskPDF arXiv

Video object removal must eliminate not only the target object but also its induced effects while maintaining high-fidelity and spatiotemporally coherent restoration. Existing methods mainly learn object-effect correspondences implicitly from predefined effect categories and fixed data distributions, limiting their generalization to complex real-world scenes involving compositional effects, spatially detached or weakly correlated effects, long-tail physical phenomena, and dynamically evolving interactions. We propose EffectLearner, a semantic-reasoning-enhanced framework that combines a VLM-based Object-Effect Reasoner with a DiT-based Video Eraser. Guided by a structured effect-analysis prompt, the Reasoner performs cross-modal reasoning over a target-highlighted video and extracts compact effect-aware context, which guides the Video Eraser toward comprehensive object-effect removal. Motion-aware mask guidance and motion-consistency supervision further improve removal coverage and spatiotemporal stability under object motion and evolving scene dynamics. To fully exploit the framework in challenging real-world scenarios, we further construct EffectWorld, a paired video dataset specifically designed for complex object-induced effects, and introduce a progressive training curriculum that combines common supervision with complex-effect data. On the standard ROSE-Bench, EffectLearner outperforms existing baselines on most metrics and achieves clear advantages on both EffectWorld-Eval and the challenging EffectWorld-Wild, demonstrating its ability to deliver high-quality video object removal in complex real-world scenes.

Abstract

Object Removal

Explore similar work

EffectLearner: World-Aware Object-Effect Reasoning for Real-World Video Object Removal

EffectErase: Joint Video Object Removal and Insertion for High-Quality Effect Erasing

From Ideal to Real: Stable Video Object Removal under Imperfect Conditions