Video object removal aims to eliminate dynamic target objects and their visual effects, such as deformation, shadows, and reflections, while restoring seamless backgrounds. Recent diffusion-based video inpainting and object removal methods can remove the objects but often struggle to erase these effects and to synthesize coherent backgrounds. Beyond method limitations, progress is further hampered by the lack of a comprehensive dataset that systematically captures common object effects across varied environments for training and evaluation. To address this, we introduce VOR (Video Object Removal), a large-scale dataset that provides diverse paired videos, each consisting of one video where the target object is present with its effects and a counterpart where the object and effects are absent, with corresponding object masks. VOR contains 60K high-quality video pairs from captured and synthetic sources, covers five effects types, and spans a wide range of object categories as well as complex, dynamic multi-object scenes. Building on VOR, we propose EffectErase, an effect-aware video object removal method that treats video object insertion as the inverse auxiliary task within a reciprocal learning scheme. The model includes task-aware region guidance that focuses learning on affected areas and enables flexible task switching. Then, an insertion-removal consistency objective that encourages complementary behaviors and shared localization of effect regions and structural cues. Trained on VOR, EffectErase achieves superior performance in extensive experiments, delivering high-quality video object effect erasing across diverse scenarios.
Figures & tables
Figure 2: Limitations of existing video object removal methods. While existing methods [ 47 , 26 ] can remove the main body within the input mask region, they often struggle to discover and remove the side effects ( e.g . , reflections) caused by the target object.
Figure 3: Removal–Insertion. Video object removal and insertion are inverse tasks that operate on the same affected regions.
Dataset
Source
Dynamic Camera
Dynamic Object
Dynamic Background
Scene Types
Object Classes
Image Pairs
Video Pairs
Average Duration (s)
Total Hours
RORD [ 31 ]
Real
✗
✓
✗
24
76
516.7K
3,106
–
5.98
Video4Removal [ 38 ]
Real
✗
✓
✗
6
–
134.3K
–
–
1.55
ROSE [ 26 ]
Synth.
✓
✗
✗
25
102
1,501.0K
16,678
6.00
27.79
VOR (Ours)
Real + Synth.
✓
✓
✓
67
366
12,556.8K
60,000
8.72
145.33
Table 1: Comparison of video object removal datasets. Our VOR dataset exceeds prior datasets in scale and diversity, offering broader object coverage and richer camera, object, and background dynamics. Further comparisons with image-level datasets are in supplementary.
Figure 4: Dataset Construction Pipeline of VOR. VOR is a hybrid dataset combining synthetic data and real-world captures. Synthetic data are generated in Blender using 3D environments, objects, and animations collected from public sources, together with carefully designed natural object and camera trajectories. Real-world data are recorded across diverse scenes and object categories using cameras, followed by the Ken Burns effect to simulate camera motion. All videos are segmented by SAM2 [ 30 ] and manually cleaned and refined by human annotators. The final dataset comprises triplet pairs of videos with and without the target object, and the corresponding mask.
Figure 5: Representative side effects in VOR dataset.
Figure 6: The framework of EffectErase. During training, removal and insertion pairs are encoded into the latent space by VAE and fused with noise via the Adaptor. Each DiT block performs cross-attention using the fused features x˙t as Query and eprompt from Task-Aware Region Guidance as Key/Value, producing attention maps that highlight affected regions. We aggregate attention maps from all blocks and apply max pooling to obtain a maximal-activation map, which is supervised by the effect consistency loss LEC to encourage both tasks to focus on the same affected area. At inference, users can flexibly switch the model between removal and insertion by modifying the inputs.
Method
ROSE-Benchmark (with GT)
VOR-Eval (with GT)
VOR-Wild (without GT)
PSNR↑
SSIM↑
LPIPS↓
FVD↓
PSNR↑
SSIM↑
LPIPS↓
FVD↓
QScore↑
User↑
ObjectClear [ 46 ]
29.535
0.920
0.076
742.829
22.583
0.787
0.190
1391.858
8.979
4.75
OmniPaint [ 43 ]
27.569
0.910
0.085
809.645
21.511
0.781
0.201
1439.867
8.942
4.38
Propainter [ 47 ]
27.200
0.915
0.095
171.020
21.975
0.800
0.225
589.012
8.860
4.88
DiffuEraser [ 21 ]
26.502
0.898
0.128
167.483
21.946
0.802
0.214
559.497
9.113
5.50
VACE [ 17 ]
20.805
0.694
0.174
254.117
17.677
0.591
0.294
806.476
8.229
1.50
Table 2: Quantitative results on ROSE and VOR. The best and second-best results are highlighted in bold and underlined, respectively.
Figure 7: Qualitative results on VOR-Eval. Inpainting models (VACE [ 17 ] , Propainter [ 47 ] ) fail to erase effects beyond the mask, while removal models (ROSE [ 26 ] , MinMax-Remover [ 48 ] ) leave artifacts. EffectErase effectively removes the target objects and their effects.
Figure 8: Qualitative results on VOR-Wild. EffectErase remains robust across in-the-wild scenarios such as multi-person occlusions, fast-moving sports, nighttime headlights, mirror reflections, and open-water boat scenes. Best viewed zoomed in.
Figure 9: Video Object Insertion by EffectErase. EffectErase seamlessly adapts to insertion, preserving background content while naturally integrating dynamic objects with realistic object-induced effects, e.g . , shadows and reflections.
Exp.
Real
LEC
TARG
Syn.
PSNR ↑
SSIM ↑
LPIPS ↓
FVD ↓
(a)
✓
20.409
0.720
0.243
368.664
(b)
✓
✓
21.020
0.737
0.224
354.545
(c)
✓
✓
✓
23.101
0.780
0.193
349.094
(d)
✓
✓
✓
✓
23.750
0.806
0.170
342.871
Table 3: Ablation study on VOR-Eval. Based on VOR real-world data (Real), the removal performance improves progressively by adding the consistency loss ( LEC ), Task-Aware Region Guidance (TARG), and synthesized training data (Syn.).
Figure I : Data capture software. Our app records aligned video pairs by locking exposure and focus, matching file names and durations, enabling reliable Bluetooth triggering for stable control, and using a tripod to remove camera shake.
Dataset
Source
Dynamic Camera
Dynamic Object
Dynamic Background
Scene Types
Object Classes
Image Pairs
Video Pairs
Average Duration (s)
Total Hours
ObjectDrop [ 39 ]
Real
✗
✗
✗
–
–
2.5K
–
–
–
Syn4Removal [ 16 ]
Synth. (paste)
✗
✗
✗
–
–
1,000K
–
–
–
LayerDecomp [ 41 ]
Synth. (paste)
✗
✗
✗
–
–
6.0K
–
–
–
OmniPaint [ 43 ]
Real
✗
✗
✗
–
–
3.3K
–
–
–
RORem [ 20 ]
Synth. (paste)
✗
✗
✗
–
–
201.1K
–
–
–
RORD [ 31 ]
Real
✗
✓
✗
24
76
516.7K
3,106
–
5.98
Table I: Comparison of object removal datasets. Image-level datasets are listed above the line, and video-level datasets are listed below. “–” denotes unreported or not applicable. Synth. (3D) denotes data generated using a graphics rendering engine, while Synth. (paste) denotes data created by directly pasting cropped foreground objects onto backgrounds.
Figure II : Scene category hierarchy. Our taxonomy organizes 67 scene types into structured outdoor and indoor groups.
Figure III : Prompt used for QScore evaluation. The prompt guides Qwen-VL to assess removal completeness and visual artifacts.
Method
PSNR ↑
SSIM ↑
LPIPS ↓
FVD ↓
ROSE
30.267
0.930
0.084
135.013
EffectErase
32.747
0.939
0.069
98.266
Table II : Effect-region erasing evaluation.
Figure IV : More insertion results of EffectErase.
Figure V : More removal results of EffectErase.
Figure VI : More comparison with state-of-the-art methods.
Figure VII : Failure cases when effects or accessories cannot be clearly attributed to the target object.
Video object removal must eliminate not only the target object but also its induced effects while maintaining high-fidelity and spatiotemporally coherent restoration. Existing methods mainly learn object-effect correspondences implicitly from predefined effect categories and fixed data distributions, limiting their generalization to complex real-world scenes involving compositional effects, spatially detached or weakly correlated effects, long-tail physical phenomena, and dynamically evolving interactions. We propose EffectLearner, a semantic-reasoning-enhanced framework that combines a VLM-based Object-Effect Reasoner with a DiT-based Video Eraser. Guided by a structured effect-analysis prompt, the Reasoner performs cross-modal reasoning over a target-highlighted video and extracts compact effect-aware context, which guides the Video Eraser toward comprehensive object-effect removal. Motion-aware mask guidance and motion-consistency supervision further improve removal coverage and spatiotemporal stability under object motion and evolving scene dynamics. To fully exploit the framework in challenging real-world scenarios, we further construct EffectWorld, a paired video dataset specifically designed for complex object-induced effects, and introduce a progressive training curriculum that combines common supervision with complex-effect data. On the standard ROSE-Bench, EffectLearner outperforms existing baselines on most metrics and achieves clear advantages on both EffectWorld-Eval and the challenging EffectWorld-Wild, demonstrating its ability to deliver high-quality video object removal in complex real-world scenes.
Feier Wu, Wanke Xia, Xu He +8
Tsinghua University · China Agricultural University · Jiangnan University +1
Video object removal aims to erase target objects while reconstructing visually plausible and temporally coherent content. However, target objects often induce shadows, reflections, illumination changes and other effects that extend beyond the provided mask, making conventional mask-conditioned completion prone to visible residuals. We therefore formulate side-effect-aware object removal as an understanding-guided process that integrates object--effect relations, affected-region localization, and context-aware reconstruction. Specifically, we introduce Object-Induced Relation Distillation to transfer token-level object--effect relations from a pretrained vision foundation model to the video diffusion model. We then design Object-aware Framewise Context Cross-Attention to combine target-object semantics with per-frame background context for removal and reconstruction, and propose Attention-guided Region Localization to derive a soft spatial prior over the target object and its affected regions. Extensive experiments across multiple benchmarks demonstrate that our method achieves more complete object-and-effect removal and outperforms existing approaches in removal quality, side-effect suppression, and temporal consistency.
Video object removal frequently struggles to simultaneously eliminate target objects and their associated physical effects (e.g., smoke, reflections, light, and ripples) in out-of-domain scenarios due to complex spatiotemporal ambiguities. While existing methods primarily rely on spatial masks, they often fail to capture weakly correlated effects, and the potential of explicit textual guidance remains underexplored. Furthermore, a fundamental optimization conflict exists in removal models between high-level semantic generalization and precise pixel-level background preservation. To address these challenges, we propose GenEraser, a novel framework for generalized and high-fidelity video object and effect removal. First, we introduce a Multi-Conditional Mixture-of-Experts (MC-MoE) paired with Bipartite Text guidance to fully exploit the multimodal priors of Diffusion Transformers, significantly enhancing the identification of complex effects. Second, a Learnable Deep ``CFG'' Fusion mechanism (LD-CFG) is developed to adaptively balance the relative dominance of mask and textual conditions across diverse scenarios. Finally, we propose a Decoupled Expert Architecture, comprising a Locator and a Preserver, to mitigate the inherent trade-off between semantic generalization and pixel alignment. Extensive experiments demonstrate that our GenEraser surpasses recent state-of-the-art approaches, achieving significant quantitative improvements (e.g., 2.16 dB and 1.44 dB on the ROSE Benchmark and VOR-Eval, respectively) while maintaining exceptionally robust generalization in open-world scenarios. https://cyqii.github.io/GenEraser.github.io/
Yuqing Chen, Lin Liu, Haisu Wu +4
Tsinghua University, China · Huawei, China · Southeast University, China +2