CLEAR: Context-Aware Learning with End-to-End Mask-Free Inference for Adaptive Video Subtitle Removal

Qingdong He; Chaoyi Wang; Peng Tang; Yifan Yang; Xiaobin Hu

cs.CVMar 23, 2026

CLEAR: Context-Aware Learning with End-to-End Mask-Free Inference for Adaptive Video Subtitle Removal

Authors: Qingdong He, Chaoyi Wang, Peng Tang, Yifan Yang, Xiaobin Hu

Organizations: University of Electronic Science and Technology of China · University of Chinese Academy of Sciences · Technical University of Munich · Shanghai Jiao Tong University ·

Abstract

Video subtitle removal aims to distinguish text overlays from background content while preserving temporal coherence. Existing diffusion-based methods necessitate explicit mask sequences during both training and inference phases, which restricts their practical deployment. In this paper, we present CLEAR (Context-aware Learning for End-to-end Adaptive Video Subtitle Removal), a mask-free framework that achieves truly end-to-end inference through context-aware adaptive learning. Our two-stage design decouples prior extraction from generative refinement: Stage I learns disentangled subtitle representations via self-supervised orthogonality constraints on dual encoders, while Stage II employs LoRA-based adaptation with generation feedback for dynamic context adjustment. Notably, our method only requires 0.77% of the parameters of the base diffusion model for training. On Chinese subtitle benchmarks, CLEAR outperforms mask-dependent baselines by + 6.77dB PSNR and -74.7% VFID, while demonstrating superior zero-shot generalization across six languages (English, Korean, French, Japanese, Russian, German), a performance enabled by our generation-driven feedback mechanism that ensures robust subtitle removal without ground-truth masks during inference.

Figures & tables

Figure 2: CLEAR framework overview. Two-stage training enables mask-free inference: Stage I learns self-supervised priors $\mathcal{M}^{prior}$ through disentangled feature learning; Stage II trains LoRA-adapted diffusion with adaptive weighting optimized by distillation, generation feedback, and sparsity losses. Inference requires only subtitled video input.

Figure 3: CLEAR Pipeline Details. Stage I: Dual encoders extract disentangled features with orthogonality and adversarial losses. Stage II: Occlusion head predicts adaptive weights from DiT encoder features, modulating generation through focal weighting $(\epsilon^{gen})^{\gamma}$ with gradient backflow.

Figure 4: Illustration of our progressive mask refinement. Despite noisy pseudo-labels from Stage I ( $\mathcal{M}^{prior}$ ), Stage II learns adaptive weights ( $\mathcal{M}^{pred}$ ) that dynamically adjust based on generation difficulty, enabling robust context-aware modulation, which are internal to training and not predicted during inference.

Method	Reconstruction			Perceptual		Temporal		Flow		Time
Method	PSNR↑	SSIM↑	LPIPS↓	DISTS↓	VFID↓	TWE↓	TC↓	Mean↓	Var↓	s/frame↓
ProPainter ( Zhou et al., 2023 )	17.24	0.658	0.3291	0.245	98.46	1.286	1.160	0.952	0.885	2.36
Minimax-Remover ( Zi et al., 2025 )	20.03	0.773	0.166	0.119	95.39	4.222	3.016	0.841	0.415	4.90
DiffuEraser ( Li et al., 2025 )	17.85	0.672	0.458	0.228	72.51	1.523	1.174	0.840	0.630	3.47
CLEAR (Ours)	26.80	0.894	0.101	0.075	20.37	1.227	1.049	0.209	0.029	4.86

Table 1: Quantitative comparison with state-of-the-art methods on Chinese subtitle test set. CLEAR uses default configuration (rank=64, steps=5, cfg=1.0, lora_scale=1.0).

Figure 5: Qualitative comparison with baseline methods. Unlike ProPainter, Minimax-Remover, and DiffuEraser that require explicit masks during inference, CLEAR achieves mask-free end-to-end removal with clean subtitle elimination and fine-grained detail preservation.

Configuration	Reconstruction			Perceptual		Temporal		Flow
Configuration	PSNR↑	SSIM↑	LPIPS↓	DISTS↓	VFID↓	TWE↓	TC↓	Mean↓	Var↓
Baseline (LoRA-only)	21.62	0.855	0.131	0.088	34.74	1.320	1.137	0.217	0.032
+ M1: Stage I Prior with Focal Weighting	23.11	0.868	0.130	0.091	38.21	1.303	1.123	0.216	0.031
+ M2: Context Distillation	24.72	0.890	0.110	0.083	31.73	1.279	1.102	0.212	0.030
+ M3: Context-Aware Adaptation	25.09	0.891	0.109	0.083	31.56	1.257	1.082	0.211	0.029
+ M4: Context Consistency (CLEAR)	26.80	0.894	0.101	0.075	20.37	1.227	1.049	0.209	0.029

Table 2: Ablation study on CLEAR’s modular design. Each row progressively adds one component.

Configuration	Reconstruction			Perceptual		Temporal		Flow		Time
Configuration	PSNR↑	SSIM↑	LPIPS↓	DISTS↓	VFID↓	TWE↓	TC↓	Mean↓	Var↓	s/frame↓
Denoising Steps (lora_scale=1.0, cfg=1.0)
steps=5	26.80	0.894	0.101	0.075	20.37	1.227	1.049	0.209	0.029	4.86
steps=10	29.43	0.884	0.101	0.081	35.70	1.358	1.243	0.215	0.030	9.92
CFG Scale (lora_scale=1.0, steps=5)
cfg=0.8	25.24	0.899	0.109	0.083	30.05	1.257	1.078	0.210	0.028	4.86

Table 3: Results of key inference hyperparameters on performance.

Figure 6: Visual comparison of output quality between the LoRA-only baseline and CLEAR (Ours). The baseline struggles with residual subtitles and background blurring, while our two-stage framework achieves clean subtitle removal and faithful preservation of structural details.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Figure A1 : Qualitative comparison with baseline methods. Unlike ProPainter, Minimax-Remover, and DiffuEraser that require explicit masks during inference, CLEAR achieves mask-free end-to-end removal with clean subtitle elimination and fine-grained detail preservation.

Figure A2 : Qualitative comparison with baseline methods. Unlike ProPainter, Minimax-Remover, and DiffuEraser that require explicit masks during inference, CLEAR achieves mask-free end-to-end removal with clean subtitle elimination and fine-grained detail preservation.

Figure A3 : Zero-shot cross-lingual generalization. CLEAR achieves robust subtitle removal across English, Korean, French, Japanese, Russian, and Arabic without language-specific training. The mask overlays visualize the learned context-aware guidance ability during Stage II training, demonstrating accurate subtitle localization; notably, these masks are not predicted during inference.

Figure A4 : Zero-shot cross-lingual generalization. CLEAR achieves robust subtitle removal across English, Korean, French, Japanese, Russian, and Arabic without language-specific training. The mask overlays visualize the learned context-aware guidance ability during Stage II training, demonstrating accurate subtitle localization; notably, these masks are not predicted during inference.

Figure A5 : Zero-shot cross-lingual generalization. CLEAR achieves robust subtitle removal across English, Korean, French, Japanese, Russian, and Arabic without language-specific training. The mask overlays visualize the learned context-aware guidance ability during Stage II training, demonstrating accurate subtitle localization; notably, these masks are not predicted during inference.

Video Editing

Video editing uses AI to alter, compose, relight, or reorganize moving footage while controlling how changes persist across frames.

See all 178 papers

SEDiT: Mask-Free Video Subtitle Erasure via One-step Diffusion Transformer
May 14, 2026Zheng Hui, Yunlong BaiDiffusion Transformer Video Editing
Baidu Inc., Beijing, China.
MFVSR: A Multi-Style and Fine-Grained Benchmark for Video Subtitle Removal
Oct 13, 2026Xiyan Huang, Qingdong He, Peng Tang +3Video Restoration Video Editing
National University of Singapore · Tencent Youtu Lab · University of Chinese Academy of Sciences
DeOverlay: A Benchmark and Source-Anchored Flow for Video Overlay Removal
Oct 19, 2026Deshui Miao, Yuhang Ma, Shaoyong Jia +4Flow Matching
ByteDance · Southern University of Science and Technology

May 14, 2026J/K move · Enter open · S save

SEDiT: Mask-Free Video Subtitle Erasure via One-step Diffusion Transformer

Zheng Hui, Yunlong Bai

Baidu Inc., Beijing, China.

Diffusion Transformer Video Editing Diffusion Models

Open paper AskPDF arXiv

Recent breakthroughs in video diffusion models have significantly accelerated the development of video editing techniques. However, existing methods often rely on inpainting video frames based on masked input, which requires extracting the target video mask in advance, and the precision of the segmentation directly affects the quality of the completion. In this paper, we present SEDiT, a novel one-stage video Subtitle Erasure approach via One-step Diffusion Transformer. We introduce a mask-free inference approach that enables direct erasure of the targeted subtitle. The proposed one-stage framework mitigates the sub-optimality inherent in the two-stage processing of prior models. Since subtitle removal is a localized editing task in which most pixels remain unchanged, the underlying distribution shift is minimal, making it well-suited to one-step generation under rectified flow. We empirically validate the reliability of one-step denoising and further provide a formal theoretical justification. Under the localized-editing structure of subtitle removal, the conditional optimal transport (OT) map and its induced rectified flow velocity field are Lipschitz continuous with respect to the latent variable, which underpins the theoretical feasibility of one-step sampling. To address the challenge of long-term temporal consistency, we adopt a hybrid training strategy by occasionally conditioning the model with a clean first-frame latent. This facilitates temporal continuity, allowing each segment during inference to leverage the output of its predecessor. To avoid visible seams caused by cropping and reinserting processed targets, particularly in scenarios involving substantial motion, we feed the original video directly into SEDiT. Thanks to one-step and chunk-wise streaming inference, our method can efficiently handle native 1440p video with infinite length.

Abstract

Video Editing

Explore similar work

SEDiT: Mask-Free Video Subtitle Erasure via One-step Diffusion Transformer

MFVSR: A Multi-Style and Fine-Grained Benchmark for Video Subtitle Removal

DeOverlay: A Benchmark and Source-Anchored Flow for Video Overlay Removal