ViTeX-Bench: Benchmarking High-Fidelity Video Scene Text Editing

Xinghao Chen; Xiangbo Gao; Jiongze Yu; Yuheng Wu; Zhengzhong Tu

cs.CVSep 30, 2026

ViTeX-Bench: Benchmarking High-Fidelity Video Scene Text Editing

Authors: Xinghao Chen, Xiangbo Gao, Jiongze Yu, Yuheng Wu, Zhengzhong Tu

Organizations: Texas A&M University

Abstract

Recent video generation is increasingly realistic and controllable, yet video editing remains less developed, particularly for precise local edits that must preserve the original scene dynamics. Video scene text editing replaces text on scene surfaces, such as storefront signs, whiteboards, and product labels, while preserving the surrounding content, motion, and camera dynamics. Although scene text editing is well studied for images, video scene text editing that achieves high visual quality, temporal consistency, and edit locality remains underexplored. Existing resources offer limited paired real-video data, and general video-editing metrics do not directly measure whether the requested text remains correct over time. We introduce ViTeX-Bench, a benchmark suite comprising ViTeX-Dataset and a three-axis evaluation protocol. The dataset contains 387 real-world 720p videos with text-region masks and editing instructions: 230 provide reviewed, pipeline-generated paired edits for training, and 157 form a frozen evaluation split. The protocol evaluates text correctness, visual and temporal quality, and edit locality through 13 metrics, with one primary metric per axis and a Pareto comparison of their trade-offs. OCR calibration, human evaluation, and annotation-sensitivity analyses support the interpretation of these scores. Across eight baselines from four editing families, accurate text, temporal stability, and scene preservation remain difficult to achieve together. We also release ViTeX-Edit-14B, an open-source reference editor fine-tuned on the paired training split with motion-aligned glyph-video conditioning. It achieves CharAcc 0.688, the highest mean among the evaluated video-native editors, and the lowest comparable text-crop Warp among raw editor outputs. ViTeX-Bench provides a reproducible foundation for studying these trade-offs in video scene text editing.

Figures & tables

Figure 1: Overview of ViTeX-Bench. Paired training examples from ViTeX-Dataset (shown on top) illustrate the high visual fidelity of the paired data across diverse text-motion conditions. In ViTeX-Bench, each task instance provides a source video $V$ , a text-region mask $M$ , and a source-target string pair $(s_{\mathrm{src}},s_{\mathrm{tgt}})$ (shown on the left). Representative baseline outputs (in the middle) exhibit distinct failure modes, while ViTeX-Bench (on the right) scores each edit along three axes (text correctness, visual quality, and edit locality) with a total of 13 metrics.

Figure 2: Training data construction pipeline. We compose the four assets ( $M$ , $(s_{\mathrm{src}},s_{\mathrm{tgt}})$ , $V_{\mathrm{clean}}$ , and $p_{1}^{\text{new}}$ ) into the paired edit $\tilde{V}$ via Strategy A (alpha composition) or Strategy B (PISCO-based inserter). The overview is in Section 3.1 and implementation details are in Appendix C .

Videos			String length		Font style (%)
Train	Eval	Scripts	Source	Target	Printed	Handwritten	Artistic
230	157	4	$8.0\pm 5.7$	$8.0\pm 5.4$	23	44	33

Table 1: ViTeX-Dataset statistics. All clips are

1280\!\times\!720

, 120 frames at 24 fps. String lengths are in characters (mean

\pm

std); source and target strings range over 1–41 and 1–36 characters. Scripts counts the writing systems in the frozen evaluation split (Latin, Chinese, Japanese, Cyrillic). Font styles are shares of all 387 videos, rounded.

Figure 3: ViTeX-Edit-14B architecture. Three streams condition the VACE backbone: target text $s_{\mathrm{tgt}}$ via frozen uMT5-XXL, source $V$ and mask $M$ via the VCU, and a target-text glyph video pooled by the glyph encoder into tokens $E_{G}$ . Every VACE block queries $E_{G}$ through an added condition cross-attention layer. Implementation details are in Appendix G .

		Text correctness			Visual quality						Edit locality
Method	Fam.	SeqAcc $\uparrow$	CharAcc $\uparrow$	TTS $\uparrow$	Flicker ${}_{f}\downarrow$	Flicker ${}_{c}\downarrow$	Warp ${}_{f}\downarrow$	Warp ${}_{c}\downarrow$	MUSIQ ${}_{f}\uparrow$	MUSIQ ${}_{c}\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	DreamSim $\downarrow$
Source video	—	0.000	0.317	0.760	3.72	3.68	1.46	1.27	70.33	45.12	$\infty$	1.000	0.000	0.000
AnyText2 [ 9 ]	A	0.280	0.633	0.382	3.34	4.95	2.04	3.95	66.68	41.65	25.56	0.905	0.091	0.043
TextCtrl [ 10 ]	A	0.475	0.734	0.511	3.80	4.29	1.59	2.09	70.32	42.77	41.14	0.994	0.008	0.004
FLUX-Text [ 11 ]	A	0.528	0.737	0.326	5.11	14.81	3.03	13.01	70.26	43.85	31.49	0.975	0.029	0.012
RS-STE [ 12 ]	A	0.354	0.626	0.534	3.73	3.66	1.61	1.81	69.57	34.26	37.00	0.983	0.024	0.007

Table 2: Main evaluation results on ViTeX-Bench . The 95% bootstrap CIs are deferred to Tables 3 , 4 and 5 . Per-column shading among raw editors only: best/ 2nd/ 3rd distinct displayed values; rounded ties share shading. The Composite row is an unranked post-processing control; all-baseline controls appear in Table 7 .

f{/}c

denotes full-frame/text-crop. † VideoPainter

\mathrm{Flicker}_{f{/}c}

and

\mathrm{Warp}_{f{/}c}

excluded from ranking ( Appendix F ). The Source video row (

\hat{V}=V

, the source video unmodified) is reported as a reference, excluded from ranking.

Figure 4: Four representative failure cases. (A) Wan2.1-VACE-14B: masked region returned essentially as the source, target string never rendered. (B) FLUX-Text [ 11 ] : per-frame editing yields legible text on individual frames, but the glyph identity is inconsistent between adjacent frames. (C) Kling Video 3.0 Omni [ 6 ] : high per-frame visual quality, but the rendered text does not match the requested target. (D) TextCtrl+AnyV2V [ 10 , 13 ] : the first-frame edit propagates while the rendered text and surrounding scene structure drift away.

Figure 5: ViTeX-Edit-14B outputs for the four source videos in Figure 4 , shown with five evenly spaced frames per video. Panels (A)–(D) correspond to the examples used to illustrate Wan2.1-VACE-14B, FLUX-Text, Kling Video 3.0 Omni, and TextCtrl+AnyV2V, respectively. On these selected examples, ViTeX-Edit-14B renders the target text correctly and maintains it across the displayed frames.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Method	SeqAcc	CharAcc	TTS
Source video	0.000 [0.000, 0.000]	0.317 [0.285, 0.350]	0.760 [0.721, 0.800]
AnyText2	0.280 [0.229, 0.333]	0.633 [0.591, 0.676]	0.382 [0.336, 0.427]
TextCtrl	0.475 [0.409, 0.539]	0.734 [0.684, 0.780]	0.511 [0.459, 0.559]
FLUX-Text	0.528 [0.483, 0.578]	0.737 [0.696, 0.778]	0.326 [0.286, 0.369]
RS-STE	0.354 [0.288, 0.412]	0.626 [0.574, 0.677]	0.534 [0.484, 0.588]
TextCtrl + AnyV2V	0.057 [0.031, 0.088]	0.308 [0.274, 0.346]	0.257 [0.222, 0.297]

Appendix

Table 3: Headline text-correctness means with 95% bootstrap confidence intervals on the

157

-video evaluation split. Text-correctness metrics are scored on source-detectable frames only ( Section 3.2 );

5

videos with no detectable source frame are excluded from the means.

Method	Flicker ${}_{f}\downarrow$	Flicker ${}_{c}\downarrow$	Warp ${}_{f}\downarrow$	Warp ${}_{c}\downarrow$	MUSIQ ${}_{f}\uparrow$	MUSIQ ${}_{c}\uparrow$
Source video	3.72 [3.14, 4.41]	3.68 [2.91, 4.63]	1.46 [1.33, 1.62]	1.27 [1.09, 1.46]	70.33 [69.55, 71.14]	45.12 [43.23, 47.09]
AnyText2	3.34 [2.89, 3.88]	4.95 [4.36, 5.64]	2.04 [1.85, 2.28]	3.95 [3.56, 4.41]	66.68 [65.82, 67.57]	41.65 [39.96, 43.45]
TextCtrl	3.80 [3.22, 4.49]	4.29 [3.53, 5.21]	1.59 [1.44, 1.76]	2.09 [1.86, 2.35]	70.32 [69.52, 71.14]	42.77 [40.96, 44.74]
FLUX-Text	5.11 [4.50, 5.80]	14.81 [13.56, 16.01]	3.03 [2.80, 3.29]	13.01 [11.71, 14.21]	70.26 [69.45, 71.07]	43.85 [42.09, 45.74]
RS-STE	3.73 [3.15, 4.40]	3.66 [2.97, 4.53]	1.61 [1.46, 1.77]	1.81 [1.60, 2.06]	69.57 [68.74, 70.43]	34.26 [32.67, 36.01]
TextCtrl + AnyV2V	4.98 [4.42, 5.67]	4.98 [4.17, 5.89]	4.11 [3.68, 4.59]	3.97 [3.46, 4.56]	69.41 [68.41, 70.46]	33.85 [32.37, 35.35]

Appendix

Table 4: Visual-quality means with 95% bootstrap confidence intervals on the

157

-video evaluation split.

f

c

denotes full-frame/text-crop scope. Lower is better for Flicker and Warp; higher is better for MUSIQ. VideoPainter Flicker and Warp values are not directly comparable ( Appendix F ).

Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	DreamSim $\downarrow$
Source video	$\infty$	1.000 [1.000, 1.000]	0.000 [0.000, 0.000]	0.000 [0.000, 0.000]
AnyText2	25.56 [25.11, 26.06]	0.905 [0.894, 0.915]	0.091 [0.087, 0.096]	0.043 [0.040, 0.046]
TextCtrl	41.14 [40.68, 41.61]	0.994 [0.994, 0.995]	0.008 [0.007, 0.009]	0.004 [0.004, 0.005]
FLUX-Text	31.49 [31.13, 31.85]	0.975 [0.973, 0.976]	0.029 [0.027, 0.030]	0.012 [0.011, 0.013]
RS-STE	37.00 [36.71, 37.30]	0.983 [0.982, 0.984]	0.024 [0.022, 0.025]	0.007 [0.007, 0.008]
TextCtrl + AnyV2V	21.08 [20.67, 21.51]	0.785 [0.769, 0.801]	0.225 [0.212, 0.239]	0.073 [0.066, 0.083]

Appendix

Table 5: Edit-locality means with 95% bootstrap confidence intervals on the

157

-video evaluation split. PSNR/SSIM are higher-is-better; LPIPS/DreamSim are lower-is-better. The Source video row satisfies

\hat{f}_{t}=f_{t}

exactly, so its PSNR is mathematically

\infty

; LPIPS and DreamSim are exactly

0

. Bounding-box-local Family-A editors (TextCtrl, RS-STE) copy exterior pixels from the source before encoding ( Appendix F ).

Editor	Scope	Working resolution	Per-frame pipeline
AnyText2	full	$1024\!\times\!576$	$1280\!\times\!720\to 1024\!\times\!576\to$ edit $\to$ Lanczos $\to 1280\!\times\!720$
FLUX-Text	full	FLUX native	$1280\!\times\!720\to$ FLUX native $\to$ edit $\to$ resample $\to 1280\!\times\!720$
TextCtrl	bounding box	$256\!\times\!256$	bounding-box crop $\to 256\!\times\!256\to$ edit $\to$ Lanczos $\to$ alpha-composite back
RS-STE	bounding box	$32\!\times\!128$	bounding-box crop $\to 32\!\times\!128\to$ edit $\to$ Lanczos $\to$ alpha-composite back

Appendix

Table 6: Family-A per-frame editor working resolutions and the resampling pipeline applied to each

1280\!\times\!720

source frame. “Full” = the editor processes the entire frame; “bounding box” = it only modifies pixels inside the dilated-mask bounding box.

Method	Raw	+Composite	Raw	+Composite	Raw SeqAcc
	Flicker ${}_{f}\downarrow$		PSNR-loc $\uparrow$
AnyText2	3.34	3.85	25.6	42.9	0.280
TextCtrl	3.80	3.78	41.1	43.0	0.475
FLUX-Text	5.11	4.47	31.5	42.9	0.528
RS-STE	3.73	3.71	37.0	43.0	0.354
TextCtrl + AnyV2V	4.98	3.77	21.1	43.0	0.057

Appendix

Table 7: Shared Composite control. Flicker is full-frame; PSNR-loc is in dB. The final column contains raw SeqAcc. Baseline post-Composite OCR was checked only on a sample (

|\Delta|\leq 0.04

); the fully re-scored ViTeX-Edit-14B pair appears in Table 2 . † VideoPainter temporal values are unranked.

Resample size	Mean Kendall $\tau$	Top method retained
40	0.892	0.79
80	0.917	0.89
120	0.932	0.92
152	0.936	0.95

Appendix

Table 8: SeqAcc ranking stability under video-level resampling of the 152 supported evaluation clips. Kendall

\tau

compares each replicate with the full-split ranking; the final column gives the fraction retaining the leading method.

Quantity	Observed value
Source exact match within $\mathcal{D}$	$0.851$
Source CharAcc within $\mathcal{D}$	$0.966$
Source decoded-string TTS	$0.760$
Source detectability $\|\mathcal{D}\|/T$	Median $1.00$ ; 25th percentile $0.97$ ; mean $0.90$
Clips with $\mathcal{D}=\emptyset$	$5$ of $157$ evaluation clips
Pipeline-rendered $\tilde{V}$	Exact match $0.585$ ; CharAcc $0.790$ ; TTS $0.768$

Appendix

Table 9: Empirical OCR reference levels. Source correctness is evaluated against

s_{\mathrm{src}}

on detectable frames; pipeline-rendered edits are evaluated against

s_{\mathrm{tgt}}

Axis	Ordinal $\alpha$	Automatic metric	Spearman $\rho$
Text correctness	0.87	SeqAcc $\uparrow$	$+0.71$
Temporal quality	0.80	Warp ${}_{c}\downarrow$	$-0.40$
Edit locality	0.37	DreamSim-loc $\downarrow$	$-0.53$

Appendix

Table 10: Human agreement and metric alignment for 70 outputs rated by three non-author raters. Human scores increase with quality; the signs of

\rho

follow the automatic metrics’ directions. All three correlations have

p<0.001

Method	On front	SeqAcc $\uparrow$	Warp ${}_{c}\downarrow$	DreamSim-loc $\downarrow$
FLUX-Text	Yes	0.528	13.010	0.012
TextCtrl	Yes	0.475	2.088	0.004
RS-STE	Yes	0.354	1.815	0.007
ViTeX-Edit-14B	Yes	0.341	1.530	0.024
Wan2.1-VACE-14B	Yes	0.000	1.561	0.007
AnyText2	No	0.280	3.952	0.043

Appendix

Table 11: Raw-output primary metrics and Pareto membership. The comparison uses mean scores; Source, Composite, and temporally adapted VideoPainter outputs are excluded.

Method	BG-Warp $\downarrow$	DINOv2-drift $\downarrow$	ArcFace-id $\uparrow$
Source	1.67	0.0036	1.00
TextCtrl	1.70	0.0038	0.99
ViTeX-Edit-14B	1.74	0.0037	0.89
RS-STE	1.76	0.0038	0.97
Wan2.1-VACE-14B	1.88	0.0039	0.96
AnyText2	2.04	0.0046	0.82

Appendix

Table 12: Supplementary preservation diagnostics. BG-Warp and DINOv2-drift use the 157-clip evaluation split; ArcFace-id uses the 131-clip source-face subset. Source is an unranked sanity anchor. † VideoPainter’s interpolated temporal scores are also unranked.

Explore similar work

Sep 29, 2026cs.CV

Beyond Legibility: Benchmarking Visual Text Rendering and In-Place Editing in Unified Video Generation

A video can exhibit convincing motion and photorealism yet fail immediately when visual text collapses. Unlike generic scene content, visual text is unforgiving in video generation: minor stroke corruption, temporal instability, or editing errors instantly break legibility and realism. Existing benchmarks overlook this challenge by treating text as incidental or using static OCR metrics that ignore temporal dynamics. We introduce VidScribe, a unified diagnostic benchmark spanning four generation regimes: writing from language (T2V), transferring text identity from a reference (R2V), sustaining text under dynamics (I2V), and localized text editing (V2V). VidScribe contains 803 human-verified samples across a 12-axis conditionally orthogonal factor space covering Intrinsic Text Properties, Physical Imaging Conditions, and Temporal Behavior. For reliable evaluation, we build a track-grounded, gated suite with 11 shared metrics and 2 task-specific probes under strict measurability conditions. Benchmarking 11 commercial and open-source systems shows that video text capability is non-monolithic, with content recognition decoupled from stroke-level glyph correctness. Performance is highly task-asymmetric: I2V sustains text most reliably, whereas V2V editing is the primary bottleneck. Counter-intuitively, degradation concentrates on a small subset of text-centric structural and temporal factors rather than adverse imaging conditions. Further probes show that visual references improve glyph and typographic fidelity rather than content accuracy, while localized editing fails to isolate target text without corrupting undeclared source text. Beyond evaluation, VidScribe also provides an actionable training signal, where benchmark-aligned preference optimization measurably improves visual text generation. https://huggingface.co/datasets/Vicky0720/VidScribe.

Ziying Zhang, Litao Li, Junchao Liao +4

Alibaba Group · Shanghai Jiao Tong University · Fudan University

Text-To-Video Models

Ask PDF

Sep 28, 2026cs.CV

Enhanced Video Text Editing with Trajectory-Aligned Glyph Rendering

Video text editing aims to replace or add text in a video while keeping the rest of the video unchanged, which requires the edited text to be correct in every frame and to move coherently with the scene. Despite the remarkable progress of video diffusion models, they struggle to reproduce exact stroke structures and often produce garbled or wrong characters, especially for characters with complex strokes. To address this, we propose a trajectory-aligned glyph rendering reference that provides explicit per-frame glyph guidance following the position and perspective of the text, and a depth-normalized recognizer feature supervision that supervises the generated text on multi-depth features of a frozen text recognizer with per-depth normalized errors, targeting stroke errors overlooked by the diffusion loss. We further build VTEdit, a benchmark of 288 real-scene clips with 440 annotated text trajectories covering text replacement and text addition, which will be publicly released to facilitate future research. Experiments on VTEdit show that our method outperforms image text editing methods, video editing methods, and commercial models in text accuracy and background preservation, achieving a sentence accuracy of 0.9408, and receives the highest preference in a user study.

Shulian Zhang, Xiangyu Shu, Wenbo Li +2

South China University of Technology · The Chinese University of Hong Kong · Huawei

Text-To-Video Models Diffusion Editing Typography Generation

Ask PDF

May 20, 2026cs.CV

TextSculptor: Training and Benchmarking Scene Text Editing

Recent advances in Multimodal Large Language Models (MLLMs) and diffusion-based generative models have substantially improved prompt-driven image editing. However, scene text editing remains challenging, as it requires models to precisely modify textual content while preserving visual realism and non-target regions. Current open-source models still lag behind proprietary systems, largely due to the scarcity of high-quality training data and the lack of standardized benchmarks tailored to text editing. To address these challenges, we present TextSculptor, a comprehensive framework for data construction and evaluation of scene text editing. We first develop an automated data construction pipeline that combines text-aware image synthesis with programmatic text rendering and compositing. Based on this pipeline, we build TextSculpt-Data, a large-scale dataset containing 3.2M training samples, including 1.2M OCR-verified text-to-image samples and 2M paired text editing samples with naturally aligned source-target images and strong background consistency. We further introduce TextSculpt-Bench, a benchmark covering four fundamental text editing tasks: text addition, text replacement, text removal, and hybrid editing. To support reliable evaluation, we design a tailored protocol that measures text accuracy, visual quality, and background preservation through OCR-based text alignment, multimodal judgment, and background-region similarity. Extensive experiments show that TextSculptor improves open-source text editing performance and narrows the gap to proprietary models. The data and benchmark are available at https://github.com/linyiheng123/TextSculptor.

Yiheng Lin, Siyu Jiao, Xiaohan Lan +12

Beijing Jiaotong University · Bytedance

Text-To-Image Multimodal Benchmarks 3D Editing Multimodal Knowledge Editing Large Image Datasets

Ask PDF