Real In, Real Out: What If We Only Use Real Data for Scene Text Editing?
Authors: Xingsong Ye, Yongkun Du, Jiaxin Zhang, Chong Sun, Zhineng Chen
Organizations: Institute of Trustworthy Embodied AI, Fudan University · Shanghai Key Laboratory of Multimodal Embodied AI · WeChat Vision, Tencent Inc.
Aiming to modify textual content while preserving the original style, Scene Text Editing (STE) has significant practical value in many applications. STE typically requires paired images before and after editing for training. However, recent approaches rely heavily on synthetic paired data and complex disentanglement strategies. They struggle to faithfully capture the diversity and complexity of natural scenes, often producing artifacts or hallucinations. In this work, we challenge the common practice and investigate whether STE can be driven by turning each real sample into its own paired supervision. Concretely, we explore multiple ways to disrupt the original image (e.g., Crop, Shuffle, Mask) to construct self-paired training data, and build a strong baseline (TextRIRO}) upon a conditional diffusion architecture. Our experiments show that combining character-column shuffling with image cropping (Shuffle&Crop) effectively destroys text semantics while preserving key style cues, enabling the model to learn editing behaviors directly from the real text image. With this concise yet real-oriented design, TextRIRO achieves promising performance in both accuracy and visual realism across real-world benchmarks, substantially mitigating the synthetic-to-real domain gap. Furthermore, we leverage TextRIRO to build TextRIRO-3M, a large-scale realistic dataset for long-text recognition by concatenating edited and original images. Experimental results show that training recognition models with TextRIRO-3M significantly improves recognition accuracy, demonstrating that TextRIRO provides highly useful training resources for downstream tasks. Overall, TextRIRO lifts STE to realistic, reliable, and widely applicable new levels. Code is available at https://github.com/YesianRohn/TextRIRO.
Figures & tables
Figure 1: A data-centric view of Scene Text Editing. The comparison of the previous methods and TextRIRO between the data utilization paradigm (left) and the editing results (right). In (e), we present the artifacts and hallucinations of the wrong editing results in (a) and (b).
Figure 2: TextRIRO pipeline. (a) At training time, a single real image Itgt is pushed through a content-destructive, style-preserving operator to produce a pseudo-source Isrc . A ViT-based style encoder emits multi-scale features that are injected into the denoising U-Net through ControlNet-style zero-convolutions, while a text encoder pre-trained on real scene text with an autoregressive STR objective supplies the target-text conditioning. (b) At inference time, the same U-Net follows the standard SD denoising schedule. (c) Our text encoder is a lightweight character tokenizer trained jointly with an STR decoder via next-token prediction. (d) The edited image is validated by an independent STR model and, if accepted, concatenated with its original to build a style-consistent long-text sample for STR. : frozen, : trainable.
Figure 3: Qualitative comparison on ScenePair. “Target Reference” is the real paired ground truth. “Rendering Tool” composites the target string on the segmented foreground/background of the source and serves as an accuracy-only reference. TextRIRO is the only method that consistently reproduces the stroke thickness, perspective and illumination of the reference without leaving a synthetic residue.
Method
ACC (%) ↑
NED ↑
FID ↓
SRNet 2019
17.84
0.478
49.22
MOSTEL 2023
37.69
0.557
49.19
AnyText 2024b
51.12
0.734
51.79
TextCtrl 2024
84.67
0.936
43.78
TextSSR 2025
79.69
0.924
45.37
RS-STE 2025
91.80
0.981
49.59
Table 1: ScenePair results. Boldface indicates the best results for each metric, while underlines highlight the second best results.
Method
IIIT 2012
SVT 2011
IC13 2013
IC15 2015
SVTP 2013
CUTE 2014
Avg
E
V
E
V
E
V
E
V
E
V
E
V
E
V
MOSTEL 2023
16.85
12.85
10.82
3.58
10.04
7.54
11.49
6.15
7.91
3.98
7.89
5.02
10.83
6.52
TextCtrl 2024
64.42
11.31
76.97
10.17
74.45
12.99
73.16
15.65
70.85
10.44
64.16
11.30
70.67
11.98
RS-STE 2025
58.80
13.21
67.08
10.73
63.71
20.11
61.29
9.10
62.17
7.59
48.39
9.62
60.24
11.73
TextRIRO (ours)
82.74
50.47
84.39
46.14
86.23
53.49
77.91
45.28
79.69
42.31
64.87
48.12
79.31
47.64
Table 2: RIROCommon results. Comparison of TextRIRO with three other representative STE models. “E (Equal)” denotes text editing with equal-length substitutions, while “V (Variable)” corresponds to editing without text length limitations.
Method
IIIT 2012
SVT 2011
IC13 2013
Regular
IC15 2015
SVTP 2013
CUTE 2014
Irregular
Avg
MOSTEL 2023
51.87
59.51
64.99
58.79
44.62
44.34
34.72
41.22
50.01
AnyText 2024b
73.27
78.83
82.61
78.24
66.08
70.46
48.27
61.60
69.92
TextCtrl 2024
79.40
84.39
89.61
84.47
72.06
77.36
53.12
67.51
75.99
RS-STE 2025
75.43
85.32
87.98
82.91
74.43
78.45
59.72
70.87
76.89
TextRIRO (ours)
90.53
89.18
92.65
90.79
81.00
84.03
82.64
82.56
86.67
Real (upper bound)
99.30
97.84
98.60
98.58
90.50
96.12
98.26
94.96
96.77
Table 3: Train-on-edits STR proxy. “Regular” indicates the average results on the IIIT, SVT, and IC13 datasets, while “Irregular” denotes the average results on the IC15, SVTP, and CUTE datasets.
Figure 4: Ablations on ScenePair. Left: visualizations of each self-pairing operator. Right: full factorial over text encoder pre-training, input shape, VAE variant and paired-data operator. Shuffle&Crop strictly dominates every other operator when joined with STR-pretrained text conditioning and a rectangular input. “None” collapses to an identity mapping (no perturbation operation).
Figure 5: Qualitative ablations. The showing setting is consistent with that in Fig. 3 .
Operator
Straight (IC13)
Curved (CUTE)
Multi-oriented (SVTP)
Shuffle only
79.00
54.83
66.67
Crop only
77.83
60.93
69.76
Shuffle&Crop
80.16
58.42
68.84
Table 4: Layout-wise attribution of different operators. Values are equal-length editing accuracy (%).
Table 6: ScenePair word accuracy (%) with three independent recognizers.
Training Data
Volume
Common Benchmarks
Union14M-Benchmark
IIIT
SVT
IC13
IC15
SVTP
CUTE
Avg
CUR
MLO
ART
CTL
SAL
MLW
GEN
Avg
U14M-Filter 2025 (UF)
3.22M
99.30
97.84
98.60
90.50
96.12
98.26
96.77
89.49
93.50
77.11
83.18
86.99
86.89
85.48
86.09
MJ 2014 + UF
1M + 3.22M
99.17
97.53
99.18
91.44
95.81
98.26
96.89
90.35
94.01
79.11
85.62
87.3
87.26
85.84
87.07
ST 2016 + UF
1M + 3.22M
99.33
97.53
98.95
90.94
96.74
98.26
96.96
90.44
94.08
78.00
86.14
87.81
87.01
85.77
87.04
TextSSR 2025 + UF
1M + 3.22M
99.07
97.53
98.83
91.28
95.50
98.61
96.80
91.06
94.52
81.22
86.26
87.56
88.35
85.95
87.85
MOSTEL 2023 + UF
1M + 3.22M
99.17
97.53
98.37
91.00
96.90
96.87
96.64
90.89
93.64
79.33
83.83
87.43
87.50
85.74
86.91
Table 7: Data quality. Performance comparison of the STR model (MAERec [ 16 ] ) with different types of synthetic data mixed with real data.
Model
Training Data
Volume
Common Benchmarks
Union14M-Benchmark
IIIT
SVT
IC13
IC15
SVTP
CUTE
Avg
CUR
MLO
ART
CTL
SAL
MLW
GEN
Avg
MAERec 2023
UF
3.22M
99.30
97.84
98.60
90.50
96.12
98.26
96.77
89.49
93.50
77.11
83.18
86.99
86.89
85.48
86.09
TextRIRO + UF
1M + 3.22M
99.37
97.84
98.72
91.33
96.59
98.96
97.13
91.18
94.67
81.00
86.65
87.49
89.08
85.80
87.98
+0.07
+0.00
+0.12
+0.83
+0.47
+0.70
+0.36
+1.69
+1.17
+3.89
+3.47
+0.50
+2.19
+0.32
+1.89
TextRIRO + UF
3M + 3.22M
99.20
97.84
98.72
91.50
95.66
98.96
96.98
91.92
94.38
80.89
84.85
87.74
90.17
86.07
88.00
-0.10
+0.00
+0.12
+1.00
-0.46
+0.70
+0.21
+2.43
+0.88
+3.78
+1.67
+0.75
+3.28
+0.59
+1.91
Table 8: Extension of the experiments in Tab. 7 , showing the performance of TextRIRO synthetic data on various STR models and data scales.
Figure 6: An interesting comic for introducing our idea. Top: Two previous STE training paradigms, both requiring paired synthetic data. Bottom: Our method leverages self-paired real data without any synthetic pairs.
pshuffle
ACC (%) ↑
NED ↑
FID ↓
0.5
80.78
0.943
42.76
0.7
82.42
0.960
43.59
1.0
81.72
0.947
44.01
Table 9: Ablation study of pshuffle on ScenePair [ 43 ] .
Metrics
VAE Version
IIIT 2012
SVT 2011
IC13 2013
IC15 2015
SVTP 2013
CUTE 2014
Avg
PSNR ↑
SD 1.5
25.90
30.79
28.41
29.87
28.58
26.43
28.33
SD 2.1
27.63
33.02
30.23
31.99
30.68
28.30
30.31
SD 3.5
29.26
32.34
30.32
32.18
31.64
28.87
30.77
FID ↓
SD 1.5
1.36
0.50
0.50
1.22
1.83
0.50
0.98
SD 2.1
0.75
0.21
0.24
0.71
1.03
0.16
0.52
SD 3.5
1.28
0.39
0.41
1.14
1.72
0.34
0.88
Table 10: Performance of different VAE variants on the quality of reconstructed scene text images.
ω
ACC (%) ↑
NED ↑
FID ↓
1.0
59.45
0.817
60.34
1.5
78.98
0.934
35.37
1.8
82.27
0.955
37.54
2.0
83.83
0.961
39.69
2.2
85.23
0.964
41.53
2.5
86.41
0.968
43.82
Table 11: Ablation study of CFG scale ω on ScenePair [ 43 ] .
Method
LPIPS ↓
SSIM ↑
PSNR ↑
FID ↓
MOSTEL 2023
0.2947
0.3311
14.05
49.19
TextCtrl 2024
0.3219
0.2610
11.68
43.78
TextSSR 2025
0.2622
0.3889
14.13
45.37
RS-STE 2025
0.2458
0.4347
15.66
49.59
TextRIRO
0.2394
0.3944
14.44
41.53
Table 12: Per-image fidelity against the ScenePair reference and distributional fidelity. Lower LPIPS/FID and higher SSIM/PSNR are better.
Training data
Odd length
Even length
Overall
Real STR data only
86.19
86.78
86.50
+ 1M symmetric (even-only)
88.12
90.27
89.25
+ 1M asymmetric-augmented
88.67
89.03
88.86
Table 13: Odd- vs. even-length word accuracy (%) under different data recipes.
Training Data
Volume
Common Benchmarks
Union14M-Benchmark
IIIT
SVT
IC13
IC15
SVTP
CUTE
Avg
CUR
MLO
ART
CTL
SAL
MLW
GEN
Avg
UF (real only)
3.22M
99.30
97.84
98.60
90.50
96.12
98.26
96.77
89.49
93.50
77.11
83.18
86.99
86.89
85.48
86.09
TextSSR + UF
1M + 3.22M
99.07
97.53
98.83
91.28
95.50
98.61
96.80
91.06
94.52
81.22
86.26
87.56
88.35
85.95
87.85
TextRIRO + UF
1M + 3.22M
99.37
97.84
98.72
91.33
96.59
98.96
97.13
91.18
94.67
81.00
86.65
87.49
89.08
85.80
87.98
TextRIRO + TextSSR + UF
1M + 1M + 3.22M
99.30
97.99
98.60
91.72
96.28
98.96
97.14
91.59
94.89
81.44
87.93
88.19
88.83
86.22
88.44
TextRIRO + ST + UF
1M + 1M + 3.22M
99.37
97.84
98.60
91.61
97.36
99.65
97.40
92.58
95.62
81.33
86.26
87.93
89.81
86.22
88.54
Table 14: Complementarity with synthetic sources. Recognition accuracy (%) of MAERec trained with different data mixtures.
Setting
ACC (%) ↑
NED ↑
FID ↓
Real only (self-paired)
82.42
0.960
43.59
+ Synthetic data
85.08
0.965
49.34
Table 15: Adding paired synthetic data to the self-paired editor under the ablation configuration.
Figure 7: t-SNE [ 23 ] visualization of text features by STR pre-trained text encoder.
Figure 8: Visualization results randomly sampled from the TextRIRO-3M dataset, which doubles the text length compared to the original scene text images.
Figure 9: Length distributions of real data (U14M-Filter) and the original symmetric TextRIRO-3M construction. The symmetric recipe doubles the source length and hence produces even-length samples, and the asymmetric-augmented recipe in Tab. 13 removes this construction artifact.
Figure 10: Comparison of STR model (MAERec [ 16 ] ) prediction outcomes trained on different datasets.
Figure 11: Bad Cases. We show several disappointing editing results produced by TextRIRO.
Scene text editing seeks to modify textual content in natural images while maintaining visual realism and semantic consistency. Existing methods often require task-specific training or paired data, limiting their scalability and adaptability. In this paper, we propose TextFlow, a training-free scene text editing framework that integrates the strengths of Attention Boost (AttnBoost) and Flow Manifold Steering (FMS) to enable flexible, high-fidelity text manipulation without additional training. Specifically, FMS preserves the structural and style consistency by modeling the visual flow of characters and background regions, while AttnBoost enhances the rendering of textual content through attention-based guidance. By jointly leveraging these complementary modules, our approach performs end-to-end text editing through semantic alignment and spatial refinement in a plug-and-play manner. Extensive experiments demonstrate that our framework achieves visual quality and text accuracy comparable to or superior to those of training-based counterparts, generalizing well across diverse scenes and languages. This study advances scene text editing toward a more efficient, generalizable, and training-free paradigm. Code is available at https://github.com/lyb18758/TextFlow
Yubo Li, Xugong Qin, Peng Zhang +3
Institute of Information Engineering, Chinese Academy of Sciences · State Key Laboratory of Cyberspace Security Defense · School of Cyber Security, University of Chinese Academy of Sciences +1
Recent advances in Multimodal Large Language Models (MLLMs) and diffusion-based generative models have substantially improved prompt-driven image editing. However, scene text editing remains challenging, as it requires models to precisely modify textual content while preserving visual realism and non-target regions. Current open-source models still lag behind proprietary systems, largely due to the scarcity of high-quality training data and the lack of standardized benchmarks tailored to text editing. To address these challenges, we present TextSculptor, a comprehensive framework for data construction and evaluation of scene text editing. We first develop an automated data construction pipeline that combines text-aware image synthesis with programmatic text rendering and compositing. Based on this pipeline, we build TextSculpt-Data, a large-scale dataset containing 3.2M training samples, including 1.2M OCR-verified text-to-image samples and 2M paired text editing samples with naturally aligned source-target images and strong background consistency. We further introduce TextSculpt-Bench, a benchmark covering four fundamental text editing tasks: text addition, text replacement, text removal, and hybrid editing. To support reliable evaluation, we design a tailored protocol that measures text accuracy, visual quality, and background preservation through OCR-based text alignment, multimodal judgment, and background-region similarity. Extensive experiments show that TextSculptor improves open-source text editing performance and narrows the gap to proprietary models. The data and benchmark are available at https://github.com/linyiheng123/TextSculptor.
We introduce Scene Text Editing with Preserved Style (STEPS), a novel diffusion model architecture for quality text replacement in images. Scene Text Editing (STE), also known as Visual Text Editing, consists of changing the textual content in an image while conserving the original style, e.g. font, colors, orientation, background, etc. STEPS advances the state of the art in STE through directed focus on improved style preservation. We introduce a style encoder for visual text that captures style independently of textual content, and a model architecture that combines the style encoder with multiple semantic conditions (target text characters encoding and rendered glyphs). STEPS achieves superior results to previous STE methods in style preservation, output readability, and subjective quality.