Real In, Real Out: What If We Only Use Real Data for Scene Text Editing?

Xingsong Ye; Yongkun Du; Jiaxin Zhang; Chong Sun; Zhineng Chen

cs.CVOct 20, 2026

Real In, Real Out: What If We Only Use Real Data for Scene Text Editing?

Authors: Xingsong Ye, Yongkun Du, Jiaxin Zhang, Chong Sun, Zhineng Chen

Organizations: Institute of Trustworthy Embodied AI, Fudan University · Shanghai Key Laboratory of Multimodal Embodied AI · WeChat Vision, Tencent Inc.

Abstract

Aiming to modify textual content while preserving the original style, Scene Text Editing (STE) has significant practical value in many applications. STE typically requires paired images before and after editing for training. However, recent approaches rely heavily on synthetic paired data and complex disentanglement strategies. They struggle to faithfully capture the diversity and complexity of natural scenes, often producing artifacts or hallucinations. In this work, we challenge the common practice and investigate whether STE can be driven by turning each real sample into its own paired supervision. Concretely, we explore multiple ways to disrupt the original image (e.g., Crop, Shuffle, Mask) to construct self-paired training data, and build a strong baseline (TextRIRO}) upon a conditional diffusion architecture. Our experiments show that combining character-column shuffling with image cropping (Shuffle&Crop) effectively destroys text semantics while preserving key style cues, enabling the model to learn editing behaviors directly from the real text image. With this concise yet real-oriented design, TextRIRO achieves promising performance in both accuracy and visual realism across real-world benchmarks, substantially mitigating the synthetic-to-real domain gap. Furthermore, we leverage TextRIRO to build TextRIRO-3M, a large-scale realistic dataset for long-text recognition by concatenating edited and original images. Experimental results show that training recognition models with TextRIRO-3M significantly improves recognition accuracy, demonstrating that TextRIRO provides highly useful training resources for downstream tasks. Overall, TextRIRO lifts STE to realistic, reliable, and widely applicable new levels. Code is available at https://github.com/YesianRohn/TextRIRO.

Figures & tables

Figure 1: A data-centric view of Scene Text Editing. The comparison of the previous methods and TextRIRO between the data utilization paradigm (left) and the editing results (right). In (e), we present the artifacts and hallucinations of the wrong editing results in (a) and (b).

Figure 2: TextRIRO pipeline. (a) At training time, a single real image $I_{\text{tgt}}$ is pushed through a content-destructive, style-preserving operator to produce a pseudo-source $I_{\text{src}}$ . A ViT-based style encoder emits multi-scale features that are injected into the denoising U-Net through ControlNet-style zero-convolutions, while a text encoder pre-trained on real scene text with an autoregressive STR objective supplies the target-text conditioning. (b) At inference time, the same U-Net follows the standard SD denoising schedule. (c) Our text encoder is a lightweight character tokenizer trained jointly with an STR decoder via next-token prediction. (d) The edited image is validated by an independent STR model and, if accepted, concatenated with its original to build a style-consistent long-text sample for STR. : frozen, : trainable.

Figure 3: Qualitative comparison on ScenePair. “Target Reference” is the real paired ground truth. “Rendering Tool” composites the target string on the segmented foreground/background of the source and serves as an accuracy-only reference. TextRIRO is the only method that consistently reproduces the stroke thickness, perspective and illumination of the reference without leaving a synthetic residue.

Method	ACC (%) $\uparrow$	NED $\uparrow$	FID $\downarrow$
SRNet 2019	17.84	0.478	49.22
MOSTEL 2023	37.69	0.557	49.19
AnyText 2024b	51.12	0.734	51.79
TextCtrl 2024	84.67	0.936	43.78
TextSSR 2025	79.69	0.924	45.37
RS-STE 2025	91.80	0.981	49.59

Table 1: ScenePair results. Boldface indicates the best results for each metric, while underlines highlight the second best results.

Method	IIIT 2012		SVT 2011		IC13 2013		IC15 2015		SVTP 2013		CUTE 2014		Avg
Method	E	V	E	V	E	V	E	V	E	V	E	V	E	V
MOSTEL 2023	16.85	12.85	10.82	3.58	10.04	7.54	11.49	6.15	7.91	3.98	7.89	5.02	10.83	6.52
TextCtrl 2024	64.42	11.31	76.97	10.17	74.45	12.99	73.16	15.65	70.85	10.44	64.16	11.30	70.67	11.98
RS-STE 2025	58.80	13.21	67.08	10.73	63.71	20.11	61.29	9.10	62.17	7.59	48.39	9.62	60.24	11.73
TextRIRO (ours)	82.74	50.47	84.39	46.14	86.23	53.49	77.91	45.28	79.69	42.31	64.87	48.12	79.31	47.64

Table 2: RIROCommon results. Comparison of TextRIRO with three other representative STE models. “E (Equal)” denotes text editing with equal-length substitutions, while “V (Variable)” corresponds to editing without text length limitations.

Method	IIIT 2012	SVT 2011	IC13 2013	Regular	IC15 2015	SVTP 2013	CUTE 2014	Irregular	Avg
MOSTEL 2023	51.87	59.51	64.99	58.79	44.62	44.34	34.72	41.22	50.01
AnyText 2024b	73.27	78.83	82.61	78.24	66.08	70.46	48.27	61.60	69.92
TextCtrl 2024	79.40	84.39	89.61	84.47	72.06	77.36	53.12	67.51	75.99
RS-STE 2025	75.43	85.32	87.98	82.91	74.43	78.45	59.72	70.87	76.89
TextRIRO (ours)	90.53	89.18	92.65	90.79	81.00	84.03	82.64	82.56	86.67
Real (upper bound)	99.30	97.84	98.60	98.58	90.50	96.12	98.26	94.96	96.77

Table 3: Train-on-edits STR proxy. “Regular” indicates the average results on the IIIT, SVT, and IC13 datasets, while “Irregular” denotes the average results on the IC15, SVTP, and CUTE datasets.

Panel of Figure 4 — Figure 4: Ablations on ScenePair. Left: visualizations of each self-pairing operator. Right: full factorial over text encoder pre-training, input shape, VAE variant and paired-data operator. Shuffle&Crop strictly dominates every other operator when joined with STR-pretrained text conditioning and a rectangular input. “None” collapses to an identity mapping (no perturbation operation).

Figure 5: Qualitative ablations. The showing setting is consistent with that in Fig. 3 .

Operator	Straight (IC13)	Curved (CUTE)	Multi-oriented (SVTP)
Shuffle only	79.00	54.83	66.67
Crop only	77.83	60.93	69.76
Shuffle&Crop	80.16	58.42	68.84

Table 4: Layout-wise attribution of different operators. Values are equal-length editing accuracy (%).

Backbone	ACC (%) $\uparrow$	NED $\uparrow$	FID $\downarrow$
OminiControl-based 2025	52.89	0.803	39.65
ControlNet-based 2023	85.23	0.964	41.53

Table 5: Backbone choice comparison on ScenePair [ 43 ] .

Method	Official	SVTRv2	PPOCR-v6-rec
MOSTEL 2023	37.69	33.91	37.42
TextCtrl 2024	84.67	85.08	88.67
TextSSR 2025	79.69	80.23	82.34
RS-STE 2025	91.80	90.39	91.80
TextRIRO	85.23	85.86	86.25
Real (upper bound)	88.44	88.67	89.14

Table 6: ScenePair word accuracy (%) with three independent recognizers.

Training Data	Volume	Common Benchmarks							Union14M-Benchmark
Training Data	Volume	IIIT	SVT	IC13	IC15	SVTP	CUTE	Avg	CUR	MLO	ART	CTL	SAL	MLW	GEN	Avg
U14M-Filter 2025 (UF)	3.22M	99.30	97.84	98.60	90.50	96.12	98.26	96.77	89.49	93.50	77.11	83.18	86.99	86.89	85.48	86.09
MJ 2014 + UF	1M + 3.22M	99.17	97.53	99.18	91.44	95.81	98.26	96.89	90.35	94.01	79.11	85.62	87.3	87.26	85.84	87.07
ST 2016 + UF	1M + 3.22M	99.33	97.53	98.95	90.94	96.74	98.26	96.96	90.44	94.08	78.00	86.14	87.81	87.01	85.77	87.04
TextSSR 2025 + UF	1M + 3.22M	99.07	97.53	98.83	91.28	95.50	98.61	96.80	91.06	94.52	81.22	86.26	87.56	88.35	85.95	87.85
MOSTEL 2023 + UF	1M + 3.22M	99.17	97.53	98.37	91.00	96.90	96.87	96.64	90.89	93.64	79.33	83.83	87.43	87.50	85.74	86.91

Table 7: Data quality. Performance comparison of the STR model (MAERec [ 16 ] ) with different types of synthetic data mixed with real data.

Model	Training Data	Volume	Common Benchmarks							Union14M-Benchmark
Model	Training Data	Volume	IIIT	SVT	IC13	IC15	SVTP	CUTE	Avg	CUR	MLO	ART	CTL	SAL	MLW	GEN	Avg
MAERec 2023	UF	3.22M	99.30	97.84	98.60	90.50	96.12	98.26	96.77	89.49	93.50	77.11	83.18	86.99	86.89	85.48	86.09
	TextRIRO + UF	1M + 3.22M	99.37	97.84	98.72	91.33	96.59	98.96	97.13	91.18	94.67	81.00	86.65	87.49	89.08	85.80	87.98
	TextRIRO + UF	1M + 3.22M	+0.07	+0.00	+0.12	+0.83	+0.47	+0.70	+0.36	+1.69	+1.17	+3.89	+3.47	+0.50	+2.19	+0.32	+1.89
	TextRIRO + UF	3M + 3.22M	99.20	97.84	98.72	91.50	95.66	98.96	96.98	91.92	94.38	80.89	84.85	87.74	90.17	86.07	88.00
	TextRIRO + UF	3M + 3.22M	-0.10	+0.00	+0.12	+1.00	-0.46	+0.70	+0.21	+2.43	+0.88	+3.78	+1.67	+0.75	+3.28	+0.59	+1.91

Table 8: Extension of the experiments in Tab. 7 , showing the performance of TextRIRO synthetic data on various STR models and data scales.

Panel of Figure 6 — Figure 6: An interesting comic for introducing our idea. Top: Two previous STE training paradigms, both requiring paired synthetic data. Bottom: Our method leverages self-paired real data without any synthetic pairs.

$p_{\text{shuffle}}$	ACC (%) $\uparrow$	NED $\uparrow$	FID $\downarrow$
0.5	80.78	0.943	42.76
0.7	82.42	0.960	43.59
1.0	81.72	0.947	44.01

Table 9: Ablation study of

p_{\text{shuffle}}

on ScenePair [ 43 ] .

Metrics	VAE Version	IIIT 2012	SVT 2011	IC13 2013	IC15 2015	SVTP 2013	CUTE 2014	Avg
PSNR $\uparrow$	SD 1.5	25.90	30.79	28.41	29.87	28.58	26.43	28.33
	SD 2.1	27.63	33.02	30.23	31.99	30.68	28.30	30.31
	SD 3.5	29.26	32.34	30.32	32.18	31.64	28.87	30.77
FID $\downarrow$	SD 1.5	1.36	0.50	0.50	1.22	1.83	0.50	0.98
	SD 2.1	0.75	0.21	0.24	0.71	1.03	0.16	0.52
	SD 3.5	1.28	0.39	0.41	1.14	1.72	0.34	0.88

Table 10: Performance of different VAE variants on the quality of reconstructed scene text images.

$\omega$	ACC (%) $\uparrow$	NED $\uparrow$	FID $\downarrow$
1.0	59.45	0.817	60.34
1.5	78.98	0.934	35.37
1.8	82.27	0.955	37.54
2.0	83.83	0.961	39.69
2.2	85.23	0.964	41.53
2.5	86.41	0.968	43.82

Table 11: Ablation study of CFG scale

\omega

on ScenePair [ 43 ] .

Method	LPIPS $\downarrow$	SSIM $\uparrow$	PSNR $\uparrow$	FID $\downarrow$
MOSTEL 2023	0.2947	0.3311	14.05	49.19
TextCtrl 2024	0.3219	0.2610	11.68	43.78
TextSSR 2025	0.2622	0.3889	14.13	45.37
RS-STE 2025	0.2458	0.4347	15.66	49.59
TextRIRO	0.2394	0.3944	14.44	41.53

Table 12: Per-image fidelity against the ScenePair reference and distributional fidelity. Lower LPIPS/FID and higher SSIM/PSNR are better.

Training data	Odd length	Even length	Overall
Real STR data only	86.19	86.78	86.50
+ 1M symmetric (even-only)	88.12	90.27	89.25
+ 1M asymmetric-augmented	88.67	89.03	88.86

Table 13: Odd- vs. even-length word accuracy (%) under different data recipes.

Training Data	Volume	Common Benchmarks							Union14M-Benchmark
Training Data	Volume	IIIT	SVT	IC13	IC15	SVTP	CUTE	Avg	CUR	MLO	ART	CTL	SAL	MLW	GEN	Avg
UF (real only)	3.22M	99.30	97.84	98.60	90.50	96.12	98.26	96.77	89.49	93.50	77.11	83.18	86.99	86.89	85.48	86.09
TextSSR + UF	1M + 3.22M	99.07	97.53	98.83	91.28	95.50	98.61	96.80	91.06	94.52	81.22	86.26	87.56	88.35	85.95	87.85
TextRIRO + UF	1M + 3.22M	99.37	97.84	98.72	91.33	96.59	98.96	97.13	91.18	94.67	81.00	86.65	87.49	89.08	85.80	87.98
TextRIRO + TextSSR + UF	1M + 1M + 3.22M	99.30	97.99	98.60	91.72	96.28	98.96	97.14	91.59	94.89	81.44	87.93	88.19	88.83	86.22	88.44
TextRIRO + ST + UF	1M + 1M + 3.22M	99.37	97.84	98.60	91.61	97.36	99.65	97.40	92.58	95.62	81.33	86.26	87.93	89.81	86.22	88.54

Table 14: Complementarity with synthetic sources. Recognition accuracy (%) of MAERec trained with different data mixtures.

Setting	ACC (%) $\uparrow$	NED $\uparrow$	FID $\downarrow$
Real only (self-paired)	82.42	0.960	43.59
+ Synthetic data	85.08	0.965	49.34

Table 15: Adding paired synthetic data to the self-paired editor under the ablation configuration.

Figure 7: t-SNE [ 23 ] visualization of text features by STR pre-trained text encoder.

Figure 8: Visualization results randomly sampled from the TextRIRO-3M dataset, which doubles the text length compared to the original scene text images.

Figure 9: Length distributions of real data (U14M-Filter) and the original symmetric TextRIRO-3M construction. The symmetric recipe doubles the source length and hence produces even-length samples, and the asymmetric-augmented recipe in Tab. 13 removes this construction artifact.

Figure 10: Comparison of STR model (MAERec [ 16 ] ) prediction outcomes trained on different datasets.

Figure 11: Bad Cases. We show several disappointing editing results produced by TextRIRO.

Diffusion-Based Image Editing

Diffusion-based image editing changes image content, style, or attributes while aiming to preserve the parts users want to keep.

See all 227 papers

Towards Training-Free Scene Text Editing
Mar 25, 2026Yubo Li, Xugong Qin, Peng Zhang +3Training-Free Image Editing Image Editing
Institute of Information Engineering, Chinese Academy of Sciences · State Key Laboratory of Cyberspace Security Defense · School of Cyber Security, University of Chinese Academy of Sciences +1
TextSculptor: Training and Benchmarking Scene Text Editing
May 20, 2026Yiheng Lin, Siyu Jiao, Xiaohan Lan +12Synthetic Data Augmentation Text-Guided Image Editing
Beijing Jiaotong University · Bytedance
STEPS: Scene Text Editing with Preserved Style Using Diffusion and Contrastive Style Encoding
Sep 29, 2026Nicolas Thiebaut, Nameer Hirschkind, Xiao Yu +1Text-Guided Image Editing Diffusion-Based Image Editing
Roblox San Mateo, CA, USA

Mar 25, 2026J/K move · Enter open · S save

Towards Training-Free Scene Text Editing

Yubo Li, Xugong Qin, Peng Zhang, Hailun Lin+2

Institute of Information Engineering, Chinese Academy of Sciences · State Key Laboratory of Cyberspace Security Defense · School of Cyber Security, University of Chinese Academy of Sciences+1

Training-Free Image Editing Image Editing Diffusion-Based Image Editing

Open paper AskPDF arXiv

Scene text editing seeks to modify textual content in natural images while maintaining visual realism and semantic consistency. Existing methods often require task-specific training or paired data, limiting their scalability and adaptability. In this paper, we propose TextFlow, a training-free scene text editing framework that integrates the strengths of Attention Boost (AttnBoost) and Flow Manifold Steering (FMS) to enable flexible, high-fidelity text manipulation without additional training. Specifically, FMS preserves the structural and style consistency by modeling the visual flow of characters and background regions, while AttnBoost enhances the rendering of textual content through attention-based guidance. By jointly leveraging these complementary modules, our approach performs end-to-end text editing through semantic alignment and spatial refinement in a plug-and-play manner. Extensive experiments demonstrate that our framework achieves visual quality and text accuracy comparable to or superior to those of training-based counterparts, generalizing well across diverse scenes and languages. This study advances scene text editing toward a more efficient, generalizable, and training-free paradigm. Code is available at https://github.com/lyb18758/TextFlow

Abstract

Diffusion-Based Image Editing

Explore similar work

Towards Training-Free Scene Text Editing

TextSculptor: Training and Benchmarking Scene Text Editing

STEPS: Scene Text Editing with Preserved Style Using Diffusion and Contrastive Style Encoding