Natural language-guided, mask-free scene text editing aims to modify text in real-world images using only textual instructions, yet remains challenged by unreliable pseudo-GT supervision, over-smoothed text generation, and the absence of GT-free evaluation. Existing methods often rely on imperfect pseudo-targets with background discrepancies, while standard MSE-based velocity regression tends to suppress high-frequency details, leading to blurred text boundaries and weakened local textures. To address these issues, we propose HarmonyText, a mask-free scene text editing framework with region- and frequency-aware supervision. HarmonyText introduces Region-Decoupling Optimization (RDO), which aligns edited text regions with pseudo-targets while anchoring preserved regions to the source image, and Cascaded High-Frequency Texture Guidance (CHFTG), which applies cascaded wavelet-domain constraints to preserve sharp boundaries and fine-grained textures of textual content. We further propose Reverse-to-Refined Pair Synthesis (RePair) to construct STE25K, a high-quality 25K paired dataset, and introduce a GT-free evaluation protocol with Color Distribution Divergence and a Harmony Assessment Model. Extensive experiments show that HarmonyText outperforms existing methods in text editing accuracy, background fidelity, color consistency, and perceptual harmony.
Figures & tables
Figure 1. Harmonious scene text editing. Given only a natural-language instruction, HarmonyText edits the target text while preserving its natural integration with the surrounding scene, including local color and texture. In contrast, strong instruction-following editors may produce the correct text but introduce color shifts or texture mismatches. High-quality STE requires text accuracy together with visual harmony between the edited text and the scene. Side-by-side scene text editing examples comparing HarmonyText with strong instruction-following editors. HarmonyText preserves the original scene appearance while changing the requested text, with more consistent local color, texture, and lighting.
Figure 2. Overview of the proposed HarmonyText framework. (Left) RePair constructs high-quality paired data from unpaired real scene text images. A base editing model first produces flawed edits with color shifts and incoherence. The editing direction is then reversed, treating flawed outputs as corrupted inputs and original real images as clean supervision targets, training the model to reconstruct artifact-free scene text. The resulting pairs are further curated via text verification, region matching, and quality filtering to form STE25K. (Right) Region-Decoupled Optimization (RDO) assigns distinct objectives to edited and non-edited regions: non-edited regions are anchored to the source image for background preservation, while edited regions are supervised against the pseudo-target for accurate text generation. Cascaded High-Frequency Texture Guidance (CHFTG) further applies region-decoupled wavelet constraints across two frequency levels, preserving fine-grained textures and enforcing crisp text strokes. Overview of HarmonyText. The left side shows the RePair data synthesis pipeline from real scene-text images through flawed edits, reverse self-refinement, forward generation, verification, region matching, and filtering. The right side shows Region-Decoupling Optimization and Cascaded High-Frequency Texture Guidance for edited and preserved regions.
Category
Method
Accuracy
Baseline
Random
49.80
Commercial Models
GPT-5.4
53.88
Gemini-3.1-Pro
55.29
Open-Source VLMs
Qwen3.6-27B
48.06
Qwen3-VL-8B
35.68
Ours
83.07
Table 1. Harmony preference alignment accuracy on HarmonyText-Bench. Accuracy measures the percentage of pairwise preference judgments that agree with human annotations.
Method
Mask-free
HarmonyText-Bench (155)
Public-STR (345)
Non-Edited
Edited
Non-Edited
Edited
PSNR ↑
SSIM ↑
LPIPS ↓
ACC ↑
NED ↑
CDD ↓
Harm. ↑
PSNR ↑
SSIM ↑
LPIPS ↓
ACC ↑
NED ↑
CDD ↓
Harm. ↑
AnyText2 ( Tuo et al., 2024 )
×
27.226
0.783
0.190
0.548
0.711
0.385
-2.967
31.853
0.891
0.127
0.736
0.880
0.420
-2.970
DreamText ( Wang et al., 2025 )
×
23.164
0.646
0.224
0.181
0.360
0.356
-2.333
25.868
0.813
0.158
0.043
0.174
0.353
-2.251
BAGEL ( Deng et al., 2025 )
✓
25.390
0.782
0.148
0.406
0.668
0.269
-1.717
28.572
0.883
0.125
0.232
0.581
0.284
-1.410
Step1X-Edit-Think ( Liu et al., 2025 )
✓
23.169
0.670
0.152
0.761
0.921
0.175
-0.207
25.289
0.805
0.115
0.707
0.913
0.178
0.347
Table 2. Quantitative comparison on HarmonyText-Bench (155 self-curated images) and Public-STR (345 public-source images) using the original short instructions. Non-edited region metrics (PSNR, SSIM, LPIPS) assess background preservation, while edited region metrics (ACC, NED, CDD, Harm.) evaluate text accuracy, color consistency, and text-background coherence. Harm. denotes Harmony score. The best and second-best results within each dataset are highlighted in bold and with underlines , respectively.
Method
HarmonyText vs. Baseline
Win
Lose
AnyText2 ( Tuo et al., 2024 )
99.9
0.1
DreamText ( Wang et al., 2025 )
100.0
0.0
Qwen-Image-Edit ( Wu et al., 2025a )
96.5
2.7
LongCat-Image-Edit ( Team et al., 2025 )
93.3
6.5
Step1X-Edit-Think ( Liu et al., 2025 )
91.9
6.8
Table 3. Blinded A/B user-study results on HarmonyText-Bench (%). Each HarmonyText–baseline comparison contains 1,240 judgments; omitted percentages correspond to ties.
Variants
Non-Edited Region
Edited Region
PSNR ↑
SSIM ↑
LPIPS ↓
ACC ↑
NED ↑
CDD ↓
Harm. ↑
Reverse Direction
27.679
0.792
0.116
0.890
0.979
0.121
1.551
RePair w/o Filtering
28.329
0.803
0.116
0.890
0.978
0.067
2.606
RePair (Ours)
28.443
0.803
0.114
0.916
0.981
0.063
2.761
Table 4. Ablation study on the RePair. Reverse Direction denotes a baseline that trains on the same STE25K pairs but in the reverse direction, taking the edited images as input to reconstruct the original real images. All variants are evaluated on HarmonyText-Bench.
RDO
CHFTG
Non-Edited Region
Edited Region
PSNR ↑
SSIM ↑
LPIPS ↓
ACC ↑
NED ↑
CDD ↓
Harm. ↑
27.658
0.798
0.120
0.903
0.979
0.145
0.952
✓
28.401
0.802
0.115
0.890
0.979
0.071
2.457
✓
27.703
0.798
0.120
0.923
0.986
0.137
1.063
✓
✓
28.443
0.803
0.114
0.916
0.981
0.063
2.761
Table 5. Ablation study on Region-Decoupling Optimization (RDO) and Cascaded High-Frequency Texture Guidance (CHFTG). RDO applies distinct optimization targets to edited and non-edited regions. When RDO is disabled, CHFTG is applied to both non-edited and edited regions. All reported scores are evaluated on the HarmonyText-Bench.
Figure 3. Qualitative comparisons of scene text editing between our HarmonyText and other SOTA methods. Arrows indicate regions where the edited result deviates from the original appearance in color, texture, or coherence. Our HarmonyText achieves the best visual harmony, with superior texture preservation, color consistency, and text-background coherence. Best viewed with zoom-in. Qualitative comparison grid for several Chinese and English scene-text editing prompts. Each row shows the source image and outputs from multiple baselines and HarmonyText; arrows mark color, texture, or coherence artifacts. HarmonyText more closely preserves the source appearance while producing the requested text.
Figure 4. Representative samples from the STE25K dataset. Each pair shows the source image (left) and the edited result (right) along with the corresponding editing instruction. STE25K covers diverse real-world scenarios including both Chinese and English text across various surface materials, demonstrating the high quality and diversity of the paired data produced by our RePair pipeline. Representative STE25K source-and-edited image pairs with text-editing instructions. The examples cover Chinese and English text on varied real-world surfaces and demonstrate diverse colors, textures, layouts, and backgrounds.
Figure 5. Visual ablation on the RePair data synthesis pipeline. Reverse Dir. trains on STE25K pairs in the reverse direction (edited → real) instead of the forward direction. w/o Filtering uses forward-direction pairs without quality filtering. Arrows indicate regions where color or texture deviates from the original image. The full RePair pipeline yields the best overall consistency. Best viewed with zoom-in. Visual ablation of the RePair data synthesis pipeline comparing Reverse Direction, forward training without filtering, and the full RePair method. Arrows indicate regions with color or texture inconsistencies; the full pipeline shows the most consistent appearance.
Figure 6. Visual ablation on Region-Decoupled Frequency Learning. w/o RDO removes Region-Decoupling Optimization, applying a single optimization target to the entire image. w/o CHFTG removes Cascaded High-Frequency Texture Guidance, relying solely on spatial-domain supervision. Arrows indicate regions where color or texture deviates from the original image. The full method yields the best overall consistency. Best viewed with zoom-in. Visual ablation of Region-Decoupling Optimization and Cascaded High-Frequency Texture Guidance. The comparison shows outputs without RDO, without CHFTG, and with the full method, with arrows highlighting color and texture inconsistencies.
Figure 7. Qualitative validation of the proposed CDD and Harmony scores. For each example, we show the source image and edited results from different methods, along with the corresponding CDD and Harmony scores. A lower CDD indicates better color consistency: edited text with noticeable color drift or illumination mismatch yields higher CDD values, while naturally blended text yields lower ones. A higher Harmony score indicates better perceptual integration: it consistently assigns higher ratings to edits that exhibit better texture adherence and lighting coherence with the surrounding scene, aligning well with human perception. Examples used to validate CDD and Harmony scores. For each source image, edited results from different methods are shown with their CDD and Harmony values, illustrating that lower CDD corresponds to better color consistency and higher Harmony scores correspond to more natural texture and lighting integration.
Institute of Information Engineering, Chinese Academy of Sciences · State Key Laboratory of Cyberspace Security Defense · School of Cyber Security, University of Chinese Academy of Sciences +1