Organizations: State Key Lab of CAD&CG, Zhejiang University (zju.edu.cn) · Alibaba Group (alibaba-inc.com) · State Key Laboratory of General Artificial Intelligence, Peking University (pku.edu.cn)
Generated images often have coherent layouts and visual content, yet local errors in small text and complex characters still limit their readability and practical use. Scene-text refinement requires accuracy, preservation, and efficiency. Existing methods based on variational autoencoders (VAEs) can lose fine strokes during reconstruction, while full-image editing can shift background colors. In this work, we introduce PixText, a model that formulates local text refinement as grouped image generation in pixel space. Conditioned on the original image, PixText generates only the target regions, preserves unedited content, and uses MeanFlow for four-step generation. We further introduce Text Refinement Bench to evaluate text correction and image preservation in generated images. On this benchmark, PixText achieves 83.06% glyph pass and 94.66% exact target-character OCR accuracy, versus 36.77% and 52.31% for the strongest baseline. Its four-step MeanFlow variant retains an 81.84% pass rate with less than one GPU-second of amortized generation cost per image.
Figures & tables
Figure 1: Overview of PixText. A main image provides context, noisy item images represent the regions to be generated, and glyph images specify the target text. Position encoding aligns each item and its glyph with the main image. Group JiT blocks predict the item pixels, which are composed into the original image.
Figure 2: Small-character degradation during VAE reconstruction. Each VAE uses full-image encoding and posterior-mode decoding in FP32.
Figure 3: Local FLUX.1-VAE decoding causes visible color shifts when pasted back into the original image, even with 32 pixels of context.
Module
SD VAE
FLUX.1 VAE
Qwen-Image VAE
FLUX.2 VAE
DC-AE f32c32
DiP XL/16
PixelDiT 1300M
DeCo XXL/16
JiT L/16
Params (M) ↓
49.49
49.55
73.30
49.62
159.22
6.97
105.06
12.61
2.89
Latency (ms) ↓
90.89
90.54
69.94
90.76
55.38
5.65
15.01
8.18
0.12
GFLOPs ↓
10470.4
10472.2
11682.6
10474.7
8368.6
513.5
868.9
142.1
6.4
Table 1: Decoder and pixel output head costs at 1024×1024 resolution (single NVIDIA H800, FP16, batch size 1). The best and second-best results in each row are bolded and underlined, respectively.
Figure 4: Color preservation and grid-artifact mitigation. (a) Full images, local close-ups, and color-gradient maps, with background color-error distributions. (b) Generated images and color-gradient maps before and after uniform sampling, with PixText and MiniT2I boundary-gradient curves. Heatmaps share a scale within each panel; shading denotes 95% bootstrap intervals.
Figure 5: Effect of the final sampling update on residual speckle. Left: scene context. Right: matched close-ups and high-frequency maps show reduced background fluctuations when returning the clean prediction. Both outputs differ only in the final update along a shared trajectory.
Figure 6: Qualitative comparisons of different methods.
Text Refinement Bench
AnyText EN
AnyText ZH
Efficiency
Method
Pass ↑
R@1 ↑
OCR Acc ↑
FID ↓
LPIPS ↓
Acc ↑
NED ↑
FID ↓
LPIPS ↓
Acc ↑
NED ↑
FID ↓
LPIPS ↓
GPU-s/img ↓
Params (B) ↓
TextDiffuser
–
–
–
–
–
54.15
0.7863
24.04
0.1443
–
–
–
–
3.126
1.066
TextCtrl
–
–
–
–
–
59.53
0.8378
8.16
0.0365
–
–
–
–
13.802
1.128
UDiffText
–
–
–
–
–
67.16
0.8599
16.94
0.1072
–
–
–
–
8.252
1.361
DiffSTE
–
–
–
–
–
61.01
0.8019
19.16
0.1258
–
–
–
–
16.132
1.315
AnyText
12.40
23.46
25.59
11.47
0.0187
64.60
0.8440
22.40
0.1111
66.54
0.8607
22.29
0.0949
4.310
1.428
Table 2: Text Refinement Bench and AnyText results. AnyText ZH uses each baseline’s higher-Acc run for all metrics; Appendix D compares inputs. Dashes mean unsupported. Params (B) include the generator, VAE, and text encoders. Bold and underline mark best and second best.
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Endpoint computation
Relative error (%) ↓
Time (s) ↓
BF16
192.64
0.551
Strict FP32 (reference)
0.00
23.895
Ours (mixed precision)
18.92
0.706
Appendix
Table 3: DDE precision and efficiency. All configurations use BF16 center forward and backward passes and differ only in endpoint computation. We report medians over three samples, taking the median of seven timing measurements for each sample.
Figure 7: Correct recognition does not imply correct glyphs. OCR confidently transcribes these malformed glyphs as the targets above, CJK-CLIP similarity scores fall below their respective acceptance thresholds μy−0.03 , yielding Error decisions consistent with human annotations.
Method
Input
Acc (%) ↑
NED ↑
FID ↓
LPIPS ↓
AnyText
Line
65.87
0.8629
21.71
0.0891
Character ⋆
66.54
0.8607
22.29
0.0949
AnyText2
Line ⋆
78.06
0.9112
28.89
0.1407
Character
71.94
0.8948
28.52
0.1353
FLUX-Text
Line ⋆
80.93
0.9233
12.22
0.0498
Character
78.49
0.9191
12.34
0.0562
Appendix
Table 4: AnyText ZH with official line inputs and PixText’s character-level inputs. All rows use the same 1,000-image benchmark and 2,092-line scoring protocol. The input with higher sentence accuracy (Acc) is marked ⋆ and supplies all four metrics in the main table; metrics are never selected independently.
Figure 8: CCPE reduces crop padding. Left and middle: the same text region with two crop boundaries; orange dashed boxes mark the original OCR regions, and the background grid has 16-pixel spacing. Right: total item tokens across 116,038 text regions.
Figure 9: Additional endpoint-update comparisons. Each row shows scene context on the left and a close-up with its high-frequency map for each update on the right. Both maps share the color scale below the scene.
Figure 11: Additional qualitative comparisons of different methods (1/5).
Figure 12: Additional qualitative comparisons of different methods (2/5), using the same panel order, crop coordinates, and display scales as Figure 11 .
Figure 13: Additional qualitative comparisons of different methods (3/5), using the same panel order, crop coordinates, and display scales as Figure 11 .
Figure 14: Additional qualitative comparisons of different methods (4/5), using the same panel order, crop coordinates, and display scales as Figure 11 .
Figure 15: Additional qualitative comparisons of different methods (5/5), using the same panel order, crop coordinates, and display scales as Figure 11 .
Figure 16: Generation beyond the main training scripts. Top: shared source image and target region. (a–l) Glyph conditions and the corresponding PixText output crops. All outputs use the same crop coordinates and display scale; pale-blue frames identify model outputs.
Method
Images
Pass ↑
R@1 ↑
OCR Acc ↑
FID ↓
LPIPS ↓
PixText
1034/1034
83.06%
92.95%
94.66%
0.34
0.0010
PixText + MeanFlow
1034/1034
81.84%
92.57%
94.58%
0.38
0.0011
Qwen-Image-3.0-Pro
1027/1034
28.17%
60.92%
70.13%
7.39
0.0188
Appendix
Table 5: Comparison on the same 1,034 images and 2,638 repair targets. Targets in the seven cases where the API refused generation count as failures for Pass, R@1, and OCR Acc; image metrics use returned images.
We present StyleText, a large-scale dataset and benchmark for localized scene-text inpainting with style preservation. StyleText contains 28,518 image-mask-prompt triplets grouped into 9,932 scene families, enabling controlled evaluation of text legibility and visual consistency under shared scene context. We construct the dataset with an automated pipeline that combines LLM prompt templating, Flux-based source generation with key-value (KV) cache injection, OCR-based semantic filtering, polygon mask extraction, and mask-conditioned FluxFill augmentation. We define a reproducible evaluation protocol using normalized OCR metrics (word accuracy and character error rate) and CLIP image-image similarity with explicit preprocessing. A FluxFill+LoRA baseline trained on StyleText improves OCR accuracy substantially over initialization while maintaining scene style consistency, establishing a strong reference point for future comparisons.
We propose TextWand, a general-purpose framework that unifies scene text removal, generation, and replacement into a single model. By decomposing complex editing tasks into the atomic primitives of rendering and erasure, TextWand achieves precise control over both text appearance and background integrity. Specifically, we introduce a novel design, Overlay-Reference Positional Encoding (ORPE), to enforce pixel-level layout fidelity and exemplar-driven style control, alongside a new strategy, Region-Adaptive Suppression (RAS), to ensure clean text erasure. To address the absence of a comprehensive benchmark for general-purpose scene text editing among existing single-task datasets, we construct TextWand-Bench. Extensive experiments demonstrate that TextWand outperforms existing leading open-source and closed-source models by delivering superior text content accuracy, layout and style consistency, and overall image quality across scene text removal, generation and replacement tasks.
Shuyu Wang, Zhile Guan, Hongxiu Chen +5
School of Electronic and Computer Engineering, Peking University, Shenzhen, China.
Text-to-image (T2I) generation models have made significant strides but still struggle with prompt sensitivity: even minor changes in prompt wording can yield inconsistent or inaccurate outputs. To address this challenge, we introduce a closed-loop, test-time prompt refinement framework that requires no additional training of the underlying T2I model, termed TIR. In our approach, each generation step is followed by a refinement step, where a pretrained multimodal large language model (MLLM) analyzes the output image and the user's prompt. The MLLM detects misalignments (e.g., missing objects, incorrect attributes) and produces a refined and physically grounded prompt for the next round of image generation. By iteratively refining the prompt and verifying alignment between the prompt and the image, TIR corrects errors, mirroring the iterative refinement process of human artists. We demonstrate that this closed-loop strategy improves alignment and visual coherence across multiple benchmark datasets, all while maintaining plug-and-play integration with black-box T2I models. Code is available at https://github.com/hafeezkhan909/Test-time-Image-Refinement.
Mohammad Abdul Hafeez Khan, Yash Jain, Siddhartha Bhattacharyya +1
Florida Institute of Technology, USA · Microsoft Research, USA