Organizations: Shanghai Jiao Tong University · Xiaohongshu Inc. · Hong Kong University of Science and Technology · Southeast University · South China University of Technology
Despite recent advances in generative models driving significant progress in text rendering, accurately generating complex text and mathematical formulas remains a formidable challenge. This difficulty primarily stems from the limited instruction-following capabilities of current models when encountering out-of-distribution prompts. To address this, we introduce GlyphBanana, alongside a corresponding benchmark specifically designed for rendering complex characters and formulas. GlyphBanana employs an agentic workflow that integrates auxiliary tools to inject glyph templates into both the latent space and attention maps, facilitating the iterative refinement of generated images. Notably, our training-free approach can be seamlessly applied to various Text-to-Image (T2I) models, achieving superior precision compared to existing baselines. Extensive experiments demonstrate the effectiveness of our proposed workflow. Associated code is publicly available at https://github.com/yuriYanZeXuan/GlyphBanana.
Figures & tables
Figure 2: The illustration of motivation. We observe that while in-distribution cases show satisfying precision-style banlance, there exists huge gap between OOD cases and deterministic rendered texts.
Figure 3: Overview of the GlyphBanana agentic pipeline. The workflow comprises four stages: (1) Extraction Stage parses the input into text content and style attributes; (2) Draft Preview Stage generates an initial image via a Layout Planner; (3) Glyph Injection Stage applies Frequency Decomposition in latent space and Attention Re-weighting inside each DiT block; (4) Style Refinement Stage employs iterative refinement with a Style Refiner and Score Judger. The bottom panel details the denoising process with the Attention Re-weighting.
Figure 4: Illustration of the GlyphBanana-Benchmark with auxiliary tools. The proposed benchmark consists of two categories. General Text for Rendering assesses standard and stylized text rendering. Formulas from Easy to Complex evaluates formula rendering across varying complexities
Datasets
Text Type
Condition
Statistics
OCR Score
Style Score
En.
Zh.
Formulas
Image
Mask
Num.
Avg.L
DrawTextExt [ 24 ]
✓
✓
✗
✗
✗
220
17.0
0.76
0.81
0.83
0.80
AnyText [ 51 ]
✓
✓
✗
✗
✗
1000
21.8
0.33
0.44
0.67
0.69
CVTG-2K [ 48 ]
✓
✗
✗
✗
✗
2000
39.5
0.49
0.51
0.75
0.67
LongText-Bench [ 7 ]
✓
✓
✗
✗
✗
320
116.7
0.38
0.72
0.69
0.74
Ours
✓
✓
✓
✓
✓
290
32.7
0.37
0.71
0.68
0.71
Table 1: Comparison of different text-rendering datasets. Num. refers to the number of samples in the dataset, and Avg.L refers to the average length of the char to be rendered in the dataset. refers to FLUX.2-klein-9B and refers to Qwen-Image-2512.
Figure 5: Qualitative comparisons with other baselines . Fail denotes the FLUX.1-dev based models unable to follow instructions to render chinese text due to its limited text-encoder. Besides, we color the quoted text in red, referring to the target text to be rendered, and color the style text related to the glyph in blue.
Method
OCR Score ↑
VLM Score ↑
ITM Score ↑
User Study ↓
Acc.
Ned.
Style
Faith.
VQA
CLIP
Aesthetic
Faith.
Ours+Z−Image
w/o re-weight
72.3
76.7
0.745
0.704
0.808
0.709
2.73
2.70
w/o refine
84.0
86.7
0.725
0.745
0.798
0.710
3.47
3.40
w/o F.D.
84.5
87.3
0.755
0.764
0.803
0.708
2.73
2.83
full
85.9
88.1
0.765
0.764
0.814
0.720
1.07
1.07
Ours+QwenImage
w/o re-weight
70.7
74.8
0.676
0.689
0.814
0.680
2.43
2.23
Table 2: Quantitative comparison results for text-rendering metrics.
Method
OCR Score ↑
VLM Score ↑
ITM Score ↑
User Study ↓
Acc.
Ned.
Style
Faith.
VQA
CLIP
Aesthetic
Faith.
AnyText2
33.8
40.5
0.661
0.438
0.641
0.637
7.80
7.62
TextCrafter
34.0
39.6
0.672
0.371
0.804
0.680
6.75
6.47
Flux.1
.dev
27.9
34.3
0.691
0.280
0.771
0.639
6.75
7.03
FluxText
25.0
28.4
0.600
0.351
0.718
0.656
6.98
6.83
Flux.2
.klein
36.7
42.1
0.676
0.521
0.821
0.686
6.38
6.08
Table 3: Quantitative comparison results for text-rendering metrics. represents for FLUX.2-klein-9B and represents for Qwen-Image-2512.
Figure 6: Qualitative comparison results.
Figure 7: Metric comparisons for multi-turn refinement.
Figure 8: Qualitative comparisons for illustrating methods of Frequency Decomposition.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 9: More qualitative results for refinement process.
Figure 10: More qualitative results for GlyphBanana, Qwen-Image as base model.
Figure 11: More qualitative results for GlyphBanana, Z-Image as base model.
Subset
Num.
Avg.∣Text∣
Avg.∣Prompt∣
English Subsets
GlyphBanana-En (Easy)
50
4.08
47.74
GlyphBanana-En (Rare)
25
8.92
56.84
Chinese Subsets
GlyphBanana-Zh (Easy)
50
2.00
19.00
GlyphBanana-Zh (Rare)
25
11.20
27.48
Appendix
Table 4: Illustration of GlyphBanana-Benchmark. The benchmark contains multimodal inputs spanning English, Chinese, and scientific-formula subsets, together with reference images and masks, and follows a ladder-shaped difficulty design. Avg.∣Text∣ denotes the average length of the target rendered text, and Avg.∣Prompt∣ denotes the average length of the corresponding prompt.
Configuration
Mean IoU ↑
Median IoU ↑
Std ↓
Improvement
VLM only)
0.2703
0.2508
0.1620
–
VLM + 3 × 3 Grid
0.4475
0.3892
0.1464
+65.6%
VLM + 5 × 5 Grid
0.5531
0.5406
0.1280
+104.6%
VLM + 8 × 8 Grid
0.3776
0.3628
0.1552
+39.7%
Appendix
Table 5: Ablation Study on VLM-Based Text Grounding with Auxiliary Tools.
Figure 12: Execution flow of the formula-rendering auxiliary tool used by GlyphBanana. The implementation prefers MathJax for rich LaTeX formulas, falls back to matplotlib mathtext when Node.js or SVG conversion is unavailable, and uses PIL-based text rendering for non-LaTeX content.
Figure 13: Schematic diagram of enhanced text rendering by injecting glyph templates in latent space.