Recent text-to-image models can generate high-quality images from natural-language prompts, yet controlling typography remains challenging: requested typographic appearance is often ignored or only weakly followed. We address this limitation with a data-centric approach that trains image generation models using targeted supervision derived from a structured annotation pipeline specialized for typography. Our pipeline constructs a large-scale typography-focused dataset, FontUse, consisting of about 70K images annotated with user-friendly prompts, text-region locations, and OCR-recognized strings. The annotations are automatically produced using segmentation models and multimodal large language models (MLLMs). The prompts explicitly combine font styles (e.g., serif, script, elegant) and use cases (e.g., wedding invitations, coffee-shop menus), enabling intuitive specification even for novice users. Fine-tuning existing generators with these annotations allows them to consistently interpret style and use-case conditions as textual prompts without architectural modification. For evaluation, we introduce a Long-CLIP-based metric that measures alignment between generated typography and requested attributes. Experiments across diverse prompts and layouts show that models trained with our pipeline produce text renderings more consistent with prompts than competitive baselines. The source code for our annotation pipeline is available at https://github.com/xiaxinz/FontUSE.
Figures & tables
Figure 2: Dataset construction pipeline. (a) Typography-focused images are collected from public design resources. (b) Hi-SAM [ 33 ] detects text regions and outputs bounding boxes. (c)-(e) An MLLM performs text recognition, font-style annotation, and use-case annotation.
Figure 3: Example of the input prompt and an MLLM responses. See Appendix A for details of ⟨example format⟩.
Figure 4: Prompt used for text recognition. The MLLM is instructed to output only the extracted word (case-sensitive), returning “-” if no text is detected and “#” for non-Roman characters.
Figure 5: Top-40 style and use-case families after consolidation, illustrating their frequency distributions.
Figure 6: Cumulative distributions of use-case and style families. Dashed lines indicate 50%, 80%, and 90% cumulative mass.
AnyText
TD-2
SD-3
base
FT
base
FT
base
FT
CLIP ↑
0.3644
0.3241
0.4364
0.4550
0.4189
0.4355
Long-CLIP ↑
0.3946
0.4464
0.4765
0.5170
0.4696
0.5618
Fine-tuned Long-CLIP ↑
0.5858
0.6368
0.6097
0.7414
0.5068
0.6297
GPT selections ↑
12/60
48/60
9/60
51/60
16/60
44/60
Gemini selections ↑
14/60
46/60
11/60
49/60
16/60
44/60
Table 1: Comparison of CLIP, original Long-CLIP and fine-tuned Long-CLIP scores (linearly normalized to [0,1] ), MLLM preference counts, and legibility (measured by the average character error rate, CER). Boldface indicates the best result within each backbone. FT denotes “fine-tuned.”
Figure 7: Qualitative comparison with AnyText [ 28 ] and FontUse-tuned model (AnyText fine-tuned using our dataset). In the text prompts, the target string to be rendered as a text image is shown in green , the font style in brown , and the font use case in blue .
Figure 8: Qualitative comparison with Stable Diffusion 3 (SD-3) [ 27 ] , TextDiffuser-2 [ 5 ] , and FontUse-tuned models (SD-3 and TextDiffuser-2 fine-tuned using our dataset).
Figure 9: Comparison of outputs across different use-case conditions. Underlined text indicates prompt differences. Input image and edit region omitted.
Figure 10: Comparison of controllability under color and effect specifications (shown in purple ). Input image and edit region omitted.
CLIP
Long-CLIP
Fine-tuned Long-CLIP
(a) corresponding
22.13
65.31
72.76
(b) non-corresponding
21.71
61.40
62.72
(c) unrelated text
19.88
38.98
33.56
Gain (a vs. b) ↑
+1.9 %
+6.4 %
+16.0 %
Gain (a vs. c) ↑
+11.3 %
+67.5 %
+116.8 %
Table 2: Validation of the alignment metrics. We report the average similarity scores, computed by each metric between images and texts under three conditions: (a) corresponding use-case text, (b) non-corresponding use-case text, and (c) unrelated text.
Method
Average CER ↓
Tesseract OCR [ 26 ]
71.88%
EasyOCR [ 14 ]
36.23%
PaddleOCR(v3) [ 16 ]
12.44%
olmOCR-7B [ 22 ]
3.78%
Ours
2.18%
Table 3: OCR performance measured by average CER.
Figure 11: Our failure cases. There is a trade-off between stylistic complexity and legibility.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 12: Details of the required ⟨example format⟩ to enforce strict compliance with the target format.
Figure 13: Prompt for the MLLM-based evaluation. The detailed instructions are crafted to produce reliable typography-focused preference judgments.
ID
Prompt
1
Suitable for romantic and playful design projects, fun and flirty which fit brandings of dating services or Valentine’s Day promotions. Cute, pretty, vague, nostalgic
2
The font has a liquid, distorted look, making it ideal for creative projects that need a rugged and organic feel
3
A hand-drawn layered font, it has a unique, crooked look with irregular strokes and outlines, making it perfect for logos & branding, stationery, book covers, magazine design, clothing prints
4
For advertising campaigns focused on youthful, casual consumers, targeting parties or social gatherings
Suitable for: modern and artistic design projects. Ideal applications: [minimalistic aesthetics] branding for lifestyle brands, cafes, or boutique stores emphasizing simplicity and elegance. Font styles: bold, modern, sans-serif, clean, minimal, elegant
Appendix
Table 4: Evaluation prompts (ID: 1-15). Font styles are indicated in brown and use cases in blue .
ID
Prompt
16
Handwritten, rough, textured and brush script. Used in social media graphics, promotional content, and blog headers for lifestyle, travel, or personal brands
17
Suitable for posters or prints with motivational quotes for decor or gifts. Creative branding for businesses in arts, crafts, or entertainment
18
Youth-oriented, for merchandise like t-shirts, stickers, and accessories for a younger audience
19
Playful and vibrant, rounded font. Used for restaurant branding, pizza menus, and promotional materials
20
Sans-serif font, modern, wide, sleek and geometric shape. Suitable for futuristic and tech-oriented design projects, as technology companies or innovative startups branding, and futuristic products
21
Elegant, serif, modern. Logos for high-end clothing brands, upscale boutiques, elegant fashion lines. Books for lifestyle or beauty brands, product packaging
Appendix
Table 5: Evaluation prompts (ID: 16-30). Font styles are indicated in brown and use cases in blue .