Blame Errant Words, Not the Canvas: Defect-Aware Spatial Credit Assignment for Visual Text Generation

Sujie Hu; Huaisong Zhang; Pujing Yang; Qian Wang; Xiaomin Li; Xinhang Leng; Chen Li; Jing Lyu

cs.CVOct 14, 2026

Blame Errant Words, Not the Canvas: Defect-Aware Spatial Credit Assignment for Visual Text Generation

Authors: Sujie Hu, Huaisong Zhang, Pujing Yang, Qian Wang, Xiaomin Li, Xinhang Leng, Chen Li, Jing Lyu,

Organizations: The Hong Kong University of Science and Technology · Weixin, Tencent · Tsinghua University · Zhejiang University ·

Abstract

Recent reinforcement learning approaches have improved visual text generation, yet generated images still suffer from incorrect wording and malformed glyphs. We identify two coupled limitations: recognition-based rewards often overlook visible glyph defects, while whole-image scalar feedback obscures the uneven quality of individual text regions. These limitations motivate our central principle: $\textit{Blame errant words, not the canvas}$ , to effectively preserve local quality distinctions. First, we introduce $\mathbf{TextDART}$ , a defect-aware region-targeted reward model that localizes individual text regions and independently evaluates their semantic correctness and glyph quality, providing fine-grained, region-precise feedback. Building on TextDART, we propose $\mathbf{SCANNER}$ , which turns these regional judgments into spatially differentiated policy feedback. It normalizes rewards for each text target across sampled images and casts the resulting regional advantages onto the corresponding latent tokens. Extensive experiments on existing benchmarks and our newly established $\textbf{PolyText-Bench}$ demonstrate that our approach achieves state-of-the-art performance in text accuracy and glyph fidelity. Project page: https://sujiehu.github.io/textdart-scanner

Figures & tables

Figure 1: Comparison of Our Method Against Current Reward Model and Flow-GRPO. (a) A outranks B despite its malformed glyph, and image-group advantages reinforce the defect while penalizing B’s correct subtitle. (b) TextDART provides defect-aware scores for individual regions, and SCANNER computes target-group advantages for spatial credit assignment, penalizing A’s malformed subtitle while reinforcing B’s correct one (green/red: positive/negative).

Figure 2: Overview of TextDART and SCANNER. (a) TextDART localizes text and evaluates semantic correctness and defects. Rewards for the same target are normalized across samples to construct spatial advantages. (b) A noise-aware schedule shifts credit from global to regional advantages as denoising proceeds. (c) Policy ratios are pooled within each text target and the background, enabling spatially aligned GRPO updates.

Figure 3: Localization and Defect-aware Recognition under Challenging Visual Conditions. TextDART accurately localizes and reads text while marking <#> as malformed glyphs, whereas PP-OCRv5 hallucinates incorrect content and TextPecker misses or mismarks defects.

Figure 4: Qualitative Comparison of TextDART with SCANNER against Baselines. Target strings are color-coded in the prompts, and circles highlight text errors of the baselines. Our method renders every target correctly while preserving the background and aesthetics of the backbone, and produces legible, well-styled spontaneous text even when no text is prescribed.

Figure 5: Component Ablations of SCANNER. Evaluation reward throughout training when varying (a) advantage grouping, (b) global reward aggregation, (c) global–regional advantage mixing, and (d) policy ratio granularity. Each curve varies a single component with all other settings fixed, and our design (solid green) attains the highest final reward in every comparison.

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Hyperparameter	Value
Initialization	Qwen3-VL-8B-Instruct
Fine-tuning strategy	Full-parameter SFT
Trainable components	Language model, vision encoder, projector
GPUs / nodes	32 / 4
Training epochs	4
Batch size per GPU	2

Appendix

Table 4: Hyperparameters for TextDART supervised fine-tuning.

Model and optimization
Parameter	Qwen-Image-2512	FLUX.1-dev
Training language	English / Chinese	English
Resolution	$1024\times 1024$	$1024\times 1024$
LoRA rank / alpha	$64$ / $128$	$64$ / $128$
Mixed precision	BF16	FP16
Optimizer	AdamW	AdamW

Appendix

Table 5: Training and Evaluation Hyperparameters. Settings for TextDART with SCANNER.

English
Method	CTR		TSAP			TRL
Method	R $\uparrow$	NED $\downarrow$	P $\uparrow$	R $\uparrow$	F1 $\uparrow$	[email protected] $\uparrow$	[email protected] $\uparrow$	[email protected] $\uparrow$	[email protected] $\uparrow$	[email protected] $\uparrow$	[email protected] $\uparrow$
PP-OCRv5	0.7967	0.1191	0.0000	0.0000	0.0000	0.9070	0.9025	0.9048	0.6281	0.6250	0.6266
MonkeyOCR	0.8637	0.0788	0.0000	0.0000	0.0000	0.7262	0.5900	0.6510	0.4338	0.3525	0.3890
GOT-OCR-2.0	0.6642	0.1903	0.0000	0.0000	0.0000	$-$	$-$	$-$	$-$	$-$	$-$
Gemini-3.1-Pro-Preview	0.8948	0.0523	0.5909	0.3588	0.4465	0.8433	0.8475	0.8454	0.4428	0.4450	0.4439

Appendix

Table 6: Evaluation of Text Recognizers on Canonical Text Recognition (CTR), Text Structural Anomaly Perception (TSAP), and Text Region Localization (TRL). The best results are marked in bold . Results of non-supporting models are marked as

-

Metric	Qwen-Image-2512	OCRv5 + Flow-GRPO	TextPecker + Flow-GRPO	TextDART + SCANNER
Text Accuracy $\uparrow$	21.30%	9.40%	22.50%	46.80%
Text Aesthetics $\uparrow$	24.80%	13.10%	22.70%	39.40%

Appendix

Table 7: Blind Randomized User Study on Visual Text Generation. We report the average win rate across all samples and annotators. The best result in each row is highlighted in bold.

Figure 6: Ablation on Spatially Weighted KL Regularization. Global KL leaves residual misspellings, text-free penalties cause hallucinated text, and equal weights induce reward hacking (covering text with plain blocks, row 3). Our background-weighted variant (R:G $=$ 1:2) renders all target text correctly while preserving the scene.

Figure 7: Supplementary Ablations. Training curves of evaluation reward for (a) region–target matching, (b) image-to-latent projection, (c) normalization of regional contributions, and (d) the regional scale

\lambda

. Our choices (green) converge the highest in all four comparisons.

Figure 8: Effect of Replacing the Text Recognizer. Training curves with the downstream pipeline fixed. (a) The OCR reward saturates and declines late in training, while TextDART keeps improving. (b) TextDART also surpasses TextPecker with a steadier curve, showing the benefit of region-precise feedback.

Figure 9: Qualitative Results of Multi-reward Optimization on Qwen-Image-2512. Each text reward is combined with the PickScore ( Kirstain et al., 2023 ) and aesthetic ( , 2025 ) rewards (PA). Red dashed boxes mark text errors. TextDART with SCANNER renders all target text correctly while producing the most appealing layout, color, and composition, whereas the baselines garble text or degrade the scene.

Figure 10: Localization and Recognition Results of TextDART. Each detected region is annotated with its defect-aware transcription and glyph-quality score (d_score), and regions matched to a target additionally receive a semantic-correctness score (s_score).

Figure 11: Qualitative Comparison on English Prompts. Rows 1–3 are generated on Qwen-Image and rows 4–6 on Flux, and circles highlight text errors of the baselines.

Figure 12: Qualitative Comparison on Chinese Prompts. All images are generated on Qwen-Image, and circles highlight text errors of the baselines.

Benchmark	Multi-block prompts	English + Chinese	Glyph-defect scoring	Text-optional evaluation
LongText-Bench ( Geng et al., 2025 )	✓	✓	$\times$	$\times$
OneIG-Bench ( Chang et al., 2026 )	✓	✓	$\times$	$\times$
CVTG-2K ( Du et al., 2025 )	✓	$\times$	$\times$	$\times$
GenTextEval ( Zhu et al., 2026 )	✓	✓	✓	$\times$
PolyText-Bench (ours)	✓	✓	✓	✓

Appendix

Table 10: Comparison of Visual Text Generation Benchmarks. The comparison concerns the released prompt sets and their original text-evaluation protocols. Multi-block prompts specify multiple spatially separated text blocks. Glyph-defect scoring explicitly assesses malformed glyphs beyond transcription accuracy. Text-optional evaluation assesses text that may emerge without prescribing its presence or content. For OneIG-Bench, we consider its text-rendering subset. ✓: included;

\times

: not included.

Branch	Entries	Share
Controlled synthesis	18,856	15.4%
Caption regeneration	30,423	24.9%
Graphic design collections	26,915	22.0%
Long-tail	13,885	11.4%
Small-text	10,273	8.4%
Text-free	21,848	17.9%

Appendix

Table 11: Construction Branches (left) and Image Sources (right) of the TextDART-123K Training Split. Percentages use all 122,200 training entries. The graphic design collections category covers existing text-rich design images, including posters, logos, infographics, and artistic lettering.

Dimension	Categories	Distribution (%)
Scene	Street views, posters, infographics, etc. (17 in total)	5.8–5.9 each
Script	Chinese, English, mixed, digits and symbols	–
Text content	Frequency-balanced characters and lexical units, spanning short labels to long phrases	–
Composition	Single, multiple, and dense regions, 1–10 lines each	49.3 / 50.7 ∗
Geometry	Horizontal, vertical, rotated, perspective	42.4 / 21.7 / 21.6 / 14.3
Typography	Standard, handwriting or calligraphy, bold, artistic lettering	39.9 / 40.0 / 20.1 †

Appendix

Table 12: Construction Dimensions of the Controlled Prompts. Distributions are recorded on the specifications of the controlled subset and listed in the order of the corresponding categories. ∗ Single-line vs. multi-line regions. † Recorded over the first three categories.

Component	Content
Role	You are an expert in perceiving structural anomalies of rendered text in AI-generated images.
Task	Given a generated image, locate and transcribe every piece of visible text, including distorted, broken, or entirely unreadable text, and report each piece as a quadrilateral region together with its transcription.
Marking	<#> for a single structurally flawed and unrecognizable Chinese character or English letter. <###> for a single structurally flawed and unrecognizable English word (never a phrase, line, or sentence). <#####> for an entirely unreadable secondary text block whose characters cannot be counted individually. Transcribe literally what is rendered: write out every readable character, never guess plausible words from context, never silently repair broken text, and keep Traditional or Simplified Chinese as rendered. Separate multiple lines within one quadrilateral by a single space.
Grounding	Every visible text piece is covered by at least one quadrilateral $[x_{1},y_{1},x_{2},y_{2},x_{3},y_{3},x_{4},y_{4}]$ , whose vertices start at the top-left corner in the semantic reading direction and proceed clockwise, including vertical, rotated, or curved text. Text that wraps over lines, curves, or is broken up by graphics is emitted as several quadrilaterals instead of one loose box.
Output	A single JSON object: {"regions": [{"quad": [x1, y1, ..., x4, y4], "recognized_text": "text with markers"}]} .

Appendix

Table 13: Instruction Template of TextDART for defect-aware text localization and transcription.

Statistic	Train (122K)
Sample entries	122,200
Text-containing	100,352
Text-free	21,848
Annotated regions	495,778
Entries with anomaly markers	55,587 (55.4%)
Multi-region entries	81.1%

Appendix

Table 14: Composition of the TextDART-123K Training Split. The upper rows report counts and the lower rows report ratios. Image-level ratios use the 100,352 text-containing entries, and the region-level ratio uses all 495,778 annotated regions.

Text Rendering in T2I Generation

See all 18 papers

SpatialReward: Verifiable Spatial Reward Modeling for Fine-Grained Spatial Consistency in Text-to-Image Generation
Mar 23, 2026Sashuai Zhou, Qiang Zhou, Junpeng Ma +9Reward Modeling T2I Generation
Zhejiang University · Alibaba Group · Fudan University
TextAlign: Preference Alignment for Text Rendering with Hierarchical Rewards
May 19, 2026Mingxuan Cui, Jingpu Yang, Fengxian Ji +7Diffusion Model Alignment Reward Modeling
Mohamed bin Zayed University of Artificial Intelligence · Chinese Academy of Sciences Institute of Automation · Northeastern University +1
STAR: SpatioTemporal Adaptive Reward Allocation for Text-to-Image RL Post-Training
Jun 16, 2026Jinjie Shen, Wei Deng, Xian Hu +2Diffusion Model Alignment Compositional T2I Generation

Mar 23, 2026J/K move · Enter open · S save

SpatialReward: Verifiable Spatial Reward Modeling for Fine-Grained Spatial Consistency in Text-to-Image Generation

Sashuai Zhou, Qiang Zhou, Junpeng Ma, Yue Cao+8

Zhejiang University · Alibaba Group · Fudan University

Reward Modeling T2I Generation T2I Generation Evaluation RL for Image Generation

Open paper AskPDF arXiv

Recent advances in text-to-image (T2I) generation via reinforcement learning (RL) have benefited from reward models that assess semantic alignment and visual quality. However, most existing reward models pay limited attention to fine-grained spatial relationships, often producing images that appear plausible overall yet contain inaccuracies in object positioning. In this work, we present SpatialReward, a verifiable reward model explicitly designed to evaluate spatial layouts in generated images. SpatialReward adopts a multi-stage pipeline: a Prompt Decomposer extracts entities, attributes, and spatial metadata from free-form prompts; expert detectors provide accurate visual grounding of object positions and attributes; and a vision-language model applies chain-of-thought reasoning over grounded observations to assess complex spatial relations that are challenging for rule-based methods. To more comprehensively evaluate spatial relationships in generated images, we introduce SpatRelBench, a benchmark covering object attributes, orientation, inter-object relations, and rendered text placement. Experiments on Stable Diffusion and FLUX show that incorporating SpatialReward into RL training consistently improves spatial consistency and overall generation quality, with results aligned more closely to human judgments. These findings indicate that verifiable reward models hold considerable potential for enabling more accurate and controllable optimization in text-to-image generation models.

Abstract

Text Rendering in T2I Generation

Explore similar work

SpatialReward: Verifiable Spatial Reward Modeling for Fine-Grained Spatial Consistency in Text-to-Image Generation

TextAlign: Preference Alignment for Text Rendering with Hierarchical Rewards

STAR: SpatioTemporal Adaptive Reward Allocation for Text-to-Image RL Post-Training