Post-training text-to-image generators with reinforcement learning usually relies on external reward models that need annotated preferences or auxiliary evaluators and are prone to reward hacking. We show that the generator's own denoising accuracy, used by a recent intrinsic reward, is a poor proxy for prompt fidelity. Within a same-prompt group, this absolute self-confidence is dominated by image complexity and ranks correct samples above incorrect ones near chance. We introduce CANDOR (Contrastive Alignment from Null-referenced Denoising lOg-Ratio), the log-ratio of unconditional to conditional denoising errors under shared noise probes, which measures how much the prompt helps the model denoise its own sample. It cancels prompt-independent difficulty that scales both errors and ranks the better of two same-prompt samples higher in 60% and 63% of pairs on compositional generation and text rendering. As the only Flow-GRPO reward for a flow-matching model, CANDOR improves text rendering, aesthetic and human-preference scores over the base model and a prior absolute self-confidence reward, with text-rendering and preference gains that stronger classifier-free guidance does not reproduce. On public Flow-GRPO adapters trained on external rewards, CANDOR further raises PickScore and ImageReward, so it can complement those rewards. Code is available at https://github.com/jhlee0619/candor.
Figures & tables
Figure 2: Overview of CANDOR. Rollout & re-noising: policy samples re-noised with antithetic probes. Contrastive denoising log-ratio: Di,t of empty-prompt to prompt errors (CFG off). GRPO: per-sample rewards (timestep mean of z-scored Di,t ) group-normalized into advantages. Error maps: latent error of this sample at t=0.60 , averaged over probes and channels, one color scale; other thumbnails: illustrative pixel-space renderings.
GenEval
OCR
Within-group correlation
Absolute S vs. complexity
−0.75
−0.74
CANDOR D vs. complexity
−0.12
−0.11
CANDOR D vs. absolute S
−0.11
−0.09
Within-group concordance with correctness ↑
Complexity (log PNG size)
0.53
0.50
Table 1: Offline analysis of intrinsic rewards. Base-model test-prompt rollouts at CFG 7.0. Concordance: fraction of same-prompt pairs ranked correctly ( 0.5 = chance); w/o complexity: log PNG size regressed out. Bold : best; underline : second best in the concordance block.
Figure 3: Qualitative comparison on four prompt sets. DrawBench, GenEval, OCR and Pick-a-Pic test prompts; shared initial noise. OCR labels: text to render. Hand-picked to show differences.
Task-specific
Text–image alignment & aesthetics
Human preference
Model
GenEval ↑
OCR ↑
CLIPScore ↑
Aesthetic ↑
PickScore ↑
HPSv2.1 ↑
ImageReward ↑
UnifiedReward ↑
SDXL
0.54
0.13
0.278
5.60
22.30
0.265
0.55
2.84
SD3.5-L
0.70
0.70
0.290
5.53
22.79
0.289
0.97
3.25
Base
0.69
0.63
0.284
5.38
22.37
0.283
0.91
3.11
+ SOLACE
0.69
0.63
0.285
5.35
22.36
0.276
0.86
3.09
+ CANDOR
0.71
0.70
0.285
5.45
22.56
0.297
1.09
3.20
Table 2: Post-training results. Measured in this work; GenEval: official task-averaged; OCR: Flow-GRPO set; rest: DrawBench; SDXL, SD3.5-L: unranked. Flow-GRPO (X): public adapter for X. + Y: Base, or the adapter above if indented, post-trained with reward Y. Bold : best; underline : second best per block; ties share the mark.
Iter.
GenEval ↑
OCR ↑
CLIP ↑
Aes. ↑
Pick ↑
HPS ↑
IR ↑
Base
–
0.687
0.626
0.284
5.38
22.37
0.283
0.91
+ SOLACE
–
0.692
0.626
0.285
5.35
22.36
0.276
0.86
+ CANDOR
1,040
0.715
0.701
0.288
5.41
22.54
0.294
1.07
Table 3: Comparison with absolute self-confidence. Measured in this work; GenEval: official task-averaged; OCR: Flow-GRPO set; rest: DrawBench. CLIP: CLIPScore; Aes.: aesthetic; Pick: PickScore; HPS: HPSv2.1; IR: ImageReward; Iter.: CANDOR training iteration; –: not applicable. Bold : best; underline : second best per column; ties share the mark.
CFG
GenEval ↑
OCR ↑
CLIP ↑
Aes. ↑
Pick ↑
HPS ↑
IR ↑
Base
4.5
0.671
0.577
0.283
5.38
22.35
0.280
0.83
7.0
0.687
0.626
0.284
5.38
22.37
0.283
0.91
9.0
0.690
0.643
0.284
5.37
22.32
0.283
0.93
11.0
0.684
0.644
0.285
5.33
22.22
0.280
0.90
13.0
0.655
0.644
0.283
5.28
22.08
0.275
0.84
Table 4: Base model across CFG scales vs . CANDOR at CFG 7.0. Measured in this work; GenEval: official task-averaged; OCR: Flow-GRPO set; rest: DrawBench. CLIP: CLIPScore; Aes.: aesthetic; Pick: PickScore; HPS: HPSv2.1; IR: ImageReward. Bold : best; underline : second best per column; ties share the mark.
GenEval ↑
OCR ↑
Reward composition
Absolute S=−logMSE(c)
0.48
0.51
Unconditional, −logMSE(∅)
0.48
0.50
S+D
0.57
0.60
D (Ours)
0.60
0.63
Reference condition
Table 5: Offline ablation of the reward design. Base-model test-prompt rollouts at CFG 7.0. Concordance: fraction of same-prompt pairs ranked correctly ( 0.5 = chance). c′ : another prompt of the same benchmark. Bold : best; underline : second best per column within each block, ranked before rounding.