Recent text-to-image generation models have achieved remarkable visual quality, but improving them through post-training remains challenging because no single reward signal captures the full range of human preference. In this work, we develop a simple and effective post-training recipe for open-domain text-to-image generation based on the composition of complementary reward signals. Our reward system consists of two main components: a preference reward, trained on large-scale human preference data using a Bradley-Terry objective to capture overall human aesthetic and perceptual preferences, and rubric-based rewards, which explicitly evaluate prompt faithfulness and other desirable properties while providing safeguards against reward hacking. A key challenge is how to combine these heterogeneous reward signals. We show that a naive weighted average leads to suboptimal optimization behavior, and propose a simple reward composition strategy that more effectively balances preference optimization with rubric satisfaction. In the Arena text-to-image leaderboard (https://arena.ai/), our RL-trained Flux2dev achieves an Elo rating 69 points above the base model, and our post-trained Ideogram-4 surpasses every open-source model on the leaderboard, reaching an Elo of 1223.5. (Claims of state-of-the-art performance are based on the Arena leaderboard snapshot as of September 4, 2026.) Our results suggest that effective rewards for frontier generative-model training require broad coverage of user intent and robustness to exploitation under optimization. To support reproducible research, we release Arena-T2I-Training, a 1K subset of training data that recovers some gains of full-scale training, providing a resource that we hope will facilitate future work on post-training for text-to-image models.
Figures & tables
Deployed model
Arena score
FLUX.2-dev
base model
1132.8±14
Ours
1202±13
Ideogram-4
Base model (best open-source model)
1204±4
Ours
1224±13
Table 1: Arena scores on the Arena text-to-image leaderboard. Public scores are from the September 4, 2026 leaderboard snapshot.
Figure 1: Visualization. Preference-only training drifts toward generic embellishment and ignores requested content; faithfulness-only training washes out detail and drops requested style and scenery; Arena-T2I-Hard recovers the requested elements but keeps unrequested decoration; the ungated constraint flattens detail even where the prompt invites it; the intent-gated Stage 1 keeps the constraint pressure only where the prompt closes the spec, yielding compliant renders without an obvious quality tax.
Figure 2: Two recurring failure modes. Left: the prompt requests no text, yet the RL policy introduces garbled strings on the napkin. Right: the prompt requests Elsa Beskow’s storybook illustration style; the base model preserves the requested style, whereas the RL policy drifts toward photorealism.
Gemini-3.5-flash
GPT-5.4
Id
Arena RM
PickScore
Arena RM
PickScore
Baseline
DiffusionNFT
.290
.290
.396
.396
faithfulness only
.366
.366
.367
.367
Arena-T2I-Hard
.589
.577
.557
.537
preference only
.507
.461
.417
.517
Table 2: All pairwise win rates vs. the frozen FLUX.2 base. Each cell reports the run’s best checkpoint among 30,60,90,120,150 , selected on the 1k Arena validation rewrite prompts. Please refer to Table 7 for 95% percentile-bootstrap confidence intervals for every cell.
Figure 3: RL outcome scales with the preference reward model’s training data. The x-axis shows the number of human preference pairs used to train the reward model, and the y-axis shows the MMRBv2 win rate of the resulting Stage 1 RL model against the frozen base.
Gemini-3.5-flash
GPT-5.4
Mechanism
Variant
Arena RM
PickScore
Arena RM
PickScore
Constrain Gate
ungated
0.6046
0.5200
0.5555
0.4985
intent-gated
0.6416
0.5810
0.5615
0.6105
Hack applicability gate
ungated
0.5591
0.5695
0.5000
0.5515
applicable-gated
0.6346
0.5710
0.5670
0.5845
Hack signal integration
additive
0.5260
0.4194
0.4730
0.4105
Table 3: Ablation study on the ensembling mechanisms. Win rate vs. the base model on the 1k Arena validation prompts; each cell is the best checkpoint among {30,60,90,120,150} , selected under the Gemini judge.
Gemini-3.5-flash
GPT-5.4
Id
Arena RM
PickScore
Arena RM
PickScore
Baseline
Arena-T2I-Hard
.401
.524
.579
0.451
preference only
.266
.334
.285
0.403
Ours
Stage 1 (w/o anti-hack)
.553
.537
.623
.525
Table 4: All pairwise win rates vs. the frozen Ideogram-4 base on the 1k Arena validation prompts, judged by Gemini-3.5-flash and re-judged by GPT-5.4.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
FLUX.2 [dev]
Ideogram 4
Rollouts (per training epoch)
Prompt groups G
48
48
Rollouts per prompt K
24
24
Images per epoch
1,152
1,152
Sampling steps
10
10
Resolution
5122
5122
Appendix
Table 5: Training hyperparameters for the two base models.
Figure 4: The complete per-prompt reward battery for two real evaluation prompts, shown alongside the as-served renders from the FLUX.2 base model and the Stage 1 model. For each prompt, we list every question posed by the reward: the presence-oriented faithfulness DAG, in which a question whose parent fails is automatically scored ‘no” without being asked; the absence-oriented constraint checklist, generated in full here because both prompts are labeled strict ; and, at the bottom, the two armed hack-prevention detector questions shared across prompts.
Id
MMRBv2 prompts
Baseline
faithfulness only
0.3357
Arena-T2I-Hard
0.5563
preference only
0.5272
Ours
Stage 1
0.5632
Appendix
Table 6: Prompt-set transfer: we swap the prompt set to the 791 public MMRBv2 T2I test prompts. The checkpoints are selected on the 1k Arena validation prompts.
Gemini-3.5-flash
GPT-5.4
Id
Arena RM
PickScore
Arena RM
PickScore
Baseline
DiffusionNFT
.290 [.266, .315]
.290 [.266, .315]
.396 [.370, .421]
.396 [.370, .421]
faithfulness only
.366 [.340, .392]
.366 [.340, .392]
.367 [.344, .390]
.367 [.344, .390]
Arena-T2I-Hard
.589 [.561, .615]
.577 [.550, .603]
.557 [.533, .580]
.537 [.513, .560]
preference only
.507 [.480, .533]
.461 [.434, .487]
.417 [.394, .441]
.517 [.492, .541]
Appendix
Table 7: 95% percentile-bootstrap confidence intervals for every cell of Table 2 . Each cell re-states the win rate vs. the frozen FLUX.2 base on the 1k Arena validation rewrite prompts together with its interval. The DiffusionNFT and faithfulness-only baselines train with neither preference reward, so their Arena-RM and PickScore columns repeat the same run. All intervals use 10,000 bootstrap replicates over prompt pairs with a fixed seed.
Gemini-3.5-flash
GPT-5.4
Variant
Arena RM
PickScore
Arena RM
PickScore
Constraint gate
ungated
.605 [.580, .630]
.520 [.493, .546]
.555 [.532, .579]
.498 [.475, .521]
intent-gated
.642 [.617, .666]
.581 [.554, .607]
.561 [.539, .584]
.611 [.587, .634]
Hack applicability gate
ungated
.559 [.533, .586]
.570 [.543, .596]
.500 [.477, .522]
.551 [.529, .574]
Appendix
Table 8: 95% percentile-bootstrap confidence intervals for every cell of Table 3 . The intent-gated variant is the Stage 1 run, and the applicable-gated and one-sided-veto variants are the Stage 2 run, so those rows repeat the corresponding intervals of Table 7 . All intervals use 10,000 bootstrap replicates over prompt pairs with a fixed seed.
Figure 5: Single-preference training vs. base across checkpoints, evaluated on the 1 k arena validation prompts with gemini-3.5-flash rewrites. The faithfulness panel shows both independently trained preference rewards below parity and degrading with training, even while aesthetics holds at or above parity for the arena-RM run: faithfulness is traded away for aesthetics.
Text-to-image generation creates images from natural-language prompts using generative models, with research focused on fidelity, control, efficiency, and safety.