APT: Feedback-Driven Agentic Post-Training for Text-to-Image Generation

Sirui Zhang; Fangyun Wei; Dong Chen; Jinjing Zhao; Yitong Wang; Yunuo Chen; Baining Guo; Zheng-Jun Zha

cs.CVOct 13, 2026

APT: Feedback-Driven Agentic Post-Training for Text-to-Image Generation

Authors: Sirui Zhang, Fangyun Wei, Dong Chen, Jinjing Zhao, Yitong Wang, Yunuo Chen, Baining Guo, Zheng-Jun Zha

Organizations: University of Science and Technology of China · Microsoft Research Asia · University of Sydney · Fudan University ·

Abstract

Post-training is crucial for improving text-to-image models, yet existing pipelines typically rely on manually curated training data and fixed training distributions. We introduce APT (Agentic Post-Training), a fully automated framework that adaptively post-trains a base model according to its evolving capability profile. APT coordinates three agents to construct evaluation sets from user requirements, diagnose model-specific strengths and failure modes, and generate reinforcement learning data with an adaptive prompt distribution that emphasizes underperforming capabilities. By repeatedly evaluating, diagnosing, and updating the generator, APT lets the current model's behavior shape its next training tasks, forming a feedback loop inspired by recursive self-improvement. For general-purpose settings without user-specified evaluation scenarios, we further develop an evaluation harness that enables broad and diverse evaluation set generation. Experiments across multiple base models, benchmarks, and generation architectures demonstrate that APT is model-agnostic, benchmark-agnostic, architecture-agnostic, and fully automated.

Figures & tables

Figure 1: APT (Agentic Post-Training) turns post-training into a fully automated, evaluation-driven adaptive loop. Given a user-specified evaluation scope, the Evaluation Set Generation Agent constructs a diverse evaluation set consisting of prompt–evaluation rubric pairs ; the Diagnosis Agent evaluates the current model on these pairs to identify its strengths and failure modes; and the RL Data Generation Agent adaptively samples next-round training prompts, prioritizing underperforming capabilities. The updated model is then re-evaluated in the next round, allowing the training distribution to continuously adapt to the model’s evolving capability profile. This design eliminates the need for manually curated post-training data and makes APT model-agnostic , benchmark-agnostic , architecture-agnostic , and fully automated .

Figure 2: Left : Statistics of the sub-categories within each category and the prompt-rubrics pairs within each category. All prompt-rubrics pairs are automatically generated by our Evaluation Set Generation Agent, providing comprehensive coverage for evaluating the current T2I model. Right : An example prompt-rubrics pair.

Lens
Model	GenEval	OneIG-EN	GenEval2	DPG	CVTG-2K	Average
Model	Overall	Average	Soft-TIFA GM	Overall	Word Accuracy	Average
Base Model	0.702	0.521	29.06	85.93	0.480	57.06
w/ Handcrafted RL Set	0.815	0.518	46.35	87.60	0.758	68.61
w/ APT	0.859	0.523	50.73	87.88	0.795	71.26
SD3.5-Medium

Table 1: Comparison of APT with the original base models and a handcrafted RL post-training baseline across four T2I models. The handcrafted RL set is manually curated by human experts to cover diverse generation scenarios, while APT uses the same model-agnostic agentic workflow for all models. APT consistently achieves the best average performance and improves multiple benchmarks with a single post-training run, demonstrating both stronger post-training effectiveness and benchmark-agnostic generalization. Detailed results for each benchmark are shown in Tables 7 – 10 in the Appendix C .

Target/Source	Lens	SD3.5	FLUX	LongCat
Lens	0.851	0.833	0.846	0.798
SD3.5	0.736	0.743	0.742	0.722
FLUX	0.679	0.726	0.751	0.729
LongCat	0.823	0.786	0.827	0.828

Table 2: APT Captures Model-Specific Training Distributions. Each row corresponds to a target model post-trained with RL data generated by APT from different source models. Diagonal entries denote the standard APT setting, where the RL data is adaptively generated for the same target model, while off-diagonal entries reuse data generated for another model. Detailed results for each benchmark are shown in Tables 11 in the Appendix C .

Lens
Model	GenEval	OneIG-EN	GenEval2	DPG	CVTG-2K	Average
Model	Overall	Average	Soft-TIFA GM	Overall	Word Accuracy	Average
Base Model	0.702	0.521	29.06	85.93	0.480	57.06
OneIG-EN Evaluation Set	0.847	0.528	47.08	87.39	0.823	70.85
GenEval Evaluation Set	0.887	0.505	41.43	84.23	0.674	66.45
APT Evaluation Set	0.859	0.523	50.73	87.88	0.795	71.26

Table 3: Ablation of the evaluation set generation agent. We compare the APT-generated evaluation set with existing benchmark evaluation sets for producing diagnosis reports during post-training. Detailed results for each benchmark are shown in Tables 12 – 13 in the Appendix C .

Ablation on the Iterative Feedback Loop
Setting	GenEval	OneIG-EN	GenEval2	DPG	CVTG-2K	Average
Setting	Overall	Average	Soft-TIFA GM	Overall	Word Accuracy	Average
APT	0.859	0.523	50.73	87.88	0.795	71.26
w/o Iterative Feedback Loop	0.844	0.523	48.27	87.61	0.791	70.34
Ablation on the RL Data Generation Agent
Reduced Diagnostic Criteria	0.800	0.520	49.58	87.53	0.763	69.08

Table 4: Ablation on the Iterative Feedback Loop and the RL Data Generation Agent. This study is conducted using the Lens base model. Detailed results for each benchmark are shown in Tables 14 – 15 in the Appendix C .

Model	GenEval							OneIG-EN						GenEval2
Model	Single	Two	Count	Color	Position	Attribute	Overall	Alignment	Text	Reasoning	Style	Diversity	Average	Soft-TIFA GM
Janus-Pro
Base Model	0.994	0.869	0.538	0.891	0.758	0.635	0.781	0.799	0.068	0.165	0.293	0.243	0.314	15.25
w/ Handcrafted RL Set	0.984	0.891	0.609	0.902	0.790	0.660	0.806	0.804	0.074	0.175	0.297	0.229	0.316	20.42
w/ APT	0.991	0.909	0.606	0.910	0.808	0.668	0.815	0.809	0.074	0.175	0.297	0.233	0.318	21.61

Table 5: Architecture Generalization to Autoregressive Generation. Comparison of APT with the original base model and a handcrafted RL post-training baseline on the AR model Janus-Pro .

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

GPT Series
Model	GenEval	OneIG-EN	GenEval2	DPG	CVTG-2K	Average
Model	Overall	Average	Soft-TIFA GM	Overall	Word Accuracy	Average
Base Model	0.702	0.521	29.06	85.93	0.480	57.06
GPT-5.4	0.859	0.523	50.73	87.88	0.795	71.26
GPT-5.6-Sol	0.832	0.519	51.43	88.12	0.791	70.75
GPT-4.1	0.851	0.520	48.94	87.93	0.796	70.71

Appendix

Table 6: Ablation on the LLM backbone used in the RL Data Generation Agent. We replace the default GPT-5.4 with LLMs from different families and scales while keeping the remaining APT pipeline unchanged. Results are reported on five benchmarks using Lens as the base model.

Lens
Model	Single	Two	Count	Color	Position	Attribute	Overall
Base Model	0.991	0.788	0.547	0.848	0.548	0.488	0.702
w/ Handcrafted RL Set	0.994	0.929	0.763	0.848	0.735	0.620	0.815
w/ APT	1.000	0.947	0.822	0.872	0.835	0.680	0.859
SD3.5-Medium
Base Model	0.994	0.854	0.593	0.844	0.260	0.560	0.684

Appendix

Table 7: Detailed comparison of APT with the original base models and a handcrafted RL post-training baseline across four T2I models on GenEval .

Lens
Model	Alignment	Text	Reasoning	Style	Diversity	Average
Base Model	0.840	0.872	0.247	0.387	0.257	0.521
w/ Handcrafted RL Set	0.868	0.949	0.274	0.314	0.184	0.518
w/ APT	0.874	0.947	0.284	0.335	0.174	0.523
SD3.5-Medium
Base Model	0.793	0.341	0.288	0.363	0.254	0.408

Appendix

Table 8: Detailed comparison of APT with the original base models and a handcrafted RL post-training baseline across four T2I models on OneIG-EN .

Lens
Model	Global	Entity	Attribute	Relation	Other	Overall
Base Model	93.47	90.44	91.05	92.16	90.49	85.93
w/ Handcrafted RL Set	89.88	92.69	91.77	93.98	91.90	87.60
w/ APT	93.46	92.94	92.73	92.46	91.73	87.88
SD3.5-Medium
Base Model	92.15	89.67	88.60	91.89	85.19	83.67

Appendix

Table 9: Detailed comparison of APT with the original base models and a handcrafted RL post-training baseline across four T2I models on DPG .

Lens
Model	Word Accuracy	NED	CLIPScore	VQAScore	Aesthetic
Base Model	0.480	0.711	0.785	0.932	4.986
w/ Handcrafted RL Set	0.758	0.878	0.786	0.959	5.926
w/ APT	0.795	0.905	0.784	0.963	5.999
SD3.5-Medium
Base Model	0.312	0.626	0.754	0.891	5.541

Appendix

Table 10: Detailed comparison of APT with the original base models and a handcrafted RL post-training baseline across four T2I models on CVTG-2K .

Data Source	GenEval							OneIG-EN
Data Source	Single	Two	Count	Color	Position	Attribute	Overall	Alignment	Text	Reasoning	Style	Diversity	Average
Target Model: Lens
Lens	1.000	0.960	0.831	0.872	0.768	0.675	0.851	0.871	0.952	0.278	0.338	0.176	0.523
SD3.5-Medium	1.000	0.934	0.813	0.862	0.748	0.640	0.833	0.869	0.941	0.277	0.343	0.181	0.522
FLUX.1 [Dev]	0.997	0.939	0.797	0.875	0.813	0.658	0.846	0.871	0.950	0.276	0.317	0.173	0.517
LongCat-Image	0.997	0.917	0.728	0.819	0.733	0.595	0.798	0.865	0.946	0.270	0.321	0.199	0.520

Appendix

Table 11: Detailed results of APT Captures Model-Specific Training Distributions on GenEval and OneIG-EN . Each block corresponds to a target model post-trained with RL data generated by APT from different source models, while rows denote the source model used for data generation. Shaded rows denote the standard APT setting.

Model	GenEval							OneIG-EN
Model	Single	Two	Count	Color	Position	Attribute	Overall	Alignment	Text	Reasoning	Style	Diversity	Average
Lens
Base Model	0.991	0.788	0.547	0.848	0.548	0.488	0.702	0.840	0.872	0.247	0.387	0.257	0.521
OneIG-EN Evaluation Set	0.997	0.929	0.788	0.912	0.770	0.688	0.847	0.869	0.969	0.292	0.344	0.166	0.528
GenEval Evaluation Set	0.984	0.957	0.900	0.864	0.930	0.685	0.887	0.823	0.908	0.273	0.330	0.193	0.505
APT Evaluation Set	1.000	0.947	0.822	0.872	0.835	0.680	0.859	0.874	0.947	0.284	0.335	0.174	0.523

Appendix

Table 12: Detailed ablation of the evaluation set generation agent on GenEval and OneIG-EN .

Lens
Model	DPG						CVTG-2K
Model	Global	Entity	Attribute	Relation	Other	Overall	Word Accuracy	NED	CLIPScore	VQAScore	Aesthetic
Base Model	93.47	90.44	91.05	92.16	90.49	85.93	0.480	0.711	0.785	0.932	4.986
OneIG-EN Evaluation Set	88.74	93.00	91.94	90.85	93.26	87.39	0.823	0.938	0.814	0.961	5.718
GenEval Evaluation Set	88.46	91.78	89.81	87.63	88.77	84.23	0.674	0.852	0.797	0.941	5.182
APT Evaluation Set	93.46	92.94	92.73	92.46	91.73	87.88	0.795	0.905	0.784	0.963	5.999

Appendix

Table 13: Detailed ablation of the evaluation set generation agent on DPG and CVTG-2K .

Setting	GenEval							OneIG-EN
Setting	Single	Two	Count	Color	Position	Attribute	Overall	Alignment	Text	Reasoning	Style	Diversity	Average
APT	1.000	0.947	0.822	0.872	0.835	0.680	0.859	0.874	0.947	0.284	0.335	0.174	0.523
Ablation on the Iterative Feedback Loop
w/o Iterative Feedback Loop	0.982	0.955	0.800	0.874	0.778	0.676	0.844	0.866	0.957	0.286	0.328	0.176	0.523
Ablation on the RL Data Generation Agent
Reduced Diagnostic Criteria	1.000	0.886	0.750	0.878	0.668	0.618	0.800	0.871	0.939	0.281	0.330	0.180	0.520

Appendix

Table 14: Detailed ablation of the Iterative Feedback Loop and the RL Data Generation Agent on GenEval and OneIG-EN .

Ablation on the Iterative Feedback Loop
Setting	DPG						CVTG-2K
Setting	Global	Entity	Attribute	Relation	Other	Overall	Word Accuracy	NED	CLIPScore	VQAScore	Aesthetic
APT	93.46	92.94	92.73	92.46	91.73	87.88	0.795	0.905	0.784	0.963	5.999
w/o Iterative Feedback Loop	94.46	91.75	92.38	93.49	88.65	87.61	0.791	0.912	0.807	0.962	5.494
Ablation on the RL Data Generation Agent
Reduced Diagnostic Criteria	93.77	91.74	92.13	89.53	94.54	87.53	0.763	0.900	0.796	0.963	5.769

Appendix

Table 15: Detailed ablation of the Iterative Feedback Loop and the RL Data Generation Agent on DPG and CVTG-2K .

Figure 3: Qualitative comparison of the base model, handcrafted RL baseline, and APT on Lens .

Figure 4: Additional qualitative comparison of the base model, handcrafted RL baseline, and APT on Lens .

Figure 5: Qualitative comparison of the base model, handcrafted RL baseline, and APT on SD3.5-Medium .

Figure 6: Qualitative comparison of the base model, handcrafted RL baseline, and APT on FLUX.1[Dev] .

Figure 7: Qualitative comparison of the base model, handcrafted RL baseline, and APT on LongCat-Image .

T2I Generation

Text-to-image generation creates images from natural-language prompts using generative models, with research focused on fidelity, control, efficiency, and safety.

See all 290 papers

Post-Training Frontier Text-to-Image Models by Composing Preference and Rubric Rewards
Oct 2, 2026Yuanhao Ban, I-Hung Hsu, Anastasios Angelopoulos +3T2I Generation Reward Design for RL
Arena Intelligence Inc · UCLA
Improving Text-to-Image Generation with Intrinsic Self-Confidence Rewards
Mar 1, 2026Seungwook Kim, Minsu ChoT2I Diffusion Models RL for Image Generation
POSTECH · GenGenAI · RLWRLD
MemoGen: Can Past Experience Improve Future Text-to-Image Generation?
Jun 2, 2026Wenshuo Chen, Kuimou Yu, Bowen Tian +10T2I Generation Image Generation
The Hong Kong University of Science and Technology (Guangzhou) · Scholarly, Guangzhou Ziyan Technology Co., Ltd. · LimX Dynamics Technology Co., Ltd. +4

Oct 2, 2026J/K move · Enter open · S save

Post-Training Frontier Text-to-Image Models by Composing Preference and Rubric Rewards

Yuanhao Ban, I-Hung Hsu, Anastasios Angelopoulos, Wei-Lin Chiang+2

Arena Intelligence Inc · UCLA

T2I Generation Reward Design for RL RL for Image Generation Reward Modeling T2I Generation Evaluation

Open paper AskPDF arXiv

Recent text-to-image generation models have achieved remarkable visual quality, but improving them through post-training remains challenging because no single reward signal captures the full range of human preference. In this work, we develop a simple and effective post-training recipe for open-domain text-to-image generation based on the composition of complementary reward signals. Our reward system consists of two main components: a preference reward, trained on large-scale human preference data using a Bradley-Terry objective to capture overall human aesthetic and perceptual preferences, and rubric-based rewards, which explicitly evaluate prompt faithfulness and other desirable properties while providing safeguards against reward hacking. A key challenge is how to combine these heterogeneous reward signals. We show that a naive weighted average leads to suboptimal optimization behavior, and propose a simple reward composition strategy that more effectively balances preference optimization with rubric satisfaction. In the Arena text-to-image leaderboard (https://arena.ai/), our RL-trained Flux2dev achieves an Elo rating 69 points above the base model, and our post-trained Ideogram-4 surpasses every open-source model on the leaderboard, reaching an Elo of 1223.5. (Claims of state-of-the-art performance are based on the Arena leaderboard snapshot as of September 4, 2026.) Our results suggest that effective rewards for frontier generative-model training require broad coverage of user intent and robustness to exploitation under optimization. To support reproducible research, we release Arena-T2I-Training, a 1K subset of training data that recovers some gains of full-scale training, providing a resource that we hope will facilitate future work on post-training for text-to-image models.

Abstract

T2I Generation

Explore similar work

Post-Training Frontier Text-to-Image Models by Composing Preference and Rubric Rewards

Improving Text-to-Image Generation with Intrinsic Self-Confidence Rewards

MemoGen: Can Past Experience Improve Future Text-to-Image Generation?