Self-Corrected Image Generation with Explainable Latent Rewards

Yinyi Luo; Hrishikesh Gokhale; Marios Savvides; Jindong Wang; Shengfeng He

cs.CVMar 26, 2026

Self-Corrected Image Generation with Explainable Latent Rewards

Authors: Yinyi Luo, Hrishikesh Gokhale, Marios Savvides, Jindong Wang, Shengfeng He

Organizations: Carnegie Mellon University · Singapore Management University · William & Mary

Abstract

Despite significant progress in text-to-image generation, aligning outputs with complex prompts remains challenging, particularly for fine-grained semantics and spatial relations. This difficulty stems from the feed-forward nature of generation, which requires anticipating alignment without fully understanding the output. In contrast, evaluating generated images is more tractable. Motivated by this asymmetry, we propose xLARD, a self-correcting framework that uses multimodal large language models to guide generation through Explainable LAtent RewarDs. xLARD introduces a lightweight corrector that refines latent representations based on structured feedback from model-generated references. A key component is a differentiable mapping from latent edits to interpretable reward signals, enabling continuous latent-level guidance from non-differentiable image-level evaluations. This mechanism allows the model to understand, assess, and correct itself during generation. Experiments across diverse generation and editing tasks show that xLARD improves semantic alignment and visual fidelity while maintaining generative priors. Code is available at https://yinyiluo.github.io/xLARD/.

Figures & tables

Figure 2: Overview of the xLARD framework. Given a prompt $p$ , the frozen backbone encodes it into a latent representation $z_{0}$ . The residual corrector $\Delta_{\theta}$ refines $z_{0}$ under multi-dimensional reward guidance, producing a corrected latent $z_{c}$ that is decoded into an image $\hat{x}$ . Image-level rewards are projected back to the latent space via a learnable reward projector $R_{\phi}$ , allowing end-to-end, interpretable correction learning. During inference, URC functions as a lightweight latent modifier with no additional sampling or retraining.

Panel of Figure 3 — Figure 3: Qualitative comparison of image generation/editing performance between HermesFlow and our proposed approach.

Method	Type	Params	DPG-Bench	GenEval
OmniGen	AR+Diffusion	7B	81.16	0.68
Show-O	AR+Diffusion	1.3B	67.27	0.68
BAGEL *	AR+RAG	14B	84.07	0.79
FLUX-dev	Diffusion	12B	84.00	0.68
UniWorld-V1	AR+Diffusion	20B	81.38	0.80
Emu3	AR	8B	80.60	0.54

Table 1: Comparison of generative and editing performance across benchmarks. * : Results are re-generated using the official pre-trained models.

Method	Single Obj	Two Obj	Cnt	Colors	Position	Color Attr	Overall
OmniGen2	100	92.93	69.12	85.88	45.52	68.75	77.03
OmniGen2 + Ours	100	93.69	78.44	92.11	48.75	73.75	81.29
Bagel	99.12	92.34	77.45	88.21	50.89	63.78	78.97
Bagel + Ours	99.85	93.95	80.55	87.33	60.72	65.89	81.38
Show-O	98.21	80.35	65.89	84.12	30.78	49.95	68.05
Show-O + Ours	98.95	90.88	74.56	85.72	41.87	55.34	74.89

Table 2: GenEval detailed evaluation metrics (%) across different baselines and backbones.

Method	Global	Entity	Attribute	Relation	Other	Overall
OmniGen2	82.07	90.22	87.82	93.26	82.26	83.48
OmniGen2 + Ours	82.76	92.41	89.90	94.07	86.60	86.45
Bagel	80.46	90.28	87.48	92.42	83.40	84.07
Bagel+Ours	81.16	91.57	88.84	93.93	85.60	85.50
Show-O	79.33	75.44	78.02	84.45	60.80	67.27
Show-O+Ours	85.11	81.69	84.17	90.44	67.21	72.92

Table 3: DPG-Bench detailed evaluation metrics (%) across different baselines and backbones.

	ImgEdit										GEdit
Method	Rep.	Bg.	Style	Adj.	Ext.	Rem.	Rep.	Add	Cmp.	Act.	SC	PQ	Overall
OmniGen2	2.82	3.50	3.66	3.15	3.03	2.27	2.82	3.50	2.43	3.87	5.05	5.53	4.46
Ours	3.04	3.62	3.57	3.20	3.17	2.69	3.04	3.62	2.36	4.22	5.38	6.77	5.52

Table 4: Comparison of ImgEdit and GEdit performance.

Figure 4: Token-level contributions for misalignment detection. Positive bars indicate tokens aligned with the image, negative bars indicate tokens driving residual corrections.

Panel of Figure 5 — Figure 5: Visualization of latent residual corrections. The high-intensity regions in the correction map indicate where the residual module most strongly adjusts the latent features. The prompt used for this example is “A skateboarder performing a jump mid-air above a concrete ramp, another person watching from the left.”

Figure 6: Illustration of improvements introduced by xLARD. From left to right: (1) Aesthetic composition. Objects are placed according to the prompt to produce visually coherent layouts; (2) Color enhancement. Colors are adjusted to better match the described scene; (3) Detail refinement. Small details such as textures and secondary objects are corrected for higher fidelity.

Method Variant	GenEval (%)	DPG-Bench (%)
RL(CLIPScore)	78.78	84.15
RL(Sentence-BERT)	78.42	84.23
Without RL	77.68	83.84
Without Confidence Map	77.94	84.21
Without Latent Anchor	76.90	83.56

Table 5: Ablation study results across benchmarks.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Qualitative results for text to image generation.

Panel of Figure 8 — Figure 8: Qualitative editing results-1.

Panel of Figure 9 — Figure 9: Qualitative editing results-2.

Panel of Figure 10 — Figure 10: Qualitative editing results-3.

Method Variant	GenEval (%)	DPG-Bench (%)
Full xLARD Framework	81.24	86.72
w/o Counting Reward	76.45 ( $-$ 4.79)	83.10 ( $-$ 3.62)
w/o Color Reward	78.92 ( $-$ 2.32)	84.55 ( $-$ 2.17)
w/o Position Reward	79.15 ( $-$ 2.09)	84.88 ( $-$ 1.84)

Appendix

Table 6: Ablation study of reward components.

Image Generation

Image generation studies models that create or edit images from text, examples, layouts, and other structured controls.

See all 463 papers

Large Language Models are Universal Reasoners for Visual Generation
May 5, 2026Sucheng Ren, Chen Chen, Zhenbang Wang +5Diffusion Model Guidance Image Generation
1Johns Hopkins University · 2Apple
Trust Your Critic: Robust Reward Modeling and Reinforcement Learning for Faithful Image Editing and Generation
Mar 12, 2026Xiangyu Zhao, Peiyuan Zhang, Junming Lin +7Reward Modeling RL for Image Editing
Shanghai Jiao Tong University · Wuhan University · BUPT +3
PRISM: Prompt Refinement via Image-grounded Self-rewarding Mechanism for Text-to-Image Generation
Jul 27, 2026Guo Tang, HongJie Luo, Tianxu Wang +2T2I Generation RL for Image Generation
Harbin Institute of Technology, Shenzhen · Sun Yat-sen University · South China University of Technology +1

May 5, 2026J/K move · Enter open · S save

Large Language Models are Universal Reasoners for Visual Generation

Sucheng Ren, Chen Chen, Zhenbang Wang, Liangchen Song+4

1Johns Hopkins University · 2Apple

Diffusion Model Guidance Image Generation Compositional T2I Generation VLM Reasoning

Open paper AskPDF arXiv

Text-to-image generation has advanced rapidly with diffusion models, progressing from CLIP and T5 conditioning to unified systems where a single LLM backbone handles both visual understanding and generation. Despite the architectural unification, these systems frequently fail to faithfully align complex prompts during synthesis, even though they remain highly accurate at verifying whether an image satisfies those same prompts. We formalize this as the understanding-generation gap and propose UniReasoner, a framework that leverages the LLM as a universal reasoner to convert its understanding strength into direct generation guidance. Given a prompt, the LLM first produces a coarse visual draft composed of discrete vision tokens. It then performs a self-critique by evaluating the draft for prompt consistency, producing a grounded textual evaluation that pinpoints what needs to be corrected. Finally, a diffusion model is conditioned jointly on the prompt, the visual draft, and the evaluation, ensuring that generation is guided by explicit corrective signals. Each signal addresses a limitation of the other: the draft provides a concrete, scene-level anchor that reduces under-specification in text-only conditioning, while the evaluation turns verification into grounded, actionable constraints that correct omissions, hallucinations, and relational errors. Experiments show that UniReasoner improves compositional alignment and semantic faithfulness under the same diffusion backbone while maintaining image quality, demonstrating a practical way to exploit LLM reasoning to close the understanding-generation gap.

Abstract

Image Generation

Explore similar work

Large Language Models are Universal Reasoners for Visual Generation

Trust Your Critic: Robust Reward Modeling and Reinforcement Learning for Faithful Image Editing and Generation

PRISM: Prompt Refinement via Image-grounded Self-rewarding Mechanism for Text-to-Image Generation