RewardFlow: Generate Images by Optimizing What You Reward

Onkar Susladkar; Dong-Hwan Jang; Tushar Prakash; Adheesh Juvekar; Vedant Shah; Ayush Barik; Nabeel Bashir; Muntasir Wahed

cs.CVApr 9, 2026

RewardFlow: Generate Images by Optimizing What You Reward

Authors: Onkar Susladkar, Dong-Hwan Jang, Tushar Prakash, Adheesh Juvekar, Vedant Shah, Ayush Barik, Nabeel Bashir, Muntasir Wahed,

Organizations: University of Illinois Urbana-Champaign · Sony Research, India

Abstract

We introduce RewardFlow, an inversion-free framework that steers pretrained diffusion and flow-matching models at inference time through multi-reward Langevin dynamics. RewardFlow unifies complementary differentiable rewards for semantic alignment, perceptual fidelity, localized grounding, object consistency, and human preference, and further introduces a differentiable VQA-based reward that provides fine-grained semantic supervision through language-vision reasoning. To coordinate these heterogeneous objectives, we design a prompt-aware adaptive policy that extracts semantic primitives from the instruction, infers edit intent, and dynamically modulates reward weights and step sizes throughout sampling. Across several image editing and compositional generation benchmarks, RewardFlow delivers state-of-the-art edit fidelity and compositional alignment.

Figures & tables

Figure 2: Gradient localization of our differentiable rewards. We visualize the image-space gradient $\nabla_{I}R_{tot}(\cdot)$ for various edit prompts. Our proposed rewards prevent semantic leakage by concentrating the gradient precisely on target semantic regions, demonstrating the fine-grained spatial control enabled by RewardFlow.

Method	Distance $\downarrow$ ( $\times 10^{3}$ )	PSNR $\uparrow$	LPIPS $\downarrow$ ( $\times 10^{3}$ )	MSE $\downarrow$ ( $\times 10^{4}$ )	SSIM $\uparrow$ ( $\times 10^{2}$ )	Whole $\uparrow$	Edited $\uparrow$	NFE $\downarrow$	Step $\downarrow$
EF [ 18 ]	8.39	27.49	44.38	29.79	85.61	25.87	22.14	70	50
ProxG [ 14 ]	8.39	28.45	38.27	25.63	85.87	25.04	21.64	100	50
P2P [ 15 ]	9.58	27.72	44.98	30.02	85.01	24.94	21.57	100	50
DI [ 19 ]	11.60	27.25	49.25	32.87	84.86	25.83	22.39	100	50
AREdit [ 41 ]	13.12	28.78	42.67	37.88	87.19	26.57	24.51	16	30
InfEdit [ 46 ]	13.87	28.63	39.80	33.19	86.28	25.84	22.44	12	12

Table 1: PIE-Bench image editing results. RewardFlow consistently improves edit fidelity and spatial localization across all metrics while maintaining competitive runtime. Best results are in bold and strong baselines are underlined .

Figure 3: Image editing qualitative comparison across diverse instruction types. RewardFlow produces edits that are both semantically accurate and spatially localized, while better preserving background structure, lighting, and identity compared to prior methods.

Model	Color	Shape	Texture	Spatial	Non-Spatial	Complex
SD v1.4 [ 36 ]	0.38	0.36	0.42	0.12	0.31	0.31
SD v2.1 [ 36 ]	0.51	0.42	0.49	0.13	0.31	0.34
SDXL [ 33 ]	0.64	0.54	0.56	0.20	0.31	0.41
PixArt- $\alpha$ [ 7 ]	0.69	0.56	0.70	0.21	0.32	0.41
DALL-E 2 [ 29 ]	0.57	0.55	0.64	0.13	0.30	0.37
DALL-E 3 [ 30 ]	0.81	0.68	0.81	–	–	–

Table 2: T2I compositional generation on T2I-CompBench . Accuracy across fine-grained attribute binding (color, shape, texture), object relationships (spatial and non-spatial), and complex compositions. RewardFlow consistently improves all base models (PixArt-

\alpha

, Flux, and Qwen Image). Best results are in bold .

Setting / Variant	Distance $\downarrow$ ( $\times 10^{3}$ )	PSNR $\uparrow$	LPIPS $\downarrow$ ( $\times 10^{3}$ )	MSE $\downarrow$ ( $\times 10^{4}$ )	SSIM $\uparrow$ ( $\times 10^{2}$ )	Whole $\uparrow$	Edited $\uparrow$
✗ Dynamic Reward Weighting.	8.47	30.77	38.78	34.67	89.37	29.02	27.01
✗ Semantic Primitives (SPs)	9.03	31.19	39.12	35.01	88.47	27.45	26.51
✗ Reward-Aware Step Size	9.15	31.38	39.56	35.21	89.12	28.32	26.92
✗ KL-Tether	9.56	29.98	40.26	35.28	87.23	27.98	26.13
Reward Flow (Full)	7.64	32.09	38.47	33.57	90.21	29.78	27.57

Table 3: Ablation on key RewardFlow components.

Figure 4: Text-to-image qualitative results. Across all prompts, RewardFlow produces images that exhibit higher alignment with the textual descriptions while also generating outputs with more visually appealing composition and aesthetics.

Figure 5: Gradient localization across reward combinations. Including all rewards concentrates gradients to accurate object contours and eliminates leakage.

Figure 6: Effect of removing reward components. ( ✗ RC, SAM, LLM, HPS, and PE) denote excluding ( $R_{\text{rg}}$ , $R_{\text{oc}}$ , $R_{\text{vqa}}$ , $R_{\text{hps}}$ , and $R_{\text{per}}$ ), respectively. RewardFlow (all rewards) achieves semantically precise edits, modifying only instruction-relevant content while maintaining background and context integrity.

Figure 7: Ablations illustrating the effect of removing key components. RewardFlow (all components) achieves the best visual consistency and instruction alignment.

$R_{\text{glb}}$	$R_{\text{per}}$	$R_{\text{hps}}$	$R_{\text{oc}}$	$R_{\text{rg}}$	$R_{\text{vqa}}$	Distance $\downarrow$ ( $\times 10^{3}$ )	PSNR $\uparrow$	LPIPS $\downarrow$ ( $\times 10^{3}$ )	MSE $\downarrow$ ( $\times 10^{4}$ )	SSIM $\uparrow$ ( $\times 10^{2}$ )	Whole $\uparrow$	Edited $\uparrow$
✓	✗	✗	✗	✗	✗	11.23	26.33	45.22	39.34	84.09	19.33	21.22
✓	✓	✗	✗	✗	✗	10.21	27.57	43.12	37.45	85.17	19.88	22.85
✓	✓	✓	✗	✗	✗	9.77	28.82	41.67	37.12	86.39	20.12	23.33
✓	✓	✓	✓	✗	✗	8.39	29.44	40.58	36.55	87.71	21.47	24.75
✓	✓	✓	✓	✓	✗	8.01	30.12	40.02	35.28	88.92	26.47	25.91
✓	✓	✓	✓	✓	✓	7.64	32.09	38.47	33.57	90.21	29.78	27.57

Table 4: Ablation on reward components. Each column indicates whether the corresponding reward is enabled ( ✓ ) or disabled ( ✗ ).

Figure 9: Overview of the RewardFlow framework.

Panel of Figure 10 — Figure 10: Prompt template used for semantic primitives and $\mathcal{R}_{\text{vqa}}$ .

Figure 11: Reward progression over time.

Model	Overall $\uparrow$	Single $\uparrow$	Two $\uparrow$	Counting $\uparrow$	Colors $\uparrow$	Position $\uparrow$	Color Attribution $\uparrow$
SD v2.1	0.50	0.98	0.51	0.44	0.85	0.07	0.17
SDXL	0.55	0.98	0.74	0.39	0.85	0.15	0.23
IF-XL	0.61	0.97	0.74	0.66	0.81	0.13	0.35
PixArt- $\alpha$	0.48	0.98	0.50	0.44	0.80	0.08	0.07
DALL-E 2	0.52	0.94	0.66	0.49	0.77	0.10	0.19
DALL-E 3	0.67	0.96	0.87	0.47	0.83	0.43	0.45

Table 5: T2I generation on GenEval .

Figure 12: Image Editing Qualitative Results with Flux + RewardFlow . For each input image on the left, RewardFlow is instructed to apply a targeted edit (text below), and the right image shows the generated result. Tasks span from global scene modifications and object-level edits to very fine-grained, localized edits.

Figure 13: Image Editing Qualitative Results with Qwen Image + RewardFlow . For each input image on the left, RewardFlow is instructed to apply a targeted edit (text below), and the right image shows the generated result. Tasks span from global scene modifications and object-level edits to very fine-grained, localized edits.

Figure 14: Image Editing Qualitative Comparisons. Comparison across a range of challenging edits, such as diverse attribute-, style-, and object-level transformations. Each row shows the source image followed by results from strong baselines and RewardFlow.

VLMs	Distance $\downarrow$ ( $\times 10^{3}$ )	PSNR $\uparrow$	LPIPS $\downarrow$ ( $\times 10^{3}$ )	MSE $\downarrow$ ( $\times 10^{4}$ )	SSIM $\uparrow$ ( $\times 10^{2}$ )	Whole $\uparrow$	Edited $\uparrow$
Qwen 2.5VL 3B $\dagger$	7.64	32.09	38.47	33.57	90.21	29.78	27.57
LLaMa-4-8B	6.57	33.43	37.19	31.31	91.33	30.44	28.82
Qwen 3 Next-34B	6.53	32.34	38.05	32.76	91.49	31.01	29.03

Table 6: VLM Comparison on PIE-Bench .

Figure 15: RewardFlow counting failure case.

Figure 16: Text-to-image qualitative results with Flux as backbone. Qualitative comparison of Flux, Flux + Global Reward, and Flux + RewardFlow across diverse prompts.

Diffusion Model Guidance

Diffusion model guidance steers denoising toward desired conditions, rewards, constraints, or away from unwanted concepts.

See all 364 papers

How to Guide Your Flow: Few-Step Alignment via Flow Map Reward Guidance
Apr 29, 2026Jerry Y. Huang, Justin Lin, Sheel Shah +2Generative Modeling Flow-Based Generative Modeling
Carnegie Mellon University
Rectify Then Diffuse: Disentangling Concepts Before Denoising Trajectory Unfolds
Aug 4, 2026Ning Zhu, An Chen, Mengfei Zhao +4T2I Generation Attention Control in Diffusion Models
Glasgow College, University of Electronic Science and Technology of China · School of Mathematical Sciences, University of Electronic Science and Technology of China
Qwen-Image-2.0-RL Technical Report
Jun 25, 2026Yixian Xu, Kaiyuan Gao, Yuxiang Chen +25Reward Modeling RL for Diffusion Models
Equal contribution. · Corresponding author.

Apr 29, 2026J/K move · Enter open · S save

How to Guide Your Flow: Few-Step Alignment via Flow Map Reward Guidance

Jerry Y. Huang, Justin Lin, Sheel Shah, Kartik Nair+1

Carnegie Mellon University

Generative Modeling Flow-Based Generative Modeling Conditional Generative Modeling

Open paper AskPDF arXiv

In generative modeling, we often wish to produce samples that maximize a user-specified reward such as aesthetic quality or alignment with human preferences, a problem known as guidance. Despite their widespread use, existing guidance methods either require expensive multi-particle, many-step schemes or rely on poorly understood approximations. We reformulate guidance as a deterministic optimal control problem, yielding a hierarchy of algorithms that subsumes existing approaches at the coarsest level. We show that the flow map, an object of significant recent interest for its role in fast inference, arises naturally in the optimal solution. Based on this observation, we propose Flow Map Reward Guidance (FMRG): a training-free, single-trajectory framework that uses the flow map to both integrate and guide the flow. At text-to-image scale, FMRG matches or surpasses baselines across inverse problems and reward-guided generation with as few as 3 NFEs, giving at least an order-of-magnitude speedup in comparison to prior state of the art. Code is available at https://github.com/jrrhuang/fmrg.

Abstract

Diffusion Model Guidance

Explore similar work

How to Guide Your Flow: Few-Step Alignment via Flow Map Reward Guidance

Rectify Then Diffuse: Disentangling Concepts Before Denoising Trajectory Unfolds

Qwen-Image-2.0-RL Technical Report