Enhancing Spatial Understanding in Image Generation via Reward Modeling

Zhenyu Tang; Chaoran Feng; Yufan Deng; Jie Wu; Xiaojie Li; Rui Wang; Yunpeng Chen; Daquan Zhou

cs.CVFeb 27, 2026

Enhancing Spatial Understanding in Image Generation via Reward Modeling

Authors: Zhenyu Tang, Chaoran Feng, Yufan Deng, Jie Wu, Xiaojie Li, Rui Wang, Yunpeng Chen, Daquan Zhou

Organizations: Peking University · ByteDance Seed

Abstract

Recent progress in text-to-image generation has greatly advanced visual fidelity and creativity, but it has also imposed higher demands on prompt complexity-particularly in encoding intricate spatial relationships. In such cases, achieving satisfactory results often requires multiple sampling attempts. To address this challenge, we introduce a novel method that strengthens the spatial understanding of current image generation models. We first construct the SpatialReward-Dataset with over 80k preference pairs. Building on this dataset, we build SpatialScore, a reward model designed to evaluate the accuracy of spatial relationships in text-to-image generation, achieving performance that even surpasses leading proprietary models on spatial evaluation. We further demonstrate that this reward model effectively enables online reinforcement learning for the complex spatial generation. Extensive experiments across multiple benchmarks show that our specialized reward model yields significant and consistent gains in spatial understanding for image generation.

Figures & tables

Figure 1: Failure of Reward Models on Spatial Understanding. Existing reward models [ 29 , 17 , 53 , 23 ] often assign higher reward values to spatially incorrect images than to spatially correct ones, thereby exposing their limited spatial reasoning capabilities.

Figure 2: Limitations of GenEval [ 8 ] as the reward model. (a) GenEval-based RL training fails to generalize to long prompts involving complex spatial relationships across multiple objects. (b) The rule-based GenEval rewards, which rely on object detectors, often produce incorrect evaluations under visual challenges like occlusion, while modern VLMs can accurately infer the correct response.

Figure 3: Overview of our SpatialReward-Dataset.

Figure 4: GRPO training pipeline for enhancing spatial unserstanding. We first samples a group of images from the policy model and uses our specialized SpatialScore to rate their spatial accuracy. After ranking based on these scores, we select the top- $k$ most accurate and bottom- $k$ least accurate examples and convert these scores into advantage signals. The policy model is updated via policy gradient optimization to directly reward correct spatial layouts and penalize errors, thereby enhancing the base model’s spatial understanding.

Figure 5: Advantage bias. For easy prompts with many high-reward samples, some high-quality samples often obtain negative advantages due to the high group mean.

Setting	Image Reward Models						Qwen2.5-VL Series			Proprietary Models		SpatiaScore
Setting	Image Reward	Pickscore	HPSv2.1	VQAScore	Unified Reward	HPSv3	7B	32B	72B	GPT-5	Gemini- 2.5 pro	SpatiaScore
1 Pert.	0.439	0.461	0.433	0.567	0.583	0.606	0.572	0.644	0.711	0.855	0.933	0.939
2–3 Pert.	0.513	0.551	0.491	0.638	0.627	0.697	0.632	0.724	0.816	0.924	0.968	0.978
Overall	0.479	0.509	0.463	0.603	0.605	0.652	0.602	0.685	0.764	0.890	0.951	0.958

Table 1: Pairwise-accuracy comparisons on the reward evaluation benchmark. “1 Pert.” and “2–3 Pert.” denote subsets with one or two–three spatial perturbations applied to the perfect prompts when constructing the perturbed prompts.

Method	SpatialScore	DPG-bench	TIIF-bench-short			TIIF-bench-long			Unibench(short)		Unibench(long)
Method	SpatialScore	Relation-Spatial	BR	AR	RR	BR	AR	RR	Lay-2D	Lay-3D	Lay-2D	Lay-3D
Flux.1-dev [ 18 ]	2.18	0.871	0.769	0.608	0.584	0.758	0.677	0.645	0.766	0.667	0.819	0.742
Flow-GRPO* [ 25 ]	3.01	0.742	0.851	0.652	0.621	0.577	0.510	0.482	0.726	0.635	0.445	0.405
Ours	7.81	0.932	0.875	0.700	0.647	0.845	0.715	0.675	0.875	0.773	0.891	0.801

Table 2: Detailed comparisons on SpatialScore, DPG-Bench, TIIF-Bench (short/long), and UnigenBench++ (short/long). * denotes training with Geneval as the reward model. BR, AR, and RR denote basic relation, attribute+relation, and relation+reasoning. Lay-2D/3D refer to layout-2D/3D. Unibench denotes UniGenBench++.

Figure 6: Qualitative comparison on prompts with complex spatial relationships across multiple objects.

Figure 7: Reward training curves for ablations on top-

k

filtering

Method	Global	Entity	Attribute	Relation	Other	Overall
GPT-Image-1	88.89	88.94	89.84	92.63	90.96	85.15
Flux.1-dev	84.70	87.29	89.29	89.44	89.46	82.91
Flow-GRPO*	69.20	72.37	69.23	73.26	69.72	57.02
Ours	87.65	90.56	91.20	91.58	86.43	85.03

Table 3: Full-dimensional comparison on DPG-Bench. The best Flux variants are in bold , and * denotes training with GenEval.

Setting	SpatialScore	DPG-bench	Unigenbench++ (long)		NFE
Setting	SpatialScore	Rel-Spatial	Layout-2D	Layout-3D	NFE
w/o top- $k$	7.73	0.919	0.891	0.793	24*6
w/ top- $k$ (k=4)	7.71	0.916	0.882	0.796	8*6
w/ top- $k$ (k=6)	7.81	0.932	0.887	0.801	12*6

Table 4: Ablations on top-

k

filtering. NFE per prompt for each training step is reported under a denoising step count of 6.

Figure 8: Distribution statistics of SpatialReward-Dataset.

Figure 9: Visualization of the preference pairs (perfect images and perturbed images) in our SpatialReward-Dataset.

Figure 10: More visualizations of the preference pairs (perfect images and perturbed images) in our SpatialReward-Dataset.

Methods	Single object	Two object	Counting	Colors	Position	Color Attribute	Overall
Flux.1-dev	0.99	0.81	0.70	0.76	0.19	0.45	0.65
Ours	1.00	0.92	0.88	0.79	0.37	0.66	0.78

Table 5: Quantitative evaluations on the Geneval benchmark. Our model is trained using SpatialScore as the reward model.

Setting\backbone	Qwen2.5-VL-3B	Qwen2.5-VL-7B	Qwen2.5-VL-32B
1 Pert.	0.861	0.939	0.955
2–3 Pert.	0.919	0.978	0.989
Overall	0.891	0.958	0.973

Table 6: Ablation study of SpatialScore backbone sizes on the reward evaluation benchmark. “1 Pert.” and “2–3 Pert.” denote subsets constructed by applying one or two–three spatial perturbations, respectively, to perfect prompts for perturbed prompts.

Method	SpatialScore	DPG-bench	TIIF-bench-short			TIIF-bench-long			Unibench(short)		Unibench(long)
Method	SpatialScore	Relation-Spatial	BR	AR	RR	BR	AR	RR	Lay-2D	Lay-3D	Lay-2D	Lay-3D
Qwen-Image [ 51 ]	6.74	0.920	0.865	0.756	0.704	0.827	0.751	0.716	0.864	0.852	0.912	0.860
Ours*	8.25	0.958	0.899	0.792	0.791	0.871	0.801	0.780	0.908	0.917	0.926	0.893

Table 7: Detailed comparisons for the Qwen-Image family on SpatialScore, DPG-Bench, TIIF-Bench (short/long), and UnigenBench++ (short/long). * denotes RL-training with our SpatialScore as the reward model. BR, AR, and RR denote basic relation, attribute+relation, and relation+reasoning. Lay-2D/3D refer to layout-2D/3D. Unibench denotes UnigenBench++.

Figure 11: More qualitative comparisons of the long prompts with complex spatial relationships across objects.

T2I Generation

Text-to-image generation creates images from natural-language prompts using generative models, with research focused on fidelity, control, efficiency, and safety.

See all 290 papers

SpatialReward: Verifiable Spatial Reward Modeling for Fine-Grained Spatial Consistency in Text-to-Image Generation
Mar 23, 2026Sashuai Zhou, Qiang Zhou, Junpeng Ma +9Reward Modeling T2I Generation
Zhejiang University · Alibaba Group · Fudan University
Getting Space Right: Agentic Rewards for Image Generation
Oct 10, 2026Xiaofeng Tan, Jiaxin Lin, Haobo Yuan +5T2I Generation RL for Image Generation
Southeast University · Tencent HY · UC Merced +1
Show, Don't Tell: Evaluating Spatial Cognition in Generative Pixels Rather Than LLM Text
Jul 23, 2026Xu Wang, Kaixiang Yao, Miao Pan +4Spatial Reasoning Benchmarks Visual Spatial Reasoning
OmniAI Group of ZJU ACES Lab

Mar 23, 2026J/K move · Enter open · S save

SpatialReward: Verifiable Spatial Reward Modeling for Fine-Grained Spatial Consistency in Text-to-Image Generation

Sashuai Zhou, Qiang Zhou, Junpeng Ma, Yue Cao+8

Zhejiang University · Alibaba Group · Fudan University

Reward Modeling T2I Generation T2I Generation Evaluation RL for Image Generation

Open paper AskPDF arXiv

Recent advances in text-to-image (T2I) generation via reinforcement learning (RL) have benefited from reward models that assess semantic alignment and visual quality. However, most existing reward models pay limited attention to fine-grained spatial relationships, often producing images that appear plausible overall yet contain inaccuracies in object positioning. In this work, we present SpatialReward, a verifiable reward model explicitly designed to evaluate spatial layouts in generated images. SpatialReward adopts a multi-stage pipeline: a Prompt Decomposer extracts entities, attributes, and spatial metadata from free-form prompts; expert detectors provide accurate visual grounding of object positions and attributes; and a vision-language model applies chain-of-thought reasoning over grounded observations to assess complex spatial relations that are challenging for rule-based methods. To more comprehensively evaluate spatial relationships in generated images, we introduce SpatRelBench, a benchmark covering object attributes, orientation, inter-object relations, and rendered text placement. Experiments on Stable Diffusion and FLUX show that incorporating SpatialReward into RL training consistently improves spatial consistency and overall generation quality, with results aligned more closely to human judgments. These findings indicate that verifiable reward models hold considerable potential for enabling more accurate and controllable optimization in text-to-image generation models.

Abstract

T2I Generation

Explore similar work

SpatialReward: Verifiable Spatial Reward Modeling for Fine-Grained Spatial Consistency in Text-to-Image Generation

Getting Space Right: Agentic Rewards for Image Generation

Show, Don't Tell: Evaluating Spatial Cognition in Generative Pixels Rather Than LLM Text