From Structural Feedback to Prompt Policies: Learning Faithful Text-to-Image Prompt Editors

Wentao Ye; Yali Ye; Zhiqing Xiao; Ru Peng; Xinpeng Ti; Liyao Li; Zhanming Shen; Peng Lu

cs.AIOct 26, 2026

From Structural Feedback to Prompt Policies: Learning Faithful Text-to-Image Prompt Editors

Authors: Wentao Ye, Yali Ye, Zhiqing Xiao, Ru Peng, Xinpeng Ti, Liyao Li, Zhanming Shen, Peng Lu,

Organizations: Zhejiang University · Tencent AI Lab · China Telecom Artificial Intelligence (Beijing) Co. Ltd

Abstract

Prompt optimization should improve the image a user receives without changing the scene the user requested. Yet stylistic rewriting can weaken object attributes and relations even when global text--image similarity remains high. We propose StruPO, a structure-aware prompt editor that uses a calibrated scene graph as a shared representation for policy conditioning and image-grounded supervision. The graph exposes what should be preserved, while deletion, reordering, and insertion provide a token-aligned way to modify the prompt. This turns structural feedback from an external critique into a training signal for a reusable editing policy. Learning from that signal requires handling both uncertain grounding and stochastic image generation. StruPO retains grounding confidence through smooth reward shaping and averages multiple images for each edit before computing group-relative advantages. Experiments span three disjoint 2K-prompt test sets and four frozen generation backbones, with comparisons to learned prompt optimizers, iterative structural feedback, diffusion-model fine-tuning, and inference-time attention control. StruPO improves the faithfulness--aesthetics trade-off, with consistent gains in relational fidelity. Independent automatic evaluators and a 15-annotator study provide complementary evidence, and human preference margins increase with relation complexity. Controlled ablations connect these gains to graph conditioning, local editing, and image-grounded learning. The resulting policy produces reusable optimized text in one pass after graph calibration, without an iterative critique--rewrite loop.

Figures & tables

Figure 1: Similar global embedding scores can conceal different structural errors. A scene graph makes object attributes and relations explicit.

Figure 2: Overview of StruPO. Calibrated structural intent conditions the DSE–MHE–EAG editor and defines image-grounded consistency feedback. Environment-aware GRPO averages renders of each candidate before comparing edit rollouts.

Figure 3: The three edit operations used by EAG. Tokens are first retained or deleted, the retained subsequence is reordered, and new tokens are inserted into the resulting slots.

Method	Lexica : In-domain					DiffusionDB : In-domain				COCO : Out-of-domain
Method	AES Δ	OR(%)	RR(%)	SGR(%)	Gemini	AES Δ	OR(%)	RR(%)	SGR(%)	AES Δ	OR(%)	RR(%)	SGR(%)
Template	0.53	89.50	59.63	74.57	77.6	0.75	91.03	62.65	76.84	0.68	90.18	65.02	77.60
Promptist	0.48	89.55	61.25	75.40	79.5	0.64	92.17	60.14	76.16	0.80	90.06	64.61	77.33
MagicPrompt	0.46	89.50	61.67	75.59	78.1	0.62	92.75	58.92	75.84	0.67	89.17	59.12	74.14
Human-crafted	0.54	89.19	56.31	72.75	78.2	0.42	92.32	59.44	75.88	–	–	–	–
VisualPrompter	0.62	90.26	53.49	71.88	79.2	0.33	83.11	23.43	53.27	-0.36	90.50	76.01	83.26

Table 1: Overall performance with Stable Diffusion v1.5. Gemini-VQA is evaluated on Lexica only and is never used for training or model selection. Best values are bold. VisualPrompter attains the highest COCO SGR but decreases aesthetics; StruPO obtains the strongest in-domain faithfulness–aesthetics trade-off and the largest AES Δ on all three datasets.

Method	Stable Diffusion v1.4				Stable Diffusion v2.1				FLUX.1-dev
Method	AES Δ	OR(%)	RR(%)	SGR(%)	AES Δ	OR(%)	RR(%)	SGR(%)	AES Δ	OR(%)	RR(%)	SGR(%)
Template	.53	89.19	58.38	73.78	.33	88.02	59.25	73.64	.83	87.88	46.03	66.96
Promptist	.47	89.75	60.64	75.19	.34	87.75	59.57	73.66	.88	90.17	53.41	71.79
MagicPrompt	.48	89.44	60.18	74.81	.32	87.98	55.65	71.81	.82	90.16	52.81	71.48
Human-crafted	.56	89.56	59.48	74.52	.39	87.75	54.28	71.01	.62	87.92	49.55	68.74
VisualPrompter	.62	88.10	47.05	67.57	.62	88.07	46.70	67.38	.63	88.08	46.53	67.31

Table 2: Transferability across T2I backbones on Lexica. The editor is trained with SD v1.5 feedback and evaluated without retraining on SD v1.4, SD v2.1, and FLUX.1-dev.

Table 3: Human preference on Lexica. Bars show StruPO wins (blue), ties (gray), and losses (sand), from left to right, with the percentages printed in each segment.

Variant	DSE	MHE	EAG	AES Δ	OR	RR	SGR
StruPO	✓	✓	✓	.63	91.84	69.64	80.74
w/o DSE	✗	✓	✓	.64	89.24	58.14	73.69
w/o MHE	✓	✗	✓	.65	89.25	61.41	75.33
w/o EAG	✓	✓	✗	.57	88.92	58.44	73.68
w/o RL	✓	✓	✓	.57	89.52	58.55	74.04
w/o $r^{\mathrm{con}}$	✓	✓	✓	.68	89.19	61.08	75.14

Table 4: Ablation study on Lexica. DSE, MHE, and EAG test the architecture; “w/o RL” and “w/o

r^{\mathrm{con}}

” test image-grounded optimization. The w/o-EAG variant replaces local editing with free-form autoregressive generation while retaining the calibrated SG, reward, and GRPO.

Panel of Figure 4 — Figure 4: Performance by constraint complexity on Lexica. Object and relation counts define separate partitions; methods share prompts and image seeds within each bucket.

Figure 5: Representative SD v1.5 comparisons. StruPO better preserves attribute-bearing phrases and object–relation constraints while maintaining visual quality.

Appendix figures & tables30 assets

Supplementary material from the paper’s appendix.

Appendix

Optimized Prompt	a fashion catwalk on a fishing boat, fashion photography, concept art, octane, redshift, high detail
Semantic Skeleton	a fashion catwalk on a fishing boat
Rewriting	on a fishing boat, there was a fashion catwalk

Appendix

Table 5: Example construction of an input–target pair from an optimized prompt.

Figure 6: Deletion, reordering, and insertion actions used by the edit-action generator.

Hyperparameter	SFT	RL
Batch size	128	16
Learning rate	$2\times 10^{-5}$	$2\times 10^{-5}$
Optimizer updates	56K	6K
Optimizer	AdamW	AdamW
AdamW $\epsilon$	$10^{-8}$	$10^{-8}$
AdamW $(\beta_{1},\beta_{2})$	$(0.9,0.999)$	$(0.9,0.999)$

Appendix

Table 6: SFT and RL optimization settings.

Hyperparameter	Value
Structural reward weight $\alpha_{1}$	0.6
Aesthetic reward weight $\alpha_{2}$	1
Global CLIP weight $\alpha_{3}$	5
KL coefficient $\beta$	0.2
Entropy coefficient $\lambda$	0.05

Appendix

Table 7: Reward and regularization weights used in RL.

	SFT	RL
Number of GPUs	4	5
Total GPU-hours	80	100

Appendix

Table 8: Compute for the reported StruPO training stages.

Figure 7: Interface shown to annotators. Method names are hidden and left–right order is randomized.

Dataset	Comparator	Aesthetics W/T/L	Consistency W/T/L
Lexica	Promptist	54/22/24	36/37/27
	VisualPrompter	48/18/34	39/32/29
	Few-shot GPT-4o	53/17/30	42/23/35
DiffusionDB	Promptist	60/19/21	44/33/23
	VisualPrompter	67/16/17	62/21/17
	Few-shot GPT-4o	59/20/21	43/33/24

Appendix

Table 9: Human preference for StruPO across all datasets. Each entry is StruPO win/tie/loss (%) against the comparator.

Comparator	W/T/L counts	Decisive win rate	95% CI for W–L	Sign-test $p$
Promptist	721/738/541	57.1%	$[+5.5,+12.5]$ pp	$4.49\times 10^{-7}$
VisualPrompter	781/638/581	57.4%	$[+6.4,+13.6]$ pp	$6.61\times 10^{-8}$
Few-shot GPT-4o	841/458/701	54.5%	$[+3.2,+10.8]$ pp	$3.97\times 10^{-4}$

Appendix

Table 10: Exact Lexica consistency results over 2,000 prompts.

Comparator	Aesthetics	Consistency
Promptist	65.2%	54.8%
VisualPrompter	57.0%	55.7%
Few-shot GPT-4o	61.7%	53.5%

Appendix

Table 11: Majority-aggregated StruPO win rates under strict forced choice on Lexica.

Comparator	$\|V_{x}\|=1$ ( $n=807$ )	$\|V_{x}\|=2$ ( $n=694$ )	$\|V_{x}\|\geq 3$ ( $n=499$ )	Raw $p$	Holm $p$
Promptist	34/38/28 (+6)	35/38/27 (+8)	41/34/25 (+16)	.0128	.0256
VisualPrompter	36/34/30 (+6)	38/33/29 (+9)	45/27/28 (+17)	.0064	.0192
Few-shot GPT-4o	40/23/37 (+3)	41/24/35 (+6)	47/20/33 (+14)	.0276	.0276

Appendix

Table 12: Lexica consistency preference by relation complexity. Entries are W/T/L (W–L percentage points).

	0 $\sim$ 20				20 $\sim$ 40				$\geq$ 40
Models	AES Δ	OR	RR	SGR	AES Δ	OR	RR	SGR	AES Δ	OR	RR	SGR
Template	0.62	92.54	61.14	76.84	0.39	85.13	59.16	72.14	0.18	70.98	45.24	58.11
Promptist	0.59	92.53	59.86	76.20	0.30	85.50	63.85	74.68	0.09	66.73	34.52	50.63
MagicPrompt	0.56	91.96	60.63	76.29	0.33	85.95	62.71	74.33	0.14	73.70	44.05	58.87
BeautifulPrompt	0.68	90.34	46.20	68.27	0.40	78.80	35.99	57.39	0.25	63.53	14.29	38.91
Human-crafted	0.66	92.53	58.75	75.64	0.37	84.27	56.12	70.19	0.23	71.13	22.62	46.88

Appendix

Table 13: Results on different lengths of input prompts.

	Rewriting				Semantic Skeleton
Models	AES Δ	OR	RR	SGR	AES Δ	OR	RR	SGR
Template	0.53	88.45	60.90	74.67	0.53	90.37	57.75	74.06
Promptist	0.46	88.39	63.44	75.92	0.49	90.51	57.99	74.25
MagicPrompt	0.46	89.35	63.14	76.24	0.47	89.46	58.46	73.96
BeautifulPrompt	0.58	83.86	40.95	62.41	0.55	87.34	38.22	62.78
Human-crafted	0.54	88.31	56.87	72.59	0.54	89.88	55.47	72.67

Appendix

Table 14: Results on different categories of input prompts.

Figure 8: DAAM attention visualization for a representative case: raw prompt (top) and StruPO-optimized prompt (bottom).

Method	AES Δ	OR	RR	SGR
Full	0.53	89.50	59.63	74.57
Tag0	0.24	90.00	60.14	75.07
Tag1	0.34	89.90	61.14	75.52
Tag2	0.30	90.22	60.04	75.13
Tag3	0.42	89.42	59.26	74.34
Tag4	0.45	90.31	62.41	76.36

Appendix

Table 15: Comparison with fixed style-suffix variants on Lexica.

Tag	Content
Full	cinematic, colorful background, concept art, dramatic lighting, high detail, highly detailed, hyper realistic, intricate, intricate sharp details, octane render, smooth, studio lighting, trending on artstation
0	dramatic lighting, smooth, hyper realistic, colorful background
1	colorful background, concept art, octane render, hyper realistic
2	smooth, highly detailed, octane render, intricate
3	trending on artstation, concept art, high detail, dramatic lighting
4	colorful background, octane render, dramatic lighting, studio lighting, high detail, smooth, concept art, intricate

Appendix

Table 16: Style suffixes used in the template comparison.

Figure 9: Prototype interface for inspecting edits and specifying object-preservation constraints.

Figure 10: Intermediate edit traces for two prompts, including residual word-order and repetition errors.

Figure 11: More cases generated by Stable Diffusion v1.5.

Figure 12: More cases generated by Stable Diffusion v1.4.

Figure 13: More cases generated by Stable Diffusion v2.1.

Dataset	Method	TIFA	VQAScore	ImageReward
Lexica	Promptist	.736	.711	.751
	VisualPrompter	.713	.705	.717
	Few-shot GPT-4o	.730	.716	.779
	StruPO	.789	.763	.804
DiffusionDB	Promptist	.748	.724	.728
	VisualPrompter	.563	.541	.492

Appendix

Table 17: Independent evaluation on three disjoint 2K-prompt test sets. These evaluators are not used in the default StruPO training pipeline.

Method	AES Δ	HPSv2.1	ImageReward
Template	.53	.262	.628
Promptist	.48	.271	.751
VisualPrompter	.62	.269	.717
Few-shot GPT-4o	.51	.276	.779
StruPO	.63	.275	.804

Appendix

Table 18: Lexica preference evaluation. HPSv2.1 and ImageReward are independent of default StruPO training; the reward-aligned AES Δ is included for comparison.

Method	AES Δ	RR	SGR	TIFA	ImageReward
Promptist	.48	61.25	75.40	.736	.751
PAE	.58	62.71	76.42	.756	.786
RePrompt	.50	67.34	78.91	.795	.799
StruPO	.63	69.64	80.74	.789	.804

Appendix

Table 19: Recent learned prompt optimizers on matched SD v1.5/Lexica evaluation.

Method / ablation	Color	Shape	Texture/material	Average
Promptist	.49	.39	.46	.45
Policy-side head-noun graph	.54	.42	.51	.49
Reward-side head-only grounding	.57	.43	.52	.51
Full attribute-bearing phrases	.60	.47	.56	.54

Appendix

Table 20: Attribute-binding diagnostic.

Variant	AES Δ	OR	RR	SGR	TIFA	Gemini-VQA
Full StruPO	$.63\pm.014$	$91.84\pm.17$	$69.64\pm.31$	$80.74\pm.21$	$.789\pm.003$	$82.4\pm.18$
Without $r_{\mathrm{CLIP}}$	$.60\pm.018$	$91.8\pm.21$	$69.1\pm.43$	$80.4\pm.24$	$.785\pm.005$	$82.1\pm.27$
T5 encoder + $r_{\mathrm{CLIP}}$	$.64\pm.024$	$91.9\pm.26$	$69.5\pm.48$	$80.6\pm.29$	$.794\pm.006$	$82.5\pm.31$
CLIP-free training	$.59\pm.033$	$91.4\pm.34$	$68.4\pm.62$	$79.8\pm.39$	$.781\pm.008$	$81.7\pm.41$

Appendix

Table 21: CLIP sensitivity on Lexica. Each entry is the reported mean

\pm

standard deviation over three training seeds. The CLIP-free variant replaces the aesthetic training reward with ImageReward.

Method	Optimized variable	Generator update	Per-image optimization	Reusable text	Offline GPU-hours
DPOK, matched reward	Diffusion parameters	Yes	No	No	100 (RL)
Attend-and-Excite	Attention/latent trajectory	No	Yes	No	0
StruPO	Prompt policy	No	No	Yes	100 (RL)

Appendix

Table 22: Cross-paradigm comparison on 2,000 Lexica prompts. Access and cost (top) and image quality (bottom) describe the same runs. StruPO’s 100-GPU-hour RL stage is preceded by 80 GPU-hours of SFT.

Method	AES Δ	OR	RR	SGR	Gemini-VQA
Template	$.53\pm.01$	$89.50\pm.14$	$59.61\pm.28$	$74.56\pm.17$	$77.6\pm.2$
Promptist	$.48\pm.01$	$89.57\pm.13$	$61.20\pm.25$	$75.39\pm.15$	$79.5\pm.2$
VisualPrompter	$.62\pm.02$	$90.21\pm.18$	$53.55\pm.39$	$71.88\pm.23$	$79.2\pm.2$
GPT-4o-SG-Iter	$.61\pm.02$	$91.08\pm.16$	$66.55\pm.32$	$78.82\pm.20$	$81.6\pm.2$
StruPO	$\mathbf{.63\pm.02}$	$\mathbf{91.82\pm.15}$	$\mathbf{69.51\pm.29}$	$\mathbf{80.67\pm.19}$	$\mathbf{82.3\pm.2}$

Appendix

Table 23: Full 2K-prompt Lexica comparison over three independent matched seed lists (mean

\pm

standard deviation).

$\alpha_{1}$	SGR	AES Δ
0.4	79.6	.69
0.6	80.7	.63
0.8	80.4	.61

Appendix

Table 24: Sensitivity to structural reward weight

\alpha_{1}

on Lexica.

Setting	SGR	Remark
Clean calibrated SG	80.7	default
Mild SG noise	79.8	distractors / predicate swaps
Medium SG noise	77.9	stronger perturbation
Text-only policy	75.3	without graph input
Without DSE	73.7	raw parsed SG

Appendix

Table 25: DSE robustness under synthetic scene-graph perturbations on Lexica.

Method	Learned policy	Calibrated SG input	Atomic SG training reward	Test-time feedback loop	Local edit actions
Promptist [ 5 ]	Yes	No	No	No	No
BeautifulPrompt [ 4 ]	Yes	No	No	No	No
RePrompt [ 19 ]	Yes	No	No	No	No
PAE [ 18 ]	Yes	No	No	No	No
OPT2I [ 8 ]	No	No	No	Yes	No
VisualPrompter [ 9 ]	No	Test-time only	No	Yes	No

Appendix

Table 26: Where structural feedback enters prompt optimization. “Test-time only” denotes graph input to an external feedback loop. The final column refers to an explicit delete/reorder/insert policy.

Prompt Optimization for T2I Generation

Prompt optimization for text-to-image generation improves how natural-language instructions control image content, style, composition, and quality.

See all 32 papers

SCOPE: Semantic Bounded Credit Optimization for Prompt Engineering in Text-to-image Generation
Oct 15, 2026Yufei Jiao, Jinfa Huang, Zeyue Xue +5Prompt Optimization for T2I Generation RL for Image Generation
Xiamen University · University of Rochester · The University of Hong Kong +1
PRISM: Prompt Refinement via Image-grounded Self-rewarding Mechanism for Text-to-Image Generation
Jul 27, 2026Guo Tang, HongJie Luo, Tianxu Wang +2T2I Generation RL for Image Generation
Harbin Institute of Technology, Shenzhen · Sun Yat-sen University · South China University of Technology +1
PE-OPSD: Internalizing Prompt Enhancement into Flow-matching Models via On-Policy Self-Distillation
Sep 29, 2026Mingfeng Lin, Chengfei Cai, Lin Xu +3On-Policy Self-Distillation T2I Generation
Harbin Institute of Technology (Shenzhen) · Zhejiang University · Peking University +1

Oct 15, 2026J/K move · Enter open · S save

SCOPE: Semantic Bounded Credit Optimization for Prompt Engineering in Text-to-image Generation

Yufei Jiao, Jinfa Huang, Zeyue Xue, Tianyu Xie+4

Xiamen University · University of Rochester · The University of Hong Kong+1

Prompt Optimization for T2I Generation RL for Image Generation Compositional T2I Generation Group Relative Policy Optimization

Open paper AskPDF arXiv

Reinforcement learning (RL) enables language models to improve text-to-image (T2I) generation by rewriting prompts for a frozen image generator. Existing methods typically apply a single signal derived from the image to individual subword tokens, although objects, counts, attributes, and relations form coherent semantic spans. We propose SCOPE, an objective based on Group Relative Policy Optimization (GRPO) that aligns update weighting with this structure. The rewriter jointly generates an enhanced prompt and its semantic partition. SCOPE pools shifts in policy log probabilities within each span to derive a shared coefficient for its token losses while retaining a single advantage for each completion. It requires neither separate span rewards nor additional image renders. Across Qwen-Image, SD3, and FLUX.1-dev, SCOPE achieves average relative gains over vanilla GRPO of 9.48% on GeneVal and 7.62% on T2I-CompBench. Ablation studies show that both semantic boundaries and pooling within spans contribute to these gains, supporting semantic structure as a practical inductive bias for prompt engineering with RL.

Abstract

Prompt Optimization for T2I Generation

Explore similar work

SCOPE: Semantic Bounded Credit Optimization for Prompt Engineering in Text-to-image Generation

PRISM: Prompt Refinement via Image-grounded Self-rewarding Mechanism for Text-to-Image Generation

PE-OPSD: Internalizing Prompt Enhancement into Flow-matching Models via On-Policy Self-Distillation