CommerceVibe: Learning to Design E-Commerce Creatives as Executable Visual Code via Dual-Feedback Reinforcement Learning

Yajiao Xu; Jin Zhang; Jiangbo Ai; Tao Jiang; Mo Xu; Lina Huang; Chengfu Huo

cs.CVAug 28, 2026

CommerceVibe: Learning to Design E-Commerce Creatives as Executable Visual Code via Dual-Feedback Reinforcement Learning

Authors: Yajiao Xu, Jin Zhang, Jiangbo Ai, Tao Jiang, Mo Xu, Lina Huang, Chengfu Huo

Organizations: Tongji University · Alibaba Group

Abstract

High-quality e-commerce creatives are essential for presenting products and conveying marketing messages. Recent diffusion models enable scalable creative generation and produce visually compelling images, but their flattened raster outputs often contain distorted text and inconsistent product details, requiring refinement before deployment. Moreover, without explicit structure, the resulting creatives are difficult to edit and reuse, while complex design requirements remain challenging to encode as verifiable training signals. To address these challenges, we present CommerceVibe, which represents creatives as executable visual code and formulates generation as conditional HTML/CSS program synthesis. Given product images, design requirements, and product information, it produces renderable, editable, and reusable creatives. We further introduce dual-feedback reinforcement learning, in which rule-based feedback evaluates rendered programs for text readability, product visibility, and layout validity, while visual feedback from a vision-language model (VLM) assesses rendered creatives against input specifications across six perceptual and commercial dimensions. Together, these complementary feedback signals improve both constraint satisfaction and perception-dependent quality. We perform supervised fine-tuning (SFT) of Qwen3.5-9B on over 28,000 e-commerce examples, followed by dual-feedback reinforcement learning. On a 1,300-case benchmark, the optimized CommerceVibe model achieves a weighted score of 94.0/100, compared with 87.3 for the SFT-only variant, and outperforms strong external models. Blind evaluations by five e-commerce design experts further validate these improvements. CommerceVibe supports controllable, editable, and scalable e-commerce creative production.

Figures & tables

Figure 1: Overview of CommerceVibe . Supervised fine-tuning establishes basic creative generation ability, while dual-feedback reinforcement learning combines rule-based feedback with VLM-based preference feedback to optimize the policy through GRPO. The resulting policy generates executable HTML/CSS creatives in one pass.

Figure 2: Dual-feedback reward design. (a) Rule-based feedback measures Text, Product, and Layout errors from rendered documents. (b) VLM-based preference feedback assesses visual and commercial quality independently of rule outputs. (c) The two terminal rewards are combined with equal weights and used for GRPO optimization.

External models
Method	Overall ( $N=1{,}300$ )				Single ( $N=1{,}140$ )				Multi ( $N=160$ )
Method	$S_{\mathrm{rule}}\uparrow$	$S_{\mathrm{pref}}\uparrow$	$S_{\mathrm{overall}}\uparrow$	$S_{\mathrm{expert}}\uparrow$	$S_{\mathrm{rule}}\uparrow$	$S_{\mathrm{pref}}\uparrow$	$S_{\mathrm{overall}}\uparrow$	$S_{\mathrm{expert}}\uparrow$	$S_{\mathrm{rule}}\uparrow$	$S_{\mathrm{pref}}\uparrow$	$S_{\mathrm{overall}}\uparrow$	$S_{\mathrm{expert}}\uparrow$
GPT-5.5	42.4	37.7	80.1	78.9	42.1	37.6	79.7	77.9	44.7	38.4	83.1	86.8
Claude Opus 4.8	41.5	36.3	77.7	78.4	41.0	36.3	77.3	78.4	44.6	36.0	80.6	78.7
Gemini 3.5 Flash	42.1	33.1	75.3	75.9	41.8	33.2	75.0	75.8	44.8	32.6	77.4	76.2
CommerceVibe variants

Table 1: Overall and Single/Multi results.

S_{\mathrm{rule}}

and

S_{\mathrm{pref}}

are out of 50; the other scores are out of 100. Indentation denotes SFT initialization; bold and underline mark the best and second-best scores in each column.

Figure 3: Nine-dimensional comparison with external models. T/P/L denote the three rule families, VA/PP/PR/MR/CU/CF denote the six Judge dimensions. All dimensions are normalized to 0–100.

Figure 4: Automatic–expert alignment on the benchmark. Gray points and colored markers denote individual cases and method means, respectively. The inset gives Spearman

\rho

with a 95% case-cluster bootstrap CI; the diagonal marks equality.

Figure 5: Qualitative comparison on two Single-image and two Multi-image cases. Paper cards summarize user requirements. Colored and patterned outline overlays localize representative issues: red, cyan, gray, and violet denote Text, Product, Layout, and VLM feedback, respectively. The underlying model outputs are otherwise unmodified; green checks indicate that no localized issue was identified in the audited categories.

Variant	Text $\Delta$	Prod. $\Delta$	Layout $\Delta$	$\Delta S_{\mathrm{pref}}$	$\Delta S_{\mathrm{overall}}$
Rule-RLVR	–	–	–	–	–
w/o Text rules	$\mathbf{-2.15}$	$+1.50$	$+1.45$	$-0.9$	$-1.6$
w/o Product rules	$-0.12$	$\mathbf{-0.50}$	$+0.38$	$0.0$	$-0.1$
w/o Layout rules	$+0.49$	$+0.33$	$\mathbf{-4.47}$	$-0.1$	$-0.2$

Table 2: Rule-family ablation on 1,300 cases. Values are paired changes from Rule-RLVR (0–100); bold marks the removed family.

Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

1	Input: product images $\mathcal{I}$ , free-form request $u$ , and optional product information $m$ .
2	PRIMARY_IDEA $\leftarrow$ summarize the requested visual concept, comparison goal, and reading order.
3	SOURCE_ANCHORS $\leftarrow$ extract only visual and product evidence grounded in $(\mathcal{I},m)$ .
4	CORE_PROOF $\leftarrow$ bind each required product, variant, and claim to its supporting source evidence.
5	COMPOSITION_AND_TITLE $\leftarrow$ specify layout order, title placement, product-safe regions, background, and blocking constraints.
6	DESIGN_PAYOFF $\leftarrow$ define PRIMARY_TREATMENT , ELEMENT_SYSTEM , VISIBLE_GAIN , and ORNAMENT .
7	TYPOGRAPHY $\leftarrow$ specify font family and weight, hierarchy, colors, background treatment, and small-screen readability.

Appendix

Schematic pseudocode for request normalization. Each field preserves supplied requirements and source-grounded relations while making the generation constraints explicit.

Figure S1: Training and evaluation distributions by input product-image count.

Category	Cases	Category	Cases
Shoes	69	Underwear	67
Storage and cleaning supplies	58	Home improvement and building materials	42
Household essentials	52	Fashion accessories and jewelry	56
Women’s clothing	54	Pet and gardening products	51
Sports and outdoors	50	Kitchenware and tableware	47
Home appliances	47	Children’s clothing	48

Appendix

Table S1: Category distribution of the 1,300-case benchmark. The 34 categories sum to 1,300 cases.

Component	Dimensions	Conditional behavior
Text	text overflow, text overlap, text on product, minimum font size, contrast	Weights and font thresholds differ between Single and Multi. Text-on-product is evaluated only when a visible <img> is available.
Product	subject prominence, image crop, image occlusion	Prominence uses the largest matched image for Single and summed matched-image area for Multi. Crop uses alpha-aware subject pixels when available. Occlusion is applicable only with at least two eligible rendered images.
Layout	element bounds, visual balance, bottom blank	Bounds and balance settings differ between Single and Multi. Multi uses ink-level text mass and tighter balance thresholds, whereas Single uses text boxes and excludes image backing panels. Bottom blank is activated and weighted only in rendered states for which it applies.
Safeguards	HTML validity, empty body, zero visible images, URL coverage, sparse text, critical violations	These checks act before or after continuous aggregation. They provide fixed negative rewards or additive penalties and are not members of the Text, Product, or Layout ablations.

Appendix

Table S2: Conditional rule families and safeguards. Inapplicable image-dependent detectors are removed from aggregation, and the remaining applicable weights are renormalized within the selected profile; shared safeguards are excluded from the three rule-family ablations.

Profile	Ov	Ol	Tp	Bd	Pr	Cr	Fs	Ct	Io	Ba	Bb	Critical dimensions
Single	9	9	6	9	9	2	2	2	–	2	–	Ov, Ol, Bd, Io
Multi	10	10	7	9	2	3	2	1	4	2	–	Ov, Ol, Bd, Io

Appendix

Table S3: Single- and Multi-profile rule weights. Each row has nominal weight 50 before rendered-state adjustments. Abbreviations are: Ov, overflow; Ol, text overlap; Tp, text on product; Bd, element bounds; Pr, prominence; Cr, crop; Fs, font size; Ct, contrast; Io, image occlusion; Ba, visual balance; Bb, bottom blank. A dash denotes no nominal profile weight; detectors such as Single-image occlusion and bottom blank receive state-conditioned weights only when applicable.

Profile	Title	Subtitle	Normal	Fallback	Prominence	W/H	Other settings
Single	56/.060/30	28/.025/15	22/.020/8	16/.014/20	.20 (max)	.78	box balance; strict bounds
Multi	48/.050/30	28/.025/15	20/.018/8	14/.012/20	.20 (sum)	.70	ink balance; clipped exclusion

Appendix

Table S4: Profile-specific font and prominence thresholds. Prominence mode max uses the largest matched input image, while sum uses the summed area of matched input images.

Item	Setting
Models	GPT-5.5; Claude Opus 4.8; Gemini 3.5 Flash
Evaluation	1,300 cases; July 16–17, 2026 (UTC+8)
Input	Shared system instruction, normalized request, product images, and product information
Sampling payload	Temperature and top- $p$ omitted; no reasoning/thinking parameter; no seed parameter
Output limit	8,192 tokens
Image encoding	Base64 data URLs; maximum 100,000 pixels per image

Appendix

Table S5: Common API inference settings for the three external models.

Item	Setting
Policy backbone	Qwen3.5-9B multimodal model
Parameterization	SFT: full language-model update; RL: LoRA (rank 8); vision encoder frozen
SFT	28,568 training and 3,174 validation examples; 3 epochs; per-device batch 1 $\times$ 8 GPUs $\times$ grad. accum. 4 $=$ effective batch 32; LR $1\times 10^{-5}$ ; max length 16,384; max pixels 100,000
GRPO (all variants)	Three independent runs (seeds 42, 25, 999); LR $5\times 10^{-5}$ ; temperature 0.7; effective batch of 64 completions (8 prompt groups with $K=8$ )
Terminal reward	Rule-RLVR: $R_{\mathrm{rule}}$ ; Preference-RL: $R_{\mathrm{pref}}$ ; Rule+Preference RL: $(R_{\mathrm{rule}}+R_{\mathrm{pref}})/2$
RL data and length	Identical normalized training contexts across variants; max completion 3,584; total max length 20,480

Appendix

Table S6: Training settings for SFT and the three GRPO variants.

Figure S2: Seed-42 terminal rewards for the three RL variants through step 900. Light lines show per-step values; dark lines show maximum-smoothed, debiased EMA trends.

	Single			Multi
Method	Text	Product	Layout	Text	Product	Layout
Qwen3.5-9B	87.2	89.0	74.7	80.3	91.3	75.5
+ SFT	96.1	98.6	95.2	94.4	89.4	95.3
+ Rule-RLVR	98.5	98.9	97.4	94.3	91.1	91.6
+ Preference-RL	98.2	98.3	94.4	94.1	92.5	93.9
+ Rule+Preference RL	98.8	98.9	98.0	96.7	96.3	95.5

Appendix

Table S7: Rule-family scores for 1,140 Single and 160 Multi cases. Each family is the equal-weight mean of its applicable dimensions; higher is better.

Figure S3: Detector-level rule scores for 1,140 Single and 160 Multi cases. Values are means on a 0–100 scale; the orange outline marks + Rule+Preference RL.

Figure S4: VLM Judge dimensions for 1,140 Single and 160 Multi cases. Values are unweighted mean ratings on the original 1–5 scale; the orange outline marks + Rule+Preference RL.

Figure S5: Paired changes in Text, Product, and Layout scores after removing each rule family. Values are means relative to Rule-RLVR on the same 1,300 cases; negative values indicate degradation.

Figure S6: Seed-42 terminal rewards for the three rule-family ablations through step 900. Light lines show per-step values; dark lines show maximum-smoothed, debiased EMA trends.

Figure S7: Seed-42 GRPO diagnostics through step 900. Rows show KL divergence and mean clipping-region ratio; light and dark lines denote per-step values and maximum-smoothed, debiased EMA trends.

Method	$S_{\mathrm{expert}}\uparrow$	95% CI
External models
GPT-5.5	78.9	[76.9, 80.9]
Claude Opus 4.8	78.4	[76.2, 80.5]
Gemini 3.5 Flash	75.9	[73.7, 78.0]
CommerceVibe variants
Qwen3.5-9B	59.0	[56.3, 61.7]

Appendix

Table S8: Five-expert scores on the 1,300-case benchmark.

S_{\mathrm{expert}}

is reported on a 0–100 scale with 95% case-bootstrap confidence intervals.

Comparator	$\Delta S_{\mathrm{overall}}$	$p_{\mathrm{Holm}}$	$\Delta S_{\mathrm{expert}}$	$p_{\mathrm{Holm}}$
+ SFT	$+6.7$	$1.3\times 10^{-91}$	$+8.4$	$2.4\times 10^{-14}$
+ Rule-RLVR	$+1.6$	$9.1\times 10^{-13}$	$+2.8$	$4.2\times 10^{-8}$
+ Preference-RL	$+3.6$	$3.1\times 10^{-39}$	$+5.6$	$2.1\times 10^{-13}$
GPT-5.5	$+13.9$	$9.0\times 10^{-165}$	$+11.1$	$1.5\times 10^{-12}$

Appendix

Table S9: Paired comparisons with Rule+Preference RL on the 1,300-case benchmark. Differences are Rule+Preference RL minus the comparator. Reported

p

-values are from asymptotic two-sided Wilcoxon signed-rank tests with Holm correction.

Figure S8: Five-expert mean scores and 95% case-bootstrap confidence intervals for all eight methods on the benchmark.

Visual Program Synthesis

See all 9 papers

Element-Aware Group Learning for E-Commerce Image Generation
Aug 1, 2026Jingtong Chen, Jiahui Wang, Xue Zhao +2Group Relative Policy Optimization Image Generation
ServImage: An Image Generation and Editing Benchmark from Real-world Commercial Imaging Services
Apr 27, 2026Fengxian Ji, Jingpu Yang, Zirui Song +5Image Editing Evaluation Image Generation Evaluation
MBZUAI, United Arab Emirates · Institute of Automation, Chinese Academy of Sciences, China
CommerceGen-Studio: An End-to-End Framework for Commercial-Grade Main-Image Generation
Oct 14, 2026Xiaoli Xu, Zhengtao Wu, Zhengze Xu +8Image Generation Conditional Image Generation
AI-Native Applications Group, Alibaba Group, Taobao App

Aug 1, 2026J/K move · Enter open · S save

Element-Aware Group Learning for E-Commerce Image Generation

Jingtong Chen, Jiahui Wang, Xue Zhao, ShaoGuo Liu+1

Group Relative Policy Optimization Image Generation RL for Image Editing RL for VLMs Fine-Grained Credit Assignment

Open paper AskPDF arXiv

Recent advances in image generation and editing have made prompt quality a key bottleneck for e-commerce creatives. Vision-language models (VLMs) can generate image-editing prompts from product images and metadata, but further improving their prompt-writing capabilities requires post-training with feedback from the generated images. Group Relative Policy Optimization (GRPO) is a natural framework for such outcome-level reward optimization. However, it assigns credit only at the full-prompt level, even though image quality often depends on specific design elements such as composition, background, and the presentation of selling points. Existing fine-grained credit assignment methods typically require step-level supervision or learned critics. To address this, we propose EAGLE-GRPO (Element-Aware Group Learning for E-Commerce Image Generation), which decomposes the group-centered reward over predefined elements. We cast element-level credit assignment as a kernel ridge regression problem and derive a closed-form solution, without additional rollouts or separate credit-assignment models. This yields interpretable per-element advantages and more precise policy updates. Experiments show that EAGLE-GRPO sustains performance gains over more training steps before plateauing and generates prompts that produce higher-quality e-commerce images than competitive VLM prompt-writing baselines.

Abstract

Visual Program Synthesis

Explore similar work

Element-Aware Group Learning for E-Commerce Image Generation

ServImage: An Image Generation and Editing Benchmark from Real-world Commercial Imaging Services

CommerceGen-Studio: An End-to-End Framework for Commercial-Grade Main-Image Generation