Learning the Loop, Not Just the Page: Execution-Grounded Loop Learning for Web Generation

Yuxin Meng; Ruixu Zhang; Junjie Wang; Yuhan Suo; Yuhan Sun; Ruining Hu; Yiyao Yu; Yubin Wang

cs.CLOct 8, 2026

Learning the Loop, Not Just the Page: Execution-Grounded Loop Learning for Web Generation

Authors: Yuxin Meng, Ruixu Zhang, Junjie Wang, Yuhan Suo, Yuhan Sun, Ruining Hu, Yiyao Yu, Yubin Wang, +5 more

Organizations: Tsinghua University · Huawei Noah’s Ark Lab · East China Normal University · Tongji University · Institute of Artificial Intelligence, Beihang University · National University of Singapore

Abstract

Functional Web generation is increasingly optimized with executable rewards, yet existing methods largely focus on the quality of the final page and leave the process of diagnosing and repairing imperfect implementations underexplored. We identify a central challenge in this setting: the Generator and Refiner produce executable artifacts with direct environment rewards, whereas the intermediate Critic influences downstream behavior without a directly executable outcome. We introduce WebLoop, an execution-grounded framework that jointly learns generation, critique, and refinement within a shared policy. WebLoop trains an execution-free Critic with complementary signals for requirement-level discriminability and downstream helpfulness, first establishing reliable diagnosis and then introducing consequence-aware credit, while all three roles are jointly optimized with group-relative policy learning. With Qwen3.5-9B, WebLoop reaches 41.5 Overall on WebRise and 38.9% accuracy on WebGen-Bench, improving the base model by 11.3 and 15.4 points, respectively. The gains transfer to first-pass generation, persist at 27B scale, and generalize from text-only training to multimodal inputs. Controlled analyses further show that the improvement cannot be explained by an additional refinement pass alone, highlighting the importance of learning the Critic and the loop itself.

Figures & tables

Figure 1: Conceptual comparison of execution-based learning paradigms for functional Web generation. (a) Generation-only RL optimizes executable outputs directly. (b) Execution-guided RL feeds external feedback back into the code policy. (c) WebLoop jointly learns Generator, Critic, and Refiner, assigning execution-derived credit to the intermediate Critic.

Figure 2: Overview of our WebLoop framework. A shared policy sequentially acts as Generator, Critic, and Refiner. The Generator and Refiner produce executable Web pages evaluated in the same browser environment, while the execution-free Critic guides subsequent refinement and receives training credit from its diagnosis and downstream effects.

Model / Method	WebRise				WebGen-Bench
Model / Method	$T$ $\uparrow$	$R_{e}$ $\uparrow$	$R_{i}$ $\uparrow$	Overall $\uparrow$	Yes (%) $\uparrow$	Partial (%) $\uparrow$	No (%) $\downarrow$	Start-Fail (%) $\downarrow$	Acc. $\uparrow$
General-Purpose Models (Open-Source & Proprietary)
Qwen3.5-122B-A10B	38.0	48.9	35.2	40.7	9.9	6.2	78.1	5.9	13.0
Kimi-K2.6	44.6	54.2	41.8	46.9	25.7	11.1	55.3	7.9	31.2
Qwen3.5-397B-A17B	45.7	57.2	42.8	48.6	16.1	9.1	70.0	4.8	20.6
GLM-5.3	59.5	77.1	54.9	63.8	20.6	11.9	67.5	0.0	26.5

Table 1: Main results on WebRise and WebGen-Bench. General-purpose and Web-generation specialized models are included as capability references, while the Qwen3.5 block provides controlled comparisons. Bold denotes the best result within each reference group or controlled backbone size according to the metric direction. † denotes proprietary models.

Model / Stage	Text			Markdown			Sketch			Image			Video			Avg. $\uparrow$
Model / Stage	$T$ $\uparrow$	$R_{e}$ $\uparrow$	$R_{i}$ $\uparrow$	$T$ $\uparrow$	$R_{e}$ $\uparrow$	$R_{i}$ $\uparrow$	$T$ $\uparrow$	$R_{e}$ $\uparrow$	$R_{i}$ $\uparrow$	$T$ $\uparrow$	$R_{e}$ $\uparrow$	$R_{i}$ $\uparrow$	$T$ $\uparrow$	$R_{e}$ $\uparrow$	$R_{i}$ $\uparrow$	Avg. $\uparrow$
Base (Qwen3.5-9B)	23.2	45.3	22.1	28.2	51.1	26.0	25.2	48.8	21.9	23.0	46.2	22.8	24.7	45.7	23.9	31.9
WebLoop (Gen)	31.7	55.4	30.0	30.5	55.0	28.4	28.7	53.4	26.2	27.7	51.9	26.1	31.8	54.1	30.9	37.5
WebLoop (Refine)	34.6	57.9	32.0	34.4	57.5	32.1	31.1	55.0	28.1	30.6	53.1	29.0	35.9	57.3	33.6	40.1

Table 2: Cross-modal generalization on WebRise. WebLoop is trained only on Text-conditioned tasks and evaluated on all five input modalities using the same checkpoint. Avg. averages the modality-level

(T+R_{e}+R_{i})/3

scores.

Training Scheme / Output	Optimized Roles	WebRise				WebGen-Bench
Training Scheme / Output	Optimized Roles	$T$ $\uparrow$	$R_{e}$ $\uparrow$	$R_{i}$ $\uparrow$	Overall $\uparrow$	Yes (%) $\uparrow$	Partial (%) $\uparrow$	No (%) $\downarrow$	Start-Fail (%) $\downarrow$	Acc. $\uparrow$
Base	–	23.2	45.3	22.1	30.2	18.4	10.2	67.1	4.3	23.5
Generation Only (Gen)	Gen	29.5	54.1	27.3	36.9	17.8	10.5	67.9	3.9	23.0
Refine w/o Critique (Gen)	Gen + Refine	28.2	53.3	25.2	35.6	16.7	11.9	70.5	0.9	22.6
Refine w/o Critique (Refine)	Gen + Refine	32.9	57.3	29.7	40.0	21.6	10.0	68.3	0.0	26.7
Staged Training (Gen)	Gen $\rightarrow$ Critic + Refine	27.6	52.0	25.9	35.2	20.7	11.1	66.0	2.2	26.3

Table 3: Where does the gain come from? Controlled comparison of training schemes on WebRise and WebGen-Bench. All variants use the same Qwen3.5-9B backbone, training data, and rollout budget, and differ only in which roles are optimized and whether they are trained jointly or sequentially. Bold denotes the best result according to the metric direction.

Critic Training / Output	$\mathbf{R_{\mathrm{crit}}}$	WebRise				WebGen-Bench
Critic Training / Output	$\mathbf{R_{\mathrm{crit}}}$	$T$ $\uparrow$	$R_{e}$ $\uparrow$	$R_{i}$ $\uparrow$	Overall $\uparrow$	Yes (%) $\uparrow$	Partial (%) $\uparrow$	No (%) $\downarrow$	Start-Fail (%) $\downarrow$	Acc. $\uparrow$
Base	–	23.2	45.3	22.1	30.2	18.4	10.2	67.1	4.3	23.5
Discriminability Only (Gen)	$D$	30.0	53.6	27.2	36.9	22.3	8.7	65.8	3.2	26.6
Discriminability Only (Refine)	$D$	33.4	57.3	30.7	40.5	27.8	10.7	59.4	2.2	33.2
Helpfulness Only (Gen)	$H$	27.6	50.7	24.7	34.3	18.2	9.0	68.2	4.6	22.7
Helpfulness Only (Refine)	$H$	31.3	54.1	28.7	38.0	28.1	13.0	56.3	2.6	34.6

Table 4: Critic reward ablation. All variants use the same Qwen3.5-9B backbone, training data, and rollout structure and differ only in the Critic reward.

D

denotes discriminability and

H

denotes downstream helpfulness. Bold denotes the best result according to the metric direction.

Figure 3: Critic learning dynamics. (a) Discriminability

D

. (b) Downstream helpfulness

H

. (c) Generator and Refiner execution rewards. Thin and thick curves denote per-step values and moving averages; the dashed line marks the reward switch after 15 updates (around 1K samples).

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Comparison	#Samples	Acc. $\uparrow$	MAE $\downarrow$	Spearman $\uparrow$	Pearson $\uparrow$	$\bm{\kappa}\uparrow$
Human (A) vs. Human (B)	300	0.920	0.079	0.924	0.923	0.838
Evaluator vs. Human (A)	300	0.880	0.095	0.901	0.902	0.758
Evaluator vs. Human (B)	300	0.900	0.094	0.902	0.898	0.799

Appendix

Table 5: Human validation of functional evaluation. Agreement between the automatic evaluator and two independent human reviewers on the same 300 sampled interaction cases. Human–human agreement is included as a reference.

Figure 4: Refinement learning and Generator–Critic checkpoint interaction. (a) Refined-page quality over training, reported as improvement from the initial training point, with refinement without learned critique as a control. The dashed line marks the Critic reward switch. (b) WebRise Overall under cross-pairing Generator and Critic checkpoints from steps 20, 30, and 45. The Refiner uses the same checkpoint as the Generator in each pairing.

General-Purpose Models
Model / Method	By Website Category			By Check Category
Model / Method	Content	User	Data	Functional	Data Display	Design
	Presentation	Interaction	Management	Testing	Testing	Validation
Qwen3.5-122B-A10B	21.3	9.6	10.6	5.2	18.0	27.0
Kimi-K2.6	42.5	26.4	28.4	20.9	37.9	49.6
Qwen3.5-397B-A17B	27.3	19.2	16.2	11.5	24.5	40.2

Appendix

Table 6: Fine-grained WebGen-Bench results. Accuracy under the same template-free setting as Table 1 . The 647 interaction checks are independently grouped by website category and check category; each grouping covers the full benchmark, so the six columns are not additive. † denotes proprietary models.

Figure 5: Critique length versus refinement quality. Each cell reports the mean WebRise Overall of refined pages within a critique-length bucket for a given Critic objective. WebLoop exhibits a stronger positive association, while alternative objectives show weaker or non-monotonic trends.

Figure 6: Successful critique-guided repair on Poll Vote (WebRise). The Critic localizes premature disclosure of voting results and a missing post identifier that blocks vote updates. The Refiner applies the corresponding fixes, changing both affected transition checks from FAIL to PASS.

Figure 7: Boundary-condition repair on File Search with Filters (WebRise). The Critic traces the empty result set to size bounds parsed as NaN after clearing filters. The Refiner adds guards for empty bounds, restoring all 12 files while preserving the already-correct control reset.

Figure 8: False-positive Critic judgment on Share Chat Link Interface (WebRise). The Critic attributes message-selection failure to an unsynchronized hidden checkbox, although the visible selection and preview are already controlled correctly. The Refiner synchronizes the checkbox without affecting either passing behavior.

Figure 9: Correct verdict but mislocalized repair guidance on Search Result Tabs (WebRise). The Critic correctly marks the per-category count requirement as unmet but targets a type-matching issue rather than the missing count labels. The Refiner applies the proposed edit, while the target predicate remains FAIL.

Figure 10: Divergent fault localization on Petition Progress Dashboard (WebRise). Both critiques correctly identify signing as broken, but only one localizes the state error that disables signing when the dialog opens. Although both suggested edits are implemented, only the correctly localized repair changes the transition from FAIL to PASS.

Figure 11: Incomplete repair guidance on Bulk Content Operations (WebRise). The Refiner adds the missing listener as recommended, but an unaddressed selector error still clears the header-checkbox state. The prescribed edit is implemented, yet the target requirement remains unsatisfied.

Figure 12: Divergent Refiner outcomes under equivalent diagnoses on Audit Timeline (WebRise). Both critiques identify the same missing event binding, but only one refinement implements the recommended listener and passes the transition check. The contrast isolates implementation fidelity from Critic diagnosis.

Figure 13: Refiner-induced regression on Topic Trending (WebRise). While addressing missing liked-state feedback, the Refiner conflates the user’s like state with the aggregate like count, causing the count-update predicate to regress from PASS to FAIL.

Panel of Figure 14 — Figure 14: Generator prompt for first-pass Web generation.

Panel of Figure 15 — Figure 15: Critic prompt for requirement-level diagnosis and repair guidance.

Panel of Figure 16 — Figure 16: Refiner prompt for critique-conditioned Web repair.

Explore similar work

WebGen-R1: Incentivizing Large Language Models to Generate Functional and Aesthetic Websites with Reinforcement Learning
Apr 22, 2026Juyong Jiang, Chenglin Cai, Chansung Park +4RL for Code Generation RL for Language Models
The Hong Kong University of Science and Technology (Guangzhou) · Alibaba · Tongyi Lab, Alibaba Group +3
WebGrader: Training LLMs for Web Development with Self-Evolving Programmatic Grader
Aug 6, 2026Boshui Chen, Huiping Liu, Shaolei ZhangRL for Code Generation Automated Software Testing
Beijing Institute of Technology, China · Beijing Jiaotong University, China · Renmin University of China, China
LiveEvalBench: Toward Open-World Evaluation for Web Generation
Aug 4, 2026Yiyao Wang, Zhen Wen, Yinghao Tang +5Web Application Generation Automated Evaluation
State Key Lab of CAD&CG, Zhejiang University · Ant Group · State Key Lab of CAD&CG, Zhejiang University; Laboratory of Art and Archaeology Image (Zhejiang University), Ministry of Education, China

Apr 22, 2026, cs.CLJ/K move · Enter open · S save

WebGen-R1: Incentivizing Large Language Models to Generate Functional and Aesthetic Websites with Reinforcement Learning

Juyong Jiang, Chenglin Cai, Chansung Park, Jiasi Shen+3

The Hong Kong University of Science and Technology (Guangzhou) · Alibaba · Tongyi Lab, Alibaba Group+3

RL for Code Generation RL for Language Models Web Application Generation LLM Code Generation

Open paper AskPDF arXiv

While Large Language Models (LLMs) excel at function-level code generation, project-level tasks such as generating functional and visually aesthetic multi-page websites remain highly challenging. Existing works are often limited to single-page static websites, while agentic frameworks typically rely on multi-turn execution with proprietary models, leading to substantial token costs, high latency, and brittle integration. Training a small LLM end-to-end with reinforcement learning (RL) is a promising alternative, yet it faces a critical bottleneck in designing reliable and computationally feasible rewards for website generation. Unlike single-file coding tasks that can be verified by unit tests, website generation requires evaluating inherently subjective aesthetics, cross-page interactions, and functional correctness. To this end, we propose WebGen-R1, an end-to-end RL framework tailored for project-level website generation. We first introduce a scaffold-driven structured generation paradigm that constrains the large open-ended action space and preserves architectural integrity. We then design a novel cascaded multimodal reward that seamlessly couples structural guarantees with execution-grounded functional feedback and vision-based aesthetic supervision. Extensive experiments demonstrate that our WebGen-R1 substantially transforms a 7B base model from generating nearly nonfunctional websites into producing deployable, aesthetically aligned multi-page websites. Remarkably, our WebGen-R1 not only consistently outperforms heavily scaled open-source models (up to 72B), but also rivals the state-of-the-art DeepSeek-R1 (671B) in functional success, while substantially exceeding it in valid rendering and aesthetic alignment. These results position WebGen-R1 as a viable path for scaling small open models from function-level code generation to project-level web application generation.

Abstract

Explore similar work

WebGen-R1: Incentivizing Large Language Models to Generate Functional and Aesthetic Websites with Reinforcement Learning

WebGrader: Training LLMs for Web Development with Self-Evolving Programmatic Grader

LiveEvalBench: Toward Open-World Evaluation for Web Generation