Generating HTML from natural-language specifications is an important application of code large language models (LLMs). Despite producing renderable pages, current models often fail to ensure functionality and visual quality during actual interaction. Existing methods typically target task-specific webpage supervision, leaving demanding interactive artifact optimization underexplored. Using mini-games as a challenging testbed, we expose limitations in interaction experience, reward signal, and credit assignment fidelity that are obscured by shallow webpage interactions. We introduce WebCatalyst, a browser-interaction reinforcement learning framework with three complementary components: a game-level GUI assessor for sustained interaction and evaluation, Visual Anchor for mitigating VLM reward hacking, and Action-Trace Credit for grounding token-level credit assignment in runtime execution traces. Trained on fewer than 1K requests, predominantly for games, WebCatalyst raises the Cookie-Bench Mini Games score from 68.8 to 75.3 and also the WebGen-Bench accuracy from 48.8 to 56.9 on Qwen3.6-35B-A3B, outperforming Claude Opus 4.7 in average rank across evaluated metrics. Further analysis shows that Visual Anchor prevents aesthetic degradation, while runtime-grounded credit assignment accelerates functional improvement. Together, our results show that improving feedback fidelity is the key to substantially advancing interactive HTML generation.
Figures & tables
Figure 1: From task-specific webpage optimization to interactive HTML capability catalysis. WebCatalyst improves fidelity in interaction experience, reward signals, and credit assignment through a Game-Level GUI Assessor, Visual Anchor, and Action-Trace Credit.
Figure 2: Overview of WebCatalyst’s browser-interaction RL framework. Training requests come from Reddit, and sampled HTML artifacts are execute in Playwright. The game-level GUI Assessor combines sustained interaction with hierarchical judgment for page-level rewards. Visual Anchor penalizes visual degradation relative to reference screenshots, while Action-Trace Credit maps successful actions’ runtime call paths to generated tokens and redistributes fine-grained advantages. GRPO optimizes the generation policy using the calibrated rewards and token-level advantages. Gameplay examples illustrate functionality and visual quality before and after training.
Model
Date
Size
WebGen-Bench
Cookie-Bench Mini Games
Avg. Rank ↓
Start Fail. Rate ↓
Acc. ↑
Content Pres. ↑
User Inter. ↑
Data Mgmt. ↑
Func. Test ↑
Data Display ↑
Design Val. ↑
Aesth. ↑
Func. ↑
Total ↑
Other LLMs
GPT-OSS-120B
25/08
116B/5B
1.4
37.6
49.7
31.3
36.6
26.0
43.0
61.5
40.3
54.4
47.4
19.5
Qwen3.5-122B-A10B
26/02
122B/10B
2.6
49.8
57.5
46.0
49.1
37.0
55.9
76.2
66.0
60.5
63.2
16.0
Qwen3.5-397B-A17B
26/02
397B/17B
0.5
48.7
59.2
43.5
47.5
33.6
57.0
77.9
69.9
65.0
67.5
14.1
Qwen3.6-35B-A3B
26/04
35B/3B
0.0
48.8
59.2
43.0
47.1
33.9
61.8
70.5
81.5
56.1
68.8
14.0
Table 1: Main results on WebGen-Bench and Cookie-Bench Mini Games. Acc., Aesth., and Func. denote accuracy, aesthetics, and functionality. Size gives amount of total/active parameters, Avg. Rank averages ranks across evaluation dimensions, and Δ gives changes from Qwen3.6-35B-A3B. Arrows indicate preferred directions. Bold marks group-wise best results, with WebCatalyst included in Other LLMs. WebCatalyst, with only 640 game development requests, improves both game and webpage generation and surpasses Claude Opus 4.7 in average rank.
Variant
WebGen
Mini-Games
Acc.
Aesth.
Func.
Total
Qwen3.6-35B-A3B
48.8
81.5
56.1
68.8
+GUI Assessor
52.2
71.3
60.5
65.9
+Self Anchor
55.5
82.7
57.3
70.0
+GPT-5.5 Anchor
56.8
82.3
58.2
70.3
+ATC
56.9
87.5
63.2
75.3
Table 2: Ablation study of components in WebCatalyst. Results show the contributions of experience, reward-signal, and credit-assignment fidelity.
Figure 3: Evaluation performance across training steps and feedback configurations. The top two rows show WebGen-Bench performance, while the bottom row shows Mini Games aesthetics, functionality, and overall scores. Anchor denotes Visual Anchor, and ATC denotes Action-Trace Credit.
Figure 4: Reward trajectories with and without runtime-grounded credit assignment. The vertical axis shows the EMA of the mean critic score, and High KL denotes stronger KL regularization. Dashed markers indicate when variants reach comparable score levels. All settings exhibit diminishing gains after approximately 40 steps. Optimization is approaching convergence.
Figure 5: User-experience and action-level gains before and after training. Before and After denote Qwen3.6-35B-A3B and WebCatalyst, respectively. The left panel shows experience-dimension scores, while the right shows action-transition scores.
Round
Qwen3.6-35B-A3B
+WebCatalyst
Aes.
Func.
Total
Aes.
Func.
Total
0
81.5
56.1
68.8
87.5
63.2
75.3
1
85.1
57.9
71.5
88.1
63.4
75.7
2
87.7
62.8
75.2
89.5
61.8
75.7
3
86.2
62.1
74.2
91.1
62.7
76.9
Table 3: Mini Games performance under iterative refinement. Round 0 denotes the initial generation, and subsequent rounds revise the artifact using feedback from the GUI Assessor. Bold indicates the better result for each metric within each round.
Appendix figures & tables23 assets
Supplementary material from the paper’s appendix.
Appendix
Component
Content
Role in action selection
Prompt
Original prompt x
Specifies the intended artifact and user objective.
Visual State
Current full-page screenshot Si,t
Provides the authoritative visible state from which the next action is chosen.
Artifact
Source of hi , truncated to 12,000 characters
Reveals input handlers and interaction rules to support precise action generation.
Action Hints
Up to 40 DOM controls, each represented by its tag, selector hint, text, and bounding rectangle
Grounds ordinary DOM actions in executable selectors. Canvas-internal targets are inferred from the screenshot and source.
History
Including previous decision summary, action, execution observation, and termination status
Discourages repetitive actions and supports multi-step behavior.
Timing
Turn index, turn limit, clock-frame interval, and post-action advance budget
Allows the agent to choose pre-action delays and held-action durations.
Appendix
Table 4: Input information provided to the Execution Agent at turn t . Dynamic values are serialized into the user message, while the action schema and execution rules are specified in the system prompt.
Activates a DOM target or page coordinate. A held click emits mouse-down, advances the clock, and then emits mouse-up.
hover
selector or text. (optional) page coordinates (x,y) , waitMs
Moves the pointer to the resolved element or coordinate without activation.
mouseMove
relative (Δx,Δy) . (optional) durationMs , waitMs
Produces smooth relative motion. Under pointer lock it dispatches relative movement events rather than absolute cursor motion.
press
key. (optional) durationMs , waitMs
Emits an instantaneous key press or a key-down/key-up pair around a held interval.
combo
multiple keys. (optional) durationMs , waitMs
Holds the keys simultaneously and releases them in reverse order.
type
selector or text. (optional) waitMs
Fills the selected text input.
Appendix
Table 5: Grounded action interface used for different types of action ai,t∈A in the browser.
Ci
Corresponding Issues
0–10
Blank, crashed, corrupted, or impossible to evaluate.
20
Page renders but no meaningful primary interaction works.
35
A required entry action is correctly targeted but nonfunctional.
45
The artifact is substantially the wrong product or lacks the requested core experience.
60
The artifact is recognizable, but its central loop or required primary mechanic is blocked.
70
The main loop works, but a prominent required feature is missing or nonfunctional.
Appendix
Table 6: Usability-cap rubric. The judge chooses the highest cap consistent with the observed severity.
Figure 6: Changes in code size and action-trace attribution after training. The solid regions denote selected-trace code, while the pale regions denote the remaining generated code. Leaf-code coverage measures the proportion of total code attributed to depth-0 frames.
Setting
Value
Backbone
Qwen3.6-35B-A3B
Optimization algorithm
GRPO
Requests per update
16
Group size
5
Batch size
16
Optimizer
Adam
Appendix
Table 7: Optimization and generation hyperparameters.
Setting
Value
GUI Assessor judge
GPT-5.6 Terra
Interaction turns
8
Visual anchor model
GPT-5.5
ATC modulation
β=0.5
Trace pause-event cap
4,096 per action
Assessor workers
6 workers
Appendix
Table 8: Reward and execution settings. TP, PP, CP, EP, and ETP denote tensor, pipeline, context, expert, and expert-tensor parallelism, respectively.
Component
Input (K)
Output (K)
Total (K)
Cost ($)
ReAct Agent
60.34
2.04
62.38
0.1452
Step-Level Judge
43.44
5.69
49.13
0.1551
Meta-Level Judge
8.49
1.55
10.05
0.0356
Total
112.27
9.28
121.56
0.3360
Appendix
Table 9: Average per-sample evaluation token usage and estimated cost for GPT-5.6 Terra, with a budget of eight interaction turns. Token counts are in thousands (K). The turn judge jointly evaluates transitions and screenshots and is counted once. Costs assume 2and12 per million input and output tokens, respectively, excluding cache discounts.
Grader
WebGen-Bench
Cookie-Bench Mini Games
Pearson r↑
Acc. ↑
Content Pres. ↑
User Inter. ↑
Data Mgmt. ↑
Func. Test ↑
Data Display ↑
Design Val. ↑
Aesth. ↑
Func. ↑
Total ↑
Vanilla (N/A)
48.8
59.2
43.0
47.1
33.9
61.8
70.5
81.5
56.1
68.8
–
Src-Gemini
–
–
–
–
–
–
–
–
–
–
0.065
Src-Rubric
–
–
–
–
–
–
–
–
–
–
0.309
GUI Assessor
Qwen3.6-27B
49.2
64.7
39.6
51.2
32.2
65.6
71.7
75.4
62.4
68.9
0.613
Appendix
Table 10: Effect of judge selection on downstream RL performance and alignment with human experience. Vanilla denotes the Qwen3.6-35B-A3B base model without RL. Src-Gemini uses Gemini 3.1 Pro to evaluate source code directly, while Src-Rubric additionally incorporates rendered screenshots and task-specific rubrics. For GUI Assessor, each row specifies the backbone used for interactive evaluation. Pearson r measures correlation with human experiential ratings. Arrows indicate the preferred direction, dashes indicate unavailable results, and green highlights judge settings with Pearson r>0.8 .
Figure 7: Compared with the Qwen3.6-35B-A3B backbone, WebCatalyst shows markedly stronger robustness after interaction begins. While the backbone often produces plausible initial interfaces, its artifacts may break during execution—for example, rendering fails after launching Krunker, the 3D collection game turns black, Minecraft terrain disappears, and movement is blocked in the tank game. In contrast, WebCatalyst preserves coherent interactive states, with complete FPS scenes, visible collection effects and score updates, stable voxel terrain, and responsive projectile effects. These examples suggest that the gains extend beyond initial appearance to maintaining functionality and visual coherence throughout interaction. Krunker comparison (1/2). The request is “Krunker Game”. The paired views show the launch interface and the first-person scene reached during interaction.
Figure 8: Krunker comparison (2/2). Continuation of Figure 7 .
Figure 9: 3D collection-game comparison (1/2). The German request for “Alles nur Schaum” asks for a small yellow character, pink beer mugs, automatic collection on approach, a +1 score effect, and coin rewards in a cheerful 3D environment.
Figure 10: 3D collection-game comparison (2/2). Continuation of Figure 9 . WebCatalyst shows a high-quality 3D scene with a +1 collection effect and an updated score.
Figure 11: Minecraft comparison (1/2). The French request asks for a web version of Minecraft that preserves its visual style and core functionality. The views show the interface and the resulting voxel world after attempting to enter the game.
Figure 12: Minecraft comparison (2/2). Continuation of Figure 11 .
Figure 13: Two-player tank-game comparison (1/2). The Chinese request asks for a console-style tank duel with distinct player colors, five health points per player, one shot per turn, ricocheting bullets, and an arena that obstructs direct shots.
Figure 14: Two-player tank-game comparison (2/2). Continuation of Figure 13 . WebCatalyst’s paired views show a projectile and impact particles around the blue tank.
Figure 15: LLM product-website comparison (1/2). The enlarged top sections show typography, color, and hero composition; the full-page views show the organization and extent of the generated content.
Figure 16: LLM product-website comparison (2/2). Continuation of Figure 15 . WebCatalyst maintains the dark theme and shows diverse features across the full page.
Figure 17: Hotel-website comparison (1/2). The hero details show the landing-page presentation; the full-page overviews show accommodations, services, visual content, and lower-page sections.
Figure 18: Hotel-website comparison (2/2). Continuation of Figure 17 . WebCatalyst combines a photographic hero with image-card layouts, resulting in a visually polished and aesthetically appealing presentation.
Reinforcement learning for code generation trains models to write, repair, and optimize programs using execution-based and other task-specific feedback.