Evaluating WebUI code generation at scale is difficult: outputs may compile and look plausible yet fail under user interaction, and prior benchmarks largely rely on free-form prompts with static checks (build success, screenshots) that miss functional correctness. We introduce WebUIProof, an execution-oriented benchmark that provides structured specifications and dense, executable interaction tests for WebUI generation across two task families: general WebUIs (e.g., dashboards, game, interactive tools) and 3D interactive simulation (e.g., particle/galaxy systems, physics dynamics). WebUIProof includes a UI-agent harness that runs executable interaction tests in a headless browser using an iterative plan--act--observe loop: it locates DOM elements, performs actions, observes resulting UI/DOM changes, and checks the specified assertions. We evaluate across eight commercial LLMs and observe frequent failures on interaction-based requirements even when pages render successfully, especially on 3D simulation interfaces. Finally, we show the UI-agent harness can provide outcome-level training signals. Training compact models (e.g., Qwen2.5 14B and MIMO 7B) with RL rewards derived from executable interaction tests improves functional completion while reducing build failures.
Figures & tables
Figure 1: Examples of WebUI outputs in WebUIProof , spanning general applications (landing pages, code IDE, social/profile pages, games) and 3D simulation interfaces (physics dynamics, planetary scenes). These examples illustrate the benchmark’s emphasis on both visual fidelity and interaction-driven functionality. Each task is evaluated with executable interaction tests that check user actions (click/type/drag) and the resulting UI behavior.
Figure 2: Benchmark instance format of WebUIProof . We show a structured spec. with paired test cases. Our benchmark construction pipeline first lets the UI agent collect a sequence of visual snapshots and HTML/DOM structure from the given seed webpage. Based on observations from webpages, the UI agent provides a website summary and test case recommendations. The red and blue boxes illustrate element-to-test alignment: a subset of entries in "element_requirement" is grouped and paired with executable test cases, making element-level coverage explicit.
Benchmark
#Instr.
Instr. Form
Instr. Type
Instr. Length
Output Scope
UI Topic
UI Agent Eval.
#Tests
TCR(%)
DesignBench Xiao et al. (2025)
900
NL prompts
Gen./Edit/Repair
∼ 80-150
Single file (HTML)
General
✗
0
0%
WebDev-Arena Zheng et al. (2024)
> 10k
NL prompts
Gen.
∼ 60
Repo (App)
General
✗
0
0%
FrontendBench Zhu et al. (2025)
148
NL prompts
Gen.
∼ 50-100
Single file (HTML)
General
✓
148
∼ 90%
WebGen-Bench Liu et al. (2024b)
101
NL prompts
Gen.
∼ 40-70
Repo (App)
General
✓
∼ 600
49.24%
WebUIProof (Ours)
219
Structured Spec.
Gen.
∼ 270-550
Repo (App)
General+3D
✓
∼ 3k
97.23%
Table 1: Benchmarks comparison for WebUI code generation. We compare the benchmark based on instruction number, form, type, length, and output scope. UI Topic indicates general WebUIs (dashboards/tools/games) or additionally includes 3D simulation interfaces (classical mechanics/particles simulation). UI Agent Eval. denotes whether the evaluation involves a UI agent that executes test suites. Test coverage rate ( TCR ) measures element-level coverage. Our WebUIProof provides diverse UI topics (general+3D) with rich test suites and high TCR.
Figure 3: VisRL Overview : Given instruction prompts, the Policy LLM generates UI code candidates, which are built and rendered to produce screenshots; rewards combination (i) code validity with error fixing loop, (ii) vision-based screenshot scoring, and (iii) UI-agent interaction tests (element detection, task proposal, and execution success), and are used to update the policy via PPO.
Table 2: General WebUI generation on WebUIProof. We report the build failure rate (% of generations that fail to build/render), and the UI-agent verified task outcome among build-successful generations: No/Partial/Full completion rate (%). Overall Accuracy is computed as Acc=N#Full+0.5⋅#Partial×100 , where build failures are counted as No completion. Rows are grouped into eight proprietary (API) models and two self-trained models, including Qwen2.5 14B Instruct and MIMO 7B under three training variants: SFT, PPO w/ vision agent, and VisRL (ours).
Table 3: 3D simulation WebUI generation on WebUIProof. Same protocol as Table 2 : we report build failure rate and UI-agent verified task outcomes (%), including No/Partial/Full completion rate, and compute Accuracy, counting build failures as No completion. Results compare eight proprietary (API) models and self-trained models, including Qwen2.5 14B Instruct and MIMO 7B under three training variants: SFT baseline, PPO w/ vision agent, and VisRL (ours).
Table 7Figure 8
Figure 6: UI-agent execution examples on WebUIProof. Top: general WebUIs (e.g., interactive dashboards and widgets). Bottom: 3D simulation WebUIs (e.g., physics-driven interactive scenes). In all cases, the UI agent executes interaction tests by grounding to DOM/UI elements and performing actions (click/type/drag) to validate functional behavior.
Appendix figures & tables29 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Next.js project structure
Action Type
Detail
Click
Click any web element identified by its numerical label (e.g., Click [0])
Type
Clear textbox and input content (automatically hits ENTER) (e.g., Type [0])
Scroll
Scroll specific element or entire window (e,g, Scroll WINDOW up / down)
Hover
Hover for tooltips/effects, with precise coordinate positioning for charts (e.g., Hover [0:75,50] for 75% from left, 50% from top)
Wait
Pause for 5 seconds (for loading processes)
GoBack
Navigate to previous webpage
Appendix
Table 7: Action space of UI agent
Figure 8: System prompt for General WebUI task. Models generate React code generation with strict styling and output constraints.
Figure 9: System prompt for Three.js HTML simulation generation.
Figure 10: WebUI Category Distribution for WebUIProof.
Figure 11: Test Case Distribution for WebUIProof.
Figure 12: Benchmark Collection Pipeline.
Figure 13: Multimodal prompt (instructions + screenshots + HTML) used for WebUI test-case generation.
Figure 14: Visualization of SFT Training Curve
Figure 15: Training Infrastructure of VisRL.
Figure 16: Comparison of PPO training dynamics with and without a UI agent. Top: average reward (with 7-step moving average). Bottom: vision score and invalid action rate.
Figure 17: Radar plot for reward decompsition analysis
Figure 18: Sensitivity analysis of α Configuration
Model Name
Build Failure Rate (%)
Test Category
Build Sucess UI-Agent Verified Task (%)
Accuracy (Cat. %)
No Completion
Partial Completion
Full Completion
Claude Sonnet 4 Anthropic (2025)
Functionality Testing
10.15
23.86
30.28
42.21
30.37
Data Display Testing
15.04
15.76
43.41
51.29
UI Design Validation
18.52
29.63
37.03
51.85
Gemini 2.5 Pro DeepMind (2025)
Functionality Testing
16.15
19.44
22.47
32.19
32.21
Data Display Testing
12.89
17.19
51.58
60.17
Appendix
Table 10: Category accuracy of General WebUI task on Proprietary API.
Training Variants of Qwen 2.5 14B Instruct
Build Failure Rate (%)
Test Category
Build Sucess UI-Agent Verified Task (%)
Accuracy (Cat. %)
No Completion
Partial Completion
Full Completion
Baseline SFT
Functionality Testing
7.75
10.71
20.66
26.01
56.25
Data Display Testing
14.33
14.33
21.49
28.65
UI Design Validation
18.52
11.11
18.52
24.07
PPO W/ Vision agent
Functionality Testing
12.87
11.05
34.38
39.90
41.42
Data Display Testing
14.33
7.30
28.65
32.31
Appendix
Table 11: Category accuracy of General WebUI task on Self-Trained Model.
Model Name
Build Failure Rate (%)
Test Category
Build Sucess UI-Agent Verified Task (%)
Accuracy (Cat. %)
No Completion
Partial Completion
Full Completion
Claude Sonnet 4 Anthropic (2025)
Functionality Testing
53.15
16.79
30.08
38.47
0.00
Data Display Testing
21.43
42.86
35.71
57.14
UI Design Validation
68.35
6.33
25.32
Qwen 3 Coder 480B Alibaba Cloud (2025)
Functionality Testing
55.25
18.87
27.27
36.71
1.43
Data Display Testing
40.00
25.71
34.29
47.14
Appendix
Table 12: Category accuracy of 3D WebUI task on Proprietary API.
Training Variants of Qwen 2.5 14B Instruct
Build Failure Rate (%)
Test Category
Build Sucess UI-Agent Verified Task (%)
Accuracy (Cat. %)
No Completion
Partial Completion
Full Completion
Baseline SFT
Functionality Testing
52.16
12.82
19.74
26.16
0.00
Data Display Testing
48.57
25.71
25.71
38.57
UI Design Validation
68.35
12.66
18.99
25.32
PPO w/ Vision agent
Functionality Testing
59.38
27.97
20.96
34.95
0.00
Data Display Testing
21.43
35.71
28.57
46.43
Appendix
Table 13: Category accuracy of 3D WebUI task on Self-Trained Model.
Figure 19: WebUI interaction examples (sample 1 and 2). Each row shows a full UI state transition with agent-labeled elements and executed actions.
Figure 20: WebUI interaction examples (sample 3 to 5). Each row shows a full UI state transition with agent-labeled elements and executed actions.
Figure 21: Six simulation samples. Each subfigure shows the UI state and interaction annotation for one sample.
Figure 22: We present three webpages generated from self-trained model Qwen 2.5 14B w/ VisRL. The left-most webpage is a music band showcase portal. The upper-right webpage is a codecraft editor. The lower-right webpage is a AI website showdown.
Figure 23: Two webpages generated from self-trained model Qwen 2.5 14B w/ VisRL. Both of them are the social media viewer.
Figure 24: Two webpages generated from self-trained model Qwen 2.5 14B w/ VisRL. The left side is a 2048 game interface and the right side is a chess game simulator.
Figure 25: Two webpages generated from self-trained model Qwen 2.5 14B w/ VisRL. Both of them are the stock report generator.
Figure 26: Three webpages generated from self-trained model Qwen 2.5 14B w/ VisRL. The left side is a travel accommodation finder, the upper-right side is an AI agent workflow manager, and the lower-right side is a landing page for tech. company.
Figure 27: Four webpage demo generated from self-trained model Qwen 2.5 14B w/ VisRL. We provide simple prompt for each to showcase how they are generated. (1.) The upper-left demo: Create a 3D animation of a circular island that has multiple trees on it and a river flowing within. (2.) The upper-right demo: Create a 3D simulation of jelly cubes bouncing in a box. (3.) The lower-left demo: Create a 3D aquarium with fishes that swim around and avoid collisions with the tank walls. (4.) The lower-right demo: Create a 3D animation of a sail boat floating in the sea.
Figure 28: Same as prior figure, we provide simple prompt for each to showcase how they are generated. (1.) The upper-left demo: Create a 3D simulation of a mars rover moving on the surface of Mars. Users should be able to accelerate and control the direction of the car. Incorporate accurate suspension dynamics as the car moves on the rugged Mars surface. (2.) The upper-right demo: Create a visually engaging animation of a double pendulum using JavaScript. The animation should demonstrate the chaotic motion of two pendulums connected together. (3.) The lower-left demo: Simulate 3D paper lanterns rising into the night sky during a festival, with buoyancy and wind physics, above a reflective lake. (4.) The lower-right demo: Create a realistic 3D ocean scene with: dynamic water movement, accurate wave simulations, and reflective surface effects.
Figure 29: Same as prior figure, we provide simple prompt for each to showcase how they are generated. (1.) The upper-left demo: Create a 3D solar system simulation using Three.js with realistic planet orbits, textures, and lighting effects. (2.) The upper-right demo: Create a breakout-style game where a paddle at the bottom of the screen reflects a bouncing ball toward a grid of colored blocks above. The paddle moves left and right with arrow keys, and blocks disappear when hit by the ball. Each block is worth different points based on its color, and the game ends when all blocks are cleared or the ball falls below the paddle. (3.) The lower-left demo: Create a 3D animation of a top spinning. Use accurate physics to demonstrate its movements. (4.) The lower-right demo: Create a dinosaur running game where: (i) A dinosaur character can jump using spacebar/up arrow, (ii) Cacti and birds appear as obstacles from the right side, (iii) Ground moves continuously to create running effect (iv) Game ends when dinosaur hits an obstacle, (v) Simple pixel art style, and (vi) Day/night cycle every 500 points.
Figure 30: Breakdown of build-time errors across models in our WebUI generation benchmark. Bars show the number of failures by error type, including missing dependencies, syntax mistakes, reference issues, and invalid default exports. Higher stacks indicate more frequent build failures.
Large language models (LLMs) have demonstrated growing competence in web page generation. However, existing text-driven approaches rely on complex prompts that impose substantial demands on users and offer limited expressivity for page layout and cross-page visual coherence. Image-driven paradigms, which take UI screenshots as input, align more closely with real development workflows. However, current benchmarks focus primarily on visual fidelity and lack a systematic evaluation of the interaction capabilities in generated artifacts. To address this gap, we introduce UI2App, the first benchmark targeting interaction inference, the ability to recover application behavior from screenshots alone, without any textual or behavioral guidance. UI2App comprises 327 screenshots grouped into 45 state-coherent screenshot sets for runnable multi-route web applications. We design an end-to-end pipeline that evaluates each artifact along four dimensions: executability, navigation reachability, visual fidelity, and interaction inference. The interaction metric (IIS) assesses inferred interactions by functional correctness and state-management complexity, crediting any valid implementation rather than matching a single reference. Experiments on six frontier vision-language models reveal a marked capability mismatch between visual reconstruction and interaction realization: the visual-fidelity leader scores only 7.5 on IIS, ranking fourth and trailing the IIS leader by 5.2x. High-complexity interactions such as cross-page state remain a pervasive bottleneck, with half of the evaluated models scoring exactly zero on this dimension. Overall, the results indicate that inferring complete interaction behavior from static screenshots remains a key challenge for models.
Grace Man Chen, Litao Guo, Yifan Wu +5
1The Hong Kong University of Science and Technology (Guangzhou) · 2The Hong Kong University of Science and Technology
Large language models can generate interactive web interfaces, but reliable generative UI requires maintaining an executable artifact as user requests evolve. We introduce EvoGenUI-Bench, a benchmark for multi-turn interface maintenance comprising 150 five-turn tasks and 750 turns across three scenarios: information presentation, executable interaction, and tool-grounded external state. We execute generated artifacts in a browser and evaluate them using screenshots, source and DOM evidence, actor traces, and runtime logs. Beyond turn-level and episode-level success, we measure cross-turn retention with Adjacent Pass Retention. Across eight models, even the strongest achieves 74.9% Turn Pass while completing only 37.3% of five-turn episodes; APR further falls to 52.4% on tool-grounded tasks. Diagnostic analysis shows that presentation failures center on information architecture, interaction failures on derived-state propagation and affordance binding, and tool-grounded failures additionally involve external-state grounding and requirement decomposition. These results reframe generative UI evaluation from judging isolated outputs to testing whether interface behavior, derived state, external state, and assistant claims remain synchronized as the artifact evolves.
AI models excel at creating content, but typically render it with static, predefined interfaces. Specifically, the output of LLMs is often a markdown "wall of text". Generative UI is a long standing promise, where the model generates not just the content, but the interface itself. Until now, Generative UI was not possible in a robust fashion. We demonstrate that when properly prompted and equipped with the right set of tools, a modern LLM can robustly produce high quality custom UIs for virtually any prompt. When ignoring generation speed, results generated by our implementation are overwhelmingly preferred by humans over the standard LLM markdown output. In fact, while the results generated by our implementation are worse than those crafted by human experts, they are at least comparable in 50% of cases. We show that this ability for robust Generative UI is emergent, with substantial improvements from previous models. We also create and release PAGEN, a novel dataset of expert-crafted results to aid in evaluating Generative UI implementations, as well as the results of our system for future comparisons. Interactive examples can be seen at https://generativeui.github.io