MiniAppBench: Evaluating the Shift from Text to Interactive HTML Responses in LLM-Powered Assistants

Zuhao Zhang; Chengyue Yu; Yuante Li; Chenyi Zhuang; Linjian Mo; Shuai Li

cs.AIMar 10, 2026

MiniAppBench: Evaluating the Shift from Text to Interactive HTML Responses in LLM-Powered Assistants

Authors: Zuhao Zhang, Chengyue Yu, Yuante Li, Chenyi Zhuang, Linjian Mo, Shuai Li

Organizations: Inclusion AI, Ant Group, Hangzhou, Zhejiang, China · Shanghai Jiao Tong University, Shanghai, China · Carnegie Mellon University, Pittsburgh, PA, USA

Abstract

With the rapid advancement of Large Language Models (LLMs) in code generation, human-AI interaction is evolving from static text responses to dynamic, interactive HTML-based applications, which we term MiniApps. These applications require models to not only render visual interfaces but also construct customized interaction logic that adheres to real-world principles. However, existing benchmarks primarily focus on algorithmic correctness or static layout reconstruction, failing to capture the capabilities required for this new paradigm. To address this gap, we introduce MiniAppBench, the first comprehensive benchmark designed to evaluate principle-driven, interactive application generation. Sourced from a real-world application with 10M+ generations, MiniAppBench distills 500 tasks across six domains (e.g., Games, Science, and Tools). Furthermore, to tackle the challenge of evaluating open-ended interactions where no single ground truth exists, we propose MiniAppEval, an agentic evaluation framework. Leveraging browser automation, it performs human-like exploratory testing to systematically assess applications across three dimensions: Intention, Static, and Dynamic. Our experiments reveal that current LLMs still face significant challenges in generating high-quality MiniApps, while MiniAppEval demonstrates high alignment with human judgment, establishing a reliable standard for future research. Our homepage is available in miniappbench.github.io.

Figures & tables

Figure 1: The shift from text to MiniApps . Unlike static text, MiniApps transforms abstract explanations into intuitive visualizations and unlocks actionable tasks (e.g., diet tracking) that were previously impossible.

Figure 2: Failure Cases in Principle Adherence. MiniApps require models to capture and instantiate relevant real-world principles, while MiniAppEval proves effective due to its multi-component system design (eval-ref, code, playwright).

Benchmark	#Data	Task	Real-User	Div.	Comp.	RW-Prin.
MBPP	500	Algorithmic Problem Solving	✗	Low	High	Low
HumanEval	164	Algorithmic Problem Solving	✗	Low	High	Low
SWE-Bench	2,294	Repository-level Bug Fixing	✗	High	High	Low
MLE-Bench	75	Repository-level Software Engineering	✓	High	High	Low
PIQA	2,000	Physical Reasoning	✗	Low	Low	High
GSM8K	1,000	Mathematical Reasoning	✗	Low	Low	High

Table 1: Comparison of representative benchmarks across three families: code generation, real-world reasoning, and web development. Real-User indicates whether queries are sourced from real users. Div. (task diversity) is bucketed by the number of primary task categories (Low:

<3

, Mid:

3

–

5

, High:

>5

). Comp. (task complexity) is approximated by the number of steps in the evaluation protocol (Low: 1, Mid: 2–5, High:

>5

). RW-Prin. indicates whether solving the queries requires real-world principles (e.g., physics or commonsense); details are provided in Appendix A.4.2 .

Figure 3: Overview of the MiniAppBench dataset and construction process. (a)–(d) illustrate the dataset construction pipeline. (e) summarizes the dataset features and distributions (domain and difficulty), with the distribution of subclasses shown in the side bar charts. (f) presents representative MiniApps examples from six domains.

Figure 4: MiniAppEval vs. Previous Methods. Unlike brittle scripts or rigid comparisons, MiniAppEval integrates code inspection with dynamic execution. It complements human evaluation by verifying underlying physical principles and automating tedious testing scenarios to ensure robust assessment.

Panel of Table 2 — Table 2: Performance of models on MiniAppBench : Pass Rate, Token Consumption, and Inference Time

Figure 5: Overall model pass rate on MiniAppBench

Figure 6: Token Length & Inference Time vs Average pass rate

Exp.	Acc.	Prec.	Rec.	F1
MiniAppEval	89.62	83.87	85.25	84.55
w/o Code	70.66 $\downarrow$ 18.96	32.73 $\downarrow$ 51.14	60.00 $\downarrow$ 25.25	42.35 $\downarrow$ 42.20
w/o Agent	66.48 $\downarrow$ 23.14	12.90 $\downarrow$ 70.97	53.33 $\downarrow$ 31.92	20.78 $\downarrow$ 63.77
w/o Eval Ref	60.12 $\downarrow$ 29.50	89.47 $\uparrow$ 5.60	46.36 $\downarrow$ 38.89	61.08 $\downarrow$ 23.47

Table 3: A blation results (%) . Metrics include accuracy (Acc.), precision (Prec.), recall (Rec.), and F1. The superscript arrows denote the absolute change relative to the MiniAppEval .

Model	Method	T/T	T/F	F/T	F/F	Acc.
Gemini-3-Pro	MiniAppEval	15	2	8	30	81.82
-Pro-Preview	Double-Blind	11	6	2	36	85.45 $\uparrow$ 3.63
GPT-5.2	MiniAppEval	16	3	8	28	80.00
	Double-Blind	12	7	2	34	83.63 $\uparrow$ 3.63
Claude-	MiniAppEval	17	3	9	26	78.18
Opus-4.5	Double-Blind	11	9	0	35	83.63 $\uparrow$ 5.45

Table 4: Evaluation accuracy comparison between MiniAppEval and double-blind methods.

Model	JPass (%)	Acc. (%)	F1 (%)
GLM-4.7	18.5	94.5	94.5
Gemini-3-Pro-Preview	28.0	90.7	90.7
GPT-5.2	46.0	92.3	92.4

Table 5: MiniAppEval –human agreement across models with different performance levels on the 200-item subset. JPass denotes the judge pass rate.

Appendix figures & tables20 assets

Supplementary material from the paper’s appendix.

Appendix

Domain	Subclass	Count	Ratio (%)
Science	Chemical	46	9.20
	Biological Systems	44	8.80
	Physics	37	7.40
	Virtual Laboratory	35	7.00
	Geometry	25	5.00
	Total (Science)	187	37.40

Appendix

Table 6: The Data Domain Classification

Domain Distribution			Difficulty Distribution			Model Pass Rate
Domain	200 (%)	500 (%)	Level	200 (%)	500 (%)	Model	500 (%)	200 (%)	$\Delta$
Science	35.0	37.4	Easy	28.5	29.4	GPT-5.2	45.46	46.00	+0.54
Games	23.0	24.2	Medium	42.0	39.2	Claude-Opus-4-5	41.14	41.50	+0.36
Tools	13.0	11.4	Hard	29.5	31.4	Gemini-3-Pro-Preview	27.52	28.00	+0.48
Humanities	8.0	9.4				GLM-4.7	18.31	18.50	+0.19
Lifestyle	7.5	6.4				Qwen3-32B	0.66	0.50	-0.16

Appendix

Table 7: Representativeness of the 200-item subset compared with the full 500-task benchmark.

Domain	Easy	Medium	Hard	Total
Science	46 (25%)	72 (38%)	69 (37%)	187
Games	23 (19%)	49 (40%)	49 (40%)	121
Tools	13 (23%)	27 (47%)	17 (30%)	57
Visualization	30 (54%)	20 (36%)	6 (11%)	56
Humanities	16 (34%)	18 (38%)	13 (28%)	47
Lifestyle	19 (59%)	10 (31%)	3 (9%)	32

Appendix

Table 8: Domain by difficulty distribution of the 500-task benchmark.

Model	Temperature	Top- $p$	Max tokens
GPT-5.2	1.0	1.0	128,000
GPT-5.1	1.0	1.0	400,000
Claude-Opus-4.5	1.0	1.0	200,000
Claude-Sonnet-4.5	1.0	1.0	200,000
Gemini-3-Pro-Preview	0.8	0.95	65,536
Gemini-3-Flash	0.8	0.95	65,536

Appendix

Table 9: Decoding settings for all evaluated models.

$T$	GPT-5.2			GLM-4.7			Gemini-3-Pro-Preview
$T$	JPass	Acc.	F1	JPass	Acc.	F1	JPass	Acc.	F1
0.6	35.5	73.0	65.8	28.0	78.5	53.8	30.5	78.0	61.4
0.7	25.0	73.5	61.3	16.5	91.0	74.3	8.5	76.0	31.4
0.8	11.0	67.5	40.4	4.5	83.0	26.1	7.0	75.5	26.9
0.9	0.0	56.5	–	0.0	81.5	–	0.0	73.5	–

Appendix

Table 10: GPT-5.2 judge: 200-sample human-agreement results across thresholds and representative generation models.

$T$	GPT-5.2			GLM-4.7			Gemini-3-Pro-Preview
$T$	JPass	Acc.	F1	JPass	Acc.	F1	JPass	Acc.	F1
0.6	53.5	70.0	69.1	43.5	78.0	68.6	48.5	70.0	55.2
0.7	42.5	82.0	79.1	20.0	76.5	49.5	36.5	82.0	67.3
0.8	28.5	85.0	79.2	9.0	76.5	33.8	21.5	91.0	77.5
0.9	23.5	80.0	70.1	2.0	74.5	10.5	6.5	87.0	48.0

Appendix

Table 11: Claude-Sonnet-4.5 judge: 200-sample human-agreement results across thresholds and representative generation models.

$T$	GPT-5.2			GLM-4.7			Gemini-3-Pro-Preview
$T$	JPass	Acc.	F1	JPass	Acc.	F1	JPass	Acc.	F1
0.6	77.0	66.5	72.2	48.0	66.5	49.6	67.0	59.5	56.7
0.7	66.0	77.5	79.5	41.0	72.5	53.8	53.0	73.5	66.7
0.8	58.0	85.5	85.7	29.0	72.5	42.1	32.0	75.5	58.1
0.9	28.0	74.5	64.3	10.0	79.5	28.1	19.5	77.0	50.0

Appendix

Table 12: GLM-4.7 judge: 200-sample human-agreement results across thresholds and representative generation models.

$T$	GPT-5.2			GLM-4.7			Gemini-3-Pro-Preview
$T$	JPass	Acc.	F1	JPass	Acc.	F1	JPass	Acc.	F1
0.6	85.5	58.0	67.4	55.5	63.0	50.0	62.0	64.5	59.9
0.7	81.5	62.0	69.6	50.5	68.0	53.6	55.0	71.5	65.0
0.8	75.5	68.0	73.1	39.0	79.5	64.3	50.5	76.0	68.8
0.9	59.5	84.0	84.5	21.5	83.0	57.5	31.5	67.0	43.1

Appendix

Table 13: Qwen3-Coder-480B-A35B-Instruct judge: 200-sample human-agreement results across thresholds and representative generation models.

$T$	GPT-5.2			GLM-4.7			Gemini-3-Pro-Preview
$T$	JPass	Acc.	F1	JPass	Acc.	F1	JPass	Acc.	F1
0.6	57.5	85.6	89.1	38.5	77.3	65.3	45.5	89.8	90.0
0.7	55.0	88.2	90.6	36.0	81.8	68.2	41.5	80.2	78.0
0.8	46.0	92.3	92.4	18.5	94.5	94.5	28.0	90.7	90.7
0.9	32.0	68.6	68.8	11.5	79.9	52.3	23.0	58.4	36.9

Appendix

Table 14: Gemini-3-Pro-Preview judge: 200-sample human-agreement results across thresholds and representative generation models.

Model	$T=0.6$	$T=0.7$	$T=0.8$	$T=0.9$
GPT-5.2	71.1 (1)	62.0 (1)	45.5 (1)	35.4 (1)
Claude-Opus-4.5	62.0 (2)	52.6 (2)	41.1 (2)	23.1 (2)
GPT-5.1	55.2 (3)	45.5 (3)	32.0 (3)	21.1 (3)
Gemini-3-Pro-Preview	47.1 (4)	36.8 (4)	27.5 (4)	13.8 (5)
Claude-Sonnet-4.5	42.4 (5)	33.6 (5)	26.4 (5)	14.2 (4)
GLM-4.7	35.5 (7)	26.8 (8)	18.3 (6)	9.3 (7)

Appendix

Table 15: Rankings across different thresholds. Pass rates are shown with ranks in parentheses.

Comparison	Spearman’s $\rho$	Kendall’s $\tau$	Max Rank Diff.
$T=0.6$ vs. $T=0.8$	0.9941	0.9667	1
$T=0.7$ vs. $T=0.8$	0.9824	0.9333	2
$T=0.9$ vs. $T=0.8$	0.9765	0.9000	2

Appendix

Table 16: Ranking stability relative to

T=0.8

$T$	GLM-4.7		GPT-5.2		Gemini-3-Pro-Preview
$T$	Acc.	F1	Acc.	F1	Acc.	F1
0.6	52.6	54.1	69.3	80.0	62.9	74.0
0.7	61.7	56.3	72.5	81.6	72.6	79.1
0.8	73.4	63.7	81.7	86.8	79.2	81.3
0.9	81.8	65.9	84.3	87.1	75.1	70.3

Appendix

Table 17: Average-score criterion: 200-sample human-agreement results across thresholds.

$T$	GLM-4.7		GPT-5.2		Gemini-3-Pro-Preview
$T$	Acc.	F1	Acc.	F1	Acc.	F1
0.6	57.1	54.8	72.5	81.6	69.5	77.6
0.7	59.1	54.0	76.5	83.8	76.1	80.5
0.8	69.5	61.2	81.0	86.4	76.1	78.7
0.9	76.0	61.9	79.7	83.9	69.5	66.7

Appendix

Table 18: Median-score criterion: 200-sample human-agreement results across thresholds.

Temp.	GPT-5.2			GLM-4.7			Gemini-3-Pro-Preview
Temp.	JPass	Acc.	F1	JPass	Acc.	F1	JPass	Acc.	F1
0.5	54.5	89.0	88.8	20.5	97.0	92.3	35.5	91.0	85.5
0.8	52.5	91.0	90.6	18.5	98.0	94.6	33.5	93.0	88.3
1.0	46.0	92.3	92.4	18.5	94.5	94.5	28.0	90.7	90.7
1.2	45.0	93.5	92.7	15.5	97.0	91.2	26.5	95.0	90.6

Appendix

Table 19: Temperature sensitivity of Gemini-3-Pro-Preview judge at threshold 0.8.

Generation Model	Run 1	Run 2	Run 3	Run 4	Run 5	Std.	CV
GPT-5.2	46.0	48.5	46.0	49.0	49.0	1.58	0.0330
GLM-4.7	18.5	17.5	18.5	16.5	17.5	0.84	0.0474
Gemini-3-Pro-Preview	28.0	30.0	28.5	30.5	28.5	1.14	0.0393

Appendix

Table 20: Pass-rate stability over five generations from the same model.

Generation Model	5/5 Consensus	$\geq$ 4/5 Consensus	Fleiss’ $\kappa$
GPT-5.2	93.5	100.0	0.948
GLM-4.7	94.5	99.5	0.924
Gemini-3-Pro-Preview	89.5	100.0	0.905

Appendix

Table 21: Per-query consistency over five generations.

Generation Model	Eval 1	Eval 2	Eval 3	Eval 4	Eval 5	Std.	CV
GPT-5.2	46.0	47.0	47.5	44.5	46.5	1.14	0.025
GLM-4.7	18.5	18.5	18.0	18.0	17.5	0.42	0.023
Gemini-3-Pro-Preview	28.0	29.0	27.5	28.5	30.0	1.00	0.035

Appendix

Table 22: Pass-rate stability over five repeated evaluations of the same samples.

Generation Model	5/5 Consensus	$\geq$ 4/5 Consensus	Fleiss’ $\kappa$
GPT-5.2	95.0	100.0	0.960
GLM-4.7	96.5	100.0	0.953
Gemini-3-Pro-Preview	95.5	100.0	0.959

Appendix

Table 23: Per-query consistency over five repeated evaluations.

Action	Description
browser_click	Simulates mouse clicks on UI control elements. Supports configurable mouse buttons (left, right, middle) and both single and double clicks. Commonly used for selecting items, activating controls, or triggering events.
browser_type	Simulates keyboard input for entering text, pressing keys, or invoking shortcuts (e.g., Ctrl+C, Enter). Enables fine-grained control over application behavior and supports both functional input and text entry.
browser_fill_form	Fills form fields with specified values, supporting various input types including text inputs, checkboxes, radio buttons, and dropdown selections. Allows batch form filling for efficient interaction with complex forms.
browser_evaluate	Executes JavaScript code to query DOM state or perform complex operations. Enables agents to extract information, manipulate page elements, or verify application state programmatically. Particularly useful for analyzing CSS styles, color schemes, and dynamic content.
browser_wait_for	Waits for specific conditions such as element appearance, text changes, or custom JavaScript predicates. Essential for handling asynchronous operations and ensuring elements are ready before interaction.

Appendix

Table 24: Summary of action types in the web-based evaluation environment.

Figure 7: Multi-dimensional trajectory analysis. The figure contains nine subplots: (a) tokens vs. step (scatter); (b) token distribution by step (boxplot); (c) average tokens vs. step with dispersion (mean/median/std); (d) cumulative time vs. step; (e) time interval vs. step (log scale); (f) tokens vs. time interval; (g) prompt tokens vs. completion tokens; (h) histogram of step values; (i) token statistics by step range.

Web Application Generation

Web application generation uses AI to turn natural-language, visual, or multimedia specifications into functional, interactive websites and applications.

See all 40 papers

WebCraftBench: Evaluating Web Application Generation from a Software Testing Perspective
Sep 14, 2026Chenxu Liu, Zilu Zou, Peizhong Gao +9Automated Software Testing Web Application Generation
Hunyuan Team, Tencent · Tsinghua University · Peking University
Cookie-Bench: Continuous On-screen Key Interaction Evaluation for Web Generation
May 28, 2026Haoyue Yang, Zhangxiao Shen, Fan Ding +8Code Generation Evaluation Automated Evaluation
Baidu Inc., Beijing, China
UI2App: Benchmarking Visual Interaction Inference in Executable Web Application Generation
Jul 7, 2026Grace Man Chen, Litao Guo, Yifan Wu +5VLM Evaluation Web Application Generation
1The Hong Kong University of Science and Technology (Guangzhou) · 2The Hong Kong University of Science and Technology

Sep 14, 2026J/K move · Enter open · S save

WebCraftBench: Evaluating Web Application Generation from a Software Testing Perspective

Chenxu Liu, Zilu Zou, Peizhong Gao, Jiawen Tao+8

Hunyuan Team, Tencent · Tsinghua University · Peking University

Automated Software Testing Web Application Generation Automated Evaluation

Open paper AskPDF arXiv

Human evaluation provides a direct measure of the quality of LLM-generated web applications. However, fitting human judgments through automated evaluation remains challenging. Static benchmarks can credit functionality that exists in source code but is unreachable at runtime. Interactive benchmarks exercise the application, yet incomplete exploration can cause them to miss implemented functionality and confound application defects with agent execution failures. To address these limitations, we propose WebCraftBench, an interactive benchmark for evaluating web application generation from a software testing perspective. WebCraftBench instruments each generated application and uses code coverage to guide an agent in exploring its functionality through user-simulated interactions. It then abstracts the interaction trace into a state-transition graph and evaluates the application along three dimensions: visual aesthetics, usability, and requirement alignment. By separating exploration from scoring, WebCraftBench collects runtime evidence without constraining exploration to predefined acceptance criteria. WebCraftBench comprises 369 real-world user requirements and 5,088 acceptance criteria. Evaluation of 17 frontier LLMs reveals distinct strengths across the three dimensions, with no model leading on every dimension. On 197 validated sessions sampled from an internal arena, WebCraftBench achieves 85.3% agreement with human preferences, with agreement generally increasing as the score difference between paired applications grows. Further experiments show that coverage guidance improves exploration coverage and the model rankings remain stable when the judge model is replaced.

Abstract

Web Application Generation

Explore similar work

WebCraftBench: Evaluating Web Application Generation from a Software Testing Perspective

Cookie-Bench: Continuous On-screen Key Interaction Evaluation for Web Generation

UI2App: Benchmarking Visual Interaction Inference in Executable Web Application Generation