WatchPoint: Executable User Feedback for Real-World Agentic Web Development

Guanqun Yang; Wei Yang; Xueqing Liu

cs.SEAug 10, 2026

WatchPoint: Executable User Feedback for Real-World Agentic Web Development

Authors: Guanqun Yang, Wei Yang, Xueqing Liu

Organizations: Stevens Institute of Technology · University of Texas at Dallas

Abstract

When a professional web developer's code fails a test, they do not simply re-read the stack trace. They open the application in a browser, click buttons, inspect computed styles, and run diagnostic commands to understand what went wrong. Existing feedback mechanisms for coding agents rely on screenshots, LLM-as-a-judge scoring, or natural-language corrections, but few interact with the live application the way a developer would. We introduce WatchPoint, a simulated-user system that mimics real developer behavior by generating and executing diagnostic scripts against the running application, producing structured observations that guide the coding model's retry. Unlike prior approaches that target single-file edits or evaluate using non-executable metrics, we operate on Web-Bench, a benchmark of 50 multi-file web projects comprising 1,000 sequentially dependent tasks, verified by deterministic end-to-end tests. WatchPoint recovers 57.6% of the tasks it diagnoses, and a controlled user study confirms the simulation's realism: human testers achieve a comparable recovery rate (54.5%), providing evidence that automated diagnostic scripts can substitute for interactive human testing on sequential web development tasks. We further identify a pattern of capability gaps that governs when simulated-user feedback is helpful and when it should be withheld.

Figures & tables

Figure 1: Overview of the WatchPoint workflow. (a) The coding agent receives a task, iteratively edits code, and runs Playwright E2E tests. On failure, WatchPoint generates diagnostic feedback via two parallel agents. (b) Detail of the diagnostic mechanism: a browser agent executes a Playwright script to inspect the live page, a terminal agent runs a bash script to check workspace state, and the combined structured observations are fed back to the coding model for a retry attempt.

ID	Coding Model	Simulated User	Scaffold	P@1	P@2
—	Xu et al. ( 2025 )	—	non-agentic †	25.1%	35.3%
C0	Sonnet 4.6	—	Claude Code	13.4%	21.5%
C1	MiniMax M2.7	—	OpenHands	18.0%	26.6%
C2	MiniMax M2.7	GLM-5	OpenHands	16.4%	28.1%
C3	MiniMax M2.7	GPT-5.4	OpenHands	15.6%	27.9%
C4	GLM-5	—	OpenHands	25.0%	44.5%

Table 1: Aggregate results on Web-Bench. P@1 is the first-attempt pass rate; P@2 is the pass rate after at most one retry. Best agentic P@1 and P@2 are bolded .

			P@2 (% of tasks)			C2 Recovery (tasks)
Category	#Projects	WatchPoint	C1	C2	$\Delta$	Retried	Rec.	Rate
Standards	18	ON	23.3%	24.4%	+1.1	44	26	59.1%
UI	6	ON	24.2%	28.3%	+4.2	15	9	60.0%
State	4	ON	11.2%	18.8%	+7.5	7	3	42.9%
ON total	28		21.8%	24.5%	+2.7	66	38	57.6%
CSS	6	OFF	30.0%	30.8%	+0.8	12	6	50.0%

Table 2: Per-category results for M2.7 baseline (C1) vs. M2.7 + GLM-5 simulated user (C2). “ON/OFF” = simulated user enabled/disabled. P@2 columns show aggregate pass rates. Recovery columns show the per-task retry success: “Retried” = tasks that failed attempt 1 and were retried (not skipped); “Rec.” = passed on attempt 2; “Rate” = Rec./Retried.

Table 3: User study overview. (a) Project selection with automated P@2 for reference. (b) Human vs. WatchPoint comparison. All numbers are tasks out of 20 per project.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Coding model
	C0	C1	C2	C3	C4	C5
Model	Sonnet 4.6	M2.7	M2.7	M2.7	GLM-5	GLM-5
API	Agent SDK	MiniMax	MiniMax	MiniMax	OpenRouter	OpenRouter
Temperature	0.0	0.0	0.0	0.0	0.0	0.0
Max iters	30	30	30	30	30	30
Max tokens	default	16384	16384	16384	16384	16384

Appendix

Table 4: Model hyperparameters for all configurations. A dash indicates the parameter is not applicable. OH = OpenHands; CC = Claude Code; warm = warm-start; cold = cold-start.

Category	Retried	Browser	Terminal	Either
Standards	44	33 (75%)	37 (84%)	37 (84%)
UI	15	1 (7%)	15 (100%)	15 (100%)
State	7	2 (29%)	7 (100%)	7 (100%)
Total	66	36 (55%)	59 (89%)	59 (89%)

Appendix

Table 5: Diagnostic script execution on ON-category retried tasks (C2). “Browser” and “Terminal” indicate the fraction of retried tasks for which each agent produced at least one valid CHECK observation.

Project	C1 P@1	C2 P@1	Tasks flipped
table	30%	0%	7
angular	25%	0%	6
react	0%	25%	6
fastify	25%	0%	5
sequelize	50%	25%	5

Appendix

Table 6: Largest P@1 discrepancies between C1 and C2, which share the same coding model (M2.7, temp. 0). Since the simulated user cannot affect attempt 1, all differences are run-to-run variance.

Category	Project	WatchPoint	C1 P@2	C2 P@2	$\Delta$
Standards	esmodule	ON	0	9	+9
	dom	ON	3	8	+5
	typescript	ON	5	9	+4
	form	ON	1	5	+4
	flex	ON	8	10	+2
	selector	ON	6	8	+2

Appendix

Table 7: Per-project P@2 (tasks out of 20) for M2.7 baseline (C1) and M2.7 + GLM-5 simulated user (C2). Projects sorted by

\Delta

P@2 within each category.

Who	Task	Feedback	Rec.?
A	calc-7	“I think it’s fine”	Yes
A	calc-11	“I think it’s fine.”	No
A	chart-1	“the UI fails to render the chart entirely, which violates the requirement.”	Yes
A	chart-5	“the UI fails to display the line chart and its data points.”	No
A	expr-1	“the site cannot be reached”	No
A	flex-8	“Item 1, Item 2, and Item 3 are still in a horizontal row on the right, not stacking vertically”	No

Appendix

Table 8: All human feedback texts from the user study. “Rec.” = task was recovered (attempt 1 failed, attempt 2 passed after feedback). Participants are anonymized as A and B.

AI-Assisted Software Development

AI-assisted software development uses language models and coding agents to help people design, write, test, review, and maintain software.

See all 109 papers

WebUIProof: Benchmarking WebUI Code Generators with UI-Agent Execution Harness
Oct 2, 2026Yun-Yun Tsai, Yuning Mao, Shiqi Wang +2Code Generation Code Generation Evaluation
Columbia University · Meta SuperIntelligence Labs
Asuka-Bench: Benchmarking Code Agents on Underspecified User Intent and Multi-Round Refinement
Jun 4, 2026Xin Wang, Liangtai Sun, Yaoming Zhu +8Code Generation Language Model Generation Evaluation
University of Science and Technology of China · 2Independent researchers
UserSWE: Evaluating Coding Agents in Multi-Turn Conversations with Dynamic Users
Oct 21, 2026Zhan-Yu Wu, Ji Luo, Alyssa Wu +5Coding Agents AI Agent Evaluation
Tencent Hunyuan

Oct 2, 2026J/K move · Enter open · S save

WebUIProof: Benchmarking WebUI Code Generators with UI-Agent Execution Harness

Yun-Yun Tsai, Yuning Mao, Shiqi Wang, Junfeng Yang+1

Columbia University · Meta SuperIntelligence Labs

Code Generation Code Generation Evaluation Web Application Generation Automated Software Testing

Open paper AskPDF arXiv

Evaluating WebUI code generation at scale is difficult: outputs may compile and look plausible yet fail under user interaction, and prior benchmarks largely rely on free-form prompts with static checks (build success, screenshots) that miss functional correctness. We introduce WebUIProof, an execution-oriented benchmark that provides structured specifications and dense, executable interaction tests for WebUI generation across two task families: general WebUIs (e.g., dashboards, game, interactive tools) and 3D interactive simulation (e.g., particle/galaxy systems, physics dynamics). WebUIProof includes a UI-agent harness that runs executable interaction tests in a headless browser using an iterative plan--act--observe loop: it locates DOM elements, performs actions, observes resulting UI/DOM changes, and checks the specified assertions. We evaluate across eight commercial LLMs and observe frequent failures on interaction-based requirements even when pages render successfully, especially on 3D simulation interfaces. Finally, we show the UI-agent harness can provide outcome-level training signals. Training compact models (e.g., Qwen2.5 14B and MIMO 7B) with RL rewards derived from executable interaction tests improves functional completion while reducing build failures.

Abstract

AI-Assisted Software Development

Explore similar work

WebUIProof: Benchmarking WebUI Code Generators with UI-Agent Execution Harness

Asuka-Bench: Benchmarking Code Agents on Underspecified User Intent and Multi-Round Refinement

UserSWE: Evaluating Coding Agents in Multi-Turn Conversations with Dynamic Users