World-Model-Augmented Web Agents with Action Correction

Zhouzhou Shen; Xueyu Hu; Xiyun Li; Tianqing Fang; Juncheng Li; Shengyu Zhang

cs.AIFeb 17, 2026

World-Model-Augmented Web Agents with Action Correction

Authors: Zhouzhou Shen, Xueyu Hu, Xiyun Li, Tianqing Fang, Juncheng Li, Shengyu Zhang

Organizations: Zhejiang University · Tencent AI Lab

Abstract

Web agents based on large language models have demonstrated promising capability in automating web tasks. However, current web agents struggle to reason out sensible actions due to the limitations of predicting environment changes, and might not possess comprehensive awareness of execution risks, prematurely performing risky actions that cause losses and lead to task failure. To address these challenges, we propose WAC, a web agent that integrates model collaboration, consequence simulation, and feedback-driven action refinement. To overcome the cognitive isolation of individual models, we introduce a multi-agent collaboration process that enables an action model to consult a world model as a web-environment expert for strategic guidance; the action model then grounds these suggestions into executable actions, leveraging prior knowledge of environmental state transition dynamics to enhance candidate action proposal. To achieve risk-aware resilient task execution, we introduce a two-stage deduction chain. A world model, specialized in environmental state transitions, simulates action outcomes, which a judge model then scrutinizes to trigger action corrective feedback when necessary. Experiments show that WAC achieves absolute gains of 1.8% on VisualWebArena and 1.3% on Online-Mind2Web.

Figures & tables

Figure 1: Human and web agent task success rates across representative web benchmarks. Despite recent progress, state-of-the-art web agents still substantially underperform humans.

Figure 2: Overview of WAC . Given the current observation and task, the agent first decides whether world-model assistance is needed for action generation. Candidate actions are then proposed and lightly filtered. Each candidate is simulated by a world model to predict potential state changes, which are evaluated by a judge model to assign confidence scores. If no action exceeds a predefined threshold, feedback derived from low-scoring simulations is used to refine action proposals in a closed loop. Once a high-confidence action is identified, it is executed in the environment, and the process repeats until termination.

Figure 3: Comparison of action generation and pre-execution decision processes used by ReAct, WebDreamer, and our method ( WAC ). While ReAct directly executes a single proposed action and WebDreamer selects among simulated candidates, WAC enables collaborative action generation and feedback-driven action refinement prior to execution, leading to more robust action choices.

Model	Method	VisualWebArena	Online-Mind2Web
Qwen3-VL-Plus	ReAct	22.7	12.7
	WebDreamer	22.7	14.7
	WAC (ours)	24.5	16.0

Table 1: Results on VisualWebArena and Online-Mind2Web.

Model	Method	SR $\uparrow$
Qwen3-VL-Plus	ReAct	30.4
	WebDreamer	32.1
	+ Feedback-Driven Action Refinement	33.9
	+ Collaborative Action Generation	35.7

Table 2: Ablation study on VisualWebArena Classifieds.

Model	Method	SR $\uparrow$
GLM-4.5v	ReAct	21.4
	WebDreamer	33.9
	WAC (ours)	37.5

Table 3: Cross-model generalization results on VisualWebArena Classifieds.

Model	Method	SR $\uparrow$
GLM-4.6v	ReAct	7.9
	WebDreamer	12.7
	WAC (ours)	14.3

Table 4: Cross-model generalization results on VisualWebArena Reddit.

Model	Method	SR $\uparrow$
Qwen3-VL-Plus	WAC	35.7
Qwen3-VL-Plus	WAC (single-proposal)	35.7

Table 5: Performance of the single-proposal variant of WAC on VisualWebArena Classifieds.

Figure 4: An illustrative case where feedback-driven action refinement corrects an initially proposed action at the first execution step. Although only a single candidate action is initially generated, simulated outcome evaluation identifies it as risky, triggering refinement and leading to a revised action that places the agent on a successful trajectory.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Action Type	Parameters	Description
click	[id]	Click on the webpage element specified by the given element id.
type	[id], [content]	Type the provided content into the input field with the specified id. By default, the Enter key is pressed after typing unless disabled.
hover	[id]	Hover the mouse cursor over the webpage element specified by the given id.
press	[key_comb]	Simulate pressing a keyboard key or key combination ( e.g. , Ctrl+V).
scroll	[up/down]	Scroll the webpage vertically in the specified direction.
new_tab	–	Open a new, empty browser tab.

Appendix

Table 6: Action space.

Web Agents

Web agents use language and vision models to navigate websites, gather information, and complete tasks through interactive interfaces.

See all 176 papers

Modeling Distinct Human Interaction in Web Agents
Feb 19, 2026Faria Huq, Zora Zhiruo Wang, Zhanqiu Guo +6User Modeling Human-AI Collaboration
Carnegie Mellon University · Duke University
Agent-Editing World Model: Rethinking World Modeling for LLM Agents
Sep 23, 2026Shuang Sun, Guoxin Chen, Fanzhe Meng +6LLM World Models World Models
Gaoling School of Artificial Intelligence, Renmin University of China
WebXSkill: Skill Learning for Autonomous Web Agents
Apr 14, 2026Zhaoyang Wang, Qianhui Wu, Xuchao Zhang +12LLM Agent Skill Learning Long-Horizon LLM Agents
University of North Carolina at Chapel Hill · Microsoft

Feb 19, 2026J/K move · Enter open · S save

Modeling Distinct Human Interaction in Web Agents

Faria Huq, Zora Zhiruo Wang, Zhanqiu Guo, Venu Arvind Arangarajan+5

Carnegie Mellon University · Duke University

User Modeling Human-AI Collaboration Web Agents

Open paper AskPDF arXiv

Despite rapid progress in autonomous web agents, human involvement remains essential for shaping preferences and correcting agent behavior as tasks unfold. However, current agentic systems lack a principled understanding of when and why humans intervene, often proceeding autonomously past critical decision points or requesting unnecessary confirmation. In this work, we introduce the task of modeling human intervention to support collaborative web task execution. We collect CowCorpus, a dataset of 400 real-user web navigation trajectories containing over 4,200 interleaved human and agent actions. We identify four distinct patterns of user interaction with agents -- hands-off supervision, hands-on oversight, collaborative task-solving, and full user takeover. Leveraging these insights, we train language models (LMs) to anticipate when users are likely to intervene based on their interaction styles, yielding a 61.4-63.4% improvement in intervention prediction accuracy over base LMs. Finally, we deploy these intervention-aware models in live web navigation agents and evaluate them in a user study, finding a 36.8% increase in user-rated agent usefulness. Together, our results show structured modeling of human intervention leads to more adaptive, collaborative agents.

Abstract

Web Agents

Explore similar work

Modeling Distinct Human Interaction in Web Agents

Agent-Editing World Model: Rethinking World Modeling for LLM Agents

WebXSkill: Skill Learning for Autonomous Web Agents