cs.AIApr 15, 2026
SaveRiskWebWorld: A Realistic Interactive Benchmark for GUI Agents in E-commerce Risk Management
Organizations: Ant International, Ant Group
Abstract
Graphical User Interface (GUI) agents show strong capabilities for automating web tasks, but existing interactive benchmarks primarily target benign, predictable consumer environments. Their effectiveness in high-stakes, investigative domains such as authentic e-commerce risk management remains underexplored. To bridge this gap, we present RiskWebWorld, the first highly realistic interactive benchmark for evaluating GUI agents in e-commerce risk management. RiskWebWorld features 1,513 tasks sourced from production risk-control pipelines across 8 core domains, and captures the authentic challenges of risk operations on uncooperative websites, partially environmental hijackments. To support scalable evaluation and agentic reinforcement learning (RL), we further build a Gymnasium-compliant infrastructure that decouples policy planning from environment mechanics. Our evaluation across diverse models reveals a dramatic capability gap: top-tier generalist models achieve 49.1% success, while specialized open-weights GUI models lag at near-total failure. This highlights that foundation model scale currently matters more than zero-shot interface grounding in long-horizon professional tasks. We also demonstrate the viability of our infrastructure through agentic RL, which improves open-source models by 16.2%. These results position RiskWebWorld as a practical testbed for developing robust digital workers.
Figures & tables
Figure 1: RiskWebWorld is a highly realistic, interactive benchmark for evaluating GUI agents in e-commerce risk management scenarios. It comprises 1,513 tasks across 8 business domains, with a scalable environment infrastructure that supports rigorous benchmarking and RL training.
Figure 2: RiskWebWorld benchmark infrastructure. Benchmark procedure is defined by the workflow (Sec. 3.4 ): First, remote browser devices are provisioned in the cloud and connected to the environment module through CDP. Then the environment is initialized with the task configuration (Sec. 3.2 ), and the agent (Sec. 3.1 ) starts to interact with the environment (Sec. 3.3 ) in a step-wise manner. After the loop terminates or truncates, evaluation results could serve for both benchmarking and training.
Figure 3: RiskWebWorld task statistics. (a) Task category distribution across 8 business domains. Exact values are given in parentheses. (b) Distributions of execution time and step count across task instances. (c) Comparison of step count distributions between SOP-accompanied and SOP-free tasks.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: An example of a real RiskWebWorld task. The task is to search for the origin and destination of a specific vessel, belonging to the Logistics and Supply Chain Tracking category. In case (a), the agent is given a task instruction and an SOP that provides step-by-step guidance. In case (b), the agent needs to complete the same task without the SOP, which requires stronger reasoning and exploration capabilities.
Figure 5: Environmental hijackments seen in RiskWebWorld tasks. Three representative cases are illustrated: (a) Verification Barrier (e.g., human verification CAPTCHA). (b) Unpredictable Pop-up (e.g., cookie consent forms). (c) Dynamic Content Shift (e.g., network load latency).
Figure 6: Agentic RL training curves. The blue line represents the raw mean reward across trajectories, while the orange line represents the smoothed reward using a moving average. The improvement is not strictly monotonic due to the inherent stochasticity of the environment (e.g., network uncertainty, website load latency). However, the overall upward trend indicates that the agent could learn to improve its performance based on the proposed RiskWebWorld Environment.
| Action | Definition |
|---|---|
| {"click": {"index": int}} | Click an interactive element by index . |
| {"input": {"index": int, "text": "string", "clear": bool}} | Type text into an element by index ; supports clear (clear first vs. append). |
| {"done": {"text": "string", "success": bool, "files_to_display": ["string"]}} | Finish the task and return final output ( text , success , optional files_to_display ). |
| {"search": {"query": "string", "engine": "duckduckgo"|"google"|"bing"}} | Run a web search with query on duckduckgo / google / bing . |
| {"navigate": {"url": "string", "new_tab": bool}} | Navigate to a URL, optionally in a new tab ( new_tab ). |
| {"scroll": {"down": bool, "pages": float, "index": int|null}} | Scroll up/down by page units ( pages ), optionally inside a container ( index ). |
Table 5: Action space used by RiskWebWorld agents.
Figure 7: Online monitoring platform. Task execution details are recorded and managed. The private information is redacted for security purposes.
Figure 8: Three main failure modes of specialized and generalist models: no action generation (action list is empty), degenerate repetitive actions (e.g., repeatedly clicking the same button), and invalid action format (e.g., missing required parameters, truncated action schema due to token limits). The correct action format is crucial for successful execution, and failure to adhere to it leads to immediate execution failures.
Figure 9: Failure trajectory of ShowUI-2B on a Customs Declaration & Clearance Status Audit task. The model correctly identifies the next subgoal and target information in its reasoning, but fails to generate valid executable actions. It repeatedly emits input actions with hallucinated text for non-input elements, and fails to output any valid click action to interact with the page. After receiving execution errors (Step 2 to Step 5), it does not update its internal state and continues to produce similar invalid actions.
Figure 10: Failure trajectory of UI-TARS-1.5-7B on a Product Risk Profile task. The model also identifies the next subgoal and target information, but fails to generate valid executable actions. It tends to output solve_slider_captcha regardless of the page state, and generates an invalid click action with a non-existent DOM index. It also misunderstands the action definition, applying search when it has already reached the target page. From Step 5 to Step 12, we could found that the model repeatedly emits invalid actions without updating its internal state based on execution feedback, which indicates a lack of error recovery and state tracking.
Figure 11: Failure trajectory of Gemini-3-Pro on a Logistics and Supply Chain Tracking task. The model successfully identifies the next subgoal and target information, while generate valid executable actions to interact with the page. However, it does not explore a feasible path to complete the task when the provided websites block access to the required information or present a 404 error. How to enable agents to proactively explore alternative paths outside of the provided candidate websites is an important future direction to enhance their robustness in real-world applications.
Figure 12: Failure trajectory of Qwen3.5-397B-A17B on a Secure Payment Channel Validation task. The model correctly identifies the candidate webpages to complete the task, but fails to generate the true answer. This failure is attributed to the model’s limited ability to perform multi-hop reasoning and information synthesis across multiple pages, where too much states and information is presented and the model struggles to identify and aggregate the relevant pieces to arrive at the correct conclusion.