Web agents--autonomous systems that navigate and execute tasks on the web on behalf of users--have the potential to transform how people interact with the digital world. However, the most capable web agents today rely on proprietary models with undisclosed training data and recipes, limiting scientific understanding, reproducibility, and community-driven progress. We believe agents for the open web should be built in the open. To this end, we introduce (1) MolmoWebMix, a large and diverse mixture of browser task demonstrations and web-GUI perception data and (2) MolmoWeb, a family of fully open multimodal web agents. Specifically, MolmoWebMix combines over 100K synthetic task trajectories from multiple complementary generation pipelines with 30K+ human demonstrations, atomic web-skill trajectories, and GUI perception data, including referring expression grounding and screenshot question answering. MolmoWeb agents operate as instruction-conditioned visual-language action policies: given a task instruction and a webpage screenshot, they predict the next browser action, requiring no access to HTML, accessibility trees, or specialized APIs. Available in 4B and 8B size, on browser-use benchmarks like WebVoyager, Online-Mind2Web, and DeepShop, MolmoWeb agents achieve state-of-the-art results outperforming similar scale open-weight-only models such as Fara-7B, UI-Tars-1.5-7B, and Holo1-7B. MolmoWeb-8B also surpasses set-of-marks (SoM) agents built on much larger closed frontier models like GPT-4o. We further demonstrate consistent gains through test-time scaling via parallel rollouts with best-of-N selection, achieving 94.7% and 60.5% pass@4 (compared to 78.2% and 35.3% pass@1) on WebVoyager and Online-Mind2Web respectively. We will release model checkpoints, training data, code, and a unified evaluation harness to enable reproducibility and accelerate open research on web agents.
Figures & tables
Figure 1: Overview of MolmoWeb . At each step, the agent receives the task instruction, current screenshot, and action history as input. It then predicts a natural language thought followed by a browser action.
Figure 2: Overview of MolmoWebMix dataset. To capture the diverse capabilities required for web agents, MolmoWebMix combines GUI perception data involving screenshot QA (top left) and referring expression grounding (top right) , with synthetic and human task trajectories demonstrating task completion on the web (bottom) . The human trajectories are further segmented into atomic skills ( Table 1 shows our skill taxonomy).
Figure 3: Data generation pipeline . Our pipeline consists of a task sampler, trajectory generation executed by either humans or AxTree agents, and success filtering.
Figure 4: Multi-agent trajectory generation pipeline. A Planner decomposes the task into sequential subgoals. For each subgoal, an Operator generates an action conditioned on the current state and subgoal. After execution, the resulting state is evaluated by a Verifier using screenshots to determine whether the subgoal is achieved. If not, the system continues attempting the same subgoal; otherwise, the Planner generates the next one. The loop repeats until the overall goal is completed.
Figure 5: Node traversal trajectory generation pipeline. Starting from a website’s homepage, we extract all reachable URLs and use an LLM to select the most informative ones for breadth-first expansion, pruning cycles to avoid loops. A deterministic, LLM-free procedure then generates trajectories by traversing the resulting paths via scrolling and clicking, with success verified through URL matching. Each validated trajectory is paired with an LLM-generated goal conditioned on the full trajectory, producing a task consistent with the produced path.
Skill
Description
go_to
Navigate directly to a specified URL
search
Enter a query into a search box and submit
find
Locate an element or information on the page
find_and_open
Locate an element or subpage and open it
find_and_click
Locate an element and click it
fill_form
Fill form fields with specified values
Table 1: Taxonomy of atomic web skills used in human annotation.
Dataset / Source
Trajectories
Steps
Domains
Avg steps
Mixture Ratio
Open-source
Mind2Web [ 11 ]
2.4K
17K
137
7.3
–
Yes
AutoWebWorld [ 58 ]
11.6K
255K
29
21.9
–
Yes
Fara [ 4 ]
145K
1.01M
–
6.9
–
No
MolmoWebMix-Traj
278.5K
2.2M
2.6K
13.2
0.80
Yes
Synthetic
AxTree Single-Agent
70K
793K
1.3K
11.4
0.35
Yes
Table 2: Statistics of MolmoWebMix . We also report its constituent sources compared to other datasets.
Action
Description
goto(url)
Navigate the browser to a URL
mouse_click(x, y, ...)
Click at a viewport coordinate
mouse_drag_and_drop(...)
Drag from one coordinate to another
scroll(delta_x, delta_y)
Scroll the page by an offset
scroll_at(x, y, dx, dy)
Scroll at a specific coordinate
hover_at(x, y)
Hover at a specific coordinate
Table 3: Action space of MolmoWeb . Spatial coordinates are normalized to [0,100] during training and denormalized to viewport pixels for browser execution. Figure 8 in the Appendix shows the distribution of actions in various subsets of MolmoWebMix .
Model
# Steps
WebVoyager [ 19 ]
Online-Mind2Web [ 61 ]
DeepShop [ 32 ]
WebTailBench [ 4 ]
API only
w/o vision
Axtree Agent (GPT-5)
30
70.6
41.9
40.7
29.2 †
Axtree Agent (Gemini-3-flash)
30
74.4
34.8
45.1
62.1 †
Axtree Agent (Gemini-3-flash)
100
85.6
44.8
55.3
63.5 †
w/ vision
Table 4: Comparison to prior work on browser-use benchmarks. For models marked with a *, the numbers reported are from Fara [ 4 ] . Italic Online-Mind2Web results are from the Online-Mind2Web auto-eval leaderboard. Since its unclear what judge was used by [ 4 ] for WebTailBench we used the WebVoyager judge to compute numbers marked with †. We highlight the best and second-best performing numbers for each benchmark in the sub-8B, open-weight model category.
Figure 6: Pass@k performance on WebVoyager and Online-Mind2Web Accuracy improves as k increases, with MolmoWeb -8B generally outperforming MolmoWeb -4B, and 100 steps outperforming 30 steps by a large margin. The MolmoWeb -8B model with max 100 steps reaches 94.7% and 60.5% at Pass@4.
Table 5: Data ablations . These ablations use an earlier version of MolmoWebMix with both fewer synthetic and human trajectories compared to the final experiments and use 30 inference steps.
Source
#trajs
DeepShop
WebV
OM2W
human
2.7K
19.8
35.4
9.0
synthetic
2.7K
24.4
53.0
16.8
Table 6: Learning from LLM-generated trajectories is more effective than learning from human demonstrations.
Strategy
WebV
greedy (temperature=0.0)
61.4
top-k (temperature=0.7, k=20)
67.4
top-p (temperature=0.7, p=0.8)
68.5
Table 7: Effect of sampling strategy. Greedy sampling performs significantly worse than random sampling strategies.
Model
ScreenSpot
ScreenSpot v2
Claude 3.7 [ 3 ]
—
87.6
OpenAI CUA [ 36 ]
—
87.9
Gemini-3-Pro [ 15 ]
—
93.7
UGround-7B [ 43 ]
86.7
76.3
Qwen2.5VL-7B [ 53 ]
85.5
89.3
Holo1-7B [ 2 ]
87.4
89.9
Table 8: We present MolmoWeb -4B’s and MolmoWeb -Ground’s (grounding specialist trained on grounding data only) performance on two prominent benchmarks: ScreenSpot and ScreenSpotV2 against open and closed baselines. Best and second-best numbers are highlighted.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Screenshot of the web browsing trajectory collection tool. The left half of the screen shows the webpage, the panel on the right is our chrome extension with annotations showing: (a) last captured screenshot; (b) instruction given to the annotator with a step-by-step breakdown; (c) input box for leaving a note when marking a step could not be completed; (d) final answer.
Figure 8: Action distribution breakdown across the dataset.
Figure 9: Sunburst chart of domains and websites in the dataset.