EverydayBench: Evaluating Satisfactory Completion of Live User-Facing Digital Work
Authors: Fangrui Huang, Bingyang Wang, Chuhan Li, Yijiang Li, Arpandeep Khatua, Austin Shen, Riki R Z Choi, Cuiyuanxiu Chen, Jin Huang, Zachary Robertson, Kate Lin, Akshay Paruchuri, Sijun Tan, Xin Liu, Zhengmin Ni, Sanmi Koyejo, Ehsan Adeli
Organizations: Stanford University · Georgia Institute of Technology · University of California, Santa Barbara · University of California, San Diego · University of Michigan · Boston University · Google Research · University of California, Berkeley · flowith AI
Everyday digital tasks, from selecting products and planning trips to conducting research and creating artifacts, require AI agents to deliver useful outcomes rather than merely retrieve web information. Yet evaluating these outcomes on the live web is challenging, as a single request may admit multiple valid results, and changing web information can alter which results remain valid over time. Although existing benchmarks address aspects of this setting, they do not fully integrate predefined success criteria, current evidence verification, and deliverable-quality assessment for agent-selected outcomes. To address this gap, we introduce EverydayBench, a benchmark of 77 human-authored tasks spanning eight domains of everyday life and work. Each task specifies success criteria before execution without prescribing an answer or workflow, allowing agents to select their own outcomes. To evaluate submissions under fixed success criteria, EverydayBench combines (1) deterministic checks of explicit constraints, (2) verification of claims against agent-cited webpages retrieved at evaluation time, and (3) rubric-based assessment of final deliverable quality. We further validate the protocol through human judgments, repeated scoring under fixed evidence, and controlled grounding failures. Across 23 reported agent-model configurations, Claude Opus 5.5 and Claude Fable 5.1 tie for the highest strict pass rate at 54.55%; a controlled GPT-6 Sol effort sweep ranges from 29.87% at low effort to 45.45% at max effort. Reliability varies substantially by task type: GPT-5.6 Sol leads lookup, Fable 5.1 leads planning, Opus 5.5 and Fable 5.1 tie on comparison, and GPT-6 Astra and Sonnet 5.5 tie on customized deliverables. Together, these findings highlight the difficulty of delivering complete, well-supported, and usable outcomes for everyday user requests.
Figures & tables
Figure 1: Overview of the EverydayBench framework. A user request is paired with fixed success criteria; agents may submit different outcomes, which are evaluated through deterministic direct tests, source-grounded checks, and rubric-based component scoring.
Benchmark
Everyday Tasks
Real Web
Open Outcomes
Deliverable Form
Executable Constraint Checks
Source Verification
Human-Curated Per-Task Rubrics
Reference-answer web assistants
GAIA [ 17 ]
✓ –
✓
✗
Answer
✗
✗
✗
AssistantBench [ 29 ]
✓ –
✓
✗
Answer
✗
✗
✗
Sandboxed execution environments
WebArena [ 31 ]
✓ –
✗
✓ –
Answer/State
✓
✗
✗
OSWorld [ 26 ]
✓
✓ –
✓ –
State/File
✓
✗
✗
Table 1: Comparison of representative AI agent benchmarks, grouped by evaluation paradigm. Everyday Tasks covers users’ everyday personal or workplace requests. Real Web indicates interaction with live real-world websites. Open Outcomes indicates that evaluation accepts substantively different valid results. Deliverable Form is the scored output: Answer (short or fixed-field answer), State (environment state), File (edited or generated file), Trajectory (action history and screenshots), Report (long-form text report), and Multi-format (a structured outcome together with user-facing reports, plans, tables, files, or tools). Executable Constraint Checks uses task-specific deterministic checks of output constraints, environment states, or artifact properties, beyond reference-answer or recorded-action matching alone. Source Verification assesses whether cited evidence supports submitted claims. Human-Curated Per-Task Rubrics indicates that human curators author judging rubrics or criteria individually for each task, rather than relying on answer matching, a generic judging prompt, or LLM-generated criteria. Evaluation columns refer to task-scoring protocols, excluding auxiliary analyses. ✓ = yes, ✓ – = partial, ✗ = no.
Figure 2: Standard fixed-answer curation and our success-criteria curation. Human curators translate an open-ended user request into criterion-level direct tests, grounding requirements, and weighted 1/3/5 rubric anchors, allowing multiple valid outcomes while making specific failures observable.
Figure 3: Panel (a) compares multiple agent runs with near-synchronized judging at different points in time to validate benchmark stability; panel (b) holds agent run fixed to show time-sensitivity to web contents under near-synchronized, 20-day-delayed, and 30-day-delayed judging.
Figure 4: Seeded-fault detection by judges.
Evaluator variant
Mean reward
Mean SD ↓
One-prompt baseline
0.6096
0.0451
Rubric-only baseline
0.6671
0.0264
Full evaluator (ours)
0.5010
0.0141
Table 2: Stability of our judge compared with two ablated judges on 50 frozen traces, with five judge reruns per trace. Lower standard deviation indicates better run-to-run stability.
Agent
Model
Effort
Overall pass (n=77)
Lookup (n=16)
Compare (n=25)
Plan (n=12)
Analysis (n=11)
Build (n=13)
Runtime mean ± SD (m:ss)
Primary leaderboard
Claude Code
Claude Opus 5.5
High
54.55
50.00
48.00
58.33
54.55
69.23
7:07 ± 5:21
Claude Code
Claude Fable 5.1
High
54.55
43.75
48.00
66.67
54.55
69.23
8:21 ± 6:16
Codex
GPT-6.1 Sol
High
49.35
50.00
44.00
58.33
54.55
46.15
6:27 ± 4:18
Codex
GPT-6 Astra
High
49.35
50.00
36.00
58.33
36.36
76.92
5:31 ± 4:06
Claude Code
Claude Sonnet 5.5
High
48.05
50.00
32.00
50.00
45.45
76.92
4:04 ± 3:23
Table 3: Strict pass rates (%) across 23 agent–model configurations and five task categories. A task passes when its overall score is at least 85% and every rubric component is at least 3. Bold marks the best pass rate in each column.
Figure 5: Deductions per 100 components by failure type for the frontier and low-cost groups over the 77 tasks. A deduction is a component that scores below 5. Numbers in parentheses are the category totals for both groups.
Figure 6: Strict pass rate against runtime and inferred model cost. The connected blue path is the controlled Codex/GPT-6 Sol effort sweep from low to max. Cost combines uncached input, cache reads and writes, and output tokens at public list prices; the logarithmic axis exposes the full price range. Subscription-authenticated runs are normalized to the corresponding API-equivalent rate rather than treated as cash expenditure.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Agent
Model
Effort
Overall-score cutoff τ
75%
80%
85%
90%
95%
Primary leaderboard
Claude Code
Claude Opus 5.5
High
85.71
75.32
54.55
33.77
11.69
Claude Code
Claude Fable 5.1
High
81.82
74.03
54.55
33.77
15.58
Codex
GPT-6.1 Sol
High
79.22
68.83
49.35
29.87
12.99
Codex
GPT-5.6 Sol
High
62.34
50.65
42.86
29.87
11.69
Appendix
Table 4: Pass-rate sensitivity to the overall-score cutoff τ , with every component required to score at least 3. Entries are percentages over each configuration’s 77 tasks. ρscore is the Spearman correlation with mean overall score and ρ85 the Spearman correlation with pass rate at τ=85% , both over the 19 configurations of the primary leaderboard. Bold marks the primary cutoff.
Configuration
Browser
Shell network
Score
Host, search only
no
blocked
77.4
Host, computer use
yes
blocked
78.4
Container, 3000s budget
no
full
79.0
Container, 900s budget
no
full
73.5
Appendix
Table 5: Scores (%) across four web-access settings on the same 25 tasks.
Figure 7: Three execution trajectories from the computer-use runs. The final row is sampled from the InsureMyTrip browser recording and shows the agent progressively entering quote-search information.
Figure 8: Human validation interface for grounding agents.
Figure 9: Human validation interface for rubric judge.
Group
Configurations
Runs
Components
Runs violating format
Deductions
Share that are a 4
Frontier
4
308
1,320
1
756 (57.3 per 100)
72%
Low-cost
6
462
1,980
33
1,508 (76.2 per 100)
44%
Appendix
Table 6: Runs, components and deductions in the failure mode analysis. A deduction is a component that scores below 5; every component of a run that violates the required format counts as one deduction.
Category
Failure type
Definition
Intent recognition
Does not look into the need
A qualification that the user needs in order to use the answer is missing, given only as a general remark, or not where the user will read it; the missing point appears neither in the delivered files nor in the model’s own messages or tool calls.
Looks into the need, does not report it
As above, but the missing point appears in the model’s own messages or tool calls, or in a delivered file in the wrong place, in weak wording, or for only some of the items.
Research
Required items or data not found in full
Fewer items than the request requires; required fields or values absent, null or placeholders; a required type of source or a required data point not obtained.
Claim says more than the cited page
The cited page supports the core of the claim, and the claim adds details, scope, precision or certainty that the page does not state, or states the agent’s own estimate or inference as if the page said it.
Value not found on the cited page
The cited page has nothing for the value, including a general page (home page, listing, search results) cited instead of the page that would show it; or no source is given for the value.
Value differs from the cited page
The cited page shows a different value, date, status, name or place for the same thing.
Appendix
Table 7: The eleven failure types. Each deduction receives exactly one.
Per 100 components
Low-cost − frontier
Share that are a 4
Deductions F / L
Failure type
Frontier
Low-cost
95% interval
Frontier
Low-cost
Intent recognition
16.5
22.3
+3.5 to +7.9
69%
50%
218 / 441
Does not look into the need
7.5
12.6
+3.4 to +6.9
66%
42%
99 / 250
Looks into the need, does not report it
9.0
9.6
-1.3 to +2.6
71%
60%
119 / 191
Research
29.5
32.5
+0.5 to +5.6
75%
46%
389 / 643
Required items or data not found in full
3.5
7.3
+2.1 to +5.7
59%
23%
46 / 145
Appendix
Table 8: Deductions by failure type over the 77 tasks. The interval is the 95% bootstrap interval over tasks of the low-cost rate minus the frontier rate. Components of outputs that violate the required format score 0 and count among the deductions that are not a 4.
Web agent benchmarks test how reliably AI agents browse, navigate, and act on websites, including under long-horizon, ambiguous, and adversarial conditions.
AI agents may be able to assist with emails and documents, but can they reliably complete everyday online workflows on real websites? Everyday online tasks offer a realistic yet unsolved testbed for evaluating the next generation of AI agents. To this end, we introduce ClawBench, an evaluation framework comprising 153 everyday online tasks that people need to accomplish regularly in their lives and work, spanning 144 platforms across 15 categories, from completing purchases and booking appointments to submitting job applications. These tasks require capabilities beyond existing benchmarks, such as obtaining relevant information from user-provided documents, navigating multi-step workflows across diverse platforms, and write-heavy operations like filling in many detailed forms correctly. Unlike existing benchmarks that evaluate agents in offline sandboxes with static pages, ClawBench operates on production websites, preserving the full complexity, dynamic nature, and interaction challenges of real-world web environments. An interception layer captures and blocks the final submission request, ensuring safe evaluation without real-world side effects. Our evaluations of 8 frontier models show that both proprietary and open-source models complete only a small portion of these tasks. For example, Claude Sonnet 4.6 achieves only 33.3%, which exposes gaps in current AI agents. Progress on ClawBench brings us closer to AI agents that can function as general-purpose assistants.
Yuxuan Zhang, Yubo Wang, Yipeng Zhu +25
University of British Columbia · Vector Institute · Etude AI +9
Large language models (LLMs) are increasingly used for everyday assistance, yet existing benchmarks only partially reflect the requests users naturally make in practice. Real-world requests are often open-ended, casually specified, and context-dependent, requiring models not only to follow explicit instructions but also to infer unstated needs from user background and situational context. We introduce xDailyBench, a benchmark of 248 carefully curated tasks spanning 51 scenarios across personal life, white-collar work, learning and research, and cross-domain activities. The tasks are grounded in requests that users have actually completed or genuinely intended to accomplish with AI, and are evaluated with fine-grained binary rubrics covering both explicit and implicit requirements. We evaluate 11 frontier models under standardized agentic settings. The best models achieve a task-level score of 75.6%, while all models perform substantially worse on implicit than explicit requirements, with gaps no less than 9 percentage points. These results reveal implicit requirement inference as a persistent bottleneck for reliably satisfying real-world everyday user needs.
Recent advances in Large Language Models(LLMs) and agents have substantially improved the ability of AI systems to execute complex tasks. Yet existing benchmarks largely rely on researcher-selected tasks, leaving uncertain whether such progress extends to the work that real-world users actually demand from AI systems. We introduce StartupBench, an E2E agent benchmark grounded in market-validated AI startup products. Rather than defining tasks from pre-defined assumptions about useful agent capabilities, we systematically study AI products with demonstrated adoption, together with their product workflows and users, to identify real-world tasks for which AI has established practical demand across diverse professional domains. We translate these workflows into complete deliverable-oriented tasks and evaluate them with fine-grained rubrics capturing their complex requirements. Across representative models evaluated under a unified agent harness, even the strongest model successfully completes only approximately 30% of StartupBench, despite making substantial partial progress on many tasks. Further analysis identifies aspects like complex instruction following and domain-specific expertise as major sources of failure. Our results reveal that many market-validated workflows remain beyond the reliable capabilities of current general-purpose agents, establishing StartupBench as an empirical measure of progress toward E2E completions of real-world user tasks.