EverydayBench: Evaluating Satisfactory Completion of Live User-Facing Digital Work
Authors: Fangrui Huang, Bingyang Wang, Chuhan Li, Yijiang Li, Arpandeep Khatua, Austin Shen, Riki R Z Choi, Cuiyuanxiu Chen, Jin Huang, Zachary Robertson, Kate Lin, Akshay Paruchuri, Sijun Tan, Xin Liu, Zhengmin Ni, Sanmi Koyejo, Ehsan Adeli
Organizations: Stanford University · Georgia Institute of Technology · University of California, Santa Barbara · University of California, San Diego · University of Michigan · Boston University · Google Research · University of California, Berkeley · flowith AI
Everyday digital tasks, from selecting products and planning trips to conducting research and creating artifacts, require AI agents to deliver useful outcomes rather than merely retrieve web information. Yet evaluating these outcomes on the live web is challenging, as a single request may admit multiple valid results, and changing web information can alter which results remain valid over time. Although existing benchmarks address aspects of this setting, they do not fully integrate predefined success criteria, current evidence verification, and deliverable-quality assessment for agent-selected outcomes. To address this gap, we introduce EverydayBench, a benchmark of 77 human-authored tasks spanning eight domains of everyday life and work. Each task specifies success criteria before execution without prescribing an answer or workflow, allowing agents to select their own outcomes. To evaluate submissions under fixed success criteria, EverydayBench combines (1) deterministic checks of explicit constraints, (2) verification of claims against agent-cited webpages retrieved at evaluation time, and (3) rubric-based assessment of final deliverable quality. We further validate the protocol through human judgments, repeated scoring under fixed evidence, and controlled grounding failures. Across 23 reported agent-model configurations, Claude Opus 5.5 and Claude Fable 5.1 tie for the highest strict pass rate at 54.55%; a controlled GPT-6 Sol effort sweep ranges from 29.87% at low effort to 45.45% at max effort. Reliability varies substantially by task type: GPT-5.6 Sol leads lookup, Fable 5.1 leads planning, Opus 5.5 and Fable 5.1 tie on comparison, and GPT-6 Astra and Sonnet 5.5 tie on customized deliverables. Together, these findings highlight the difficulty of delivering complete, well-supported, and usable outcomes for everyday user requests.
Figures & tables
Figure 1: Overview of the EverydayBench framework. A user request is paired with fixed success criteria; agents may submit different outcomes, which are evaluated through deterministic direct tests, source-grounded checks, and rubric-based component scoring.
Benchmark
Everyday Tasks
Real Web
Open Outcomes
Deliverable Form
Executable Constraint Checks
Source Verification
Human-Curated Per-Task Rubrics
Reference-answer web assistants
GAIA [ 17 ]
✓ –
✓
✗
Answer
✗
✗
✗
AssistantBench [ 29 ]
✓ –
✓
✗
Answer
✗
✗
✗
Sandboxed execution environments
WebArena [ 31 ]
✓ –
✗
✓ –
Answer/State
✓
✗
✗
OSWorld [ 26 ]
✓
✓ –
✓ –
State/File
✓
✗
✗
Table 1: Comparison of representative AI agent benchmarks, grouped by evaluation paradigm. Everyday Tasks covers users’ everyday personal or workplace requests. Real Web indicates interaction with live real-world websites. Open Outcomes indicates that evaluation accepts substantively different valid results. Deliverable Form is the scored output: Answer (short or fixed-field answer), State (environment state), File (edited or generated file), Trajectory (action history and screenshots), Report (long-form text report), and Multi-format (a structured outcome together with user-facing reports, plans, tables, files, or tools). Executable Constraint Checks uses task-specific deterministic checks of output constraints, environment states, or artifact properties, beyond reference-answer or recorded-action matching alone. Source Verification assesses whether cited evidence supports submitted claims. Human-Curated Per-Task Rubrics indicates that human curators author judging rubrics or criteria individually for each task, rather than relying on answer matching, a generic judging prompt, or LLM-generated criteria. Evaluation columns refer to task-scoring protocols, excluding auxiliary analyses. ✓ = yes, ✓ – = partial, ✗ = no.
Figure 2: Standard fixed-answer curation and our success-criteria curation. Human curators translate an open-ended user request into criterion-level direct tests, grounding requirements, and weighted 1/3/5 rubric anchors, allowing multiple valid outcomes while making specific failures observable.
Figure 3: Panel (a) compares multiple agent runs with near-synchronized judging at different points in time to validate benchmark stability; panel (b) holds agent run fixed to show time-sensitivity to web contents under near-synchronized, 20-day-delayed, and 30-day-delayed judging.
Figure 4: Seeded-fault detection by judges.
Evaluator variant
Mean reward
Mean SD ↓
One-prompt baseline
0.6096
0.0451
Rubric-only baseline
0.6671
0.0264
Full evaluator (ours)
0.5010
0.0141
Table 2: Stability of our judge compared with two ablated judges on 50 frozen traces, with five judge reruns per trace. Lower standard deviation indicates better run-to-run stability.
Agent
Model
Effort
Overall pass (n=77)
Lookup (n=16)
Compare (n=25)
Plan (n=12)
Analysis (n=11)
Build (n=13)
Runtime mean ± SD (m:ss)
Primary leaderboard
Claude Code
Claude Opus 5.5
High
54.55
50.00
48.00
58.33
54.55
69.23
7:07 ± 5:21
Claude Code
Claude Fable 5.1
High
54.55
43.75
48.00
66.67
54.55
69.23
8:21 ± 6:16
Codex
GPT-6.1 Sol
High
49.35
50.00
44.00
58.33
54.55
46.15
6:27 ± 4:18
Codex
GPT-6 Astra
High
49.35
50.00
36.00
58.33
36.36
76.92
5:31 ± 4:06
Claude Code
Claude Sonnet 5.5
High
48.05
50.00
32.00
50.00
45.45
76.92
4:04 ± 3:23
Table 3: Strict pass rates (%) across 23 agent–model configurations and five task categories. A task passes when its overall score is at least 85% and every rubric component is at least 3. Bold marks the best pass rate in each column.
Figure 5: Deductions per 100 components by failure type for the frontier and low-cost groups over the 77 tasks. A deduction is a component that scores below 5. Numbers in parentheses are the category totals for both groups.
Figure 6: Strict pass rate against runtime and inferred model cost. The connected blue path is the controlled Codex/GPT-6 Sol effort sweep from low to max. Cost combines uncached input, cache reads and writes, and output tokens at public list prices; the logarithmic axis exposes the full price range. Subscription-authenticated runs are normalized to the corresponding API-equivalent rate rather than treated as cash expenditure.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Agent
Model
Effort
Overall-score cutoff τ
75%
80%
85%
90%
95%
Primary leaderboard
Claude Code
Claude Opus 5.5
High
85.71
75.32
54.55
33.77
11.69
Claude Code
Claude Fable 5.1
High
81.82
74.03
54.55
33.77
15.58
Codex
GPT-6.1 Sol
High
79.22
68.83
49.35
29.87
12.99
Codex
GPT-5.6 Sol
High
62.34
50.65
42.86
29.87
11.69
Appendix
Table 4: Pass-rate sensitivity to the overall-score cutoff τ , with every component required to score at least 3. Entries are percentages over each configuration’s 77 tasks. ρscore is the Spearman correlation with mean overall score and ρ85 the Spearman correlation with pass rate at τ=85% , both over the 19 configurations of the primary leaderboard. Bold marks the primary cutoff.
Configuration
Browser
Shell network
Score
Host, search only
no
blocked
77.4
Host, computer use
yes
blocked
78.4
Container, 3000s budget
no
full
79.0
Container, 900s budget
no
full
73.5
Appendix
Table 5: Scores (%) across four web-access settings on the same 25 tasks.
Figure 7: Three execution trajectories from the computer-use runs. The final row is sampled from the InsureMyTrip browser recording and shows the agent progressively entering quote-search information.
Figure 8: Human validation interface for grounding agents.
Figure 9: Human validation interface for rubric judge.
Group
Configurations
Runs
Components
Runs violating format
Deductions
Share that are a 4
Frontier
4
308
1,320
1
756 (57.3 per 100)
72%
Low-cost
6
462
1,980
33
1,508 (76.2 per 100)
44%
Appendix
Table 6: Runs, components and deductions in the failure mode analysis. A deduction is a component that scores below 5; every component of a run that violates the required format counts as one deduction.
Category
Failure type
Definition
Intent recognition
Does not look into the need
A qualification that the user needs in order to use the answer is missing, given only as a general remark, or not where the user will read it; the missing point appears neither in the delivered files nor in the model’s own messages or tool calls.
Looks into the need, does not report it
As above, but the missing point appears in the model’s own messages or tool calls, or in a delivered file in the wrong place, in weak wording, or for only some of the items.
Research
Required items or data not found in full
Fewer items than the request requires; required fields or values absent, null or placeholders; a required type of source or a required data point not obtained.
Claim says more than the cited page
The cited page supports the core of the claim, and the claim adds details, scope, precision or certainty that the page does not state, or states the agent’s own estimate or inference as if the page said it.
Value not found on the cited page
The cited page has nothing for the value, including a general page (home page, listing, search results) cited instead of the page that would show it; or no source is given for the value.
Value differs from the cited page
The cited page shows a different value, date, status, name or place for the same thing.
Appendix
Table 7: The eleven failure types. Each deduction receives exactly one.
Per 100 components
Low-cost − frontier
Share that are a 4
Deductions F / L
Failure type
Frontier
Low-cost
95% interval
Frontier
Low-cost
Intent recognition
16.5
22.3
+3.5 to +7.9
69%
50%
218 / 441
Does not look into the need
7.5
12.6
+3.4 to +6.9
66%
42%
99 / 250
Looks into the need, does not report it
9.0
9.6
-1.3 to +2.6
71%
60%
119 / 191
Research
29.5
32.5
+0.5 to +5.6
75%
46%
389 / 643
Required items or data not found in full
3.5
7.3
+2.1 to +5.7
59%
23%
46 / 145
Appendix
Table 8: Deductions by failure type over the 77 tasks. The interval is the 95% bootstrap interval over tasks of the low-cost rate minus the frontier rate. Components of outputs that violate the required format score 0 and count among the deductions that are not a 4.
Web agent benchmarks test how reliably AI agents browse, navigate, and act on websites, including under long-horizon, ambiguous, and adversarial conditions.