DAYJOB: A Benchmark for Long-Horizon Professional Work

Stephanie Finley; Liudas Panavas; Thomas Mikkelson; Cam Hinton; Stacey Ganss; Bradley Monton; Emily Kendall; Michelle Spradlin

cs.AIOct 1, 2026

DAYJOB: A Benchmark for Long-Horizon Professional Work

Authors: Stephanie Finley, Liudas Panavas, Thomas Mikkelson, Cam Hinton, Stacey Ganss, Bradley Monton, Emily Kendall, Michelle Spradlin, +7 more

Organizations: Surge AI

Abstract

Professional work often starts with a brief request that leaves the professional to work out what is needed, which documents matter, and whether the request's premise holds. We introduce DAYJOB, a benchmark of 130 tasks built by professionals in healthcare (50) and finance (80). The tasks are estimated to take a professional 13.6 hours on average in healthcare and 16.6 in finance. Each task is a containerized Harbor environment with an expert rubric of binary criteria (median 47.5 and 57.5 per task) that an agentic judge applies to the delivered files, and an attempt passes only if it meets every criterion. Across 30 model configurations from 13 developers, the strongest, Claude Opus 5.5, passes 24.7% of healthcare and 23.9% of finance attempts, and the median configuration passes 0.6% and 2.5%. In case studies, agents accept premises that the record contradicts and carry wrong inputs through otherwise consistent analyses. We release all healthcare tasks, 50 of the 80 finance tasks, the evaluation harness, and the leaderboard.

Figures & tables

Request. “Caldervane Emerging Markets Fund is a new client’s first fund with us and we are delivering our first reports for it today for COB June 30th. The client wants the initial reports as soon as they are available, so they can review these BAU deliverables in detail. Go through the attached pack and let me know if there are any issues stopping me sending the reports to the client as they are. Write it up as a short .md memo.”
Criteria
Identifies that the Bloomberg screen quotes the Nerovexa price of 261.200 in South African cents rather than rand.
States that the Nerovexa position is overstated by about $25.7 million (accept$ 25.5m–$25.9m).
Recommends not sending the reports until the Nerovexa position is corrected; a hold that lifts once the quotation basis is confirmed does not satisfy the criterion.
Identifies that the Kingdom of Saudi Arabia 4.500% 2046 bond was priced from BVAL when ICE supplied a current evaluation.
Does not call the Dravena position quantity or its 1,242.000 CZK price incorrect, and does not treat the stock split as a reason to withhold the reports.

Table 1: Excerpt of a DAYJOB: Finance task ( finance_9148d239 ): six of its 20 criteria, abridged.

	DAYJOB: Healthcare	DAYJOB: Finance	GDPval gold subset
Tasks	50	80	220
Request length, words (mean / median)	48.9 / 42.5	80.7 / 62.5	337.2 / 307
Input files per task (mean / median)	19.8 / 16.5	25.7 / 20.5	1.19 / 1
Input files per task (range)	8–50	8–122	0–17
Rubric criteria per task (mean / median)	55.7 / 47.5	66.1 / 57.5	47.5 / 47
Estimated hours per task (mean) ‡	13.6	16.6 §	6.7

Table 2: DAYJOB compared with the GDPval gold subset. Counts come from each benchmark’s task files (request text, input workspace, and rubric). Finance statistics describe the 50 public tasks. ‡ Healthcare hours average the task creators’ range estimates at range midpoints, with “more than 40 hours” counted as 40; the GDPval figure is the mean validated self-reported completion time on the gold subset (404 minutes) reported by Patwardhan et al. [22] . § Average reported on the finance dataset card [ 26 ] .

Figure 1: Cumulative distributions of (a) request length and (b) the number of input files per task. The shortest GDPval request (100 words) is longer than 94% of healthcare requests and 70% of public finance requests, and 43% of GDPval tasks have no input file.

Configuration	Developer	Strict	Rank	Strict	Rank
		DAYJOB: Healthcare		DAYJOB: Finance
Claude Opus 5.5 (adaptive/max)	Anthropic	24.7	1	23.9	1
GPT-6 Astra (max)	OpenAI	11.6	2	21.5	2
Claude Fable 5.1 (adaptive/max)	Anthropic	9.6	3	19.8	3
Grok 4.7 (xhigh)	xAI	8.4	4	14.5	5
Muse Spark 1.3 (max)	Meta	7.8	6	14.8	4

Table 3: Launch leaderboard (September 2026): strict pass@1 (%) and rank on 50 healthcare and 80 finance tasks, five attempts per task. Rows are ordered by the mean of the two scores, and equal displayed scores share a rank.

Figure 2: (a) Strict pass@1 of the 30 configurations, sorted within each domain. (b) Pass@1 as the required share of criteria is relaxed, for the leader and for the median across configurations at each threshold.

Figure 3: Strict pass@1 against mean cost per attempt. Colored points are the configurations on the cost frontier, and the step line gives the best strict score available at or below each cost; gray points are the other configurations. Costs exclude the judge.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Figure 4: Strict pass@1 for all 30 configurations, in the order of Table 3 .

Explore similar work

JobBench: Aligning Agent Work With Human Will
May 25, 2026Yuetai Li, Yichen Feng, Zhangchen Xu +21Computer-Use Agent Benchmarks LLM Agent Evaluation
University of Washington · 10Bake AI · University of California, Santa Barbara +10
xDailyBench: Benchmarking LLMs on Professional Consultation for Real-Life Problems
Sep 7, 2026Yongchang Peng, Qingshui Gu, Liya Zhu +31Pragmatic Reasoning in Language Models LLM Agent Evaluation
ByteDance Seed · M-A-P
FinProBench: Evaluating Financial AI Agents with Role-Grounded Rubrics Derived from Professional Deliverables
Aug 4, 2026Ben Wang, Kang Zhou, Lifan Guo +2AI Agent Evaluation AI Agent Benchmarks
Qwen DianJin Team, Alibaba Cloud Computing

May 25, 2026, cs.AIJ/K move · Enter open · S save

JobBench: Aligning Agent Work With Human Will

Yuetai Li, Yichen Feng, Zhangchen Xu, Zixian Ma+20

University of Washington · 10Bake AI · University of California, Santa Barbara+10

Computer-Use Agent Benchmarks LLM Agent Evaluation AI Agent Benchmarks Human-AI Collaboration AI and Labor

Open paper AskPDF arXiv

Current benchmarks for occupational AI agents are scoped primarily by economic values, telling a replacement story. We introduce JobBench, which evaluates AI agents on the workflows that experts identify as high-priority for delegation, empowering humans based on their needs instead of replacing them with GDP value. JobBench covers 130 agentic tasks across 35 occupations. Each task is packaged as a workspace of heterogeneous reference files, requiring the agent to reason through the cluttered information streams of real professional work. Outputs are graded by a fact-anchored chain of rubrics, averaging 35.6 binary criteria per task. We evaluate 36 models; the strongest, Claude Opus~4.7 under Claude Code, reaches only 45.9 %. We hope JobBench shifts the community's target labour-market effect from replacement to enhancement: building agents that do what humans actually want delegated, not only what is most economically valuable.

Abstract

Explore similar work

JobBench: Aligning Agent Work With Human Will

xDailyBench: Benchmarking LLMs on Professional Consultation for Real-Life Problems

FinProBench: Evaluating Financial AI Agents with Role-Grounded Rubrics Derived from Professional Deliverables