GDP.xlsx: Benchmarking Frontier Models on Professional Workbooks
Organizations: Surge AI
Abstract
We introduce GDP_xlsx, a benchmark of 70 realistic workplace requests written by professionals in 13 domains, each paired with an expert rubric of binary criteria and with the complete workbook or CSV file it refers to. Together the files hold 2.18 million populated cells, and the rubrics 791 criteria. Agents work in a container with spreadsheet tools, and an agentic judge that can open the file applies the rubric. Completeness and exclusivity criteria require set-valued answers to be found in full and with nothing extra. We rank models by strict pass@1, the share of attempts that meet every criterion. Across 20 model configurations, including frontier models from Google, Anthropic, and OpenAI, the top-scoring one, Gemini 4 Argon, passes only 38.3% of attempts, and the median score is 19.3%, leaving wide headroom. We release the evaluation harness and three public tasks; the other 67 are held out to protect the leaderboard from contamination.
Figures & tables
| Family | Representative work | What is evaluated |
|---|---|---|
| Formulas and manipulation | SpreadsheetCoder, FLAME, InstructExcel, SheetCopilot, SheetRM, SpreadsheetBench, SpreadsheetBench 2 | Whether a specified formula, script, edit, or workflow is produced or executed correctly |
| Table and spreadsheet question answering | WikiTableQuestions, HiTab, TAT-QA, TableBench, MiMoTable, SheetBench, FRTR-Bench | Answers to questions about one or more tables or whole workbooks, scored against references |
| Data analysis | InfiAgent-DABench, DA-Code, DSBench, Spider 2.0 | Code-driven analysis of data files or databases, scored against exact answers or task metrics |
| Finance spreadsheet workflows | Finch, BlueFin | Building, editing, and querying finance workbooks |
| Professional deliverables | GDPval, DAYJOB | End-to-end work products over many file types |
| This work | GDP.xlsx | Answers to professionals’ requests about complete workbooks and CSV files in 13 domains, graded in full against expert criteria; the evidence is often spread across tabs or held in layout, formatting, comments, or embedded objects |
| Input files | Worksheets | |||||||
|---|---|---|---|---|---|---|---|---|
| Domain | Tasks | Excel | CSV | Median | Max | Populated cells | Request words | Criteria |
| Manufacturing & supply chain | 10 | 9 | 1 | 4 | 8 | 227,207 | 92.5 | 152 |
| Construction | 8 | 6 | 2 | 6 | 8 | 273,118 | 84 | 85 |
| Healthcare | 8 | 5 | 3 | 2 | 7 | 120,698 | 31.5 | 64 |
| Human resources | 8 | 7 | 1 | 4 | 26 | 148,396 | 87.5 | 107 |
| STEM research | 7 | 5 | 2 | 5 | 19 | 331,888 | 23 | 66 |
| The response should state … | Form | |
|---|---|---|
| 1 | only the correct housebound codes (eight listed) | Exclusivity |
| 2 | all eight housebound codes | Completeness |
| 3 | the correct value for all housebound codes | Completeness |
| 4 | that the baseline form has no arthritis field | Affirmative |
| 5 | only the correct arthritis fields (two listed) | Exclusivity |
| 6 | all the arthritis fields | Completeness |
| Demand | Tasks | What it requires, with examples |
|---|---|---|
| Authoritative source | 24 | Choosing the tab, series, or record that governs the answer: a summary total that omits a row, a tab whose name gives the wrong year, projections next to confirmed figures |
| Cross-sheet linkage | 41 | Joining tabs through IDs, lookups, and formulas: a rule stated on one tab applied to records on another, or one entity split across several tabs |
| Rules and conditions | 50 | Applying filters, dates, thresholds, exceptions, and domain conventions from the request or the workbook: notice periods, eligibility rules, testing standards |
| Structure and format | 21 | Reading meaning from layout and format: font colors, fills, strikethrough, stacked headers, drawn shapes, data validation, comments |
| Premise and abstention | 14 | Recognizing that a figure in the request is wrong or that the file cannot answer part of the request, and saying so |
| Multi-step computation | 29 | Deriving values through several dependent steps: compounding, straight-line schedules, regressions, cost roll-ups |
| Form | Criteria | Tasks | Requires | Example, abridged from the public tasks |
|---|---|---|---|---|
| Affirmative | 357 | 65 | a stated fact or value | “should correctly identify the sheet where the location is found: Sheet 1” |
| Completeness | 97 | 48 | every item of a set (full recall) | “should correctly state all of the housebound-related codes: W1_OUTSI, B_HOUSB, …” |
| Exclusivity | 72 | 40 | no item outside the set (full precision) | “should only include the correct tenancies in the Tenancy reference column …” |
| Conditional ∗ | 124 | 34 | correct attributes for included items | “If the response includes any correctly identified Tenancy, then for each one it includes, it should correctly state its demise …” |
| Prohibition ∗ | 140 | 52 | avoiding a named error | “should avoid using GBP amounts in place of USD amounts” |
| Permissive ∗ | 1 | 1 | nothing (may state) | “may state that there is only one DC auxiliary power cabinet in the asset” |
| Task | Configuration | What the answer did | What it missed, and where that evidence sits | Retrieval failure |
|---|---|---|---|---|
| Rent forecast (public) | Muse Spark 1.3 | Built both tables with the two US properties, all six discount endings, and every estimated rental value in USD | The three tenancies that qualify only through a tenant break, listed on the BreakClauses tab | Incomplete recall: 6 of 9 tenancies, precision 6 of 6 |
| Clinical data dictionary (public) | GPT 6 Astra | Listed only real housebound codes and arthritis fields, each with the right value | Five of the eight housebound codes, all in the same dictionary, and the absence of a baseline arthritis field | Incomplete recall: 3 of 8 codes, precision 3 of 3; an unstated absence |
| Equipment register (public) | Gemini 3.8 Flash | Named +MDY80UC001, the transition-piece cabinet that the IO List gives for its auxiliary-power trip signals | The cabinet +MDY50UC017, labeled only in a diagram pasted onto Sheet1 | Non-text evidence: a pasted diagram |
| Certification tracker (held out) | GPT 6 Astra | Recommended both products, which look equally ready on the main status sheet, and stated that the workbook did not identify who handles a pending certification | The tab recording which party handles each certification, which shows that only one of the two products is cleared | Multi-hop evidence across tabs; a false claim of absence |
AI Agent Benchmarks
AI agent benchmarks measure how reliably autonomous systems plan, use tools, interact, and complete tasks across controlled and real-world settings.
See all 1,583 papers