The Last Mile Is the File: OfficeEditBench for Preservation-Aware Office Editing

Zhiwen Wu; Chengxu Wu

cs.SESep 28, 2026

The Last Mile Is the File: OfficeEditBench for Preservation-Aware Office Editing

Authors: Zhiwen Wu, Chengxu Wu

Organizations: Peking University

Abstract

A small Office edit creates two obligations: propagate every required update and leave protected state untouched. Updating too little leaves dependencies inconsistent; updating too much changes content the user did not authorize. We introduce OfficeEditBench, a 170-task benchmark for change-scoped maintenance of spreadsheets, presentations, and documents. Task contracts specify required updates, protected state, native structures, and applicable interaction requirements. Across 510 archived task-system outcomes from WorkBuddy, Doubao, and Codex, we distinguish file delivery, target completion, and verifier-defined acceptance. Hard package-valid delivery ranges from 92% to 100%, yet no selected output satisfies the complete contract. Case analysis highlights why local correctness is insufficient: an updated value can lose its generating formula, a revised rule can fail to reach related conclusions, and a new deadline can omit a retained prerequisite. These mechanisms connect artifact-level checks to the continued maintainability of Office files. We analyze maintenance failures while distinguishing frozen automatic verdicts from human acceptability. OfficeEditBench provides a testbed for completing required changes while preserving the logic and scope of existing work.

Figures & tables

Figure 1: One request, two ways to miss its scope (OBM-032). The approved value must propagate to the workbook, native PPT chart, and conclusions. Updating only the workbook omits required changes; also changing raw Sales violates preservation. The right panel satisfies the depicted obligations, assuming other protected state and native editability are retained. These are task-derived alternatives, not actual agent outputs, Office screenshots, or an empirical failure-frequency claim. The task requires XLSX and PPTX only.

Figure 2: OfficeEditBench evaluation architecture. Requests, starting files, and visible evidence define the agent input; submitted native artifacts are checked against the task contract. Strict success requires all applicable predicates, including interaction when specified. Agent steps are illustrative. The dashed path marks intended evaluator-only access, not demonstrated historical isolation.

Tier	Tasks	Artifact coverage	Contract emphasis
Easy v2	50	20 XLSX, 20 PPTX, 10 DOCX; one file	Local target/type correctness; non-target content and structure
Medium	70	30 spreadsheet-, 20 presentation-, 20 document-primary	Contextual business edits; linked native objects and protected narrative
Hard	50	XLSX + PPTX + DOCX per task; 2–3 evidence files	Cross-file consequences, approval boundaries, source authority

Table 1: Active task inventory. Tiers express construction intent, not a calibrated difficulty scale. Format denotes the primary artifact for Medium; some tasks include linked pairs.

Figure 3: Three systems, three task tiers: strict acceptance and partial completion. (a) Selected-output strict pass rates with pass/task counts. (b) Mean composite scores over every task in each tier, including non-delivery zeros. All 510 frozen outcomes are represented. Partial scores are tier-specific diagnostics, not a calibrated difficulty scale. WorkBuddy Hard is exploratory and Doubao Hard uses the corrected snapshot; submission-selection details are in Appendix A .

Tier	System	Strict pass	Rate (%)	Mean $C$	$S$ [ $n$ ]	$P$ [ $n$ ]
Easy	WorkBuddy †	43/50	86.00	91.60	–	–
Easy	Doubao	29/50	58.00	63.60	–	–
Easy	Codex	42/50	84.00	98.40	–	–
Medium	WorkBuddy †	37/70	52.86	82.43	85.97 [70]	71.81 [70]
Medium	Doubao	35/70	50.00	78.36	–	–
Medium	Codex	44/70	62.86	88.96	88.08 [70]	91.63 [70]

Table 2: Verifier-defined acceptance and partial scores. Means are on a 0–100 scale within each tier; brackets give each component’s observed denominator. Product order is fixed across tiers.

Task	$C$	Reported outcome	Maintenance obligation
OBM-032	100	Approved total synchronized across workbook, conclusions, native chart, and embedded data	Complete the authorized chain while retaining other states, months, and source records.
OBM-034	76.92	Current values correct; operator reports replacing AVERAGE with a constant	Preserve the relationship to source data, not only today’s displayed value.
OBM-058	62.50	Rule says delay $\geq 15$ minutes; a 15-minute flight still labeled on time in summary/timeline	Propagate a changed definition to its dependent statements.
OBM-062	0	Deadline changed to 30 days after acceptance; compliant-invoice prerequisite omitted	Update the requested term without deleting a retained condition.

Table 3: From local correctness to maintainable files. WorkBuddy Medium cases combine atomic-check records with operator case descriptions.

C

is the unchanged frozen composite, not a new human rating.

Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

System / tier	Inspected evidence	Selection and uncertainty
WorkBuddy / Easy	Original score/check JSON and generated task report; 50 tasks	Final formal UI file cards; retries included. 50 readable outputs in stored checks.
WorkBuddy / Medium	Original score/check JSON with $C/S/P$ ; 70 tasks	Formal delivery for all 70; DNS retries on 055/056; authorized candidate B on 063.
WorkBuddy / Hard	Original score/check JSON; 50 tasks; exploratory designation retained	46 package-valid triples; four missing-submission zeros.
Doubao / Easy	Score/check JSON and progress metadata	Fresh conversation; first delivery selected according to metadata.
Doubao / Medium	Two progress files: 20 + 50 task scores	Completed 001–070; component means not available in the imported records.
Doubao / Hard	Final score/check JSON and historical summary	Final corrected snapshot after reruns and interaction corrections.

Appendix

Table 4: Evidence available for the September 26 freeze. “First delivery” is a recorded selection rule, not a verified universal first-attempt claim.

Scenario	Task IDs	Spreadsheet rows	Slides	DOCX pages
Public-company reporting	13	160,000	112	86
Retail operations	13	220,000	128	92
Capital/procurement	12	220,000	118	90
Environmental compliance	12	220,000	142	104

Appendix

Table 5: Hard fixture sizes reported by the repository. Pages describe construction metadata, not renderer-invariant pagination; rows and slides describe shared fixtures, not independent samples.

Quantity	Recorded value
Target-check specifications / task	9–43
All gold operation specifications / task	14–43
Explicit preservation specifications / task	0–8
Declared depth labels (tasks)	4 (16); 5 (28); 7 (6)
Dependency edges / distinct edit sites	Not enumerated
Protected-object count	Not enumerated

Appendix

Table 6: Hard contract specifications and stored check records. Counts are not numbers of edited or protected objects. Source hashes and per-task distributions are provided in contract_inventory.json .

Figure 4: Delivery versus verifier-defined acceptance on Hard. (a) Package-valid triples versus zero strict passes. (b) All 50 stored composites per system, including missing-output zeros; diamonds denote means and point offsets aid readability. *WorkBuddy is the exploratory snapshot, corroborated by operator-supplied atomic records;

\dagger

Doubao is the corrected snapshot. Package validity does not certify native-application editability.

	WorkBuddy		Doubao		Codex
Format	Pass	Mean	Pass	Mean	Pass	Mean
XLSX	19/20	96.00	19/20	99.50	12/20	96.00
PPTX	19/20	96.00	1/20	14.50	20/20	100.00
DOCX	5/10	74.00	9/10	90.00	10/10	100.00

Appendix

Table 7: Easy v2 by file format. Each pair is strict passes and mean score. All three systems have archived check-level evidence. Equal means can conceal different pass counts.

	WorkBuddy		Doubao		Codex
Task IDs	Pass	Mean	Pass	Mean	Pass	Mean
001–010	8/10	97.00	8/10	98.48	10/10	100.00
011–030	11/20	80.40	11/20	82.31	13/20	88.53
031–050	13/20	94.84	8/20	78.08	11/20	94.29
051–070	5/20	64.75	8/20	64.63	10/20	78.56

Appendix

Table 8: Medium cohort results: strict passes and mean composites. All cohorts are complete; WorkBuddy values match its atomic records. Cohorts do not isolate a single causal difficulty variable.

Figure 5: Target-perfect failures in Medium. Each bar partitions all 70 tasks into strict passes, perfect semantic scores that still fail strict success, and remaining outcomes. WorkBuddy counts match the supplied atomic records. Doubao is omitted because comparable component scores were not imported. These are verifier outcomes, not human acceptability.

Failed predicate	WorkBuddy ( $n=46$ )	Doubao ( $n=50$ )	Codex ( $n=49$ )
Non-target document text	34	45	49
Non-target spreadsheet cells/formulas	33	44	35
Non-target presentation notes	25	23	18
Non-target presentation text	20	26	30

Appendix

Table 9: Non-exclusive Hard preservation failures among package-valid triples . Counts use stored predicate verdicts; WorkBuddy’s atomic records corroborate its report. A predicate failure is not a count of damaged objects.

Source	System	$D/n$	Strict/ $n$	Mean $C$	$S_{D}$	$P_{D}$
Public-company	WorkBuddy	13/13	0/13	52.48	44.37	83.93
	Doubao	13/13	0/13	46.70	39.28	71.61
	Codex	12/13	0/13	51.10	48.83	81.96
Retail	WorkBuddy	10/13	0/13	36.91	38.36	84.40
	Doubao	13/13	0/13	41.97	32.91	73.27
	Codex	13/13	0/13	53.44	45.42	82.21

Appendix

Table 10: Hard results grouped by shared source scenario.

D

is the number of package-valid triples;

n

counts all tasks. Composite

C

includes all

n

stored outcomes.

S_{D}/P_{D}

condition on the same

D

deliveries (their component denominator is

D

, not

n

Omitted source	Retained $n$	WorkBuddy	Doubao	Codex
None (full snapshot)	50	45.89	43.52	51.29
Public-company	37	43.58	42.40	51.36
Retail	37	49.05	44.06	50.54
Capital/procurement	38	45.57	43.99	51.14
Environmental	38	45.40	43.61	52.12

Appendix

Table 11: Descriptive leave-one-source-scenario-out sensitivity of all-task Hard composite means. A row removes the complete named source group and reaggregates retained records; it is not a held-out generalization experiment or a confidence interval.

Figure 6: Evaluation evidence and its limits. The top row contains three separate Easy v2 construction-QA probes, not an agent execution sequence. The middle row is the archived-record analysis workflow; atomic checks or operator reports are used according to availability. Dashed cards mark proposed independent validation that has not been conducted in this study. Construction QA is not a human baseline.

Check family	Measured property	Not established by that check alone
Readability/package	Expected file is parseable or ZIP-valid under the checker	Native Office opening, recalculation, visual fidelity, accessibility
Target	Specified value/type, text, formula, or native object meets a predicate	Every business claim is true; all equivalent answers are accepted
Preservation	Selected non-target state matches the starting artifact	Severity of a mismatch; byte or pixel identity of the full file
Structure	Selected sheets, objects, paragraph/table structure or identifiers match	A failed structural signature means the output is unusable
Interaction	Required concepts and fixed reply satisfy the transcript check	Full pre-reply behavior, historical isolation, universal first attempt

Appendix

Table 12: What a check can establish, and what requires additional evidence.

Figure 7: Construction probe, not an agent failure. In archived E2-013 QA, both requested updates pass, but an injected overwrite of Purchases!A1 fails non-target preservation: 90/100, strict failure. Cells are schematic; the original Chinese Summary sentence is paraphrased in English.

ID	Score	Pass	Score	Pass	Score	Pass	ID	Score	Pass	Score	Pass	Score	Pass
	WorkBuddy		Doubao		Codex			WorkBuddy		Doubao		Codex
001	100.00	Y	100.00	Y	90.00	N	026	100.00	Y	10.00	N	100.00	Y
002	100.00	Y	100.00	Y	90.00	N	027	100.00	Y	100.00	Y	100.00	Y
003	100.00	Y	100.00	Y	90.00	N	028	100.00	Y	10.00	N	100.00	Y
004	100.00	Y	100.00	Y	90.00	N	029	100.00	Y	10.00	N	100.00	Y
005	100.00	Y	100.00	Y	100.00	Y	030	20.00	N	10.00	N	100.00	Y

Appendix

Table 13: All 50 Easy v2 tasks, three systems (150 outcomes).

ID	Score	Pass	Score	Pass	Score	Pass	ID	Score	Pass	Score	Pass	Score	Pass
	WorkBuddy		Doubao		Codex			WorkBuddy		Doubao		Codex
001	85.00	N	100.00	Y	100.00	Y	036	87.50	N	100.00	Y	100.00	Y
002	85.00	N	100.00	Y	100.00	Y	037	100.00	Y	100.00	Y	83.93	N
003	100.00	Y	100.00	Y	100.00	Y	038	100.00	Y	100.00	Y	100.00	Y
004	100.00	Y	100.00	Y	100.00	Y	039	76.56	N	35.27	N	90.62	N
005	100.00	Y	100.00	Y	100.00	Y	040	100.00	Y	100.00	Y	100.00	Y

Appendix

Table 14: All 70 Medium tasks, three systems (210 outcomes).

ID	Score	Pass	Score	Pass	Score	Pass	ID	Score	Pass	Score	Pass	Score	Pass
	WorkBuddy		Doubao		Codex			WorkBuddy		Doubao		Codex
001	72.14	N	70.29	N	66.71	N	026	72.95	N	43.00	N	52.09	N
002	61.82	N	46.55	N	56.73	N	027	47.76	N	33.85	N	33.21	N
003	49.63	N	39.25	N	48.27	N	028	28.25	N	33.14	N	55.11	N
004	49.63	N	39.25	N	64.27	N	029	40.60	N	35.21	N	40.60	N
005	62.88	N	56.06	N	61.21	N	030	47.12	N	34.91	N	47.12	N

Appendix

Table 15: All 50 Hard tasks, three systems (150 outcomes).

Benchmark	Evaluation unit	Relationship to the present study
SWE-bench (ICLR 2024)	Repository patch evaluated with issue-resolution and regression tests	Prior maintenance principle; our protected observations concern native Office state.
WebArena (ICLR 2024)	Web task evaluated by functional outcomes	Persistent-environment evaluation; our deliverables are existing files after scoped edits.
$\tau$ -bench (ICLR 2025)	User/tool episode, database end state, repeated-trial reliability	State and interaction constraints; our fixed clarification checks do not measure the same reliability statistic.
MLE-bench (ICLR 2025)	ML competition submission evaluated against held-out targets	Submission-based interface and explicit execution reporting; nominal limits do not replace per-attempt records.
ScienceAgentBench (ICLR 2025)	Scientific programming task with automatic and human assessment	Validity evidence beyond automatic outcomes; our construction QA is not independent human validation.
OfficeEditBench (this work)	Authorized edit to native Office artifacts with protected state	170-task suite, decomposed predicates, and a descriptive audit of selected archived outputs.

Appendix

Table 16: Evaluation settings relevant to preservation-aware Office editing. ICLR status refers to the cited formal publication, not an arXiv submission or workshop.

Benchmark	Artifact scope and scoring	Preservation and validity evidence
PPTArena ( Ofengenden et al., 2026 )	Existing PPTX editing; structure and visual judges	Instruction/quality assessment with human alignment; focused on presentations.
PPT-Eval ( Gandhi et al., 2026 )	PPTX creation/editing; code/model rubrics and partial credit	Extra-change penalties; meta-evaluation on human-constructed completion attempts.
DeckEdit-Bench ( Kim et al., 2026 )	Native PPTX editing; page targeting and object preservation	Preservation of out-of-scope objects, distinct from task-level conjunctive acceptance.
OmegaUse-OfficeVal ( Zhou et al., 2026 )	Multi-format Office/PDF deliverables; usability gate then weighted rubrics	Negative criteria for unintended damage; expert–code calibration and a shared execution scaffold.
OfficeEditBench	Native XLSX/PPTX/DOCX; scoped updates and protected-state predicates	Conjunctive acceptance plus component diagnostics; construction QA, but no completed independent agent-output adjudication.

Appendix

Table 17: Closest preservation-aware Office evaluations. Descriptions indicate the reported scope, not an exhaustive feature checklist. Different units and denominators preclude comparing their published scores directly.

Explore similar work

May 3, 2026cs.CL

EditPropBench: Measuring Factual Edit Propagation in Scientific Manuscripts

Local factual edits in scientific manuscripts often create non-local revision obligations. If a dataset changes from 215 to 80 documents, claims such as 'medium-scale' or 'a few hundred items' may also become stale, even though they do not repeat the edited number. In an audit of recent arXiv cs.CL benchmark and dataset papers, we find fact-dependent qualitative claims in 37.2% of papers, suggesting that this dependency pattern is common in the target genre. We introduce EditPropBench, a benchmark for measuring whether LLM editors propagate factual edits through dependent manuscript claims. Each item contains an ML/NLP-style synthetic manuscript, a targeted edit, and a controlled fact graph with sentence-level labels for direct targets, required downstream updates, and unrelated text that should remain unchanged. We summarize cascade success with Edit-Ripple Adherence (ERA), the fraction of required downstream updates correctly revised, and validate the metric with adversarial probes and stress-test variants. On the hardest cases, where dependent claims use implicit or free-form wording rather than repeating the edited value, five LLM editing systems span ERA 0.148-0.705. Even the strongest misses roughly 30% of required cascade updates. This advantage persists in a mixed evaluation that includes easy cases solvable by deterministic substitution. EditPropBench shows that current LLM editors can repair many implicit consequences of factual edits, but reliable scientific revision still requires cascade-aware checking.

Garvin Kruthof

Technical University of Munich

Multimodal Knowledge Editing Edit Generation Provenance Scientific Discovery

Ask PDF

Jun 29, 2026cs.SE

SpreadsheetBench 2: Evaluating Agents on End-to-End Business Spreadsheet Workflows

Spreadsheets are widely used for business analysis, financial modeling, reporting, and decision-making. However, most existing spreadsheet benchmarks evaluate isolated operations such as single-formula generation or local cell edits, and therefore fail to capture end-to-end workflows in realistic business settings. We introduce \textsc{SpreadsheetBench 2}, a workflow-level benchmark for spreadsheet agents that covers three task categories: generation, debugging, and visualization. The benchmark is constructed from authentic business data, including financial reports and corporate filings, and is annotated and validated by domain experts. The benchmark contains 321 tasks; each instance averages 11.8 worksheets and requires 593.5 cell modifications, reflecting large multi-sheet workbooks with cross-sheet dependencies. We evaluate eight frontier large language models under a unified multi-turn agent scaffold, and additionally include several LLM-based spreadsheet products as complementary baselines. Results show that current systems remain far from reliable on real-world workflows: the best model achieves 34.89% overall task accuracy, and debugging accuracy is as low as 12.00%. Trajectory analysis and a failure taxonomy further indicate that insufficient spreadsheet inspection and incorrect target-cell selection are the dominant bottlenecks. Together, these findings position \textsc{SpreadsheetBench 2} as a challenging testbed for advancing reliable spreadsheet automation. Project page: https://spreadsheetbench.github.io/

Jian Zhu, Yuzheng Zhang, Zeyao Ma +11

School of Information, Renmin University of China · Aptura AI · Shortcut AI +1

Spreadsheets Agentic Workflow Design

Ask PDF

May 29, 2026cs.CL

Office Comprehension Benchmark

We introduce Office Comprehension Bench (OCB), the first public benchmark to jointly evaluate LLM systems on Word, Excel, and PowerPoint comprehension over native file formats (.docx, .xlsx, .pptx) and their variants. OCB consists of two tracks. File Fidelity Q&A tests structural and visual perception of office artifacts - tables, charts, embedded images, formulas, and app-specific elements such as headers, speaker notes, and named ranges. Domain Q&A tests expert-level reasoning grounded in real-world industry documents across 12 professional domains, with queries requiring multi-step analysis and synthesis across documents. Each reference answer is decomposed into atomic, binary-gradable claims, and an ensemble of LLM judges scores responses against each claim independently. Even the strongest frontier system in its default reasoning mode reaches only about 59.3% on Domain Q&A; increasing thinking depth within a tier does not move performance materially, while moving to a higher product tier yields modest gains. We release the dataset, evaluation tooling, judge prompt, and a public leaderboard.

Firoz Shaik, Mateus Picanço Lima Gomes, Tanvir Aumi +17

Microsoft · † Work done while at Microsoft.

Topic pending

Ask PDF