CRAFT: An Agentic Spreadsheet Form Filling System with Template Awareness

Leyao Gu; Yingjie Xiong; Zirui Tang; Jiangtao Zhou; Yeye He; Chunwei Liu; Xuanhe Zhou; Fan Wu

cs.AIOct 3, 2026

CRAFT: An Agentic Spreadsheet Form Filling System with Template Awareness

Authors: Leyao Gu, Yingjie Xiong, Zirui Tang, Jiangtao Zhou, Yeye He, Chunwei Liu, Xuanhe Zhou, Fan Wu

Organizations: Shanghai Jiao Tong University · Microsoft Research · Purdue University

Abstract

Spreadsheet form filling requires agents to consolidate external evidence, ground values to precise cells, and preserve irregular template structure. Errors in early edits can overwrite labels or misalign fields, undermining later decisions. We propose CRAFT, a template-aware agent framework that connects reflective validation to constrained local repair. Instead of treating reflection as a free-form request to regenerate the workbook, CRAFT grounds detected errors to spreadsheet regions, restores corrupted template state when necessary, and re-grounds plausible writable slots before subsequent edits. A Rectangle-Aware Slot Grounder (RASG) proposes writable cells, while label-slot hints and protected regions constrain subsequent edits. We introduce FormFillBench, with 327 forms across Instruction-Only and Multi-File tracks. Compared with the strongest baselines, CRAFT improves pair accuracy by 8.51 and 23.38 percentage points on these tracks, respectively. Component-removal experiments support structural adjudication and slot re-grounding within the pipeline, and the framework retains its relative advantage among the methods evaluated with a second backbone. The code and benchmark FormFillBench are available at https://github.com/Glllllly/CRAFT.

Figures & tables

Figure 1: Example of spreadsheet form filling and its key challenges. The agent needs to gather evidence from auxiliary files, satisfy field-level constraints, preserve cross-field consistency, and interpret diverse form layouts.

Figure 2: Pilot Study of Spreadsheet Form Filling. Samples are from two common categories of real spreadsheet forms (i.e., mixed label-slot layout and repeated-section forms).

Figure 3: CRAFT combines evidence-based prefilling, reflective region adjudication, and constrained local repair. The planner carries validation evidence and protected cells between stages.

Figure 4: Rectangle-Aware Slot Grounder (RASG). A SegFormer-B1 backbone supports rectangle prediction and auxiliary horizontal and vertical boundary heads.

	Instruction-Only		Multi-File
Method	Pair	E2E	Pair	E2E
VLM+HTML+Image	51.17	13.5	32.60	12.60
VLM+HTML	49.26	11.0	26.31	7.87
VLM+Excel Skills+Coding	10.66	0.0	0.26	0.0
LangGraph	44.28	10.0	33.65	12.60
OpenAI Agents SDK	45.64	9.5	43.14	16.54

Table 1: Primary benchmark results (%). Pair accuracy requires both the value and target range to be correct; E2E requires every pair in a form to be correct. All 11 baselines are shown. Complete metrics appear in Table 5 .

	Instruction-Only		Multi-File
Method	Pair Acc. $\uparrow$	E2E Acc. $\uparrow$	Pair Acc. $\uparrow$	E2E Acc. $\uparrow$
CRAFT	59.68	15.5	66.52	43.31
CRAFT w/o SALA	52.97	10.0	25.48	10.24
CRAFT w/o PSR	50.45	12.5	24.86	3.15
CRAFT w/o Re.	53.71	10.5	24.96	7.87
CRAFT w/o Re. & Pl.	51.63	12.5	24.17	6.30

Table 2: Ablation results (%). SALA: Structure-Aware Local Adjudication, PSR: Progressive Slot Re-grounding, Re.: Whole Reflection module, Pl.: planner.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Risk Signal	Illustrative Example
Label-cell overwrite	The value “11-2045-5-601002-4310” is written into merged range K19:T19 , which originally contains the static template label “Account Number”, thereby overwriting template-side content.
Semantic ambiguity	The value “Robert J. Miller” is written into cell B20 , even though this slot should be associated with the first occurrence of Name rather than the second one, creating ambiguity between duplicated label groups.
Template inconsistency	Cell K20 is left-aligned, while the neighboring filled cells in the same local group are all center-aligned, indicating a local formatting inconsistency.
Label–value layout abnormality	The slot for Signature is located above its label, but the value is written into a nearby cell below or beside the label, violating local top-slot / bottom-label layout.

Appendix

Table 4: Representative risk signals used in structure-aware local adjudication. These examples illustrate template-integrity and layout-grounding risks.

Figure 5: Qualitative visualization of RASG on representative spreadsheet forms. From left to right, we show the rendered input, horizontal boundary cues, vertical boundary cues, reconstructed RASG predictions, and ground-truth writable regions.

Figure 6: Statistics and Structural Diversity of FormFillBench .

Figure 7: Comparison of average pair accuracy across five representative form types for CRAFT and the top-5 baselines. The reported scores are computed by first measuring pair accuracy on individual forms and then averaging within each form type.

	Instruction-Only					Multi-File
Method	4	29	38	84	88	226	238	284
VLM+HTML+Image	✓	$\blacktriangle$	✓	$\blacktriangle$	✓	$\blacktriangle$	$\blacktriangle$	$\blacktriangle$
VLM+HTML	$\blacktriangle$	$\blacktriangle$	✓	$\blacktriangle$	✓	✗	$\blacktriangle$	$\blacktriangle$
VLM+Excel Skills+Coding	✗	✗	✗	✗	✗	✗	✗	✗
LangGraph	✓	$\blacktriangle$	✓	$\blacktriangle$	✓	✗	$\blacktriangle$	$\blacktriangle$
OpenAI Agents SDK	✓	$\blacktriangle$	✓	✓	$\blacktriangle$	✗	$\blacktriangle$	$\blacktriangle$

Appendix

Table 6: Representative case studies across both tracks. Outcomes are success ( ✓ ), partial success (

\blacktriangle

), or failure ( ✗ ). Case metadata are listed below; L/R/U/D/ML denotes the original layout distribution.

Spreadsheet Automation

Spreadsheet automation uses AI to understand, create, edit, and validate workbooks while preserving their formulas, structure, and intended use.

See all 20 papers

SheetMind: Actions Set Accuracy, Agents Set the Failure Mode
Jun 14, 2025Lyuhao Chen, Xi Cheng, Yanming Kang +11Multi-Agent LLM Systems LLM Agent Evaluation
Carnegie Mellon University · Cornell University · University of California, Berkeley +2
SpreadsheetBench 2: Evaluating Agents on End-to-End Business Spreadsheet Workflows
Jun 29, 2026Jian Zhu, Yuzheng Zhang, Zeyao Ma +11Benchmark Design Spreadsheet Automation
School of Information, Renmin University of China · Aptura AI · Shortcut AI +1
MBABench-2.0: Evaluating AI Agents on Constructing Financial Spreadsheets Beyond Simple Verifiability
Oct 22, 2026Julian Poeltl, Thomson Yen, Patrick Shea +5Spreadsheet Automation AI Agent Benchmarks
ESB Reutlingen · Columbia Business School · TensorBlock

Jun 14, 2025J/K move · Enter open · S save

SheetMind: Actions Set Accuracy, Agents Set the Failure Mode

Lyuhao Chen, Xi Cheng, Yanming Kang, Ruiyan Zhu+10

Carnegie Mellon University · Cornell University · University of California, Berkeley+2

Multi-Agent LLM Systems LLM Agent Evaluation Spreadsheet Automation LLM Agent Reliability

Open paper AskPDF arXiv

Spreadsheet agents are converging on elaborate multi-agent designs, yet it is unclear how much of their performance comes from the agents rather than from the action interface they share. We answer this with SheetMind, a Manager-Action-Reflection framework, in a controlled study over all 221 tasks of the SheetCopilot Benchmark: five architectural variants, four backbones, exact McNemar tests on paired outcomes, and a checker reproducing the official chart and pivot comparisons. Replacing the high-level action API with primitive cell operations costs 47.1 points (p < 0.0001) and leaves the agent below a do-nothing baseline, whereas both extra agents together are worth 3.2 points: the Reflection Agent adds +4.5 (p = 0.013), the Manager +1.4 (p = 0.68). Decomposition instead changes how the system fails, cutting silently wrong outputs from 33% to 25% of tasks (p = 0.010). Capability saturates: GPT-5 and the five-times-cheaper GPT-5-mini are not significantly different (61.1% vs. 58.4%, p = 0.15), while GPT-3.5 loses 16.3 points and fails differently. A reflector must judge the step it just took, not the subtask. SheetMind reaches 61.1% Pass@1 with GPT-5 on the full SCB-221, against a do-nothing baseline of 9.0%. Accuracy comes from the operations an agent can name; the agents decide how it fails.

Abstract

Spreadsheet Automation

Explore similar work

SheetMind: Actions Set Accuracy, Agents Set the Failure Mode

SpreadsheetBench 2: Evaluating Agents on End-to-End Business Spreadsheet Workflows

MBABench-2.0: Evaluating AI Agents on Constructing Financial Spreadsheets Beyond Simple Verifiability