cs.AINov 11, 2026

WordInferBench: Can Agents Infer How to Edit from Word Document Context?

Authors: Jiayu Ding, Yupan Huang, Jinyoung Park, Yilin Jia, Wenshan Wu, Tengchao Lv, Shaohan Huang, Yuzhong Zhao, 

Organizations: Xi’an Jiaotong University · Microsoft · KAIST

Abstract

Document editing is among the most common knowledge-work tasks, yet existing benchmarks for evaluating autonomous agents on Word document editing provide fully specified attribute-modification instructions. They do not evaluate agents' ability to infer the appropriate attributes from document context, a capability essential in real-world editing where users expect consistency without spelling out every attribute. We introduce WordInferBench, a benchmark comprising 280 base document-editing instances, each paired with three instruction conditions: descriptive (the instruction explicitly lists every attribute change), referential (the instruction points to another object as an exemplar), and inferential (the agent must examine document context to infer what attributes to apply). To evaluate contextual attribute inference, each base instance is constructed so that the target formatting is recoverable from an in-document exemplar. The instances are derived from real Word documents collected from Common Crawl and cover 13 object types and 81 attributes. We evaluate six recent coding-agent systems and find that the best-performing evaluated system, Claude Code with Opus 4.6, achieves a 62.9% task-level pass rate on descriptive variants but only 40.7% on inferential variants, indicating that current systems complete tasks less reliably when the required formatting changes must be determined from document context.

Figures & tables

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. WordEval: Evaluating Word-Native Operation Fidelity in Document Editing

    Nov 7, 2026Jinyoung Park, Yupan Huang, Shimin Zhang +18Computer-Use Agent EvaluationAutomated Agent Evaluation

  2. SWE-Edit: Rethinking Code Editing for Efficient SWE-Agent

    Apr 28, 2026Yikai Zhang, Jiaxin Pei, Kenan Li +9AI Coding AgentsMulti-Agent Coding