PPTBench: Can Coding Agents Reconstruct the Visual World through Structured, Editable Slides

Xiaoqiu Wang; Yizhe Chi; Wenyi Li; Deyao Hong; Zhihan Shan; Mingju Gao; Kaisen Yang; Youjie Zheng

cs.CLAug 31, 2026

PPTBench: Can Coding Agents Reconstruct the Visual World through Structured, Editable Slides

Authors: Xiaoqiu Wang, Yizhe Chi, Wenyi Li, Deyao Hong, Zhihan Shan, Mingju Gao, Kaisen Yang, Youjie Zheng,

Organizations: Navers Lab, Einsia.AI · Tsinghua University

Abstract

Coding agents are increasingly moving beyond text-based software tasks to reconstruct visual targets through code. This capability, visual coding, requires agents to translate their understanding of visual targets into executable code. Slides provide a natural testbed for this capability, combining rich visual structure with objects that can be programmatically created and edited. To measure this capability, we introduce PPTBench, a benchmark for reconstructing scientific flow diagrams as editable PowerPoint slides. PPTBench covers 500 scientific flow-diagram tasks across 10 presentation domains, drawn from real research papers, and is evaluated with a four-stage agentic judge covering artifact validity, process and connector fidelity, rendering quality, and visual fidelity. Evaluation of ten models across 36 model--harness--effort configurations shows a substantial gap in reliable visual coding: the best configuration achieves 77.34, while the median across configurations is 24.38. Fine-grained analysis shows that agents can generally produce valid slide files, but struggle to produce high-quality reconstructions that faithfully recover the semantics and visual structure of the target. Further analysis shows that increasing reasoning effort primarily improves hard-gate passage rather than mean detail quality on each configuration's passing tasks, while configurations with more inspection tend to achieve higher overall scores. PPTBench establishes a measurable testbed for studying visual coding and advancing agents toward more reliable visual creation.

Figures & tables

Figure 1: PPTBench task domains and representative model scores on 500 scientific flow diagrams.

Figure 2: PPTBench construction: source retrieval, preprocessing, human screening, and task assembly. Screening retains 500 of 1,000 candidate diagrams.

Figure 3: The four-stage Agentic Judge: artifact validation ( $G_{\mathrm{art}}$ ), semantic correctness ( $G_{\mathrm{sem}}$ ), rendering and text quality ( $G_{\mathrm{rt}}$ ), and visual detail. Stages 1–3 run in three independent rounds. At least two rounds must pass both judged gates before merged detail findings determine the score (Eq. ( 1 )).

Figure 4: Failure analysis over 18,000 model–task pairs. Left: number of rounds triggering a hard gate. Right: the twelve largest detail penalties among gate-passing cases; dimension shares include all issue types.

Figure 5: Reasoning-effort sweeps across six model families: mean score over all tasks (left), gate-pass rate (center), and mean score on passed tasks (right).

Human–human pairwise
Pair	Gate agr.	Cohen’s $\kappa$
Judge 1–Judge 2	90.5%	0.781
Judge 1–Judge 3	89.0%	0.766
Judge 2–Judge 3	83.5%	0.645

Table 2: Reported human agreement on 200 cases. Gate failure is the positive class.

Single-round consistency (3 repeats)
Judge	Unanimous	Fleiss’ $\kappa$	Flipped
	gate rate		cases
Luna High	87.5%	0.809	25
Sol Medium	86.5%	0.797	27
Terra High	81.0%	0.714	38

Table 3: Repeatability of single-round and three-round gate decisions on 200 cases.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Setting	Representation	Evaluation target
Chart reconstruction [ 71 , 66 , 59 , 78 ]	Plotting code	Reproduce chart appearance and numerical content.
Visual code reconstruction [ 2 , 52 , 51 ]	TikZ, SVG, or other source	Recover a rendered target in an executable representation.
Diagram parsing and evaluation [ 25 , 58 , 35 ]	Diagram constituents and relations	Recover or compare semantic structure.
Slide generation and editing [ 84 , 73 , 5 , 14 ]	Slide decks or PPTX edits	Generate an open-ended design or execute specified edits.
Reference-image slide reconstruction [ 60 ]	Editable PPTX	Reconstruct a slide design; assess content, layout, execution, and visual fidelity.
PPTBench [ 20 ]	Answers or PowerPoint API sequences	Evaluate detection, understanding, modification, and generation with task-specific metrics.

Appendix

Table 5: Related evaluation settings, output representations, and evaluation targets.

Figure 6: Outcome proportions by model, averaged across tested effort settings. Dots mark the gate-pass rates of individual configurations.

Figure 7: Examples of hard-gate failures: (a) incorrect process connections from Astra High on task 0017; (b) illegible connector labels from Luna Max. Both fail all three judge rounds.

Figure 8: Semantic failure on task 0104 (Astra High): (a) full reference and reconstruction; (b) enlarged control-flow region showing the altered feedback connection.

Issue type	Dimension	Severity	Cases	Category	Total	Single-
				ratio	penalty	round (%)
unexpected_wrap	Text	Medium	1,864	0.577	5,723.9	6.8
outside_text_box	Text	High	658	0.319	3,441.8	35.7
node_aspect_ratio	Graphics	Medium	802	0.655	2,614.5	53.7
horizontal_alignment	Text	Medium	737	0.694	2,446.7	42.1
text_connector_overlap	Text	High	385	0.342	2,061.8	55.3

Appendix

Table 7: The fifteen largest aggregate detail penalties among 5,714 gate-passing reconstructions. Cases may contain multiple issue types.

Model	Higher effort	$N$	$\Delta$ detail	95% CI	$\Delta p$ (pp)
Opus 5	max	147	1.76	[0.15, 3.43]	0.0
Sonnet 5	xhigh	35	0.58	[-2.75, 3.87]	0.4
Luna	max	85	4.39	[1.88, 7.01]	12.6
Sol	max	109	4.13	[2.43, 5.91]	20.8
Terra	max	66	2.83	[0.22, 5.47]	18.0
Astra	max	289	0.98	[0.18, 1.75]	-9.8

Appendix

Table 8: High versus highest effort on shared passing tasks.

N

is common-pass count;

\Delta p

uses all 500 tasks.

Weighting / selection	Mean	95% CI	Median
36 configurations, equal weight	28.18	[26.72, 29.62]	24.38
10 families, mean over available efforts	31.91	[30.40, 33.48]	27.14
10 families, highest available effort	38.26	[36.39, 40.10]	33.50
10 families, best observed effort	39.36	[37.49, 41.25]	34.69

Appendix

Table 9: Aggregate score under alternative configuration menus. Median is over configuration or family means, as specified.

Figure 9: Mean score versus review turns (left) and revision turns (right) for 27 configurations. Connected points follow each model’s reasoning-effort sweep.

		Review turns		Revision turns
Model	Configurations	Pearson $r$	Spearman $\rho$	Pearson $r$	Spearman $\rho$
All matched configurations	27	0.881	0.897	0.255	0.462
Claude Opus 5	5	0.947	0.900	0.927	0.900
Claude Sonnet 5	4	0.922	1.000	0.913	1.000
GPT-5.6 Luna	6	0.946	1.000	0.920	0.943
GPT-5.6 Sol	5	0.998	1.000	0.979	0.900

Appendix

Table 10: Correlations between mean score and generation behavior. A dash indicates an undefined within-model correlation for the single GPT-5.5 configuration.

Figure 10: Gate validation on 200 cases: (a) chance-corrected agreement; (b) raw agreement; (c) confusion matrix; (d) single-round gate consistency across three repeats.

Effort	Artifacts	Median range	95th percentile	Mean pairwise difference
Low	279	2.50	8.52	2.03
Medium	324	1.87	7.76	1.84
High	366	1.50	7.50	1.62
XHigh	354	1.06	7.50	1.30
Max	322	0.62	7.50	1.27
All five	1,645	1.25	7.50	1.60

Appendix

Table 11: Detail-score variation within existing Astra judge triplets, conditional on all three rounds passing.

Effort	At least one pass	Majority (reported)	Unanimous pass
Low	63.15 (335)	59.42 (315)	52.81 (279)
Medium	72.60 (382)	68.16 (358)	61.78 (324)
High	80.56 (421)	77.34 (404)	70.20 (366)
XHigh	75.94 (392)	72.60 (374)	68.85 (354)
Max	72.10 (373)	68.76 (355)	62.59 (322)

Appendix

Table 12: Astra scores under three gate-vote thresholds. Parentheses give passing cases out of 500.

Policy	Kendall $\tau$	Median $\Delta$	Max. $\|\Delta\|$	Reversed pairs
Two-round issue support	0.975	1.77	4.53	8/630
Unit affected-ratio multiplier	0.971	1.70	4.15	9/630
Equal dimension weights	0.997	0.13	0.55	1/630
Maximum per issue category	0.981	1.21	2.74	6/630

Appendix

Table 13: Sensitivity of 36 configuration scores to detail policies.

\Delta

measures score change from the original rubric.

Figure 11: Ranking stability under paired task resampling: (a) Kendall’s

\tau

, with a 5–95% band; (b) exact top-

k

set recovery rates.

Figure 12: Luna Medium on task 0852 ( $S=79.99$ ). Enlarged regions show text overflow, flattened subscripts, and the reoriented cylinder.

Figure 13: Three additional gate-passing reconstructions with different detail errors. Table 14 reports their dimension scores.

Category	Severity base	Issue subtype
Layout and composition
Panel misalignment	High (3.0)	missing_panel , extra_panel , duplicate_panel , substituted_panel , panel_order , panel_overlap , panel_clipping , conspicuous_blank_region
Panel misalignment	Medium (1.5)	canvas_crop , canvas_aspect_ratio , canvas_orientation , panel_position , panel_size , panel_proportion , panel_spacing , panel_alignment , separator_position , other_panel_layout
Panel misalignment	Diagnostic (0.0)	global_scale , global_centering , outer_whitespace
Node layout deviation	High (3.0)	missing_node , extra_node , duplicate_node , substituted_node , reading_order , grouping , nesting , hierarchy , node_overlap , node_occlusion
Node layout deviation	Medium (1.5)	node_position , node_spacing , node_alignment , node_distribution , grid_arrangement , padding , z_order , local_scale , local_mirroring , local_rotation , other_node_layout

Appendix

Table 15: Stage 3 issue taxonomy and severity bases. Diagnostic findings carry zero penalty.

Coding Agents

Coding agents use language models, tools, and feedback to modify software and complete engineering tasks.

See all 131 papers

Sep 16, 2026

ReFigBench: Benchmarking Scientific Figure Reconstruction as Editable PowerPoint Artifacts

Multimodal coding agents are expected to turn visual inputs into usable artifacts, and they act through a harness, the layer of tools, context management, and execution environment around the model. Existing evaluations often isolate short tool calls, API traces, or screenshot resemblance, and a low score under these proxies cannot say whether the model saw poorly, planned poorly, or was failed by its harness. We study scientific overview figure reconstruction, an agent task in which a source image must become an editable PowerPoint slide that preserves text, topology, layout, and native document structure. We introduce ReFigBench, a benchmark and evaluation framework built on 1,000 real overview figures retrieved from arXiv papers with full provenance. Coding agents from four model families reconstruct every figure under two workflows, direct code generation and a specialized PPTX workflow, and the strongest model runs inside two commercial harnesses, yielding ten configurations. Evaluation combines deterministic artifact checks, repeated automated scoring by judges from two model families, and blinded human comparisons. Perception remains a bottleneck that iterative rendering only partly repays. Whether workflow effort converts into quality depends on the model together with its harness, since the same model gains from the specialized workflow inside one harness and loses inside the other, and the harness shifts scores even under an identical direct prompt. The specialized workflow erases native connectors in every configuration, human judges still prefer its renderings in most matchups, and even the strongest agent falls short of the rubric ceiling. These results expose the tension between fidelity and editability as the central challenge for practical multimodal document agents.

Liyang Fan, Chi Wei, Yitai Li +8

SZU · SIAT, CAS · SUAT +2

Scientific Visualization AI Agent Benchmarks Multimodal Agents Multimodal Model Evaluation

Ask PDF

Dec 2, 2025

PPTArena: A Benchmark for PowerPoint Editing

We introduce PPTArena, a benchmark for PowerPoint editing that evaluates how agents modify real slides from natural-language instructions. Unlike benchmarks that rely on image-PDF renderings or text-to-slide generation, PPTArena features 100 decks with over 1,300 human-curated edits across 2,125 slides, spanning text, charts, animations, and professional master styles. Each edit pairs a ground-truth deck with a target rubric and is scored by two Vision-Language Model (VLM) judges: one rates instruction following from structural diffs, the other visual quality from slide images. On top of this benchmark, we present PPTPilot, a structure-aware agent that plans semantic edit sequences, routes between programmatic tools and deterministic XML operations, and verifies each result in an iterative plan-edit-check loop. PPTPilot outperforms strong VLM-based agents by more than 10 percentage points on compound, layout-sensitive, and cross-slide edits, with large gains in visual fidelity and deck-wide consistency. Despite this, all agents still struggle on long-horizon, document-scale tasks, underscoring how hard reliable PowerPoint editing remains. We publicly release our code at https://github.com/michaelofengend/PPTArena .

Michael Ofengenden, Yunze Man, Ziqi Pang +2

University of Illinois Urbana-Champaign · University of California, Berkeley

LLM Agent Evaluation AI Agent Benchmarks Tool-Using Agents

Ask PDF

Oct 9, 2026

VisualCoding: Learning Executable Skills for Reliable Visual Reasoning

Learning reusable skills can improve visual reasoning agents, yet applying a skill still requires the agent to implement or invoke a procedure and produce the final answer. Errors and hallucinations at these steps can prevent acquired experience from translating into reliable single-run performance. We propose VisualCoding, which combines execution-guided acquisition of reusable visual programs with direct adoption of their answers on applicable inputs. Execution-Guided Program Acquisition composes visual primitives and additional code into family-level solvers, then iteratively tests and revises them on labeled training instances. Scope-Aware Direct Execution runs a retained solver and returns its valid in-scope answer. We evaluate VisualCoding on four models across four benchmarks spanning 69 task families. VisualCoding exceeds the strongest evaluated baseline in mean accuracy by 4.45 percentage points with Gemini-3.1-Pro and by 14.78 points when reusing the frozen Pro-acquired artifacts with Flash-Lite. Ablations support execution-guided acquisition and show that direct execution outperforms textual or callable reuse of the same programs. Code is available at https://github.com/jiaq-liu/VisualCoding.

Liu Jiaqi, Jin Ruinan, Li Xiaoxiao

Department of Electrical and Computer Engineering, University of British Columbia · Vector Institute

Visual Program Synthesis Visually Grounded Reasoning

Ask PDF

Abstract

Coding Agents

Explore similar work

ReFigBench: Benchmarking Scientific Figure Reconstruction as Editable PowerPoint Artifacts

PPTArena: A Benchmark for PowerPoint Editing

VisualCoding: Learning Executable Skills for Reliable Visual Reasoning