ChartRevise: A Dataset and Evaluation Protocol for Exact Chart Editing via Code

Jiaxiang Tang; Yi Zhou; Chad DeLuca; Rogerio Feris; Ahmed Khalil Omran; Zhi-Li Zhang; Pengyuan Li; Ali Anwar

cs.AISep 29, 2026

ChartRevise: A Dataset and Evaluation Protocol for Exact Chart Editing via Code

Authors: Jiaxiang Tang, Yi Zhou, Chad DeLuca, Rogerio Feris, Ahmed Khalil Omran, Zhi-Li Zhang, Pengyuan Li, Ali Anwar

Organizations: University of Minnesota, Twin-Cities · IBM Research · Horizon School of Digital Technologies

Abstract

Chart editing requires cross-modal edit grounding, realizing a requested visual change in the code that draws it, with necessary related updates and without altering unrelated content. Existing benchmarks emphasize either code executability or chart quality, but their metrics do not clearly distinguish request completion from missed coupled updates and gratuitous changes. We introduce ChartRevise, a structured dataset and evaluation protocol for exact program-grounded chart editing. For dataset construction, we build on the grammar of graphics to systematically cover chart-editing operations, using source-program checks to verify their applicability across chart types and libraries. To improve edit exactness, our pipeline checks individual requirements and guides repair or exclusion when they are unmet. The resulting dataset contains 92,438 records covering 344 edit types across 20 chart types and three plotting libraries. For evaluation, our reference-free protocol separately measures atomic requirement completion, identifies gratuitous changes, and detects missed coupled updates. These checks are combined with successful execution and rendering to determine exact-edit success. Across five models and four external benchmarks, fine-tuning yields relative gains of 16% in mean requirement recall and 22% in mean exact-edit rate.

Figures & tables

Figure 1: Failure modes in Chart2Code-L2 task 0565. Under-edited and exact panels are constructed; the over-edited panel is the released reference.

Resource	Dataset scope			Construction checks	Evaluation checks
Resource	Tasks	Chart types	Instruction types		Atomic requirements	Preservation	Coupled updates
ChartEdit	1,405	19	6	Human (request, code, chart)	—	—	—
ChartM 3	1K eval. 24K train	10	—	Model (charts)	—	✓ (code-space)	—
ChartEditVista	7,964	31	6	Model (charts) Human (code, charts)	—	—	—
ChartEditBench	4,142	37	35	Model (code, charts) Program (assertions)	✓ (assertions)	—	—
Chart2Code L2	1,010	19	—	—	—	—	—

Table 1: Task scope, construction checks, and explicit evaluation checks of chart-editing resources.

Figure 2: Dataset construction pipeline. Source-grounded instructions guide program generation. Execution and target validation produce repair feedback, and post-processing audits assess cross-modal edit grounding by checking completion, consistency, and preservation before release.

Figure 3: Record counts aggregated by category and chart type (left) and plotting library (right).

Figure 4: Dataset size and edit-category coverage under our taxonomy.

Benchmark	Family	Requirement recall (CR μ ) $\uparrow$	Full completion $\uparrow$	Gratuitous per task $\downarrow$	Coupled-update recall $\uparrow$	Exact-edit rate $\uparrow$
ChartEdit 1,405 tasks	Phi-3.5-V	.773 +.138	.674 +.155	.346 -.205	.540 +.076	.425 +.119
	InternVL2.5	.707 +.234	.593 +.215	.373 -.007	.500 +.188	.378 +.162
	Qwen3.5-4B	.867 +.047	.804 +.050	.165 -.091	.688 +.004	.572 +.055
	Qwen3.5-9B	.895 +.068	.846 +.085	.135 +.021	.731 +.064	.616 +.032
	Granite	.812 +.114	.733 +.128	.459 -.482	.622 +.077	.439 +.168
	rel. $\Delta$ of mean	+17.4%	+21.0%	-34.1%	+15.3%	+28.3%

Table 2: External transfer under the evaluation protocol. Green numbers mark improvements over the base model, and red numbers mark declines.

rel.\ \Delta

measures the mean change across models.

Family	ChartEdit		Chart2Code-L2		ChartM 3		ChartSync
	Exec.	Code.	Exec.	Code.	Exec.	Code.	Exec.	Code.
Phi-3.5-V	0.860 [-1pt] +0.000	75.4 [-1pt] +11.6	0.334 [-1pt] -0.066	38.0 [-1pt] +10.4	0.650 [-1pt] -0.010	46.3 [-1pt] +11.2	0.930 [-1pt] -0.010	85.5 [-1pt] +1.4
InternVL2.5	0.860 [-1pt] -0.040	71.6 [-1pt] +20.4	0.380 [-1pt] -0.214	32.6 [-1pt] +11.9	0.740 [-1pt] +0.050	43.3 [-1pt] +10.6	0.930 [-1pt] -0.030	88.2 [-1pt] +8.4
Qwen3.5-4B	0.910 [-1pt] +0.010	86.7 [-1pt] +3.6	0.444 [-1pt] +0.030	62.1 [-1pt] +1.6	0.700 [-1pt] +0.040	66.8 [-1pt] +13.1	0.980 [-1pt] -0.010	92.6 [-1pt] +0.3
Qwen3.5-9B	0.920 [-1pt] -0.020	89.8 [-1pt] +6.0	0.540 [-1pt] -0.027	73.4 [-1pt] +6.7	0.720 [-1pt] -0.010	74.8 [-1pt] +13.6	0.980 [-1pt] -0.010	94.8 [-1pt] -0.9
Granite	0.860 [-1pt] -0.010	79.7 [-1pt] +11.2	0.373 [-1pt] -0.223	43.1 [-1pt] +13.4	0.630 [-1pt] -0.070	48.4 [-1pt] +4.5	0.900 [-1pt] +0.030	83.4 [-1pt] +3.7

Table 3: Results under the inherited benchmark-facing metrics. Each cell reports the fine-tuned value and its change from the base model.

Figure 5: Editing states pooled across benchmarks. (a) Executable tasks grouped by completion ratio. (b) Requirements judged from code in non-executable outputs. (c) Explicitly requested and additional changes (coupled and gratuitous) in tasks satisfying all explicit requirements.

Figure 6: Phi-3.5-Vision outcomes grouped by explicit requirement count across benchmarks. In shaded bins, at least 85% of tasks come from Chart2Code-L2.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Category	Semantic objects
data	dataset, field, record, series, category
transform	aggregation, bin, stack, derived metric, sort, window
encoding	position, colour, size, shape, opacity, and text channels
layer	primary and overlay layers, error bars, reference line or band, secondary axis
mark	line, point, bar, area, arc, box, violin, cell
position	margins, anchor, z-order, aspect ratio, legend and annotation placement

Appendix

Table 4: Semantic objects of the twelve edit categories.

Edit	Chart	Template $\rightarrow$ realized instruction
E026	Bubble (seaborn)	Add subgroup by {subgroup_field} within {primary_group_field} $\rightarrow$ “For the subgroup, add subgroup by ‘Crop‘ within ‘Region‘ .”
E311	Bubble (seaborn)	Map {facet_row_field} to rows and {facet_column_field} to columns $\rightarrow$ “For the facet layout, map ‘Region‘ to rows and ‘Crop‘ to columns.”
E308	Pie (matplotlib)	Set the multi-facet grid to {facet_rows} rows and {facet_columns} columns $\rightarrow$ “For the subplot grid, set the multi-facet grid to 2 rows and 2 columns.”
E100	Bubble (plotly)	Remove facet mapping for {facet_field} $\rightarrow$ “For the facet channel, remove facet mapping for ‘Gender‘ .”

Appendix

Table 5: Template

\rightarrow

grounded instruction examples.

Category	Train	Test	Total
guide	14,356	253	14,609
data	11,223	218	11,441
style	10,994	252	11,246
position	8,521	223	8,744
annotation	8,385	187	8,572
transform	7,171	177	7,348

Appendix

Table 6: Per-category composition of the released dataset. Compound records are training-only.

Figure 7: Preservation is a second axis of difficulty, not a proxy for the first. (a) The bar joining the two markers is the applied/strict gap; dotted lines are the earlier export means. (b) Strict success by collection subset. Tier 2 contains 4–6 actions and tier 3 contains 8–12.

(a) Task level: records
Subset	Audited units	Correct	Correctness	95% CI
Single-action	300	255	85.0%	[80.5, 88.6]
Compound	200	139	69.5%	[62.8, 75.5]
Weighted	500	—	84.2%	[80.4, 88.1]
(b) Requirement level: compound requirements
Compound	1,071	—	95.6%	[94.2, 96.7]

Appendix

Table 7: Human validation of accepted training records. Panel (a) reports task-level correctness by subset and the training-set-weighted estimate. Panel (b) reports correctness of individual requirements in the same 200 compound records.

	Phi-3.5-V	InternVL2.5-8B	Qwen3.5-4B	Qwen3.5-9B	Granite-4.1-4B
Effective batch	$1\times 16\times 2=32$	$1\times 8\times 4=32$	$4\times 4\times 4=64$	$4\times 2\times 8=64$	$1\times 16\times 4=64$
Optimizer steps	2,830	2,805	1,416	1,416	1,416
Warmup steps	20	20	60	60	60
Optimizer	AdamW (8-bit)	AdamW (8-bit)	AdamW (8-bit)	AdamW	AdamW (8-bit)
Max sequence length	8,192	6,144	8,192	8,192	8,192
Image input	1 crop	$\leq$ 6 tiles + thumbnail	$\leq$ 1,280 visual tokens	$\leq$ 1,280 visual tokens	$384\times 384$

Appendix

Table 8: Per-model fine-tuning settings. Effective batch is per-device batch

\times

gradient accumulation

\times

number of GPUs. Settings shared by all runs are given in the text.

Metric	Source	Meaning / use
ExecRate	Mechanical	Fraction of all tasks whose generated program executes and renders.
CodeScore	LLM judge	Continuous editing-quality score over the generated program; can score non-running programs.
ChartScore	LLM judge	Overall rendered-chart quality; not target-edit success.
AppliedRate	LLM judge	Whether the requested edit reached the named target.
EditFidelity	LLM judge	Applied edit with no unrequested change, unintended reposition, or destroyed object.
Requirement recall (CR μ )	GT-free judge	Micro-aggregated fraction of explicit requirements satisfied across all tasks.

Appendix

Table 9: Metric inventory and provenance.

Benchmark	Family	ExecRate	CodeScore	ChartScore	8D F1	LLM score	LMM score
ChartEdit	Phi-3.5-V	$0.860\to 0.860$	$63.8\to\mathbf{75.4}$	$91.6\to\mathbf{92.4}$	$91.60\to\mathbf{92.54}$	$90.91\to\mathbf{91.69}$	$44.88\to\mathbf{54.80}$
	InternVL2.5	$0.900\to 0.860$	$51.2\to\mathbf{71.6}$	$90.2\to\mathbf{92.0}$	$90.03\to\mathbf{92.07}$	$90.53\to 90.29$	$41.82\to\mathbf{51.60}$
	Qwen3.5-4B	$0.900\to\mathbf{0.910}$	$83.1\to\mathbf{86.7}$	$94.0\to\mathbf{94.5}$	$93.69\to\mathbf{94.13}$	$94.21\to 93.79$	$59.16\to\mathbf{64.36}$
	Qwen3.5-9B	$0.940\to 0.920$	$83.8\to\mathbf{89.8}$	$94.7\to\mathbf{95.1}$	$93.81\to\mathbf{94.40}$	$94.72\to 94.32$	$65.24\to\mathbf{65.98}$
	Granite	$0.870\to 0.860$	$68.5\to\mathbf{79.7}$	$90.7\to\mathbf{93.6}$	$90.26\to\mathbf{93.17}$	$90.09\to\mathbf{92.29}$	$49.72\to\mathbf{58.16}$
Chart2Code-L2	Phi-3.5-V	$0.400\to 0.334$	$27.6\to\mathbf{38.0}$	$54.2\to\mathbf{58.9}$	$23.99\to 21.17$	$63.77\to\mathbf{67.18}$	$3.31\to\mathbf{3.41}$

Appendix

Table 10: Complete out-of-domain benchmark-facing transfer results.

Benchmark	Family	AppliedRate $\uparrow$	EditFidelity $\uparrow$	CR μ $\uparrow$	CR M $\uparrow$
ChartEdit	Phi-3.5-V	.625 +.127	.389 +.121	.773 +.138	.758 +.145
	InternVL2.5	.573 +.224	.345 +.143	.707 +.234	.703 +.241
	Qwen3.5-4B	.711 +.024	.470 +.052	.867 +.047	.871 +.049
	Qwen3.5-9B	.743 +.059	.498 +.022	.895 +.068	.899 +.073
	Granite	.666 +.131	.384 +.110	.812 +.114	.811 +.117
Chart2Code-L2	Phi-3.5-V	.234 +.136	.032 +.015	.691 +.120	.699 +.156

Appendix

Table 11: Complementary holistic outcomes.

Metric	Phi-3.5-V	InternVL2.5	Qwen3.5-4B	Qwen3.5-9B	Granite
TESR	$81.2\to\mathbf{88.0}$	$74.3\to\mathbf{90.7}$	$89.4\to\mathbf{91.7}$	$94.7\to 94.1$	$83.7\to\mathbf{87.9}$
VLCS	$64.8\to\mathbf{78.5}$	$76.5\to\mathbf{84.6}$	$90.0\to 87.6$	$87.3\to\mathbf{92.3}$	$79.0\to\mathbf{84.9}$
BFS	$95.4\to 93.6$	$96.2\to 94.7$	$98.1\to 97.4$	$98.5\to 98.3$	$91.1\to\mathbf{94.3}$
OCR F1	$93.3\to 92.8$	$94.0\to 92.5$	$94.1\to 93.9$	$94.6\to 94.3$	$93.7\to 92.9$
SSIM	$88.4\to 87.8$	$88.4\to 88.3$	$88.8\to 88.8$	$88.9\to\mathbf{89.0}$	$88.2\to 88.2$

Appendix

Table 12: ChartSync diagnostic metrics.

Figure 8: Terminal Python exception counts in the 4,285-task diagnostic run for all five models.

Benchmark	ExecRate	CodeScore	ChartScore	AppliedRate	EditFidelity
ChartEdit	1.00	95.41	99.88	0.796	0.552
Chart2Code L2	0.97	98.04	95.92	0.757	0.455
ChartSync	0.99	99.79	98.58	0.956	0.901
ChartM 3	0.91	89.52	89.26	0.345	0.297

Appendix

Table 13: Reference-program calibration. These calibrate the renderer–judge pipeline and are not model-specific capability ceilings.

Benchmark	Pairs	Agreement	$\Delta_{P}$	$\Delta_{H}$
ChartEdit	69	91%	$+0.232$	$+0.203$
ChartM 3	50	91%	$+0.180$	$+0.160$
ChartSync	64	98%	$+0.031$	$+0.078$
Chart2Code-L2	17	91%	$+0.059$	$+0.118$
All	200	93%	$+0.140$	$+0.145$

Appendix

Table 14: Paired human–protocol exact-edit agreement and fine-tuning gains on rendered outputs. Agreement is measured per output;

\Delta_{P}

and

\Delta_{H}

are the gains under automatic and human judgments, respectively. Gains are absolute rate differences.

Benchmark	Model	Training data	Requirement recall $\uparrow$	Full completion $\uparrow$	Gratuitous per task $\downarrow$	Coupled-update recall $\uparrow$	Exact-edit rate $\uparrow$
ChartEdit	Phi-3.5-V	Before filtering	.764	.670	.348	.527	.418
	Phi-3.5-V	Released	.773	.674	.346	.540	.425
	Qwen3.5-4B	Before filtering	.873	.804	.140	.666	.566
	Qwen3.5-4B	Released	.867	.804	.165	.688	.572
Chart2Code-L2	Phi-3.5-V	Before filtering	.689	.034	1.404	.489	.001
	Phi-3.5-V	Released	.691	.032	1.433	.493	.002

Appendix

Table 15: Fine-tuning on the corpus before and after exactness filtering across all four external benchmarks.

Explore similar work

ChartSync: A Benchmark for Visuo-Logical Cascading Chart Editing
Jul 11, 2026Jiakang Yu, Yixuan Chai, Tianci Wang +7Multi-Image Editing Benchmarks Renderer
1Jianghan University · †Work done during an internship at HiThink Research. · 2HiThink Research +3
Charts Are Beyond Pixels: Probing for Layer-Wise Chart Understanding and Editing
Sep 8, 2026Xiaochuan Zhong, Yifan Hou, Chenxi Pang +1Chart Layer-Wise
DeepDelta Lab, School of Artificial Intelligence, Shanghai Jiao Tong University · ETH Zurich · Google DeepMind
VisEditBench: Can Vision-Language Models Edit Visualization Code from Multimodal Feedback?
Aug 11, 2026Mizanur Rahman, Arshia Azimlu, Shadikur Rahman +4Image-To-Code Generation Visualization
‡York University · $Nanyang Technological University · ¶Salesforce AI Research

Jul 11, 2026, cs.CVJ/K move · Enter open · S save

ChartSync: A Benchmark for Visuo-Logical Cascading Chart Editing

Jiakang Yu, Yixuan Chai, Tianci Wang, Rihui Jin+6

1Jianghan University · †Work done during an internship at HiThink Research. · 2HiThink Research+3

Multi-Image Editing Benchmarks Renderer Image-To-Cad Generation

Open paper AskPDF arXiv

Generative image editing models struggle with structured statistical charts when data modifications require geometric synchronization. We formalize this task as Visuo-Logical Cascading Editing (VLCE). However, existing methods remain confined to localized text substitutions and struggle with dependency-aware cascading updates. To systematically evaluate this capability, we introduce ChartSync, an expert-validated benchmark constructed via a programmatic rendering pipeline that guarantees deterministic visuo-logical coupling for the ground truth. ChartSync comprises 870 triplets across 9 chart categories and 4 task types, including 235 geometry-coupled VLCE instances that specifically test cascading text-to-geometry synchronization. We further evaluate these instances via a two-tier framework combining objective visual metrics with a vision-language model judge paradigm to assess low-level fidelity alongside multimodal comprehension and reasoning. Evaluating 14 image editing models and one code-mediated pipeline reveals a nuanced capability gap: most open-source models suffer severe drops in geometric synchronization, while only two frontier proprietary models show emerging VLCE capability, with their residual errors mainly involving semantic isolation and background corruption. Our detailed error analysis deconstructs these failure paradigms to identify core meta-abilities for guiding future multimodal architectures. The ChartSync dataset and code are publicly released at https://github.com/kaka-yjk/ChartSyncCodebase.

Abstract

Explore similar work

ChartSync: A Benchmark for Visuo-Logical Cascading Chart Editing

Charts Are Beyond Pixels: Probing for Layer-Wise Chart Understanding and Editing

VisEditBench: Can Vision-Language Models Edit Visualization Code from Multimodal Feedback?