Multimodal agents are increasingly used for data visualization tasks but remain limited in autonomous review. Unlike humans, they may fail to recognize when a visualization is incorrect, determine what to change, repair it without disrupting correct content, and verify whether the intervention succeeded. Existing benchmarks largely evaluate predefined individual capabilities such as chart generation, instruction-guided editing, or defect detection, and therefore do not capture this gap in autonomous review. We introduce VisAudit, a benchmark for evaluating visualization diagnosis, repair, and verification. Given a rendered chart and configurable auxiliary evidence, including its source data table, intended text summary, and visualization code, an agent iteratively diagnoses potential defects, modifies and executes visualization code, inspects execution and visual feedback, and determines when no further intervention is needed. VisAudit defines three tracks spanning diagnosed repair, autonomous repair, and open-world verification, and contains 1,900 flawed instances across 21 chart types and 10 flaw categories, together with 300 initially correct charts. We construct the benchmark through controlled perturbations of validated source visualizations, with systematic verification and human-aligned quality control to ensure that injected defects are well-defined and recoverable from the available evidence. Experiments with leading multimodal models reveal a substantial gap from reliable autonomous review: the strongest evaluated model fully recovers only 47.4% of flawed charts in the autonomous-repair setting.
Figures & tables
Figure 1: Overview of VisAudit. Given a chart and optional auxiliary modalities, an agent iteratively edits, executes, and verifies visualization code in a multi-turn environment. We evaluate three tracks: (A) diagnosed repair, (B) autonomous repair, and (C) open-world verification.
Category
Injected discrepancy
Wrong title
The title contradicts the source table or summary.
Wrong axis label
An axis names a different quantity from the one plotted.
Missing legend
The mapping identifying plotted series is absent.
Swapped legend labels
Two legend labels identify each other’s series.
Value perturbation
A plotted value disagrees with its source value.
Swapped series
Two series display each other’s data.
Table 2: The ten flaw categories used to construct single-, double-, and triple-flaw instances.
Task group
Scenarios
Instances per scenario
Instances
Single flaw
10
40
400
Two flaws
36
25
900
Three flaws
60
10
600
Flawed total
106
–
1,900
Additional correct charts
–
–
300
Table 3: Evaluation-set composition. Track C contains the same flawed instances as Tracks A and B, plus 300 initially correct instances.
Model
Exec.
Rec-B
Rec-S
Pre-B
Pre-S
Turns
In
Out
Gemma
99.8
39.7
70.1
50.3
85.5
13.2
56.3
2.9
Mistral
98.4
36.8
66.0
59.2
88.2
8.5
29.0
2.3
Claude
99.9
30.8
62.5
69.0
91.7
9.0
48.5
2.6
Qwen
99.5
27.7
58.6
66.7
91.1
9.2
67.0
2.7
Ministral
87.7
25.1
52.2
37.8
70.1
10.5
73.1
3.0
DeepSeek
96.8
45.6
68.8
74.1
94.0
6.1
21.1
8.4
Table 4: Track B: Autonomous Repair on 1,900 flawed instances.
Model
Track
Rec-B
Rec-S
Pre-B
Pre-S
Turns
In/Out
Gemma
A
57.1
82.2
71.6
92.6
12.2
50.0/2.8
C
35.2
66.8
37.5
78.5
13.0
54.1/2.6
Qwen
A
60.2
83.8
84.5
96.4
8.1
43.2/2.8
C
26.4
55.9
66.1
89.9
9.0
65.2/2.7
Ministral
A
35.1
57.2
55.9
79.2
9.2
45.0/2.9
C
26.4
51.9
45.8
75.3
9.9
61.7/3.1
Table 5: Repair performance on the 1,900 flawed instances in Tracks A and C.
Model
Exec.
CCR
Pre-B
Pre-S
Turns
In/Out
Gemma
99.7
34.3
50.7
87.4
12.1
45.6/2.5
Qwen
100.0
72.0
88.0
96.4
7.5
42.7/1.9
Ministral
94.7
33.0
64.7
83.9
9.9
54.8/2.5
Claude
100.0
61.0
70.0
92.7
9.4
52.0/2.8
Mistral
99.0
32.7
59.7
90.5
7.8
20.4/1.8
DeepSeek
94.0
63.3
81.3
95.9
6.0
19.5/9.3
Table 6: Behavior on the 300 initially correct instances in Track C.
Flaw
Gemma
Qwen
Ministral
Claude
DeepSeek
Gemini
Mistral
Axis truncation
95.0/85.0/85.0
77.5/37.5/35.0
45.0/47.5/47.5
80.0/32.5/32.5
92.5/70.0/65.0
92.5/50.0/60.0
40.0/67.5/72.5
Color contradiction
62.5/52.5/37.5
62.5/30.0/17.5
22.5/12.5/10.0
72.5/57.5/45.0
65.0/65.0/62.5
72.5/52.5/57.5
47.5/25.0/37.5
Dropped series
82.5/47.5/35.0
87.5/65.0/70.0
40.0/27.5/32.5
95.0/75.0/65.0
87.5/60.0/65.0
87.5/77.5/75.0
65.0/30.0/32.5
Missing legend
75.0/30.0/30.0
75.0/15.0/20.0
40.0/30.0/30.0
77.5/27.5/40.0
77.5/55.0/60.0
75.0/45.0/47.5
52.5/25.0/45.0
Swapped legend labels
70.0/50.0/42.5
72.5/65.0/55.0
50.0/32.5/32.5
82.5/62.5/55.0
77.5/55.0/67.5
77.5/60.0/62.5
57.5/65.0/65.0
Swapped series
72.5/65.0/55.0
80.0/67.5/67.5
52.5/50.0/45.0
85.0/67.5/65.0
92.5/65.0/65.0
95.0/75.0/72.5
72.5/65.0/60.0
Table 7: Full recovery (Rec-B) by flaw category across Tracks A/B/C. Each category contains 40 single-flaw instances.
Model
Omitted
Rec-B
Rec-S
Pre-B
Pre-S
Turns
Gemma
None
39.7
70.1
50.3
85.5
13.2
Code
9.4
38.9
0.9
36.7
12.5
Table
38.8
69.7
57.5
89.2
12.0
Summary
13.1
39.5
61.1
91.5
13.5
Qwen
None
27.7
58.6
66.7
91.1
9.2
Code
10.2
41.8
0.8
33.8
9.3
Table 9: Track-B modality ablation on 1,900 instances.
Model
Budget
Rec-B
Rec-S
Pre-B
Pre-S
Gemma
3
0.0
0.0
100.0
100.0
5
0.0
0.0
100.0
100.0
10
39.3
67.6
56.9
88.0
Unrestricted
39.7
70.1
50.3
85.5
Qwen
3
0.0
0.0
100.0
100.0
5
28.5
59.0
62.5
86.9
Table 10: Track-B ablation over interaction budgets.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Chart type
Flawed instances
Clean instances
3D Bar Chart
2
1
Area Chart
148
37
Bar Chart
182
28
Box Plot
71
0
Bubble Chart
107
14
Funnel Chart
84
24
Appendix
Table 11: Instance-weighted source chart types in the flawed and additional clean populations. A zero denotes a source chart type absent from the clean selection.
Flaw category
Diagnosis string
Wrong title
the chart title contradicts the provided data or summary
Wrong axis label
an axis label names a quantity other than the one plotted
Missing legend
the legend that identifies the plotted series is missing
Swapped legend labels
two legend entries are attached to each other’s series
Value perturbation
one plotted value does not match its number in table.csv
Swapped series
two series plot each other’s data
Appendix
Table 12: Diagnosis strings appended to the Track-A task instruction.
Vision-language models (VLMs) have shown strong capabilities in generating visualization code from textual or visual specifications. However, real-world visualization authoring is inherently iterative: users frequently revise existing visualizations to repair flawed charts or adapt them to desired styles. Existing benchmarks primarily evaluate generation from scratch, leaving visualization code editing from multimodal feedback largely unexplored. We introduce VisEditBench, a benchmark of 1,395 human-annotated visualization code-editing tasks grounded in realistic visualization workflows and failure cases. VisEditBench covers two practical settings: feedback-guided repair, where models revise visualization code using buggy or marked charts together with textual feedback, and reference-guided restyling, where models modify code to match a target chart image. Evaluating 20 state-of-the-art VLMs reveals that visualization code editing remains challenging: Claude-4.6-Sonnet achieves the best overall pass rate of 74.46%, while most open-source models remain below 50%. Performance is particularly weak on visually grounded style adaptation, where Claude-4.6-Sonnet achieves only 55.71%. To establish a strong baseline, we further propose VisEditAgent, a render-grounded editing framework that iteratively generates, executes, validates, and refines candidate edits. Built on GPT-4o, VisEditAgent improves overall pass rate from 55.75% to 67.99%, demonstrating the importance of render-grounded feedback for faithful visualization editing. We will release VisEditBench at https://github.com/vis-nlp/VisEditBench.
Mizanur Rahman, Arshia Azimlu, Shadikur Rahman +4
‡York University · $Nanyang Technological University · ¶Salesforce AI Research
Web agents increasingly browse, interpret, and act on web content on behalf of users. In many realistic workflows, however, the visual evidence they consume is not neutral: charts and dashboards can be misleading, shaping judgments without issuing any explicit malicious instruction. Unlike chart question answering, Web Agents treat charts as intermediate evidence whose misinterpretation can propagate into concrete downstream actions --- a reliability risk that remains underexplored. We introduce MisVisAgentBench, the first paired executable benchmark for evaluating how misleading visualizations affect Web Agent execution. It comprises 280 task instances in matched clean--misleading pairs that differ only in chart evidence. We further provide a diagnostic framework that isolates misleading-induced failures from baseline task difficulty (Zhu et al., 2025) and captures execution-level effects such as extra browsing steps and decision hesitation. Across thirteen vision-language Web Agents, replacing clean charts with misleading counterparts reduces the average task success rate by more than 34 percentage points. Moreover, these failures predominantly follow the targeted misleading action branch, suggesting that deceptive charts systematically redirect rather than randomly disrupt agent decisions. These results reveal that visualization robustness is a critical yet understudied dimension of Web Agent safety and reliability.
Yisheng Li, Dongming Han, Hongkun Pan +5
State Key Laboratory of Novel Software Technology, Nanjing University · HiThink Research · Zhejiang University
Real-world multimodal agents solve multi-step workflows grounded in visual evidence. For example, an agent can troubleshoot a device by linking a wiring photo to a schematic and validating the fix with online documentation, or plan a trip by interpreting a transit map and checking schedules under routing constraints. However, existing multimodal benchmarks mainly evaluate single-turn visual reasoning or specific tool skills, and they do not fully capture the realism, visual subtlety, and long-horizon tool use that practical agents require. We introduce AgentVista, a benchmark for generalist multimodal agents that spans 25 sub-domains across 7 categories, pairing realistic and detail-rich visual scenarios with natural hybrid tool use. Tasks require long-horizon tool interactions across modalities, including web search, image search, page navigation, and code-based operations for both image processing and general programming. Comprehensive evaluation of state-of-the-art models exposes significant gaps in their ability to carry out long-horizon multimodal tool use. Even the best model in our evaluation, Gemini-3-Pro with tools, achieves only 27.3% overall accuracy, and hard instances can require more than 25 tool-calling turns. We expect AgentVista to accelerate the development of more capable and reliable multimodal agents for realistic and ultra-challenging problem solving.
Zhaochen Su, Jincheng Gao, Hangyu Guo +10
Hong Kong University of Science and Technology · University of North Carolina at Chapel Hill · Zhejiang University +1