LLMs have significantly advanced code generation, enabling the synthesis of functional programs. While recent systems achieve strong performance on many coding benchmarks, tasks involving programs such as TikZ that generate visual artifacts remain challenging, in particular on visual code customization. Unlike generation from scratch, customization requires localized, semantics-preserving edits: the model must locate relevant code, modify it according to the instruction, and preserve the remaining structure and rendering. Approaches based on post-hoc iterative refinement/correction where a verifier provides feedback to guide corrections, have shown promise. However, in the case of programs with a visual outcome such as in TikZ, where correctness is harder or likely impossible to formalize and evaluate automatically, deterministic verifiers do not exist. Hence, developers can only rely on imperfect verifiers. In this paper, we conduct an empirical study to answer:to what extent can iterative refinement remain effective when the verifier itself is unreliable?} We use TikZ as a focused case study that isolates the core difficulties of the problem (weak code structure, fine-grained visual semantics, and difficult feature localization) in a controlled and challenging setting. We define visual code customization as an iterative editing problem with an imperfect oracle, and introduce a framework for analyzing such iterative refinements. We conduct a large-scale study and evaluate multiple LLM-based and tool-augmented visual verifiers within iterative refinement pipelines, and perform extensive manual annotation of refinement trajectories to assess verifier behavior and feedback quality. Our findings show that even imperfect verifiers can determine with moderate accuracy whether visual instructions are applied to code, achieving F1-scores up to 0.815. Feedback improves iterative refinement, especially for weaker models, adding 11--20 perfect customizations for Qwen3-vl-30b-a3b-Instruct, while stronger models like Gemini-3 gain fewer improvements (+5) but benefit more from accurate verification that prevents premature acceptance. Feedback is effective only when it precisely identifies image issues, provides actionable guidance, addresses all relevant problems, and remains grounded in the original instruction.
Figures & tables
Figure 1. Excerpt of a TikZ code. Excerpt of a TikZ code that creates a shark drawing
Figure 2. Post-hoc Iterative Refinement. Overview of the Post-hoc Iterative Refinement in the context of TikZ code customization
Figure 3. Complete evaluation protocol. Overview the complete evaluation protocol used in the study
#Annotators
8
Total number of distinct items
3907
Total number of items
15345
Total number of distinct annotated items
4289
Total number of annotated items
17100
Avg. annotations per Item
1.11
Avg. distinct #items per annotator
536
Table 1. Human Annotation Statistics and Inter-Annotator Agreement
Figure 4. PR curves for each verifier. Visualization of the Precision-Recall curves for all verifiers
Verifier
Generator
Text
Text+Visual
Visual
Visual Property
Model
Threshold
Precision
Recall
F1
Threshold
Precision
Recall
F1
Threshold
Precision
Recall
F1
Threshold
Precision
Recall
F1
Gem.
Gem.
1.000
0.683
0.967
0.801
0.980
0.712
0.952
0.815
1.000
0.762
0.822
0.791
0.667
0.645
0.970
0.775
Gem.
Qwen
1.000
0.533
0.937
0.680
1.000
0.691
0.857
0.765
1.000
0.674
0.919
0.778
0.875
0.484
0.715
0.578
Qwen
Qwen
1.000
0.302
0.950
0.458
1.000
0.277
0.897
0.423
1.000
0.316
0.783
0.450
0.880
0.348
0.531
0.421
Table 2. Precision/Recall/F1 Scores for the best thresholds for llm verifiers
Generator
Agentic Vision
Segmentation-Tool Verifier
Threshold
Precision
Recall
F1
Threshold
Precision
Recall
F1
Gem.
1.000
0.712
0.949
0.814
0.000
0.601
1.000
0.751
Qwen
0.980
0.519
0.943
0.669
0.920
0.609
0.568
0.587
Table 3. Precision/Recall/F1 Scores for the best thresholds for fixed verifiers
Verifier Model
Generator
Text
Text+Visual
Visual
Visual Property
Gem.
Gem.
0.172
0.448
0.456
0.287
Gem.
Qwen
0.718
0.765
0.709
0.530
Qwen
Qwen
0.620
0.616
0.487
0.364
Table 4. Kendall correlations for LLM and Fixed verifiers
Figure 5. Total number of perfect customizations per step per verifier. Visualization of the score reached at each step for each configuration evaluated
Figure 6. Observed Customization Trajectories, and subsequent interpretation/actions by the generator. Decision tree describing the customization Trajectories studied
Vision-language models (VLMs) have shown strong capabilities in generating visualization code from textual or visual specifications. However, real-world visualization authoring is inherently iterative: users frequently revise existing visualizations to repair flawed charts or adapt them to desired styles. Existing benchmarks primarily evaluate generation from scratch, leaving visualization code editing from multimodal feedback largely unexplored. We introduce VisEditBench, a benchmark of 1,395 human-annotated visualization code-editing tasks grounded in realistic visualization workflows and failure cases. VisEditBench covers two practical settings: feedback-guided repair, where models revise visualization code using buggy or marked charts together with textual feedback, and reference-guided restyling, where models modify code to match a target chart image. Evaluating 20 state-of-the-art VLMs reveals that visualization code editing remains challenging: Claude-4.6-Sonnet achieves the best overall pass rate of 74.46%, while most open-source models remain below 50%. Performance is particularly weak on visually grounded style adaptation, where Claude-4.6-Sonnet achieves only 55.71%. To establish a strong baseline, we further propose VisEditAgent, a render-grounded editing framework that iteratively generates, executes, validates, and refines candidate edits. Built on GPT-4o, VisEditAgent improves overall pass rate from 55.75% to 67.99%, demonstrating the importance of render-grounded feedback for faithful visualization editing. We will release VisEditBench at https://github.com/vis-nlp/VisEditBench.
Mizanur Rahman, Arshia Azimlu, Shadikur Rahman +4
‡York University · $Nanyang Technological University · ¶Salesforce AI Research
Although multimodal large language models (MLLMs) have shown substantial potential in visual understanding and graphic code generation, editing scientific figures through code presents a greater challenge: a model must jointly recover visual structure, ground the requested change, generate compilable code, and preserve all unrelated content. While existing TikZ benchmarks mainly focus on figure reconstruction and generation, few systematically evaluate instruction-guided scientific figure editing with compilable code. We introduce Edit2TikZ, a comprehensive benchmark for scientific figure editing tasks, featuring 1,548 diverse and high-quality samples. Edit2TikZ combines real-world and controlled synthetic edit cases, supports both textual and visual localization request, and contains multi-step editing, each with step-level annotations. We further construct a human-aligned evaluation framework to measure whether a requested edit is completed while irrelevant content is preserved. Utilizing Edit2TikZ, we evaluate 14 mainstream MLLMs and find that current systems remain unreliable: on average, proprietary models achieve a compilation success rate of merely 75% and remain limited in both figure restoration and edit correctness, while compact models below 9B struggle further with instruction following and complete figure generation. Therefore, we build a mixed training set TikZEditMix and adopt reconstruction-then-editing curriculum learning for compact models. On Qwen3.5-4B, this training improves the compilation success rate from 45.35% to 83.40% and yields an average improvement of 18.7 points across our proposed evaluation metrics. The code and data will be released at https://github.com/Solunny/Edit2TikZ.
While Large Language Models (LLMs) have substantially advanced text-to-code synthesis, many real programming tasks specify intent through visual artifacts such as screenshots, charts, vector drawings, videos, and interactive states. These tasks require models to connect visual perception to executable programs, because correctness depends not only on syntax but also on layout, data semantics, interaction behavior, and domain-specific constraints that apply after execution. This survey examines Multimodal Code Intelligence, covering systems that generate, edit, refine, or reason with code under visually grounded inputs and outputs. We first formulate the field by the role that code plays in each task, distinguishing code as a rendered artifact, an editable symbolic structure, a scientific representation, an intermediate reasoning trace, or an executable policy or tool interface. We then organize benchmarks and methods into four domains: Graphical User Interface, Scientific Visualization, Structured Graphics, and Frontier Tasks and Frameworks. This taxonomy connects mature artifact-generation problems to emerging agentic and unified settings and allows us to compare how different tasks treat evidence of correctness. Looking ahead, we argue that future research may benefit from four verification-centered directions. Multi-signal validation can combine complementary evidence of correctness, multi-state verification can test behavior across execution trajectories, cross-task transfer testing can probe reusable visual-code skills, and verifiable agent traces can reveal whether agent actions are grounded in visual evidence. Together, these directions may move this field from single-output imitation toward evidence-grounded executable systems. An ongoing project and resources are available on GitHub.
Xuanle Zhao, Qiushi Sun, Jingyu Xiao +16
Meituan · The University of Hong Kong · The Chinese University of Hong Kong +7