cs.CLNov 2, 2026

VisCodeBench: a new benchmark dataset for Text-to-Vis reflecting real-world practice

Authors: Hy Nguyen, Thomas James Elton, Ella Wilson, Andrew Reeson, Cecile Paris, Josiah Poon, Jonathan K. Kummerfeld

Organizations: The University of Sydney · CSIRO’s Data61

Abstract

Generating visualisations from natural language requests (text-to-vis) enables users to explore data more efficiently. While LLMs can complete simple visualisation requests, their performance on challenging ones is not well covered by existing benchmarks. We introduce VisCodeBench, an end-to-end text-to-vis benchmark with 2,672 samples from three different domains: educational, scientific, and general audience scenarios with varying levels of complexity. We benchmark three visualisation systems---Codex, Claude Code, and LIDA---paired with a range of open and commercial LLMs across seven metrics covering execution, chart content, and visual quality. While basic execution is near-solved for strong systems, no system comes close to reproducing the reference data, with the best displayed data similarity reaching only 0.810.81 and falling to 0.690.69 on our hardest subset for this metric. We further find that frontier models separate on chart correctness and efficiency, with ClaudeCode-Opus-4.7 improving chart correctness by around 5 points over Codex-GPT-5 while running roughly 20 seconds faster. Separately, we find that system design matters as much as the underlying model: despite both running on GPT-5, Codex clearly outperforms LIDA, showing that the surrounding prompting, execution, and refinement pipeline can shape outcomes as much as the model itself. Finally, latency remains a practical obstacle, as even the strongest systems take 39-78 seconds to produce results. VisCodeBench is available at https://github.com/SAIPLL/VisCodeBench and accessed via a leaderboard; the full dataset is kept private to prevent LLM training use.

Figures & tables

Appendix figures & tables26 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. VisEditBench: Can Vision-Language Models Edit Visualization Code from Multimodal Feedback?

    Aug 11, 2026Mizanur Rahman, Arshia Azimlu, Shadikur Rahman +4VLM EvaluationData Visualization

  2. VisInteract: Towards Dynamic Interactive Text-to-Visualization under Imperfect Queries

    Sep 14, 2026Wenxin Xu, Jinwei Lu, Hwanhee Kim +4Data VisualizationMonte Carlo Tree Search