cs.CVFeb 23, 2026

Do Large Language Models Understand Data Visualization Principles?

Authors: Martin Sinnona, Valentin Bonas, Viviana Siless, Emmanuel Iarussi

Organizations: Universidad Torcuato Di Tella, Buenos Aires, Argentina · Universidad de Buenos Aires, Buenos Aires, Argentina · Consejo Nacional de Investigaciones Científicas y Técnicas, Buenos Aires, Argentina

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Data visualization principles, derived from decades of research in design and perception, ensure proper visual communication. While prior work has shown that large language models (LLMs) can generate charts or flag misleading figures, it remains unclear whether they and their vision-language counterparts (VLMs) can reason about and enforce visualization principles directly. Constraint based systems encode these principles as logical rules for precise automated checks, but translating them into formal specifications demands expert knowledge. This motivates leveraging LLMs and VLMs as principle checkers that can reason about visual design directly, bypassing the need for symbolic rule specification. In this paper, we present the first systematic evaluation of both LLMs and VLMs on their ability to reason about visualization principles, using hard verification ground truth derived from Answer Set Programming (ASP). We compiled a set of visualization principles expressed as natural-language statements and generated a controlled dataset of approximately 2,000 Vega-Lite specifications annotated with explicit principle violations, complemented by over 300 real-world Vega-Lite charts. We evaluated both checking and fixing tasks, assessing how well models detect principle violations and correct flawed chart specifications. Our work highlights both the promise of large (vision-)language models as flexible validators and editors of visualization designs and the persistent gap with symbolic solvers on more nuanced aspects of visual perception. They also reveal an interesting asymmetry: frontier models tend to be more effective at correcting violations than at detecting them reliably.

Figures & tables

Figure 1: Bridging symbolic and language-based reasoning for visualization principle checking. Given a chart specification and its corresponding rendering ( A ), we automatically check compliance with visualization principles using two complementary reasoning approaches: formal verification via handcrafted ASP constraints and language/image-based reasoning from natural-language prompts and chart renderings ( B ). The detected infractions are then compared ( C ), with performance metrics evaluated across both checking and fixing tasks.
Figure 2: Overview of the benchmark datasets and generation pipeline. (A) The synthetic dataset is produced by sampling Draco chart specifications from tabular data, automatically generating Vega-Lite renderings and ground-truth annotations of design principle violations. A Kullback–Leibler (KL) divergence filter ensures balanced coverage across violation types by accepting only candidates that improve uniformity in the distribution of problems. (B) The real visualization dataset complements the synthetic corpus by translating human-authored Vega-Lite specifications from GitHub into Draco grammar, enabling principle-level analysis of authentic visualization practices.
Figure 3: Evaluation setup for assessing LLM understanding of visualization principles. Each chart instance is defined by its Vega-Lite specification and annotated with one or more principle violations (e.g., high_cardinality_shape , stack_discrete , number_categorical ). Structured prompts (five semantically equivalent variants per principle) are used to query each model. Model outputs, expressed in a fixed JSON schema, are validated and compared to the reference annotations to compute F1-scores across all principles.
 Synthetic DatasetReal Dataset
ModelF1 ↑STD ↓F1 ↑STD ↓
Open-source    
Deepseek-v2-16B0.1720.1720.0050.0050.2020.2020.0090.009
Llava-13B0.1740.1740.0050.0050.1820.1820.0070.007
Llama-3.1-8B0.1950.1950.0090.0090.2170.2170.0150.015
Gemma-3-12B0.3240.3240.0070.0070.4480.4480.1140.114
Table 1: Comparison of open- and closed-source models across the Synthetic and Real datasets in text-only modality. We report mean F1 (↑) and standard deviation STD (↓) across runs. Best within each block in bold .
Figure 4: Model performance across mark types and problem categories. (A) Mean F1-scores by mark type , showing how different visual encodings influence model accuracy. (B) Mean F1-scores by principle sorted by Gemini-2.5-Flash performance.
 Synthetic DatasetReal Dataset
ModelF1 ↑STD↓F1 ↑STD↓
Open-source    
Llava-13B0.1720.1720.0150.0150.1840.1840.0080.008
Gemma-3-12B0.3190.0120.0120.4600.1530.153
Closed-source    
GPT-4o0.5640.5640.0180.0180.6570.6570.0200.020
Table 2: Comparison of open- and closed-source models across the Synthetic and Real datasets in text+image modality. We report mean F1 (↑) and standard deviation STD (↓) across runs. Best within each block in bold .
 Synthetic Dataset
ModelCO ↑\uparrowER ↑\uparrowCR
Open-source   
GPT-OSS-20B99%86.3%0.722
Closed-source   
Gemini-2.5-Flash100%94.3%0.724
Table 3: Comparison of open- and closed-source models across the Synthetic dataset in text-only modality . We report mean Enforcement Rate (ER↑) and Compliance Ratio (CR) across runs.
More in this topic

LLM Evaluation

Large language model evaluation measures what models can do, how reliably they do it, and whether benchmark scores predict real-world performance.

See all 2,408 papers

Explore similar work

CardsList
Feb 23, 2026

Do Large Language Models Understand Data Visualization Rules?

Data visualization rules-derived from decades of research in design and perception-ensure trustworthy chart communication. While prior work has shown that large language models (LLMs) can generate charts or flag misleading figures, it remains unclear whether they can reason about and enforce visualization rules directly. Constraint-based systems such as Draco encode these rules as logical constraints for precise automated checks, but maintaining symbolic encodings requires expert effort, motivating the use of LLMs as flexible rule validators. In this paper, we present the first systematic evaluation of LLMs against visualization rules using hard-verification ground truth derived from Answer Set Programming (ASP). We translated a subset of Draco's constraints into natural-language statements and generated a controlled dataset of 2,000 Vega-Lite specifications annotated with explicit rule violations. LLMs were evaluated on both accuracy in detecting violations and prompt adherence, which measures whether outputs follow the required structured format. Results show that frontier models achieve high adherence (Gemma 3 4B / 27B: 100%, GPT-oss 20B: 98%) and reliably detect common violations (F1 up to 0.82),yet performance drops for subtler perceptual rules (F1 < 0.15 for some categories) and for outputs generated from technical ASP formulations.Translating constraints into natural language improved performance by up to 150% for smaller models. These findings demonstrate the potential of LLMs as flexible, language-driven validators while highlighting their current limitations compared to symbolic solvers.

Martin Sinnona, Valentin Bonas, Emmanuel Iarussi +1

Universidad Torcuato Di Tella, Buenos Aires, Argentina · Universidad de Buenos Aires, Buenos Aires, Argentina · Consejo Nacional de Investigaciones Científicas y Técnicas, Buenos Aires, Argentina

Data VisualizationLLM Evaluation
AskPDF
Feb 23, 2026

De-rendering, Reasoning, and Repairing Charts with Vision-Language Models

Data visualizations are central to scientific communication, journalism, and everyday decision-making, yet they are frequently prone to errors that can distort interpretation or mislead audiences. Rule-based visualization linters can flag violations, but they miss context and do not suggest meaningful design changes. Directly querying general-purpose LLMs about visualization quality is unreliable: lacking training to follow visualization design principles, they often produce inconsistent or incorrect feedback. In this work, we introduce a framework that combines chart de-rendering, automated analysis, and iterative improvement to deliver actionable, interpretable feedback on visualization design. Our system reconstructs the structure of a chart from an image, identifies design flaws using vision-language reasoning, and proposes concrete modifications supported by established principles in visualization research. Users can selectively apply these improvements and re-render updated figures, creating a feedback loop that promotes both higher-quality visualizations and the development of visualization literacy. In our evaluation on 1,000 charts from the Chart2Code benchmark, the system generated 10,452 design recommendations, which clustered into 10 coherent categories (e.g., axis formatting, color accessibility, legend consistency). These results highlight the promise of LLM-driven recommendation systems for delivering structured, principle-based feedback on visualization design, opening the door to more intelligent and accessible authoring tools.

Valentin Bonas, Martin Sinnona, Viviana Siless +1

Universidad Torcuato Di Tella, Buenos Aires, Buenos Aires, Argentina · Facultad de Ciencias Exactas y Naturales Universidad de Buenos Aires, Buenos Aires, Argentina · Consejo Nacional de Investigaciones Científicas y Técnicas, Buenos Aires, Argentina

Data VisualizationChart Understanding
AskPDF
Mar 23, 2026

When Visuals Aren't the Problem: Evaluating Vision-Language Models on Misleading Data Visualizations

Visualizations help communicate data insights, but deceptive data representations can distort their interpretation and propagate misinformation. While recent Vision Language Models (VLMs) perform well on many chart understanding tasks, their ability to detect misleading visualizations, especially when deception arises from subtle reasoning errors in captions, remains poorly understood. Here, we evaluate VLMs on misleading visualization-caption pairs grounded in a fine-grained taxonomy of reasoning errors (e.g., Cherry-picking, Causal inference) and visualization design errors (e.g., Truncated axis, Dual axis, inappropriate encodings). To this end, we develop a benchmark that combines real-world visualization with human-authored, curated misleading captions designed to elicit specific reasoning and visualization error types, enabling controlled analysis across error categories and modalities of misleadingness. Evaluating many commercial and open-source VLMs, we find that models detect visual design errors substantially more reliably than reasoning-based misinformation, and frequently misclassify non-misleading visualizations as deceptive. Overall, our work fills a gap between coarse detection of misleading content and the attribution of the specific reasoning or visualization errors that give rise to it.

Harsh Nishant Lalai, Raj Sanjay Shah, Hanspeter Pfister +2

Birla Institute of Technology and Science, Pilani · Georgia Institute of Technology · Harvard University

VLM EvaluationChart Understanding
AskPDF