Infographic charts communicate data through cohesive compositions of visual elements such as charts, text, and images. Existing infographic chart datasets primarily annotate individual elements with bounding boxes and category labels. Such element-level annotations do not capture hierarchical and spatial relationships, limiting their use for understanding how elements are organized and related within an infographic and for generating well-organized designs. To address this limitation, we introduce ChartGalaxy++, a richly annotated dataset of 217,195 infographic charts. At the core of this dataset is a scene graph representation, in which nodes represent visual elements and semantic groups, and edges encode their hierarchical and spatial relationships. For real infographics, we construct scene graphs by detecting and progressively grouping visual elements; for synthetic ones, we derive them directly from the generation process. We demonstrate the utility of ChartGalaxy++ through three applications: 1) image-to-scene-graph prediction, 2) scene-graph-augmented infographic question answering, and 3) evaluation of scene graph preservation in image generation. Together, these applications highlight the value of ChartGalaxy++ for advancing chart understanding and generation.
Figures & tables
Figure 1: ChartGalaxy++ provides scene graph annotations for 217,195 infographic charts. These annotations support (a) image-to-scene-graph prediction, (b) scene-graph-augmented infographic question answering, and (c) evaluation of scene graph preservation in image generation.
Figure 2: Comparison of flat and scene graph representations for infographic understanding. Flat representations lack explicit structural information, potentially leading to incorrect element associations. Our scene graph representation resolves such ambiguities by explicitly grouping semantically related elements and encoding their hierarchical and spatial relationships.
Figure 3: Overview of the scene graph annotation pipeline.
Node-level
Edge-level
Model
Overall
Type
Box
Text
Color
Description
Hierarchical
Spatial
Molmo2-8B
8.7
67.3
32.0
43.2
65.0
34.9
4.0
0.9
InternVL3.5-8B
27.7
79.6
34.6
75.1
68.7
36.2
9.8
10.0
Gemma 4-12B-IT
45.8
94.1
51.1
91.8
77.5
53.7
18.1
29.5
Qwen3.5-4B
27.8
90.5
25.3
86.4
80.1
51.3
9.0
10.6
Qwen3.8-27B
60.0
95.6
65.7
96.5
87.3
54.8
27.3
47.1
Table 1: Comparison of our image-to-scene-graph model with representative LVLMs.
Figure 4: Qualitative comparisons of our image-to-scene-graph model and GPT-6 Astra.
Open-source models
Proprietary models
Prompting
Molmo2 8B
InternVL3.5 8B
Gemma 4 12B-IT
Qwen3.5 4B
Qwen3.8 27B
Qwen3.8 Flash-Next FP8
GLM-5.3 Flash
Gemini 3.8 Flash
GPT-5.6 Sol
GPT-6 Astra
Direct
37.5
31.9
32.4
35.9
75.1
72.7
74.0
90.0
81.4
95.9
Evidence-guided
40.4
33.8
40.2
46.7
76.9
76.0
79.0
88.5
85.1
94.0
Scene-graph-augmented
46.7
45.3
67.4
53.4
86.9
86.1
83.9
90.6
91.6
97.6
Table 2: Comparison of QA accuracy under different prompting methods.
Annotations
Accuracy
No scene graph
76.9
+ node types & + bounding boxes
77.8
+ relationships
78.9
+ labels
80.2
+ text without digits
81.5
+ text with digits (best)
86.9
Table 3: QA accuracy on Qwen3.8-27B with different scene graph inputs.
Node-level
Edge-level
Model
Overall
Type
Box
Text
Color
Description
Hierarchical
Spatial
HiDream-O1-Image
18.7
92.1
32.8
20.8
75.9
40.1
16.6
6.6
GLM-Image
35.9
83.7
25.0
19.1
74.3
35.9
20.8
14.0
Qwen-Image-2512
40.1
82.5
28.2
15.9
77.8
36.4
28.0
19.1
Qwen-Image-SP
44.9
85.9
31.7
24.9
78.7
38.1
35.2
25.3
ERNIE-Image-Turbo
45.3
89.6
30.3
34.2
78.6
45.1
36.9
23.2
Table 4: Comparison of scene graph preservation across representative image-generation models.
Figure 5: Qualitative comparison of Ideogram 4 and GPT-Image-2.5 Sunburst.
Appendix figures & tables18 assets
Supplementary material from the paper’s appendix.
Appendix
Field
Applies to
Definition
label
All
Unique node identifier within the image, combining its type and one-based position in the node array, e.g., <text 4> .
parent
All
Label of the direct parent group, or the reserved token ROOT .
node_kind
All
element or group .
element_type
Element
text , image , or shape .
group_type
Group
One of the seven semantic group types.
bbox
All
Four integers [y0,x0,y1,x1] in the range [0,1000] .
Appendix
Table 5: Node fields in the JSON representation. Fields marked “element” or “group” apply only to that node kind.
Group type
Meaning
chart
A chart component containing data marks or data items and their associated reading references.
axis
An axis and its associated components, such as ticks, tick labels, and an axis title.
legend
A legend containing its entries and any associated title.
legend_item
One legend entry associating a visual key with its label.
data_item
Elements jointly representing one observation, such as a mark with its category label, value, or identifying icon.
series
Elements representing one data series, including its marks and associated labels.
Appendix
Table 6: Group types and their intended meanings.
Figure 6: A real infographic and its scene-graph annotation. (a) Original four-bar Statista infographic by Felix Richter (2022), with its source and attribution retained. (b) Enlarged view of the video-games data item: group 2 contains bar 3, illustration 4, revenue label 5, category label 6, and explanatory text 7. The annotation boxes and numbered tags are added for this example and correspond to the JSON excerpt below.
Stage
Input
Output
Element annotation
Image, crops, and localized candidates
Leaf nodes with type, box, palette, description, and text where applicable.
Hierarchical relationship extraction
Image and annotated elements
Semantic groups and parent–child assignments.
Spatial relationship extraction
Element and group boxes
Direction, alignment, overlap, and proximity relationships.
Iterative refinement
Image, current graph, and validation findings
Corrected annotations, rechecked graph, and remaining unresolved findings.
Appendix
Table 7: Inputs and outputs of the image-based annotation pipeline. Hierarchical relationships are inferred from the image and annotated elements; spatial relationships are computed from node geometry.
Component
Setting
HunyuanOCR-1.5
spotting_json ; temperature 0, top- p 1, repetition penalty 1.08; at most 32,768 output tokens.
DAD
Fixed layer-localization instruction; temperature 0, top- p 0.9, top- k 20; deployed output limit 16,384 tokens.
Image and crop inputs
Up to 3×3 ownership cores; grid size depends on image dimensions and candidate count, with a 900-pixel size threshold and 80-pixel context padding.
Element attributes / scene description
Gemini 3.8 Flash; at most 64 candidates per attribute request; LOW thinking; at most 32,768 output tokens.
Localization recovery / crop correction
Gemini 3.8 Flash; LOW thinking; at most 32,768 output tokens.
Whole-image review
Gemini 3.8 Flash; MEDIUM thinking; at most 32,768 output tokens; at most two audit rounds in the recorded extraction configuration.
Appendix
Table 8: Model settings for the image-based annotation pipeline. Token limits are per request.
Complexity
Images
Low ( ≤50 nodes)
191
Medium (51–91 nodes)
344
High ( ≥92 nodes)
465
Total
1000
Appendix
Table 9: Quality-assessment sample by scene graph complexity.
Measure
Result
Non-root nodes
99,010
Element nodes
82,042
Element precision
99.81%
Element recall
99.78%
Mean box IoU
0.9976
Text transcription errors
134/34,242
Appendix
Table 10: Node-level assessment before audit corrections. Error rows report affected nodes / eligible nodes. Numeric transcription errors are a subset of text errors. Precision, recall, and IoU are computed against the expert-adjudicated reference.
Name
URL
License
Statista (Daily Data)
statista.com
CC BY-ND 4.0
Visual Capitalist
visualcapitalist.com
Customized
Our World in Data
ourworldindata.org
CC BY
OECD
oecd.org
CC BY 4.0
Office for National Statistics
ons.gov.uk
OGL
Kaiser Family Foundation
kff.org
CC BY-NC-ND 4.0
Appendix
Table 11: Selected chart sources and their licenses, following ChartGalaxy ( Li et al., 2026b ) . Source names and license entries link to official policies.
Node kind
Type
Count
Percentage
Element
text
6,044,031
38.76%
image
1,107,791
7.11%
shape
8,439,834
54.13%
Group
chart
221,006
7.16%
axis
72,699
2.35%
legend
88,976
2.88%
Appendix
Table 12: Element and group-type distributions. Percentages use all 15,591,656 elements or all 3,088,632 non-root groups as the denominator, respectively.
Quantity
Min
P25
Median
P75
P90
P99
Max
Mean
Elements
1
35
54
85
121
336
3,682
71.79
Non-root groups
0
7
10
18
31
57
234
14.22
Total nodes
2
44
67
108
151
344
3,725
87.01
Hierarchy depth
2
4
4
5
6
7
8
4.54
Appendix
Table 13: Per-chart distributions over all 217,195 charts. Total nodes include one implicit root per chart; non-root groups exclude it. Quantiles use linear interpolation of the ordered observations.
Figure 7: Empirical cumulative distributions of (a) element counts and (b) group counts for real and synthetic infographic charts. Both count axes are logarithmic and include all charts. Markers identify the median element count and the 90th-percentile group count. Each distribution is normalized within its real or synthetic subset. Group counts include the implicit root, matching the main-text convention.
Model (revision prefix)
Context
Output
Decoding
Molmo2-8B ( e28fa28 )
36,864
24,576
Sampling; inventory first
InternVL3.5-8B ( 9bb6a56 )
32,768
24,576
Temperature 0
Gemma 4-12B-IT ( 707f0a3 )
65,536
24,576
Temperature 0
Qwen3.5-4B ( 851bf6e )
32,768
24,576
Sampling; thinking off
Qwen3.8-27B ( 1d4bf0f )
65,536
24,576
Temperature 0; thinking off
Qwen3.8-Flash-Next-FP8 ( 236dfdf )
65,536
24,576
Temperature 0; thinking off
Appendix
Table 14: Inference settings for image-to-scene-graph prediction. Context and output limits are in tokens; an API dash denotes a context limit not explicitly set by the client. Open-model revision prefixes identify the archived weight revisions; full identifiers are retained in the accompanying inference configuration files.
Figure 8: Performance by reference node count. The top row compares our model with the three proprietary baselines; the bottom row includes all seven open-source baselines. All panels use equal image weights within each bin.
Figure 9: QA examples of within-group correspondences and across-group reasoning. Boxes on the full charts mark the regions enlarged on the right.
Figure 10: QA examples of connections and spatial adjacency. Boxes on the full charts mark the regions enlarged on the right.
Model
Checkpoint
Sampling
HiDream-O1-Image
HiDream-ai/HiDream-O1-Image
50 steps; CFG 5; shift 3; UniPC
GLM-Image
zai-org/GLM-Image
50 steps; CFG 1.5
Qwen-Image-2512
Qwen/Qwen-Image-2512
50 steps; CFG 4
Qwen-Image-SP
Qwen/Qwen-Image with heheyas/Qwen-Image-SP transformer and text-encoder overlays
25 steps; CFG 4
ERNIE-Image-Turbo
baidu/ERNIE-Image-Turbo
8 steps; CFG 1; prompt enhancement disabled
SenseNova-U1 Infographic V3
sensenova/SenseNova-U1-8B-MoT-Infographic-V3
50 steps; CFG 4; shift 3; thinking disabled
Appendix
Table 15: Recorded checkpoints and generation settings. CFG denotes classifier-free guidance.
Figure 11: Generation performance by input node count. The top row compares the GPT-Image and Nano Banana variants with Ideogram 4; the bottom row includes the remaining six models, with GPT-Image-2.5 Sunburst repeated for reference. Each point is the mean score over the images in its bin.
Model
Direction
Alignment
Overlap
Proximity
HiDream-O1-Image
6.7
3.5
13.5
5.0
GLM-Image
16.0
3.6
19.7
7.5
Qwen-Image-2512
21.9
5.2
25.8
11.1
Qwen-Image-SP
28.2
9.1
34.1
15.2
ERNIE-Image-Turbo
25.5
9.5
34.0
15.2
SenseNova-U1 Infographic V3
28.2
11.1
36.3
16.8
Appendix
Table 16: Spatial relationship scores by component, averaged over all 1,000 images per model. Each column is multiplied by 100. The overall Spatial score in Table 4 pools relationship counts within each image; it is not the arithmetic mean of these four columns.
Understanding charts requires models to jointly reason over geometric visual patterns, structured numerical data, and natural language -- a capability where current vision-language models (VLMs) remain limited. We introduce ChartNet, a high-quality, million-scale multimodal dataset designed to advance chart interpretation and reasoning. ChartNet leverages a novel code-guided synthesis pipeline to generate 1.5 million diverse chart samples spanning 24 chart types and 6 plotting libraries. Each sample consists of five aligned components: plotting code, rendered chart image, data table, natural language summary, and question-answering with reasoning, providing fine-grained cross-modal alignment. To capture the full spectrum of chart comprehension, ChartNet additionally includes specialized subsets encompassing human annotated data, real-world data, safety, and grounding. Moreover, a rigorous quality-filtering pipeline ensures visual fidelity, semantic accuracy, and diversity across chart representations. Fine-tuning on ChartNet consistently improves results across benchmarks, demonstrating its utility as large-scale supervision for multimodal models. As the largest open-source dataset of its kind, ChartNet aims to support the development of foundation models with robust and generalizable capabilities for data visualization understanding. The dataset is publicly available at https://huggingface.co/datasets/ibm-granite/ChartNet
Faithful chart generation in real-world data-science workflows requires grounding visualizations in scattered evidence, computing chart-ready quantities, and rendering them accurately. Modern LLMs can produce visually plausible, instruction-compliant charts, yet data-level hallucinations remain difficult to detect in long, noisy, and multimodal contexts. To measure this gap, we introduce DEEPCHART, an expert-annotated benchmark of 1,482 task-conditioned chart-generation instances drawn from real-world scientific papers, financial filings, and ecosystem reports. DEEPCHART formulates chart generation as an Extract--Reason--Visualize pipeline and evaluates source-data extraction, derived-data reasoning, and chart rendering stage by stage. Experiments with state-of-the-art models show that visually plausible charts often conceal data-level hallucinations, with extraction and reasoning errors common in realistic long and multimodal settings. These findings suggest that larger context windows alone are insufficient; faithful chart generation also requires reliable evidence extraction and quantitative reasoning before rendering. Our benchmark and associated resources are available at https://github.com/tangdouer1005/DeepChart.
Jiahui tang, Kuicai Dong, Dexun Li +7
University of Science and Technology of China · Huawei Technologies Co., Ltd.
Annotations are essential to communicative visualization, helping explain data, emphasize key findings, and guide attention. While multimodal large language models (MLLMs) offer new opportunities for automatic chart annotation authoring, their capabilities in this task remain underexplored. To address this gap, we introduce ChartAnno, a comprehensive benchmark for evaluating MLLMs on chart annotation generation. ChartAnno contains 1,200 real-world charts with paired annotated and unannotated executable code, along with 3,600 annotation instructions spanning three levels of specificity. We also develop a multidimensional evaluation framework combining rule-based and LLM-judged metrics to assess execution, structural compliance, semantic consistency, and design effectiveness. We evaluate 10 representative MLLMs under two primary chart input settings: (1) chart code alone and (2) both code and chart image. Results reveal that proprietary models lead overall, though open-source models narrow the gap. While higher instruction specificity improves annotation quality, inferring abstract communicative intent remains difficult across all models. Providing chart images yields marginal benefit when code is available. We also examine the effect of chart code through an image-only ablation and analyze the effects of multiple task complexity indicators and instruction-level transitions. Further analyses characterize common failure modes and validate the reliability of the LLM-based judge. Experiments with D3 and SVG demonstrate the generalizability of ChartAnno beyond its primary Python setting.
Zhenghan Chen, Zekai Shao, Lidan Tan +10
Fudan University · Sun Yat-sen University · The Hong Kong University of Science and Technology (Guangzhou) +1