Chart2SVG: Editable SVG Generation from Raster Chart Images

Jinning Cui; Lu Chen; Haoyan Shi; Yue He; Chenglong Wang; Mengyu Zhou; Weidong Huang; Yunhai Wang

cs.LGAug 27, 2026

Chart2SVG: Editable SVG Generation from Raster Chart Images

Authors: Jinning Cui, Lu Chen, Haoyan Shi, Yue He, Chenglong Wang, Mengyu Zhou, Weidong Huang, Yunhai Wang

Organizations: University of Technology Sydney, Sydney, NSW, Australia

Abstract

We present Chart2SVG, a multimodal large language model that converts static raster charts into structurally organized, semantically enriched SVGs that support programmatic editing. By incorporating chart-specific semantic tokens into a vision-language model, Chart2SVG captures both geometric primitives and their functional roles. To support robust structural recovery, we introduce Beagle+, a dataset of 33K canonicalized and structurally distilled chart samples. Our approach combines specialized training objectives with a rendering-aware post-training phase, producing SVGs that are both visually accurate and structurally consistent. To facilitate higher-level manipulations, we construct a Chart Structure Graph (CSG) that exposes visual dependencies, enabling tasks such as interactive exploration, chart repurposing, and layout reuse. Experiments show that Chart2SVG substantially outperforms baselines in reconstruction fidelity and downstream editing utility, advancing the development of intelligent and interactive visualization tools.

Figures & tables

Figure 1: Training pipeline. (Left) SFT: a vision encoder and LLM are aligned via parameter-efficient adapters to generate structured SVG sequences. (Right) Rendering-aware GRPO: SVG candidates are evaluated by code validity and visual fidelity rewards, with KL divergence against a reference model for regularization.

Figure 2: CSG construction pipeline. From a reconstructed SVG (a), chart components are extracted (b) and organized into a graph (c) with data constraints (governing mark-axis-legend relationships) and spatial constraints (defining layout relative to marks).

Model	ChartBlocks Subset						Fusion Subset						Plotly Subset
	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	$\mathbf{IoU}_{\mathbf{fg}}\uparrow$	Edge	Failure	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	$\mathbf{IoU}_{\mathbf{fg}}\uparrow$	Edge	Failure	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	$\mathbf{IoU}_{\mathbf{fg}}\uparrow$	Edge	Failure
	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	$\mathbf{IoU}_{\mathbf{fg}}\uparrow$	Cons. $\uparrow$	Rate $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	$\mathbf{IoU}_{\mathbf{fg}}\uparrow$	Cons. $\uparrow$	Rate $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	$\mathbf{IoU}_{\mathbf{fg}}\uparrow$	Cons. $\uparrow$	Rate $\downarrow$
Adobe Illustrator $\dagger$	\new 22.9034	\new 0.9107	\new 0.1736	\new 0.9187	\new 0.7276	\new 0.0000	\new 21.3574	\new 0.8813	\new 0.2461	\new 0.9257	\new 0.5082	\new 0.0000	\new 20.8586	\new 0.8838	\new 0.2174	\new 0.9217	\new 0.5202	\new 0.0000
OmniSVG [ 40 ]	2.3979	0.1757	0.7943	0.2498	0.3705	0.2667	4.1156	0.2162	0.7501	0.3027	0.2814	0.1615	2.7260	0.0605	0.8878	0.1872	0.1359	0.5455
StarVector [ 26 ]	9.9510	0.4935	0.6870	0.4454	0.2497	0.2667	9.1777	0.4795	0.7316	0.3304	0.2161	0.1385	9.6040	0.4933	0.6955	0.2882	0.2240	0.1273
ChartCoder [ 42 ]	6.3438	0.4462	0.7897	0.1946	0.2872	0.1917	3.5305	0.2095	0.8778	0.2129	0.1008	0.5077	2.1856	0.1358	0.8838	0.2543	0.1172	0.4091

Table 1: In-distribution evaluation on Beagle+ test subsets. ↑: higher is better; ↓: lower is better. Bold: best among model-based methods; underline: second-best among model-based methods.

Model	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	$\mathbf{IoU}_{\mathbf{fg}}\uparrow$	Edge	Failure
Model	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	$\mathbf{IoU}_{\mathbf{fg}}\uparrow$	Cons. $\uparrow$	Rate $\downarrow$
Adobe Illustrator $\dagger$	\new 15.8100	\new 0.7440	\new 0.4726	\new 0.3873	\new 0.3691	\new 0.0000
OmniSVG [ 40 ]	3.6713	0.2736	0.7947	0.1345	0.2755	0.2390
StarVector [ 26 ]	8.6618	0.4116	0.7714	0.1214	0.1685	0.3558
ChartCoder [ 42 ]	7.8008	0.4115	0.7898	0.0988	0.1864	0.4026
GPT-4o	12.6482	0.6014	0.6066	0.2552	0.2423	0.0597

Table 2: Out-of-distribution evaluation on VisAnatomy [ 6 ] . ↑: higher is better; ↓: lower is better. Bold: best among model-based methods; underline: second-best among model-based methods.

Model	Bar Chart		Pie Chart		Line Chart
Model	Len. $\uparrow$	Acc. $\uparrow$	Len. $\uparrow$	Acc. $\uparrow$	Len. $\uparrow$	Acc. $\uparrow$
StarVector [ 26 ]	\new 0	\new 0	\new 0	\new 0	\new 0	\new 0
OmniSVG [ 40 ]	\new 0.0210	\new 0	\new 0.1039	\new 0.0018	\new 0	\new 0
GPT-4o	\new 0.5944	\new 0.0651	\new 0.7013	\new 0.0914	\new 0.0870	\new 0.0046
o3	\new 0.7413	\new 0.1456	\new 0.6623	\new 0.1243	\new 0.3913	\new 0.1763
Gemini-2.5-Flash	\new 0.6643	\new 0.1545	\new 0.7922	\new 0.1664	\new 0.6087	\new 0.2158

Table 3: \new Data accuracy by chart type on VisAnatomy [ 6 ] . For each chart, we retrieve the encoded data array and report the array-length matching rate (Len.) and the numerical accuracy over matched data (Acc.).

Figure 3: Qualitative comparison of our model with general-purpose and state-of-the-art chart and SVG foundation MLLMs on chart vectorization.

Variant	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	$\mathbf{IoU}_{\mathbf{fg}}\uparrow$	Edge	Failure
Variant	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	$\mathbf{IoU}_{\mathbf{fg}}\uparrow$	Cons. $\uparrow$	Rate $\downarrow$
Chart2SVG-4B	14.0211	0.6951	0.5758	0.2640	0.2919	0.0052
w/o SVG tokens	12.2325	0.6028	0.6358	0.1801	0.2585	0.1481
w/o GRPO	6.8122	0.4672	0.6263	0.3771	0.2579	0.0260
w/o $\mathrm{IoU}_{\mathrm{fg}}$ in GRPO	12.5594	0.6342	0.6255	0.1795	0.2616	0.0987

Table 4: Ablation studies on the 4B model. We take the full model as the reference and remove one component at a time. Higher is better for PSNR, SSIM, Foreground IoU (

\mathrm{IoU}_{\mathrm{fg}}

), and Edge Cons., while lower is better for LPIPS and Failure Rate.

\new CSG	\new Semantic Label	\new Success Rate $\uparrow$	\new CSG Acc. $\uparrow$	\new Propagation Precision $\uparrow$	\new Propagation Recall $\uparrow$	\new Propagation F1 $\uparrow$
\new ✗	\new ✗	\new 0.5000	\new —	\new —	\new —	\new —
\new ✓	\new ✗	\new 0.6300	\new 0.8389	\new 0.6194	\new 0.4798	\new 0.5407
\new ✓	\new ✓	\new 0.7400	\new 0.8919	\new 0.7025	\new 0.4913	\new 0.5782

Table 5: \new Controlled editing study on 100 charts sampled from VisAnatomy. Each chart is paired with four manually specified editing tasks covering color, axis, data-value, and layout edits. CSG denotes whether CSG-inferred constraints are used during editing, and Semantic Label denotes whether the SVG is heuristically tagged before CSG construction. Success rate, construction accuracy, and edit-propagation F1 score are computed against manually verified annotations and reference edits.

Figure 4: Interactive chart exploration. (a) Data extraction: Chart2SVG enables precise value recovery; baseline hallucinates ranges. (b) Semantic querying: correct mark highlighting vs. baseline misidentification. (c) Rescaling: CSG updates geometric paths; baseline only edits text labels.

Figure 5: Chart repurposing (a) and layout reuse (b). (a) CSG enables pie-to-pictorial-bar conversion; baseline hallucinates data (red). (b) CSG preserves x-axis alignment under new data; baseline fails.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Token	Description	Token	Description
1. Structural Containers & Anchors
[<\|START_OF_SVG\|>] / [<\|END_OF_SVG\|>]	SVG start/end	[<\|START_OF_GROUP\|>] / [<\|END_OF_GROUP\|>]	Group ( <g> ) start/end
[<\|clipPath\|>] / [<\|/clipPath\|>]	Clipping path start/end	[<\|defs\|>] / [<\|/defs\|>]	Reusable elements start/end
[<\|linearGradient\|>] / [<\|/linearGradient\|>]	Linear gradient start/end	[<\|radialGradient\|>] / [<\|/radialGradient\|>]	Radial gradient start/end
[<\|stop\|>] / [<\|/stop\|>]	Gradient stop start/end	[<\|filter\|>] / [<\|/filter\|>]	Filter element start/end
[<\|feOffset\|>] / [<\|/feOffset\|>]	feOffset start/end	[<\|feColorMatrix\|>] / [<\|/feColorMatrix\|>]	feColorMatrix start/end

Appendix

Table 6: Full Definition of the 115 SVG-Specific Tokens

Figure 6: \new Failure case of chart-to-code methods on a highly customized infographic. (Left) Original raster chart using rose-shaped glyphs to jointly encode GDP by size, DESI score by color, and country ranking by position. (Center) A chart-to-code approach attempts to reproduce the customized rose marks with Matplotlib; because the library lacks native support for such pictorial glyphs, the petals must be manually approximated using parameterized ellipses. (Right) The resulting rendering captures only the coarse layout and data encoding, while failing to preserve the distinctive rose-petal geometry and visual style of the original chart. This example illustrates the difficulty of chart-to-code methods in reconstructing highly customized pictorial marks with predefined plotting primitives.

Figure 7: \new Qualitative comparison of SVG code structure across different methods on a grouped bar chart. Chart2SVG produces a clear hierarchical structure and uniform color and coordinate representations, enabling direct programmatic editing. Adobe Illustrator Image Trace generates a flat list of anonymous <path> elements with opaque Bézier sequences and no semantic structure. GPT-4o produces partially readable code but lacks proper grouping and contains data overlaps. StarVector outputs path-based geometry with numerical redundancy and no semantic labels.

Vector Graphics Generation

See all 26 papers

REChart: Reasoning-Efficient Chart Editing with Large Reasoning Models
Aug 18, 2026Yuanbang Liu, Chenxi Ruan, Yihan Hou +2Efficient Language Model Reasoning Large Reasoning Models
HKUST(GZ)
Back2Struct: Making Structured Images Editable Again
Sep 29, 2026Pengyu Yan, Yixin Wu, Yunjie Tian +1Image-to-Code Generation Vector Graphics Generation
University at Buffalo, SUNY
ChartNet: A Million-Scale, High-Quality Multimodal Dataset for Robust Chart Understanding
Mar 28, 2026Jovana Kondic, Pengyuan Li, Dhiraj Joshi +24Chart Understanding Data Visualization
MIT · IBM Research · MIT-IBM Watson AI Lab +1

Aug 18, 2026J/K move · Enter open · S save

REChart: Reasoning-Efficient Chart Editing with Large Reasoning Models

Yuanbang Liu, Chenxi Ruan, Yihan Hou, Qiong Luo+1

HKUST(GZ)

Efficient Language Model Reasoning Large Reasoning Models RL for Language Model Reasoning

Open paper AskPDF arXiv

Chart editing requires inferring and modifying visualization code from a reference chart image based on an editing instruction, challenging fine-grained visual reasoning, instruction following, and executable code synthesis capabilities of MLLMs. Large reasoning models (LRMs) with extended Chain-of-Thought (CoT) reasoning are suitable for tackling such complex multimodal tasks. However, our preliminary study reveals an inverted-U'' relationship between reasoning length and chart-editing performance: Excessive reasoning often leads to overthinking,'' where models drift toward hallucinated visual details or get stuck in redundant reasoning loops. To address the gap, we introduce REChart, a two-stage training framework that provides process-level supervision over intermediate reasoning steps, improving both editing fidelity and reasoning efficiency. First, we synthesize 200k high-quality reasoning trajectories for supervised fine-tuning from a large image-instruction-code pool, using a role-specialized agentic Reason-Score-Refine workflow that iteratively refine the chart code toward higher quality. Second, we optimize the model via reinforcement learning with two complementary rewards: a fidelity reward evaluating code correctness, visual fidelity, and structural consistency, and an efficiency reward that assigns each rollout a random thinking budget, truncates the reasoning process, and credits the final reasoning segment according to its contribution to the output. On the ChartEdit and ChartMIMIC benchmarks, our model achieves state-of-the-art chart-editing performance among open-source models of comparable scale, while mitigating overthinking and reducing average reasoning token usage by 79.0% under a maximum thinking budget of 16,384 tokens compared with the base model.

Abstract

Vector Graphics Generation

Explore similar work

REChart: Reasoning-Efficient Chart Editing with Large Reasoning Models

Back2Struct: Making Structured Images Editable Again

ChartNet: A Million-Scale, High-Quality Multimodal Dataset for Robust Chart Understanding