DesigNet: Learning to Draw Vector Graphics as Designers Do
Authors: Tomas Guija-Valiente, Iago Suárez
Organizations: Machine Learning Circle, Spain · Universidad Politécnica de Madrid, Departamento de Inteligencia Artificial, Spain · Qualcomm XR Labs, Spain
AI-driven content generation has made remarkable progress in recent years. However, neural networks and human designers operate in fundamentally different ways, making collaboration between them challenging. We address this gap for Scalable Vector Graphics (SVG) by equipping neural networks with tools commonly used by designers, such as axis alignment and explicit continuity control at command junctions. We introduce DesigNet, a hierarchical Transformer-VAE that operates directly on SVG sequences with a continuous command parameterization. Our main contributions are two differentiable modules: a continuity self-refinement module that predicts C0, G1, and C1 continuity for each curve point and enforces it by modifying Bézier control points, and an alignment self-refinement module with snapping capabilities for horizontal or vertical lines. DesigNet produces editable outlines and achieves competitive results against state-of-the-art methods, with notably higher accuracy in continuity and alignment. These properties ensure the outputs are easier to refine and integrate into professional design workflows. Source Code: https://github.com/TomasGuija/DesigNet.
Figures & tables
Figure 1: Overview of DesigNet . A subset of characters ("H", "a", "m", "b", "u", "r", …) from a font is encoded to extract style features. The decoder then generates the remaining glyphs by combining the learned style with the embedding of the target letters. Finally, our self-refinement modules adjust control points and endpoints to enhance continuity and axis alignment, yielding cleaner SVG outputs.
Figure 2: Illustration of different continuity types . C0 : only geometric continuity, G1 : collinear tangents, and C1 : collinear tangents with equal magnitude.
Figure 3: Examples of Continuity and Alignment Modules . We show the predicted glyphs before and after the self-refinement modules. LineFromTo commands are shown in blue and CurveFromTo commands in green. The predicted continuity is represented by the pink squares, circles, and diamonds. We highlight with orange arrows the junctions corrected by the Self-refinement modules based on the pink predictions.
Figure 4: Qualitative comparison of reconstructed words using Latin fonts . The first row presents the ground truth (GT) glyphs, including their joints and control points. The second and third rows compare the outputs of DeepSVG and DesigNet against the GT, where black indicates overlapping regions, green denotes GT regions not covered by the prediction, and red marks predicted regions that do not correspond to the GT.
Figure 5: Qualitative comparison of reconstructed words using Chinese fonts .
Model
IoU ↑
L1 ↓
RE ↓
Cont. Acc. ↑
Align. Acc. ↑
DeepSVG
0.789
0.069
8.782
0.567
0.603
+ cont. args., +sin. pos. enc., +centered glyphs
0.943
0.016
2.242
0.567
0.368
+ hierarchical latent space
0.963
0.010
1.477
0.651
0.380
+ relaxed rep. and aux. loss
0.970
0.009
1.137
0.686
0.376
+ self-refinement 75% Conf. Trh.
0.969
0.009
1.138
0.886
0.969
Table 1: Ablation study of our VAE model on the test split of our proprietary dataset. Using continuous drawing parameters instead of the quantized representation in DeepSVG leads to a substantial improvement in reconstruction quality. Subsequent architectural enhancements further increase reconstruction fidelity, while the continuity and alignment self-refinement modules markedly improve continuity and alignment accuracy.
Figure 6: Font Interpolation : Examples of latent space interpolation in DesigNet between two fonts with different weights and slants.
Figure 7: Qualitative comparison of icon reconstructions : Each pair shows ground truth (GT) and our generated glyph.
Model
IoU ↑
L1 ↓
RE ↓
Cont. Acc. ↑
Align. Acc. ↑
DualVector [ 17 ]
0.564
0.137
-
-
-
DeepVecFont-v2 [ 31 ]
0.681
0.117
13.124
0.444
0.391
DeepVecFont-v2 + Self-Ref.
0.675
0.120
13.269
0.528
0.391
DesigNet w/o Self-Ref.
0.711
0.106
12.665
0.276
0.282
DesigNet
0.693
0.115
13.126
0.482
0.531
DeepVecFont-v2 10 shots
0.735
0.091
11.848
0.452
0.383
Table 2: Comparison on our proprietary dataset for Latin typeface generation (52 letters) on the cross-reconstruction task.
Figure 8: Reconstruction quality across encoding (left) and decoding (right) glyph sets. From top to bottom: ground truth (GT), DeepVecFont-v2, and our method after refinement.
Model
IoU ↑
L1 ↓
RE ↓
Cont. Acc. ↑
Align. Acc. ↑
DeepVecFont-v2 [ 31 ]
0.380
0.216
21.414
0.499
0.198
DeepVecFont-v2 + Self-Ref.
0.378
0.218
21.423
0.501
0.205
DesigNet w/o Self-Ref.
0.397
0.219
19.604
0.501
0.247
DesigNet
0.391
0.221
19.651
0.512
0.351
DeepVecFont-v2 10 shots
0.417
0.199
20.211
0.498
0.202
Table 3: Comparison on the Chinese font dataset [ 31 ] . Metrics are computed on the cross-reconstruction task.
Recent large language models have shifted SVG generation from differentiable rendering optimization to autoregressive program synthesis. However, existing approaches still rely on generic byte-level tokenization inherited from natural language processing, which poorly reflects the geometric structure of vector graphics. Numerical coordinates are fragmented into discrete symbols, destroying spatial relationships and introducing severe token redundancy, often leading to coordinate hallucination and inefficient long-sequence generation. To address these challenges, we propose HiVG, a hierarchical SVG tokenization framework tailored for autoregressive vector graphics generation. HiVG decomposes raw SVG strings into structured atomic tokens and further compresses executable command--parameter groups into geometry-constrained segment tokens, substantially improving sequence efficiency while preserving syntactic validity. To further mitigate spatial mismatch, we introduce a Hierarchical Mean--Noise (HMN) initialization strategy that injects numerical ordering signals and semantic priors into new token embeddings. Combined with a curriculum training paradigm that progressively increases program complexity, HiVG enables more stable learning of executable SVG programs. Extensive experiments on both text-to-SVG and image-to-SVG tasks demonstrate improved generation fidelity, spatial consistency, and sequence efficiency compared with conventional tokenization schemes. Our code is publicly available at https://github.com/ximinng/HiVG
Scalable Vector Graphics (SVG) are essential for modern industrial Computer-Aided Design (CAD). However, existing autoregressive SVG generation models are predominantly tailored for artistic creation and struggle to maintain the rigorous geometric fidelity and cross-view spatial alignment required for engineering drawings. To bridge this gap, we introduce DrawingsDreamer, a unified Large Language Model (LLM)-driven framework for multi-view vector-based engineering drawings generation. By formulating the generation of multi-view engineering drawings purely as a sequence modeling task, we eliminate the need of raster image encoders. We propose a Streamlined Representation utilizing hierarchical postfix tokenization, which guides the model to establish local geometric coordinates before assigning semantic boundaries. Optimized via a progressive task-aware curriculum schedule, DrawingsDreamer effectively transitions from localized structural repair to macroscopic generation in a unified model. Extensive experiments demonstrate that our unified model achieves strong performance in both geometric fidelity and syntactic accuracy across diverse conditional and unconditional generation tasks.
Shurui Liu, Weide Chen, Changwang Yi +1
School of Computer Science and Engineering, Sun Yat-sen University, China · School of Intelligent Systems Engineering, Shenzhen Campus of Sun Yat-sen University, China
Scalable Vector Graphics (SVGs) power much of the modern visual ecosystem, yet state-of-the-art generative models focus almost entirely on rasterized images. We explore whether inference-time methods can unlock SVG generation capabilities in off-the-shelf vision-language models (VLMs). We systematically evaluate a constrained iterative refinement harness that combines visual feedback, structured editing, and constrained decoding to characterize the capabilities and limitations of current VLMs for SVG generation. Across multiple VLMs and generation settings, we find that constrained decoding improves compilation success rates, while iterative refinement reveals a deficit in visual reasoning and self-correction. Our results highlight both the promise and current limitations of using inference-time methods to adapt general-purpose VLMs for SVG generation.
Matthew Perlman, James Beetham, Niels Da Vitoria Lobo +2
University of Massachusetts Amherst · University of Central Florida