Vector sketches remain one of the most concise and immediate mediums for abstract human expression. However, generating high-quality vector strokes that exhibit human-like drawing styles remains an open challenge due to the severe scarcity of fine-grained, high-quality text-to-sketch paired data. Existing text-conditioned generation methods often rely on unstable, time-consuming optimization or struggle to generalize to unseen categories in a zero-shot manner. To address these limitations, we present SketchFlow, a novel generative framework rooted in Optimal Transport (OT) theory and flow matching. By leveraging pre-trained CLIP models to bypass labor-intensive image-level text annotations, we formulate cross-modal alignment as a continuous mapping problem directly within the CLIP latent space. To bridge the inevitable modality gap between discrete text concepts and continuous sketch features, we first inject noise into discrete category embeddings to construct a continuous Gaussian Mixture Model (GMM) prior. We then utilize an Optimal Transport Conditional Flow Matching (OT-CFM) model to learn a deterministic vector field mapping from this continuous GMM prior to the target sketch feature distribution. Finally, a Hybrid Diffusion Decoder, fusing 1D U-Net and Transformer architectures, is designed to decode these features into fast and high-fidelity stroke trajectories. Extensive experiments demonstrate that SketchFlow substantially outperforms existing baselines in visual quality and adherence to natural human drawing styles. Furthermore, our geometry-preserving framework demonstrates promising local zero-shot synthesis for prompts beyond the QuickDraw training vocabulary, including unseen concept labels and semantic modifiers, while enabling smooth, continuous semantic interpolation between distinct concepts. Source code is available at: https://github.com/doudin404/SketchFlow.
Figures & tables
Figure 1. Local zero-shot vector sketch generation by SketchFlow. SketchFlow leverages drawing priors from the CLIP latent space to synthesize prompts beyond its 345-class QuickDraw training vocabulary. Left: Text-prompted generation of letters, numbers, and landmarks in Malaysia. Right: Diverse zero-shot generation including pop-culture characters, creatures, abstract emotions, and dynamic poses. SketchFlow produces visually appealing sketches with human-like drawing behaviors without category-specific training examples for these prompts. A grid of computer-generated sketches showing a composite scene on the left and various characters like Kirby and Mickey on the right.
Figure 2. Overview of the SketchFlow generative framework. (a) Initial Discrete Embeddings: The framework extracts discrete text embeddings ztextk and sketch embeddings for known categories in the latent space. (b) Vector Field Construction: A noise injection mechanism is introduced to transform sparse concept anchors into a continuous probability density field, establishing a Gaussian Mixture Model (GMM) prior (Sec. 3.2 & 3.4). (c) Learned Vector Field vθ : Optimal Transport Conditional Flow Matching establishes a deterministic mapping between the GMM prior p0(z) and the target sketch distribution, naturally yielding straight-path trajectories (Sec. 3.3). (d) Generalizing to an Unseen Query: During inference, the CLIP embedding of an unseen query (e.g., "kirby") follows the learned transport field, informed by known training concepts, toward the rendered-sketch CLIP feature distribution. (e) Decode Sketch: Conditioned on the acquired target sketch embedding, a Hybrid Diffusion Decoder utilizes a reverse diffusion process to decode pure noise XT into a final high-fidelity vector stroke sequence X0 (Sec. 3.1). Five-stage pipeline diagram. Discrete CLIP text anchors and rendered-sketch embeddings form colored clusters; noise expands the text anchors into a GMM prior, and OT flow matching learns non-crossing paths to the sketch distribution. At inference, a Kirby query follows a nearby transport path to a target embedding that conditions the diffusion decoder, which converts noise into a vector sketch.
Figure 3. Architecture of the Hybrid Diffusion Decoder. The model employs an interleaved generative backbone combining U-Net-style bottlenecks for local geometric extraction with Transformer blocks for global dependency modeling. Condition embeddings (diffusion timestep t and target CLIP embedding) are injected via AdaLN to predict the added noise. Architecture diagram showing an ordered cat trajectory represented by coordinate and pen-state triples, its noisy version, and its rendering. Sequence tokens enter a multiscale encoder-decoder with interleaved Transformer blocks and skip connections; timestep and CLIP conditions are injected throughout, and the network predicts trajectory noise.
Figure 4. Qualitative comparison on QuickDraw. Generated samples for four categories ( Bus , Cat , Chair , and Train ) from baseline methods and ours. A five-row comparison grid for SketchRNN, StrokeFusion, SketchAgent, NeuralSVG, and SketchFlow. Each row contains two generated examples of buses, cats, chairs, and trains; SketchFlow outputs use concise object-level strokes, while several baselines show missing parts, rigid geometry, or dense overlapping curves.
Figure 5. Zero-shot generation beyond the training vocabulary. All prompts shown here lie outside the QuickDraw taxonomy, yet the model produces coherent and style-consistent vector sketches. Three-row gallery of blue vector sketches labeled with prompts outside the training vocabulary, including fire, heart, sun and cloud, pizza clock, twins, tie, rocket, Earth, moon and star, potion, Kinabalu, Pikachu, Tetromino, two eyes, Goblin, ship, slime, Sonic, Mario, water, and the letter A.
Bus
Cat
Chair
Train
Method
FID ↓
CLIP ↑
RDP ↓
FID ↓
CLIP ↑
RDP ↓
FID ↓
CLIP ↑
RDP ↓
FID ↓
CLIP ↑
RDP ↓
SketchRNN
66.103
0.266
79.571
70.704
0.210
68.591
88.294
0.227
25.351
83.578
0.236
78.830
StrokeFusion
59.329
0.278
86.690
52.940
0.260
85.730
33.463
0.277
36.460
59.418
0.247
102.910
SketchAgent
–
0.258
124.429
–
0.214
93.684
–
0.267
40.200
–
0.234
98.400
NeuralSVG
–
0.224
278.446
–
0.261
267.091
–
0.304
225.663
–
0.225
256.168
Gaussian-Prior
9.556
0.265
98.53
24.568
0.148
97.15
17.572
0.165
38.45
15.180
0.183
99.06
Table 1. Quantitative results on QuickDraw Bus , Cat , Chair , and Train . Bold and underlined values indicate the best and second-best performances, respectively.
Method
Mean CLIP Score ↑
Gaussian Prior
0.2024
Interp CLIP
0.2034
SketchFlow (Ours)
0.2480
Table 2. Mean CLIP Score across 44 prompts absent from the QuickDraw training vocabulary. Each value averages 32 samples per prompt without best-of- N selection ( γ=1.0 ).
Metric
SketchFlow
NeuralSVG
SketchAgent
Abstraction Fidelity
72.85%
12.52%
14.63%
Semantic Alignment
56.75%
32.52%
10.73%
Human-likeness
70.73%
16.59%
12.68%
Stroke Rationality
61.79%
26.18%
12.03%
Aesthetic Quality
51.71%
37.24%
11.05%
Table 3. User study results comparing SketchFlow with NeuralSVG and SketchAgent across five metrics. The scores indicate the average percentage of user preference for each method.
Figure 6. Zero-Shot Generation with Semantic Modifiers. Generation results for specific action prompts. Three distinct results are shown for each prompt. Nine blue vector cat sketches arranged in three groups. Running cats have extended legs and horizontal motion, jumping cats have lifted or arched bodies, and stretching cats have elongated bodies and tails.
Figure 7. Ablation of Flow Mapping. Results of linear interpolating the condition from raw text embeddings ( λ=0.0 ) to Flow-mapped embeddings ( λ=1.0 ) at intervals of 0.2. The raw text embedding alone is insufficient to guide structured sketch synthesis. Two rows show cat and spiral generations at interpolation values from 0.0 to 1.0. At low values the strokes are fragmented and disorganized; as the flow-mapped condition increases, they progressively form a recognizable cat face and a clean spiral.
Figure 8. Ablation of Gaussian-Prior baseline Latent interpolation results produced by the baseline. Unlike our approach, this baseline struggles to maintain structural coherence during the transition. The intermediate sketches exhibit abrupt topological shifts and a loss of visual semantics, highlighting the necessity of our GMM-based OT-Flow Matching design for establishing a continuous generative space. Two interpolation rows produced by the Gaussian-prior baseline. The apple-to-leaf row passes through irregular mixed objects before reaching a leaf, and the moon-to-sun row contains abrupt shape changes and unrelated intermediate structures.
Figure 9. Representative failure cases. (a) The outputs remain recognizable as the prompted concepts but contain incomplete structures. (b) The generated trajectories form text-like strokes resembling the prompt words instead of the intended objects. Four generated sketches showing incomplete structures for a brachiosaurus and a seahorse, followed by text-like outputs for the prompts ratchet and spinach.
Figure 10. Semantic Interpolation in CLIP Space. Each row visualizes a linear interpolation path between the discrete text embeddings of two distinct concepts (e.g., smoothly morphing from an “Apple” to a “Leaf”, or from a “Palm” to a “Pine”). The intermediate generations maintain exceptionally uniform structural transitions and logical stroke topologies without disjointed visual jumps. This progressive morphing provides compelling evidence that our model successfully establishes a continuous generative manifold, rather than stochastically snapping to isolated known modes. Nine-stage interpolation rows transform angel to bat, apple to leaf, crown to castle, happy face to sad face, moon to sun, palm to pine, pineapple to fire, torch to water, tree to cloud, black to white, and rain to tornado. Adjacent sketches change gradually in outline and internal strokes.
Figure 11. Zero-shot Image-Conditioned Vector Sketch Synthesis. As an intriguing extension, our model can seamlessly accept CLIP visual embeddings in place of textual prompts. This substitution enables a zero-shot stylized redrawing effect, where the generated vector sketches effectively capture the core semantic features and structural characteristics of the reference images. Reference photographs are paired with four generated vector sketches each. The examples cover buses, foods, portraits, flowers, old and modern cars, dogs, cats, drink containers, laboratory glassware, and birds; the outputs preserve distinguishing silhouettes and parts while simplifying appearance into blue strokes.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Method
Creative Birds
Creative Creatures
Creative Mix
Doodleformer
58.118
63.212
58.219
Ours
34.444
32.938
32.181
Appendix
Table 4. Quantitative comparison of FID on Creative datasets. Lower scores indicate better visual fidelity and structural similarity to the target distribution.
Figure 12. Qualitative results on the TU-Berlin dataset. Our model effectively learns the dataset’s unique freehand style and produces coherent sketches, even for categories absent from the original dataset. Dataset references and three SketchFlow samples are shown for chandelier, hamburger, piano, revolver, windmill, and person dancing. For person dancing, the dataset reference is marked unavailable while the generated figures show several dynamic poses.
Figure 13. Qualitative results on the Creative-Mix dataset. Our method successfully adapts to the highly creative style while maintaining strict semantic control, visibly outperforming Doodleformer in overall structure and clarity. Six creative bird prompts are compared between a Doodleformer reference and three SketchFlow samples: big head, big beak, flying high, walking forward, just chilling, and stay focused. Generated birds vary in pose while retaining the requested exaggerated trait or action.
Bus
Cat
Chair
Train
Method
CLIP ↑
RDP ↓
CLIP ↑
RDP ↓
CLIP ↑
RDP ↓
CLIP ↑
RDP ↓
NeuralSVG-4
0.208
36.891
0.212
36.374
0.247
29.980
0.213
33.941
NeuralSVG-8
0.215
72.218
0.228
70.475
0.271
58.554
0.215
66.099
NeuralSVG-16
0.219
142.376
0.248
137.232
0.291
115.030
0.221
129.941
NeuralSVG-32
0.224
278.446
0.261
267.091
0.304
225.663
0.225
256.168
Ours ( γ =0.60)
0.283
86.310
0.261
83.720
0.290
21.240
0.259
87.140
Appendix
Table 5. Additional quantitative comparisons on the QuickDraw dataset ( Bus , Cat , Chair , and Train ), specifically evaluating NeuralSVG with varying stroke counts. Bold and underlined values indicate the best and second-best performance, respectively.
Figure 14. Drawing Order Visualization. To clearly illustrate the sequential nature of our generated vector sketches, stroke trajectories are color-coded according to their generation sequence, progressing from light blue (initial strokes) to dark blue (final strokes). Note that all generated sketches presented in this paper adopt this color-coding scheme to indicate the underlying drawing progression. Five rows show six cumulative drawing stages for Malaysian landmark sketches. Each sequence begins with a few light-blue structural strokes and gains darker later strokes until the mosque, towers, and related building silhouettes are complete.
Prompt and nearest anchor
Mean CLIP Score
Prompt and nearest anchor
Mean CLIP Score
Prompt
Anchor
Cos.
SketchFlow
G. Prior
Interp.
Prompt
Anchor
Cos.
SketchFlow
G. Prior
Interp.
Tetromino
square
0.8021
0.2740
0.1997
0.2242
Tie
toe
0.8861
0.2339
0.1936
0.1875
Pikachu
banana
0.8369
0.2354
0.1800
0.1840
Box
book
0.8943
0.2770
0.2176
0.2139
Dancing-human
hand
0.8451
0.2680
0.1870
0.1959
a stretching cat
cat
0.8971
0.2529
0.2248
0.2274
Slime
brain
0.8534
0.2310
0.1993
0.1953
Ghost
saw
0.8993
0.2449
0.1909
0.1892
Goblin
brain
0.8537
0.1961
0.1770
0.1778
a jumping cat
cat
0.9035
0.2632
0.2234
0.2423
Appendix
Table 6. Complete unseen-label quantitative evaluation. CLIP Score is used as a label-alignment proxy and averaged over 32 samples per prompt without sample selection. The nearest anchor and cosine similarity are computed in CLIP text space over the 345 QuickDraw training labels. Prompts are arranged in two side-by-side blocks for compact presentation. Bold indicates the best result among the three evaluated methods.
Figure 15. Comparison with the two-stage SDXL+CLIPasso baseline. The SketchFlow row reuses samples already presented in the paper; light-to-dark blue indicates drawing order. The bottom row shows the intermediate SDXL targets supplied to CLIPasso. SketchFlow prioritizes abstract, concept-defining strokes over perspective and volumetric realism, whereas T2I+CLIPasso often preserves more plausible 3D structure from its raster target but may lose simple visual cues essential to conveying the concept. Qualitative comparison on Pikachu, Tetromino, Goblin, Rocket, Potion, and Ghost. Each column shows a SketchFlow result, the corresponding CLIPasso result, and the intermediate SDXL target image.
Figure 16. Extended zero-shot sketch generation beyond the training vocabulary. We show clear, visually appealing sketches for multiple prompts, with samples demonstrating generation diversity and a recurring hand-drawn style across random seeds. Two-column gallery with six generated samples for each labeled prompt. The left column includes fire, heart, sun and cloud, pizza clock, egg, twins, tie, happy face, rocket, box, and lightning cloud; the right includes Earth, moon and star, potion, spiral, Star of David, Pikachu, Tetromino, Goblin, water, slime, and smiley sunflower.
We invert the typical formulation of sketch generation: instead of drawing strokes in order, we predict a 2D field that defines the order in which strokes are drawn. We use a pretrained latent flow-matching transformer to supply the image prior to predict an intermediate representation, while training the VAE's decoder to predict the order field, stroke mask, and stroke segmentation. We vectorize the predicted segmentation into polylines and sort them by the field, producing an ordered vector sketch. Our model can predict an ordered vector sketch from a text description or derender an image into ordered vectors; for either, it follows text instructions specifying the order of drawing.
Sketching is inherently sequential: strokes are drawn progressively to explore and refine ideas. Yet most generative approaches treat sketches as static images, ignoring the temporal process underlying creative exploration. Modeling this sequential structure remains challenging: prior methods either rely on large-scale human-drawn datasets with limited diversity, or use large language models (LLMs) to produce drawing instructions, often at the cost of visual fidelity. We present VideoSketcher, a method for generating high-quality sketching processes by adapting pretrained text-to-video diffusion models to the sparse, continuous nature of sketch formation. Our key insight is that LLMs and video diffusion models offer complementary strengths: LLMs act as semantic planners that decompose concepts into step-by-step instructions, while video diffusion models serve as powerful "renderers" that translate them into temporally coherent sketch sequences. We introduce a two-stage fine-tuning strategy that decouples temporal structure from visual appearance: stroke ordering is learned from synthetic shape compositions, while style is distilled from as few as seven hand-drawn examples. Despite minimal supervision, our method can generate diverse, high-quality sequential sketches that faithfully follow specified drawing orders. Our framework naturally extends to brush style control and autoregressive generation, supporting artistic applications.
Existing vector-sketch models treat recognition and generation as separate tasks, leaving a gap for streaming interfaces that must understand a drawing as it is being made. We present SketchMamba, a single causal sequence model that continuously classifies a sketch from any partial prefix while simultaneously generating its continuation. We achieve this by applying a dense per-step classification loss to a selective state-space backbone. Evaluated on a 58-class subset of the Quick, Draw! dataset, SketchMamba yields 94.93% final-step accuracy and a progressive-accuracy Area Under the Curve (AUC) of 0.706, crossing 90% of its final accuracy by the time 70% of the strokes are drawn. In a matched-budget comparison, the 1.55 million-parameter backbone ties a causal Transformer while outperforming recurrent and convolutional baselines. Ablations confirm that the dense supervision regime, rather than the architecture alone, drives the early-prediction capability. The results demonstrate that a single causal hidden state can unify progressive recognition and autoregressive generation without auxiliary encoders or task-specific branching.
Kavish Jhaveri, Arya Shah
The National High School, Ahmedabad, India · Indian Institute of Technology, Gandhinagar, India