SketchBloom: Training-Free Vector Sketch Outpainting with Stroke-Level Style Control
Authors: Quang-Thinh Nguyen, Duy-Khang Do, Tam V. Nguyen, Minh-Triet Tran, Trung-Nghia Le
Organizations: University of Science, Ho Chi Minh City, Vietnam · Vietnam National University, Ho Chi Minh City, Vietnam · University of Dayton, Ohio, United States
Outpainting sparse sketches while preserving contour coherence and stroke style remains challenging. We present SketchBloom, a training-free framework that combines pretrained raster outpainting and style-transfer guidance with per-instance vector optimization. Given a sketch, a scene prompt, and an optional reference style, it produces an editable Scalable Vector Graphic (SVG) composed of Bezier strokes. Adaptive latent blending balances content contours and reference texture without model retraining. A reference-derived stroke vocabulary initializes contour- and DINO-attention-guided strokes, which are optimized separately for structure and texture before contour-emphasizing multiply fusion. As in prior optimization-based vector sketching, no task-specific network is trained; our contribution is the synthesis design specialized to sketch outpainting. On LineArtOut-1K, comprising 1,024 input and style combinations, SketchBloom improves perceptual, semantic, and structural fidelity over the evaluated vector baselines when all methods vectorize the same outpainted raster. This protocol evaluates vector synthesis rather than end-to-end outpainting or reference-style superiority. Runtime is approximately six minutes per pair on a single NVIDIA H200 GPU. Project page: https://khangcntn123.github.io/SketchBloom/
Figures & tables
Figure 2: Raster guidance pipeline. ControlNet expands Isketch into Icnt , from which Iedge is extracted. DDIM inversion, adaptive latent blending, and style-sourced attention features combine content structure with Iref , producing the stylized target Isty .
Figure 3: Vector-native synthesis. A reference-derived stroke vocabulary initializes two passes: structure optimizes geometry against Iedge , whereas texture optimizes stroke attributes against Isty . Multiply blending places Vstruct over Vtex to produce the final SVG V∗ .
Figure 4: Qualitative comparison under the same raster input. SketchBloom (bottom row) better retains object boundaries, perspective, and scene detail.
Method
Aesthetic [ 22 ] ↑
DINO [ 5 ] ↑
DreamSim [ 9 ] ↓
Edge F1 ↑
CLIPasso [ 25 ]
4.1690
0.2064
0.7256
0.5262
CLIPascene [ 24 ]
4.3904
0.2022
0.7024
0.5797
SwiftSketch [ 2 ]
3.9784
0.1566
0.8305
0.3139
ControlSketch [ 2 ]
4.1493
0.1470
0.7960
0.3299
DiffSketcher [ 27 ] (adapted)
4.8020
0.3760
0.5351
0.8325
SketchBloom (Ours)
5.2186
0.5053
0.4983
0.8866
Table 1: Quantitative comparison on 64 shared outpainted content images. All methods receive the same Icnt . DINO and DreamSim use Icnt as reference, Edge F1 uses Iedge , and Aesthetic is no-reference. Results are computed per content image. Best values are in bold .
Init.
MAE ↓
SSIM [ 26 ] ↑
Chamfer [ 4 ] ↓
Edge F1 ↑
Random
0.4384
0.1120
0.0006
0.7731
Hybrid
0.2041
0.8068
0.0003
0.8841
Table 2: Structure-only initialization ablation against Iedge . Best values are in bold .
Variant
Gram [ 10 ] ↓
DINO [ 5 ] ↑
DreamSim [ 9 ] ↓
Edge F1 ↑
Aesthetic [ 22 ] ↑
Texture only
1.82×10−4
0.7328
0.1744
0.8067
5.0715
Structure only
2.76×10−4
0.5800
0.2918
0.8841
4.8718
Fusion
2.19×10−4
0.7372
0.1712
0.8629
5.3507
Table 3: Fusion ablation. Gram uses Iref , DINO and DreamSim use Isty , and Edge F1 uses Iedge . Best values are in bold .
Figure 5: Qualitative ablations. Top: hybrid tiling converges earlier than random initialization. Bottom left: patch loss preserves local detail without the gray halos of L2 . Bottom right: multiply fusion combines geometric structure with tonal texture.
Converting a raster image into a resolution-independent, stroke-editable vector sketch while faithfully preserving both structural contours and tonal texture remains challenging: raster-space rendering pipelines are difficult to edit at the stroke level, while generation-from-scratch vectorization methods struggle to remain faithful to an arbitrary input image. We present SketchForge, a training-free framework that converts an input raster image into a resolution-independent SVG sketch through per-instance test-time optimization of Bezier curves, requiring no additional training or fine-tuning. Strokes are seeded via a contour and attention-guided initialization, then refined by a vector-native differentiable rendering pipeline that decouples structural contour strokes from free-form texture strokes under dedicated objectives, composited via multiply-blend layering. Extensive experiments show that SketchForge consistently outperforms existing vector-based baselines in structural fidelity and generation quality. Project page: https://khangcntn123.github.io/SketchForge
Quang-Thinh Nguyen, Duy-Khang Do, Hien Trinh +3
University of Science, Ho Chi Minh, Vietnam · Vietnam National University, Ho Chi Minh, Vietnam · University of Dayton, Dayton, Ohio, US
Vector sketches remain one of the most concise and immediate mediums for abstract human expression. However, generating high-quality vector strokes that exhibit human-like drawing styles remains an open challenge due to the severe scarcity of fine-grained, high-quality text-to-sketch paired data. Existing text-conditioned generation methods often rely on unstable, time-consuming optimization or struggle to generalize to unseen categories in a zero-shot manner. To address these limitations, we present SketchFlow, a novel generative framework rooted in Optimal Transport (OT) theory and flow matching. By leveraging pre-trained CLIP models to bypass labor-intensive image-level text annotations, we formulate cross-modal alignment as a continuous mapping problem directly within the CLIP latent space. To bridge the inevitable modality gap between discrete text concepts and continuous sketch features, we first inject noise into discrete category embeddings to construct a continuous Gaussian Mixture Model (GMM) prior. We then utilize an Optimal Transport Conditional Flow Matching (OT-CFM) model to learn a deterministic vector field mapping from this continuous GMM prior to the target sketch feature distribution. Finally, a Hybrid Diffusion Decoder, fusing 1D U-Net and Transformer architectures, is designed to decode these features into fast and high-fidelity stroke trajectories. Extensive experiments demonstrate that SketchFlow substantially outperforms existing baselines in visual quality and adherence to natural human drawing styles. Furthermore, our geometry-preserving framework demonstrates promising local zero-shot synthesis for prompts beyond the QuickDraw training vocabulary, including unseen concept labels and semantic modifiers, while enabling smooth, continuous semantic interpolation between distinct concepts. Source code is available at: https://github.com/doudin404/SketchFlow.
Jin Zhou, Hongliang Yang, Pengfei Xu +1
Guangdong Provincial Key Laboratory of Visual Media and Multidimensional Intelligence, CSSE, Shenzhen University, Shenzhen, China
Sketch-guided image inpainting provides intuitive structural control, yet real sketches often mix reliable global intent with locally crowded, displaced, incomplete, or deliberately unconventional strokes. Existing approaches typically either retain the input sketch as a fixed condition throughout denoising or refine it into a clean structure before RGB synthesis. The former assumes uniformly reliable strokes and can propagate local errors throughout generation; the latter must resolve ambiguous structure before emerging appearance and semantic context become available. We propose SketchSense, a framework that interprets imperfect sketch guidance by synchronously denoising interacting RGB and structure streams. Bidirectional Attention Fusion couples appearance generation with structural recovery, producing a refined structure that exposes the model's evolving sketch interpretation. A phrase-level objective aligns the semantic grounding of the two streams. Sketch-Aware Spatial Regulation further adapts sketch use to local generation states by modulating attention and the fusion process, while an optional signed prior injects preserve-versus-correct intent into feature representations and attention behavior. Experiments on natural and structurally complex imagery show substantial gains over existing methods in both restoration quality and structural fidelity.