cs.CVApr 3, 2026
SaveDifferentiable Stroke Planning with Dual Parameterization for Efficient and High-Fidelity Painting Creation
Organizations: Shanghai Jiao Tong University
Abstract
In stroke-based rendering, search methods often get trapped in local minima due to discrete stroke placement, while differentiable optimizers lack structural awareness and produce unstructured layouts. To bridge this gap, we propose a dual representation that couples discrete polylines with continuous Bézier control points via a bidirectional mapping mechanism. This enables collaborative optimization: local gradients refine global stroke structures, while content-aware stroke proposals help escape poor local optima. Our representation further supports Gaussian-splatting-inspired initialization, enabling highly parallel stroke optimization across the image. Experiments show that our approach reduces the number of strokes by 30-50%, achieves more structurally coherent layouts, and improves reconstruction quality, while cutting optimization time by 30-40% compared to existing differentiable vectorization methods.
Figures & tables
Figure 1: Comparison with existing stroke-based rendering approaches. (a) Search-based methods suffer from structural fragmentation and require numerous strokes, leading to slow and time-consuming optimization. (b) Differentiable optimization methods converge faster but often produce visually disorganized and unstructured layouts due to the lack of explicit structural priors. (c) Our method combines discrete structure-aware search with differentiable refinement, producing coherent, visually harmonious, and high-fidelity paintings with significantly fewer strokes.
Figure 2: Framework. Given a target image , our method iteratively reconstructs it through a dual-stage search-and-optimize pipeline. At iteration , the structure-aware search step samples seeds from high-residual regions using NMS and traces geometry-aligned polylines along image gradients. These polylines are then mapped, via a bidirectional differentiable dual representation, to Bézier curves and refined in the optimization step using gradients from our differentiable polyline splatting renderer. All strokes are splatted in parallel to form the canvas, and the process rapidly converges, with most structure recovered in the first iteration.
Figure 3: Qualitative comparison with state-of-the-art methods. Our method produces clearer contours, cleaner textures, and more coherent stroke flows across both natural and synthetic domains. In contrast, baseline approaches often generate noisy strokes or lose structural consistency, particularly under limited stroke budgets.
| Method | PSNR↑ | SSIM↑ | LPIPS↓ | Time (s)↓ |
|---|---|---|---|---|
| DIV2K Validation | ||||
| Im2Oil [ 28 ] | 27.59 | 0.72 | 0.211 | 727.8 |
| CNP [ 12 ] | 27.91 | 0.64 | 0.296 | 125.5 |
| Paint Transformer [ 21 ] | 23.83 | 0.63 | 0.378 | 414.1 |
| Learning to Paint [ 14 ] | 27.19 | 0.70 | 0.330 | 167.9 |
| SNP [ 32 ] | 20.63 | 0.42 | 0.405 | 5837.2 |
Table 1: Quantitative comparison with SOTA SBR methods.
| Method | Structure↑ | Texture↑ | Color↑ | Overall↑ |
|---|---|---|---|---|
| Im2Oil [ 28 ] | 3.42 | 3.28 | 3.55 | 3.38 |
| CNP [ 12 ] | 3.65 | 3.12 | 3.08 | 3.31 |
| Paint Transformer [ 21 ] | 2.98 | 3.05 | 2.87 | 2.97 |
| Learning to Paint [ 14 ] | 3.51 | 3.42 | 3.47 | 3.47 |
| SNP [ 32 ] | 2.45 | 2.38 | 2.31 | 2.41 |
| Ours | 4.32 | 4.28 | 4.41 | 4.35 |
Table 2: User study results (1–5 Likert ratings; higher is better). Our method achieves the best rank across all perceptual criteria.
Figure 4: Progressive painting process with height-field relighting.
| Iter. | Stage | PSNR↑ | SSIM↑ | LPIPS↓ | Time (s) |
|---|---|---|---|---|---|
| Search | 20.3 | 0.48 | 0.323 | 35.6 | |
| Optimize | 28.4 | 0.82 | 0.214 | 47.2 | |
| Search | 29.6 | 0.83 | 0.215 | 59.0 | |
| Optimize | 30.4 | 0.85 | 0.194 | 65.5 | |
| Search | 29.9 | 0.84 | 0.197 | 75.6 | |
| Optimize | 32.2 | 0.93 | 0.076 | 87.6 |
Table 3: Iteration-by-iteration analysis.
Figure 5: Qualitative visualization of intermediate canvases at iterations . Iteration 1 establishes coarse structural coverage via search, while later iterations progressively refine geometric precision, stroke smoothness, and color consistency.
| Variant | PSNR↑ | SSIM↑ | LPIPS↓ | Time (s)↓ |
|---|---|---|---|---|
| Search-only | 27.8 | 0.76 | 0.227 | 57.2 |
| Polyline-only optimization | 31.2 | 0.89 | 0.126 | 31.6 |
| Bézier-only optimization | 30.4 | 0.86 | 0.138 | 3978.4 |
| Full model (ours) | 32.16 | 0.93 | 0.076 | 87.6 |
Table 4: Ablation on the dual stroke representation.
| Variant | PSNR | SSIM | LPIPS |
|---|---|---|---|
| Search components | |||
| w/o residual seeding | 26.6 | 0.68 | 0.263 |
| w/o gradient tracing | 26.3 | 0.71 | 0.249 |
| Full search | 27.8 | 0.76 | 0.227 |
| Optimization components | |||
| w/o opacity reinit | 30.3 | 0.85 | 0.158 |
Table 5: Ablation study on search and optimization components.
Figure 6: Overview of our height field construction pipeline. We fuse a depth-based height field, representing global scene structure, with a texture-based height field, capturing local surface details, to generate the final physically-plausible height field .
Figure 7: Relighting results using our height field representation. Our rendered result with canvas texture and impasto effects under directional lighting. The height field captures both macroscopic paint accumulation and microscopic canvas weave.
Figure 8: Trajectory comparison. Direct optimization (orange) overfits to noise. Bézier proxy (magenta) projects the solution onto the cubic polynomial subspace.
Figure 9: Gradient Spectrum. Spectral analysis confirms that the proxy operator nullifies gradient components associated with high-frequency spatial variation.