DiffBMP: Differentiable Rendering with Bitmap Primitives

Seongmin Hong; Junghun James Kim; Daehyeop Kim; Insoo Chung; Se Young Chun

Figure 2: An illustration of the algorithm flow of DiffBMP. Our core contribution is the renderer for bitmap images, introduced in Sec. 3.1 , which enables scalable and efficient differentiable rendering using a custom CUDA kernel. Additionally, we employ soft rasterization ( Sec. 3.2.1 ) for improved gradient propagation and an initialization technique ( Sec. 3.2.2 ) for better target approximation. After optimization, a native, layered file is generated, which enables diverse post-processing creation.

2D Differentiable rendering method	Primitive type	Applications	Opacity Support	Para- llelism	Engine / Interface
Differentiable Composition by Reddy et al. [44]	Bitmap	Specialized for patterns e.g . , manipulation, tiling	✗	✗	Pytorch / Python
DiffVG [ 27 ] and its extensions [ 43 , 33 , 4 , 5 , 60 , 59 ]	Vector	Very diverse e.g . , vectorization, painterly, etc.	✓	✓	C++ / Python
DiffBMP (Ours)	Bitmap	Very diverse. e.g . , painterly, assemblage, video, etc.	✓	✓	CUDA / Python

Table 1: Comparison with Prior Work. Prior work can be broadly classified into two main categories. DiffVG [ 27 ] is high-performing and broadly applicable, but it is limited to vector primitives. Due to the difficulty of matching DiffVG’s performance using bitmap primitives, only a few studies, such as Reddy et al. [44] , have attempted implementations for specialized tasks, but their efficiency and performance fall short of DiffVG. To put simply, DiffBMP aims to be the bitmap primitive counterpart to DiffVG.

Figure 3: Blur enriches the spatial gradients of primitives. The figure displays the per-pixel gradients with respect to

x_{i}

and

y_{i}

for three different primitives (a), extracted during the experiments of Tab. 3 , both (b) with and (c) without applying blur ( Sec. 3.2.1 ). Color indicates the direction of the gradient, and opacity represents its magnitude on a log scale (from

10^{-6}

to

10^{-3}

). Applying blur results in richer and more coherent gradients. This, in turn, leads to the better optimization results shown in Tab. 3 .

Figure 4: Heuristics for Dynamic DiffBMP. Sequential optimization ( i.e . , initializing

\Theta^{f}

with

\Theta^{f-1\star}

) can lead to local minima, as in b , where a large primitive is stuck on the face. c shows our remedy: reducing the opacity of such overly dominant primitives, resulting in PSNR gain.

Image res / Tile size		512 $\times$ 512 / 16 $\times$ 16	1024 $\times$ 1024 / 16 $\times$ 16	1024 $\times$ 1024 / 32 $\times$ 32	2048 $\times$ 2048 / 32 $\times$ 32
PyTorch	RTX 3090	1360ms / 2337ms / 6.4 GB	5514ms / 9811ms / 12.0 GB	1393ms / 2477ms / 5.0 GB	5405ms / 9483ms / 9.0 GB
PyTorch	L40S	1342ms / 2413ms / 6.6 GB	4570ms / 8151ms / 12.2 GB	1423ms / 2507ms / 5.2 GB	4914ms / 8707ms / 9.2 GB
CUDA-32bit	RTX 3090	3.9ms / 11.6ms / 1.0 GB	7.2ms / 9.7ms / 2.1 GB	7.6ms / 9.3ms / 2.0 GB	16.1ms / 10.0ms / 6.1 GB
CUDA-32bit	L40S	4.7ms / 2.9ms / 1.2 GB	10.5ms / 2.8ms / 2.3 GB	8.5ms / 2.9ms / 2.2 GB	23.3ms / 3.8ms / 6.3 GB
CUDA-16bit	RTX 3090	2.3ms / 6.2ms / 1.1 GB	4.2ms / 5.8ms / 1.6 GB	4.3ms / 5.5ms / 1.6 GB	9.0ms / 6.4ms / 3.8 GB
CUDA-16bit	L40S	2.0ms / 2.1 ms / 1.2 GB	5.4ms / 2.3ms / 1.8 GB	4.5ms / 2.3ms / 1.8 GB	12.8ms / 3.0ms / 4.0 GB

Table 2: Per-iteration performance across implementations and resolutions. Each cell reports a triple “forward / backward / VRAM” in the order of forward runtime, backward runtime, and total VRAM usage per iteration. Rows list the compared implementations ( e.g . , PyTorch baseline and our custom CUDA kernels in FP32/FP16), and columns specify the target image resolution and tile size.

Figure 5: Our DiffBMP works well for complex SVG primitives (second row) and bitmap primitives (third row), whereas existing DiffVG [ 27 ] fails or is not available, even though it works well for simple SVG primitives (first row). Note that SVG exportation is included in runtimes for row 1-2, while bitmap exports only PNG, so the third row takes less time (36s).

SoftRas Sec. 3.2.1	SA-Init. Sec. 3.2.2	Scenario 1 (512 $\times$ 512)	Scenario 2 (512 $\times$ 512)	Scenario 3 (1024 $\times$ 1024)
✗	✗	24.4	20.6	25.9
✓	✗	24.7	21.5	26.5
✗	✓	25.5	21.0	27.1
✓	✓	25.7	21.7	27.4

Table 3: Ablation of our optimization helpers: soft rasterization ( Sec. 3.2.1 ) and structure-aware initialization ( Sec. 3.2.2 ). We report PSNR on three targets with indicating whether each component is enabled. Soft rasterization enriches gradients and yields consistent gains, while structure-aware initialization further boosts fidelity; enabling both gives the best PSNR across all scenarios.

Figure 6: Uniform noise ( Sec. 3.2.3 ) makes canvas fully covered by primitives, as shown in (c). (b) Conversely, using the canvas as white removes the need for white primitives to be positioned.

Init. $\Theta^{f-1\star}$	Rem. Stk.	Freez. Unch.	Frame-wise Fidelity		Anti-flicker
Init. $\Theta^{f-1\star}$	Rem. Stk.	Freez. Unch.	PSNR $\uparrow$	SSIM $\uparrow$	tLP $\downarrow$	tOF $\downarrow$
✗	✗	✗	24.19 $\pm$ 2.36	0.616 $\pm$ 0.106	7.41 $\pm$ 8.89	2.23 $\pm$ 1.10
✓	✗	✗	24.26 $\pm$ 1.73	0.629 $\pm$ 0.073	5.39 $\pm$ 4.64	1.88 $\pm$ 1.13
✓	✓	✗	24.66 $\pm$ 2.03	0.647 $\pm$ 0.088	4.98 $\pm$ 4.93	1.89 $\pm$ 1.11
✓	✗	✓	24.23 $\pm$ 2.13	0.617 $\pm$ 0.099	3.50 $\pm$ 2.52	1.91 $\pm$ 1.32
✓	✓	✓	24.38 $\pm$ 2.23	0.630 $\pm$ 0.069	3.49 $\pm$ 2.26	1.84 $\pm$ 1.19

Table 4: Dynamic DiffBMP ablations. We observed that improvements in flicker suppression can trade off with per-frame fidelity; using all three of our heuristics in Sec. 3.3.1 targets a balanced solution, having competitive fidelity and the best anti-flicker.

Figure 7: Comparison of rendered output

I

and primitives with spatial constraint under different settings: (a) Ours (default), (b) Ours with opacity loss, and (c) Ours with both opacity loss and re-initialization. Applying both opacity loss and re-initialization produces the best results.

Figure 8: DiffBMP can also be combined with CLIP for creation. Each caption shows the text prompt and the primitive used.

Figure S1 : Coordinate transformation and primitive sampling. Color and opacity from

i

-th primitive at

(x,y)

in canvas (black dot) is sampled from

(U,V)

in the primitive’s coordinate (red dot). Since

U

and

V

are not integers, bilinear interpolation from four nearest lattice points (green dots).

Figure S2 : DiffBMP computation tree (forward/backward). Forward uses premultiplied-alpha Over in front-to-back order; per-primitive opacity is a sigmoid-scaled value modulated by a template mask sampled at rotated/scaled coordinates. Learnable leaves (gray) are

c_{i},\nu_{i},x_{i},y_{i},s_{i},\theta_{i}

. Red dashed arrows indicate gradient flow. See Sec. 3.1.1 for equations.

Figure S3 : Frame-wise Fidelity Heuristic of Dynamic DiffBMP. Warm-starting from

\Theta^{f-1\star}

can leave an over-dominant primitive sitting over a high-change area; rather than relocating, it gets recolored and suppresses local detail, as in b . We apply Removing Stuck Primitives : adaptively decaying the opacity of large, opaque, front-ordered strokes so finer primitives behind them take over. This restores facial detail and improves fidelity under the same budget.

Figure S4 : Anti-flicker via freezing unchanged regions. We visualize inter-frame change with an error map

E=5\!\cdot\!(I_{1}-I_{17})+127

, where

I_{1}

is the first frame and

I_{17}

the last frame. Mid-gray (

\approx 127

) indicates no change; brighter/darker values denote larger differences. If all primitives are freely optimized, spurious updates appear even in static regions, producing widespread flicker, as seen in b . Our remedy c freezes primitives, concentrating updates only where the video truly changes.

Category	Parameter	Value / Description
Initialization	$\mathtt{initializer}$	$\mathtt{structure\_aware}$ (default for most; $\mathtt{random}$ for Fig. 8)
	$\mathtt{v\_init\_bias}$	$-4.0$ (yields $\sigma(-4)\approx 1.8\%$ initial opacity)
	$\mathtt{std\_c\_init}^{\dagger}$	$0.02$ (color initialization noise std)
	$\mathtt{variance\_window\_size}^{\dagger}$	$7$ (local variance computation window)
	$\mathtt{variance\_base\_prob}^{\dagger}$	$0.1$ (base sampling probability for low-variance areas)
	$\mathtt{max\_prims\_per\_pixel}^{\dagger}$	$100$ (200 for spatially constrained cases)

Table S1 : Common hyperparameters used across experiments. These settings are applied to all experiments unless explicitly overridden in figure-specific configurations. Values marked with † come from

\mathtt{constants.py}

when not specified in config files.

Figure	Primitive(s)	$N$	Resolution	Scale Range	Iter.	Special Settings
Figure 1: Teaser examples
1(a)-fingerprint	fingerprint.jpg	2000	1024	[2, 8]	300	$\mathtt{radial\_transparency{=}true}$ , $\mathtt{c\_blend{=}0.0}$
1(a)-autograph	autograph_seurat.png	2000	1024	[2, 10]	300	$\mathtt{c\_blend{=}0.0}$
1(b)	logos/*.png (300 logos)	300	512	[4, 20]	100	$\mathtt{c\_blend{=}1.0}$ , grayscale L1 + MSE loss
1(c)-flowers	5 flower types	1000	512	[2, 31]	300	$\mathtt{exist\_bg{=}false}$ , mask, pruning
1(c)-girl	2 flower types	1000	512	[2, 20]	300+100	$\mathtt{exist\_bg{=}false}$ , initial+sequential (17 frames), Algorithm 2 , freeze unchanged

Table S2 : Figure-specific experimental configurations. This table shows parameters that differ across experiments, including primitive details, number of primitives (

N

), image resolution, scale range, iterations, and special settings.

Aspect	Naive PyTorch	Our CUDA
Spatial culling	None	Tile-based binning
Primitive loop	Sequential ( $N$ )	Parallel per-pixel
Memory per iteration	$O(N\times HW)$	$O(k\times HW)$ , $k\ll N$
Precision	FP32 only	FP16 + FP32 mixed
Intermediate tensors	$(H,W)$ per primitive	Small per-pixel cache
Atomic operations	Many (unoptimized)	Packed $\mathtt{\_\_half2}$

Table S3 : Comparison of naive PyTorch vs. our CUDA implementation

Figure S5 : Additional scaling statistics for the Seurat composition in (a). To expose a broader primitive size distribution, this analysis widens the allowed scale range to

[2,50]

while sweeping the primitive count

N

. Fingerprint primitives retain radial transparency as in (a). From left to right, the panels show the CDF of the normalized primitive footprint

A_{i}/(HW)

for fingerprint , the CDF of

A_{i}/(HW)

for autograph_seurat , the CDF of the primitive tile-hit degree

\tau_{i}

for fingerprint , the CDF of

\tau_{i}

for autograph_seurat , and the measured runtime versus

N

. Here,

A_{i}

denotes the area of the padded screen-space bounding box of primitive

i

, normalized by the image area

HW

.

\tau_{i}

denotes the number of tiles intersected by that primitive.

Figure S6 : Spatial distribution of per-tile primitive density

d_{t}

for the same scaling analysis setting as Fig. S5 , with scale range

[2,50]

and increasing primitive count

N

. Each heatmap visualizes the number of primitives assigned to tile

t

during CPU binning. The top row shows fingerprint , and the bottom row shows autograph_seurat . In each row,

N\in\{1000,2000,3000,4000,5000\}

increases from left to right. Each panel uses its own color scale for readability, so absolute density values should be interpreted using the corresponding colorbar.

Figure S7: Optimization progress across iterations for three different examples. Each row shows the evolution from iteration 1 (left) to iteration 100 (right), illustrating how coarse structure emerges early and fine details are refined in later iterations. All three rows reuse the same primitive sets and target images as the ablation study in Table 3 of the main paper, enabling direct visual comparison with the quantitative results.

Figure S8: Effect of primitive count on reconstruction quality across diverse targets and primitive types. Each row uses a different primitive set (left) to approximate a different target image (right), while columns sweep the number of primitives

N

from 1000 to 4000. As

N

increases, our optimizer consistently sharpens details and improves PSNR across movie posters, portraits, and natural scenes, demonstrating robust approximation capability over varied image content and primitive shapes.

DiffBMP: Differentiable Rendering with Bitmap Primitives

Abstract

Figures & tables

Differentiable Rendering

XPR: An Extensible Cross-Platform Point-Based Differentiable Renderer

DiffSoup: Direct Differentiable Rasterization of Triangle Soup for Extreme Radiance Field Simplification

Gaussian Mesh Renderer for Lightweight Differentiable Rendering

Abstract

Differentiable Rendering

Explore similar work

XPR: An Extensible Cross-Platform Point-Based Differentiable Renderer

DiffSoup: Direct Differentiable Rasterization of Triangle Soup for Extreme Radiance Field Simplification

Gaussian Mesh Renderer for Lightweight Differentiable Rendering