cs.CVFeb 24, 2026
SaveSPRITETOMESH: Automatic Mesh Generation for 2D Skeletal Animation Using Learned Segmentation and Contour-Aware Vertex Placement
Abstract
We present SPRITETOMESH, a fully automatic pipeline for converting 2D game sprite images into triangle meshes compatible with skeletal animation frameworks such as Spine2D. Creating animation-ready meshes is traditionally a tedious manual process requiring artists to carefully place vertices along visual boundaries, a task that typically takes 15-60 minutes per sprite. Our method addresses this through a hybrid learned-algorithmic approach. A segmentation network (EfficientNet-B0 encoder with U-Net decoder) trained on over 100,000 sprite-mask pairs from 172 games achieves an IoU of 0.87, providing accurate binary masks from arbitrary input images. From these masks, we extract exterior contour vertices using Douglas-Peucker simplification with adaptive arc subdivision, and interior vertices along visual boundaries detected via bilateral-filtered multi-channel Canny edge detection with contour-following placement. Delaunay triangulation with mask-based centroid filtering produces the final mesh. Through controlled experiments, we demonstrate that direct vertex position prediction via neural network heatmap regression is fundamentally not viable for this task: the heatmap decoder consistently fails to converge (loss plateau at 0.061) while the segmentation decoder trains normally under identical conditions. We attribute this to the inherently artistic nature of vertex placement - the same sprite can be meshed validly in many different ways. This negative result validates our hybrid design: learned segmentation where ground truth is unambiguous, algorithmic placement where domain heuristics are appropriate. The complete pipeline processes a sprite in under 3 seconds, representing a speedup of 300x-1200x over manual creation. We release our trained model to the game development community.
Figures & tables
Figure 1: Overview of the SpriteToMesh pipeline in six panels. (1) Input sprite image. (2) Binary segmentation mask obtained from the alpha channel or our EfficientNet-B0 network. (3) Exterior contour vertices (blue) placed via Douglas-Peucker simplification with adaptive arc subdivision. (4) Interior boundary vertices (yellow) detected via multi-channel Canny edge detection, combined with exterior vertices (blue). (5) Delaunay triangulation wireframe with centroid-based filtering. (6) Final mesh overlaid on the original sprite.
Figure 2: Exterior contour extraction process in four panels. (a) Binary segmentation mask. (b) Raw contour extracted with findContours , showing thousands of contour pixels. (c) Douglas-Peucker simplification ( ) identifies structural keypoints at corners and concavities (red dots). (d) Final hull vertices (red dots with white outlines) after adaptive arc subdivision adds uniformly-spaced vertices between keypoints along the actual contour path.
Figure 3: Interior vertex placement pipeline shown as a grid. (a) Original sprite. (b) Bilateral-filtered image (texture noise reduced, structural edges preserved). (c) Multi-channel Canny edge map (union of per-channel detections). (d) Masked edges after erosion and morphological closing. (e) Interior vertices (yellow) placed along surviving contours using Douglas-Peucker simplification and uniform subdivision. (f) All vertices combined: exterior (blue) and interior (yellow), ready for Delaunay triangulation.
Figure 4: Example of binary-to-JSON conversion using SkelToJson. The binary .skel format encodes the complete skeleton as a dense byte stream (left). Our converter produces structured JSON (right) from which mesh vertex positions, UV coordinates, and triangle indices can be directly extracted.
Figure 5: Dataset preparation pipeline shown in four panels. (1) A full texture atlas from a game. (2) An individual sprite attachment cropped from the atlas using region definitions parsed from the .atlas file. (3) The binary mask extracted from the sprite’s alpha channel ( ). (4) The sprite composited onto a random gradient background, as used during training to force the network to learn foreground–background separation from visual features rather than trivial alpha thresholding.
Figure 6: Representative samples from our dataset, shown as a grid where each sprite is drawn from a different game. The dataset spans 172 games and exhibits wide diversity in artistic style, subject matter, complexity, and resolution—including characters, objects, UI elements, effects, and accessories.
Figure 7: Dataset statistics shown as four panels. (a) Histogram of the number of samples per game. (b) Pie chart of attachment type breakdown: region (mask only) vs. mesh (mask + vertex annotations) with percentages and counts. (c) Distribution of vertex counts for mesh samples, with a median line. (d) Distribution of max image dimension (in pixels) across all samples, with a median line.
Figure 8: Data augmentation via background composition, shown as five horizontal panels. From left to right: the original RGBA sprite on a checkerboard (showing transparency), then the same sprite composited onto a solid color, a gradient, and a noise pattern. The rightmost panel shows the ground truth binary mask derived from the alpha channel. This augmentation forces the segmentation network to learn foreground–background separation from visual features.
| Metric | Value |
|---|---|
| IoU | 0.8697 |
| Dice Score | 0.9304 |
| Pixel Accuracy | 0.9741 |
| BCE Loss | 0.0523 |
| Dice Loss | 0.0696 |
Table 1: Segmentation performance on the validation set.
Figure 9: Training dynamics shown on a single dual-axis plot. The left axis displays the segmentation loss (BCE + Dice) for both training and validation sets, which decrease steadily with a minimal overfitting gap. The right axis shows the validation IoU, which increases during training and reaches its best value, after which overfitting begins.
Figure 10: Detailed training loss decomposition shown as a grid (five panels, one empty). Each panel plots training and validation curves for a single loss component: total loss, segmentation loss (BCE + Dice combined), BCE loss, Dice loss, and heatmap MSE loss. The segmentation-related losses decrease steadily on both sets, while the heatmap loss plateaus early, confirming the infeasibility of direct vertex prediction.
Figure 11: Mesh prediction results on eight diverse sprites arranged in a grid. Each cell shows a sprite with the automatically generated mesh overlaid as a wireframe, demonstrating that our pipeline handles a variety of art styles, shapes, and complexities.
| Segmentation | Heatmap | |
|---|---|---|
| Final loss | 0.122 | 0.061 (plateau) |
| IoU / Peak recall | 0.87 | 0 |
| Converged? | ✓ | |
| Prediction quality | Accurate masks | Uniform blur |
Table 2: Comparison of segmentation vs. heatmap decoder convergence. The heatmap head plateaus without producing meaningful predictions.
Figure 12: Training loss comparison between segmentation and heatmap decoder heads. Four curves are shown: segmentation loss (green, train and val) and heatmap loss (orange, train and val). The segmentation loss decreases steadily throughout training, while the heatmap loss plateaus at approximately 0.061 after the first few epochs and never improves, demonstrating the fundamental non-learnability of vertex positions from pixel values alone.
Figure 13: Heatmap decoder failure analysis. The heatmap MSE (left axis) remains elevated throughout training with no convergence trend, while the peak recall metric (right axis) drops from an initial value near 1.0, indicating the network’s inability to localize vertex positions. This contrasts sharply with the segmentation decoder’s steady improvement under identical training conditions.
| Metric | Value |
|---|---|
| Exterior vertices | |
| Interior vertices | |
| Total vertices | |
| Triangles | |
| Processing time | s |
| Mask coverage | % |
Table 3: Mesh generation statistics across test sprites. Values reported as mean std.
Figure 14: Qualitative results on four diverse test sprites, arranged with one sprite per row. Each row shows four panels from left to right: input sprite image, predicted binary segmentation mask, detected vertices (blue = exterior contour, yellow = interior boundary), and final triangulated mesh overlaid on the sprite. Interior vertices are placed along detected visual boundaries within the sprite.
| Configuration | Boundary | Vertices | Angle |
| Adherence | Count | Ratio | |
| Full pipeline | 78.3% | 484 | 0.42 |
| No bilateral filter | 71.2% | 623 | 0.38 |
| Single-channel Canny | 64.8% | 412 | 0.41 |
| No DP simplification | 76.5% | 1,847 | 0.31 |
| No arc subdivision | 73.1% | 389 | 0.40 |
Table 4: Ablation study. We disable individual components and measure their impact on mesh quality.
Figure 15: Comparison of four mesh generation approaches on the same sprite, shown as a grid. Top row: vertex placement (blue = exterior, yellow = interior) with vertex counts. Bottom row: resulting triangulated mesh wireframe with triangle counts. (a) Hull only—exterior contour vertices with no interior, yielding a single deformable region. (b) Uniform grid—regular interior vertices that ignore visual structure. (c) Shi-Tomasi corners—feature points that do not follow edge contours. (d) Our contour-aware method—vertices placed along internal visual boundaries, enabling independent deformation of visually distinct regions.
| Method | Boundary | Vertices | Articulable |
|---|---|---|---|
| Adherence | Regions | ||
| Alpha-only hull | 0% | 64 | 1 |
| Uniform grid | 42.6% | 520 | Partial |
| Shi-Tomasi corners | 55.3% | 667 | Partial |
| SpriteToMesh (ours) | 78.3% | 484 | Full |
Table 5: Comparison with baseline methods.
Figure 16: Comparison of artist-created and automatically generated meshes, shown as three columns per row: input sprite (left), ground truth artist mesh in green (center), and our predicted mesh in orange with blue exterior and yellow interior vertices (right). While the exact vertex positions differ—reflecting the subjective nature of the task—both meshes cover similar visual boundaries. Our method tends to produce denser meshes along detected edges, while artists place vertices more selectively based on anticipated animation needs.
Figure 17: Examples of current limitations, shown as three mesh prediction results side by side with vertex counts. (a) A highly textured sprite that produces excessive interior vertices due to many spurious Canny edges. (b) A sprite with subtle visual boundaries (soft color gradients) where edges fall below the Canny threshold and are missed. (c) A sprite with unusual proportions or an art style underrepresented in the training data, leading to imperfect segmentation or vertex placement.
Figure 18: Gallery of automatically generated meshes arranged in a grid. Each cell shows a single sprite with the predicted mesh overlaid as a wireframe. Our pipeline handles characters, objects, effects, and UI elements from diverse games without any manual intervention or parameter tuning.