Organizations: College of Computer, Qinghai Normal University, Xining, 810001, Qinghai, China · College of Computer, Nanjing University, Nanjing, 210023, Jiangsu, China · College of Computer, Qinghai Minzu University, Xining, 810007, Qinghai, China
This work investigates large-scale sketch recognition from a graph-native perspective, where free-hand sketches are directly modeled as structured graphs rather than raster images or stroke sequences. We propose SketchGraphNet, a hybrid graph neural architecture that integrates local message passing with a memory-efficient global attention mechanism, without relying on auxiliary positional or structural encodings. To support systematic evaluation, we construct SketchGraph, a large-scale benchmark comprising 3.44 million graph-structured sketches across 344 categories, with two variants (A and R) to reflect different noise conditions. Each sketch is represented as a spatiotemporal graph with normalized stroke-order attributes. On SketchGraph-A and SketchGraph-R, SketchGraphNet achieves Top-1 accuracies of 83.62% and 87.61%, respectively, under a unified training configuration. MemEffAttn further reduces peak GPU memory by over 40% and training time by more than 30% compared with Performer-based global attention, while maintaining comparable accuracy.
Figures & tables
Figure 1: Architecture of SketchGraphNet.
Figure 2: Architecture of MemEffAttn.
Figure 3: Statistical Comparison Between the A and R Versions of the SketchGraph Dataset: (a) Path-Length Distributions Across Categories; (b) Category-Level Density in the Space of Stroke Count Versus Path Length.
Figure 4: Structural Statistics Aggregated Across All Categories in the Dataset.
Figure 5: Mutually Exclusive Visualization of A − R and R Samples in the SketchGraph Dataset.
Models
Param (M)
Train time
Latency (ms)
DSSG
SketchGraph-A
SketchGraph-R
TOP-1
TOP-3
TOP-5
TOP-1
TOP-3
TOP-5
S3Net
5.33
1.410 h
4.206
w/o
0.8024
0.9246
0.9465
0.8496
0.9640
0.9804
1.405 h
4.237
w/
0.8070
0.9272
0.9486
0.8544
0.9663
0.9820
SketchGNN
0.79
1.047 h
5.208
w/o
0.6936
0.8551
0.8934
0.7457
0.9076
0.9425
1.058 h
5.132
w/
0.6986
0.8582
0.8964
0.7511
0.9105
0.9447
MGT
39.98
6.251 h
6.086
w/o
0.7029
0.8625
0.9009
0.7571
0.9164
0.9495
Table 1: Classification Accuracy on Both Dataset Versions.
Figure 6: Training Accuracy and Loss Curves.
Figure 7: Confusion Matrices for Best and Worst Performing Categories.
Figure 8: Visualization of Easily Confused Categories.
Global Attention
Conv Blocks
Avg. Memory
Avg. Train Time
Top1 Accuracy
SketchGraph-A
SketchGraph-R
PerformerAttention
4
5.03 GB
2.12 h
0.8344
0.8756
6
6.44 GB
3.03 h
0.8404
0.8804
8
7.93 GB
3.93 h
0.8423
0.8823
MemEffAttn
4
2.87 GB
1.38 h
0.8362
0.8761
6
3.60 GB
1.93 h
0.8414
0.8813
Table 2: Evaluation of ConvBlock Count and Global Attention Methods.
Local GNN
DSSG
TOP-1 Accuracy
SketchGraph-A
SketchGraph-R
GINConv
w/o
0.8313
0.8731
w/
0.8362
0.8761
DenseSAGEConv
w/o
0.8312
0.8709
w/
0.8337
0.8753
SAGEConv
w/o
0.8302
0.8725
Table 3: Evaluation of Local GNN Operator Replaceability.
Configuration
TOP-1 Accuracy
SketchGraph-A
SketchGraph-R
Full SketchGraphNet
0.8362
0.8761
w/o Global Attention
0.7589
0.8048
w/o Temporal Feature
0.8171
0.8632
Table 4: Ablation Study on Global Attention Structures.
MemEffAttn
Number of ConvBlock
TOP-1 Accuracy
SketchGraph-A
SketchGraph-R
w/ Kernel
4
0.8362
0.8761
6
0.8419
0.8815
8
0.8432
0.8822
w/o Kernel
4
0.8376
0.8773
6
0.8430
0.8584
Table 5: Ablation Study on Nonlinear-Kernels.
Attention Implementation
Implementation Type
Peak Memory
Convergence Status
First NaN Epoch
Standard Attention
Explicit NN Matrix
4.07 GB
Failed
1
MemEffAttn
Tiling-based (xFormers)
2.87 GB
Stable
None
Table 6: Stability analysis under identical ReLU-kernel mapping.
SketchGraphNet
TOP-1 Accuracy
SketchGraph-A
SketchGraph-R
Default settings
0.8362
0.8761
Learning Rate (0.0001)
0.8136
0.8575
Label Smoothing (0.0)
0.8317
0.8741
DropOut (0.2)
0.8291
0.8698
Table 7: Parameter Sensitivity Analysis of SketchGraphNet.
Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, Guangdong, China · Control and Simulation Center, Harbin Institute of Technology, Harbin, Heilongjiang