Organizations: VCIP, College of Computer Science, Nankai University · Academy for Advanced Interdisciplinary Studies, Nankai University · School of Computer Science and Engineering, Tianjin University of Technology · School of Information and Communication Engineering, UESTC · School of Computer Science and Technology, Zhejiang Normal University · Nankai International Advanced Research Institute, Shenzhen Futian
Edge detection is a fundamental image analysis task that underpins numerous high-level vision applications. Recent advances in Transformer architectures have significantly improved edge quality by capturing long-range dependencies, but this often comes with computational overhead. Achieving higher pixel-level accuracy requires increased input resolution, further escalating computational cost and limiting practical deployment. Building on the strong representational capacity of recent Transformer-based edge detectors, we propose an Adaptive Multi-stage non-edge Pruning framework for Edge Detection(Amped). Amped identifies high-confidence non-edge tokens and removes them as early as possible to substantially reduce computation, thus retaining high accuracy while cutting GFLOPs and accelerating inference with minimal performance loss. Moreover, to mitigate the structural complexity of existing edge detection networks and facilitate their integration into real-world systems, we introduce a simple yet high-performance Transformer-based model, termed Streamline Edge Detector(SED). Applied to both existing detectors and our SED, the proposed pruning strategy provides a favorable balance between accuracy and efficiency-reducing GFLOPs by up to 40% with only a 0.4% drop in ODS F-measure. In addition, despite its simplicity, SED achieves a state-of-the-art ODS F-measure of 86.5%. The code will be released.
Figures & tables
Figure 1: Non-edge pruning flowchart. As the backbone network extracts features, our method adaptively generates a binary decision mask by thresholding the intermediate edge score maps to determine which tokens should be pruned.
Figure 2: Overview of the proposed Amped framework . Amped trims high-confidence non-edge tokens by computing the edge score map and comparing it against a stage-specific threshold. The proposed SED utilizes a simple linear decoder, a design that effectively reduces model complexity while maintaining high precision. Z(l) and M(l) denote the feature map and binary decision mask of pruning stage l , and Z~(l) represents the recovered feature map of stage l .
Pruning Scores
SED-SViT
GFLOPs
SED-ViT
GFLOPs
ODS
OIS
ODS
OIS
origin
0.849
0.868
484.5
0.826
0.847
663.7
[0.25, 0.35, 0.45]
0.843(-0.6)
0.864(-0.4)
355.4( ↓ 26.65%)
0.820(-0.6)
0.838(-0.9)
407.3( ↓ 38.63%)
[0.3, 0.4, 0.5]
0.843(-0.6)
0.863(-0.5)
348.5( ↓ 28.07%)
0.818(-0.8)
0.837(-1.0)
406.2( ↓ 38.80%)
[0.35, 0.45, 0.55]
0.840(-0.9)
0.860(-0.8)
339.6( ↓ 29.91%)
0.816(-1.0)
0.837(-1.0)
390.2( ↓ 41.21%)
[0.4, 0.5, 0.6]
0.824(-2.5)
0.846(-2.2)
216.6( ↓ 55.29%)
0.812(-1.4)
0.831(-1.6)
332.3( ↓ 49.93%)
Table 1: Ablation study of non-edge pruning scores for SED-SViT and SED-ViT on BSDS500 [ 1 ] . All results are obtained using models with Base-scale backbones.
Pruning Layers
SED-SViT
GFLOPs
SED-ViT
GFLOPs
ODS
OIS
ODS
OIS
origin
0.849
0.868
484.5
0.826
0.847
663.7
3
0.844(-0.5)
0.864(-0.4)
411.7( ↓ 15.03%)
0.820(-0.6)
0.839(-0.8)
566.2( ↓ 14.69%)
3, 6
0.841(-0.8)
0.861(-0.7)
356.0( ↓ 26.52%)
0.816(-1.0)
0.836(-1.1)
446.9( ↓ 32.67%)
3, 6, 9
0.843(-0.6)
0.863(-0.5)
348.5( ↓ 28.07%)
0.818(-0.8)
0.837(-1.0)
406.2( ↓ 38.80%)
Table 2: Ablation study of multi-stage pruning for SED-SViT and SED-ViT on BSDS500 [ 1 ] . All results are obtained using models with Base-scale backbones and pruning scores of [0.3, 0.4, 0.5].
Figure 3: Qualitative comparisons between our method and baselines on three challenging samples from the BSDS500 test set [ 1 ] .
Methods
Origin
GFLOPs
Pruned
GFLOPs
ODS
OIS
AP
ODS
OIS
AP
BSDS500
EDTER
0.829
0.845
0.883
356.4*
0.825(-0.4)
0.839(-0.6)
0.878(-0.5)
206.8*( ↓ 41.98%)
SED-ViT-B
0.826
0.847
0.884
663.7
0.818(-0.8)
0.837(-1.0)
0.877(-0.7)
406.2( ↓ 38.79%)
SED-ViT-L
0.835
0.854
0.890
2069.6
0.833(-0.2)
0.850(-0.4)
0.888(-0.2)
1316.1( ↓ 36.41%)
SED-SViT-B
0.849
0.868
0.902
484.5
0.843(-0.6)
0.863(-0.5)
0.897(-0.5)
348.5( ↓ 28.07%)
SED-SViT-L
0.865
0.882
0.911
1489.2
0.863(-0.2)
0.881(-0.1)
0.912(+0.1)
1169.1( ↓ 21.49%)
Table 3: Results of Amped applied to the prior method EDTER [ 41 ] and our models SED-SViT and SED-ViT on BSDS500 [ 1 ] and NYUDv2 [ 46 ] .
Encoder Types
Methods
Pub.’Year
ODS
OIS
AP
Traditional
Canny [ 5 ]
TPAMI’86
0.600
0.640
0.580
gPb-UCM [ 1 ]
TPAMI’10
0.726
0.757
0.696
SE [ 12 ]
TPAMI’14
0.746
0.767
0.803
OEF [ 19 ]
CVPR’15
0.746
0.770
0.820
CNN
DeepEdge [ 3 ]
CVPR’15
0.753
0.772
0.807
DeepContour [ 45 ]
CVPR’15
0.757
0.776
0.800
Table 4: Quantitative comparison on the BSDS500 test set [ 1 ] . All results are evaluated with single scale input.
Figure 4: Precision-recall curves on the BSDS500 test set [ 1 ] .
Figure 5: Visualization of progressive non-edge pruning for SED-SViT (top) and SED-ViT (bottom). Most pruned tokens lie in smooth background regions, while tokens near object boundaries are largely preserved, especially for SED-SViT.
Encoder Types
Methods
Pub.’Year
ODS
OIS
AP
Traditional
gPb-UCM [ 1 ]
TPAMI’10
0.632
0.661
0.562
SE [ 12 ]
TPAMI’14
0.695
0.708
0.679
OEF [ 19 ]
CVPR’15
0.751
0.767
–
CNN
HED [ 57 ]
ICCV’15
0.720
0.734
0.734
RCF [ 31 ]
CVPR’17
0.729
0.742
-
AMH-Net [ 59 ]
NeurIPS’17
0.744
0.758
0.765
Table 5: Quantitative comparison on the NYUDv2 test set [ 46 ] . All results are evaluated with single scale input.
Department of Electro-Optics, National Formosa University, Taiwan · Department of Electrical Engineering, University of Muhammadiyah Malang, Indonesia · Department of Electrical Engineering, National Taipei University, Taiwan +1