Authors: Julia Dietlmeier, Vayangi Ganepola, Oluwabukola G. Adegboro, Mayug Maniparambil, Claudia Mazo, Noel E. O'Connor
Organizations: Insight Research Ireland Centre for Data Analytics, Dublin City University, Glasnevin 9, Dublin, Ireland · Research Ireland Centre for Research Training in Machine Learning (ML-Labs), Dublin City University, Glasnevin 9, Dublin, Ireland · School of Computing, Dublin City University, Glasnevin 9, Dublin, Ireland
Directly inspired by findings in biological vision, high-dimensional hypercolumns are feature vectors built by concatenating multi-scale activations of convolutional neural networks for a single image pixel location. Together with powerful classifiers, they can be used for image segmentation i.e. pixel classification. However, in practice, there are only very few works dedicated to the use of hypercolumns. One reason is the computational complexity of processing concatenated dense hypercolumns that grows linearly with the size N of the training set. In this work, we address this challenge by applying stratified subsampling to the VGG16 based hypercolumns. Furthermore, we investigate the performance of ensemble learning on sparse hypercolumns. Our experiments on a brain tumor dataset show that stacking and voting ensembles deliver competitive performance, but in the extreme low-shot case of N≤20, a simple Logistic Regression classifier is the most effective method. For 10% stratified subsampling rate, our best average Dice score is 0.66 for N=20. This is a statistically significant improvement of 24.53% over the standard multi-scale UNet baseline (p-value = [3.07e−11], Wilcoxon signed-rank test), which is less effective due to overfitting.
Figures & tables
Figure 1: Our hypercolumn-based processing pipeline for binary pixel classification.
Base classifiers
Number of estimators
Max iterations
Random Forest Classifier (RF)
10
n/a
Linear Support Vector Classifier (SVC)
n/a
1000
Logistic Regression Classifier (LR)
n/a
1000
Table 1: Base classifier configuration.
Subsampling rate
N
hypercolumn+LR ↑
UNet ↑
Dice gain ↑
p -value ↓
\Block 3-11%
2
0.11 ± 0.05
0.34 ± 0.16
−67.65 %
2.23e−05
10
0.53 ± 0.05
0.43 ± 0.21
23.26 %
1.59e−07
20
0.61 ± 0.03
0.56 ± 0.26
8.93 %
0.663
\Block 3-110%
2
0.45 ± 0.05
0.33 ± 0.18
36.36 %
1.52e−14
10
0.56 ± 0.06
0.39 ± 0.21
43.59 %
1.40e−20
20
0.66 ± 0.02
0.53 ± 0.25
24.53 %
3.07e−11
Table 2: Comparison between the hypercolumn+LR and the UNet models. Dice performance gains are computed based on the mean values.
Models
Accuracy ↑
Precision ↑
Recall ↑
Jaccard ↑
Dice ↑
\Block 6-1 \rotate N=2
hypercolumn+LR
0.98 ± 0.00
0.35 ± 0.12
0.08 ± 0.04
0.08 ± 0.04
0.11 ± 0.05
hypercolumn+RF
0.98 ± 0.00
0.04 ± 0.03
0.00 ± 0.00
0.00 ± 0.00
0.00 ± 0.00
hypercolumn+SVC
0.98 ± 0.00
0.02 ± 0.01
0.00 ± 0.00
0.00 ± 0.00
0.00 ± 0.00
hypercolumn+stacking
0.93 ± 0.04
0.08 ± 0.05
0.20 ± 0.12
0.05 ± 0.03
0.08 ± 0.04
hypercolumn+voting
0.98 ± 0.00
0.08 ± 0.04
0.01 ± 0.01
0.01 ± 0.01
0.01 ± 0.01
UNet
0.79 ± 0.38
0.40 ± 0.20
0.57 ± 0.23
0.25 ± 0.12
0.34 ± 0.16
Table 3: Selected quantitative results (mean and standard deviation) for the case of 1% stratified subsampling rate. Best results are in red and second best in blue.
Figure 2: Dice score coefficient performance of LR versus UNet (left) and stacking versus voting (right). All models were trained on the same amount of images for 1% stratified subsampling rate. Graphs are showing mean (lines) ± standard deviation (filled areas).
Models
Accuracy ↑
Precision ↑
Recall ↑
Jaccard ↑
Dice ↑
\Block 6-1 \rotate N=2
hypercolumn+LR
0.99 ± 0.00
0.69 ± 0.03
0.39 ± 0.06
0.34 ± 0.05
0.45 ± 0.05
hypercolumn+RF
0.98 ± 0.00
0.21 ± 0.12
0.03 ± 0.03
0.03 ± 0.05
0.05 ± 0.05
hypercolumn+SVC
0.98 ± 0.00
0.27 ± 0.13
0.06 ± 0.03
0.06 ± 0.03
0.08 ± 0.04
hypercolumn+stacking
0.94 ± 0.02
0.18 ± 0.06
0.41 ± 0.12
0.13 ± 0.05
0.20 ± 0.08
hypercolumn+voting
0.99 ± 0.00
0.36 ± 0.15
0.10 ± 0.06
0.10 ± 0.06
0.13 ± 0.07
UNet
0.79 ± 0.39
0.41 ± 0.21
0.57 ± 0.29
0.26 ± 0.14
0.33 ± 0.18
Table 4: Selected quantitative results (mean and standard deviation) for the case of 10% stratified subsampling rate. Best results are in red and second best in blue.
Figure 3: Dice score coefficient performance of LR versus UNet (left) and stacking versus voting (right). All models were trained on the same amount of images for 10% stratified subsampling rate. Graphs are showing mean (lines) ± standard deviation (filled areas).
Models
Total parameters
Inference time
Training time
Device
hypercolumn model
14,714,688
-
-
GPU
+LR
1,473
0.48
7.66
CPU
+RF
5,290
0.17
13.56
CPU
+SVC
2,536,257
63.38
1062.97
CPU
+stacking
1,486
0.61
254.95
CPU
+voting
2,886,604
80.40
1347.26
CPU
Table 5: Computational details. The total number of parameters for base and ensemble classifiers are obtained after training on N=20 images with the 10% subsampling rate. Inference time is given per image. Times are in seconds.
Figure 4: Selected qualitative results for N=10 training images. The first three rows show results for 1% subsampling rate and the last three rows show the results for 10% subsampling rate. Note that UNet is trained on N=10 images without subsampling.
Department of Mathematics, Rhodes University, Makhanda 6140, South Africa · Department of Statistics, Rhodes University, Makhanda 6140, South Africa · National Institute for Theoretical and Computational Sciences, Stellenbosch 7600, South Africa +8