Organizations: Computer Science and Engineering, Wright State University, Dayton, OH, USA · Computer Science, Kansas State University, Manhattan, KS, USA
Convolutional Neural Networks (CNNs) can classify images using background-dominated patches that contain little or none of the target object, showing that they exploit contextual information. Previous studies detected this behavior through classification accuracy but say little about what CNN hidden layers represent. We investigate whether Concept Induction can reveal interpretable visual properties associated with hidden-neuron activations on contextual image patches. Using five agricultural datasets with high, moderate, and low contextual bias, we analyze four CNN architectures at three network depths and five patch locations. Quantitative image descriptors of each patch are encoded in a knowledge graph and used as background knowledge for Concept Induction, which derives logical expressions characterizing neuron activation. These expressions are tested on held-out images using the Mann-Whitney U test and Cliff's delta. Of 48,013 tested neuron-feature pairs, 13,164 (27.4%) were validated. Validation was most frequent at intermediate depths (mean 40.2% at mid layers versus 12.0% at deep layers), and no patch location dominated across datasets. Among the tested descriptors, edge, Gabor, and Tamura texture features had the highest validation rates. Stronger contextual bias did not correspond to higher neuron-feature validation rates: moderately biased Sugarcane had the highest rate (40.5%) and highly biased Cassava the lowest (19.2%). These results suggest that behavioral context dependence and the semantic interpretability of hidden neurons capture different aspects of CNN behavior, and that Concept Induction can complement patch-based bias detection.
Figures & tables
Feature family
#Values
Visual property quantified
Intensity statistics
8
Global distribution of pixel intensities (e.g., mean, median, standard deviation, entropy)
Color histogram
19
Distribution of colors in the image
Combined first four moments
48
Mean, standard deviation, skewness and kurtosis of intensity over image regions
Multiscale histograms
24
Intensity histograms computed at several bin resolutions
Edge statistics
28
Magnitude and direction of intensity gradients (means, medians, variances, histograms, homogeneity)
Object statistics
34
Properties of segmented objects: count, area, distances, Euler number, centroid
Table 1: UDAT feature families considered in this study, the number of numerical descriptors associated with each family, and the visual properties they quantify.
Dataset
Tasks
# Images
Acquisition Setting
# Classes
Random Chance
Patch Acc. Range
Margin
Context-Bias Category
PlantVillage
Disease classification
∼ 54,000
Controlled
15
6.7%
9.1–19.1%
6.2 pp
Moderate
PlantDoc
Disease classification
2,569
Field
27
3.7%
2.0–7.1%
0.03 pp
Low
DeepWeeds
Species classification
17,509
Field
9
11.1%
51.3–55.0%
40.6 pp
High
Cassava
Disease classification
21,397
Field
5
20%
16.0–55.8%
13.6 pp
High
WeedCrop
Crop/weed classification
2,822
Field/UAV
2
50%
85.6–92.4%
41.1 pp
High
OpenSprayer
Weed classification
∼ 10,000
UAV / aerial
2
50%
58.4–80.6%
20.2 pp
High
Table 2: Summary of dataset characteristics and patch-based context-bias results reported in the previous study. “Chance Acc.” denotes random-chance accuracy, and “Patch Acc.” denotes the reported patch accuracy range.
Figure 1: Overview of the neuro-symbolic analysis pipeline for interpreting CNN hidden-neuron activations from contextual image patches.
Architecture
Mid
Mid-deep
Deep
ResNet50V2
conv3_block4_out
conv4_block6_out
conv5_block3_out
DenseNet121
conv3_block12_concat
conv4_block24_concat
relu
InceptionV3
mixed5
mixed7
mixed10
VGG16
block3_conv3
block4_conv3
block5_conv3
Table 3: Hidden layers selected to represent mid, mid-deep, and deep stages of each CNN architecture
Architecture
Depth
Layer
Initial neurons/ channels
After variance filter
After correlation ranking
Final selected
ResNet50V2
Mid
conv3_block4_out
512
384
100
50
ResNet50V2
Mid-deep
conv4_block6_out
1,024
768
100
50
ResNet50V2
Deep
conv5_block3_out
2,048
1,536
100
50
DenseNet121
Mid
conv3_block12_concat
512
384
100
50
DenseNet121
Mid-deep
conv4_block24_concat
1,024
768
100
50
DenseNet121
Deep
relu
1,024
768
100
50
Table 4: Reduction in candidate neurons across the three-stage neuron-selection procedure for the analyzed CNN layers.
Figure 2: Ontology pattern for one patch image and one UDAT sub-feature (Gabor filters, sub-feature Bin2). The patch individual is linked by hasGaborFilters_Bin2 to a value individual (value 0.1225), which is typed with the interval class for [0.105,0.150] ; the five equal-width interval classes are subclasses of the sub-feature class GaborFilters_Bin2. The same pattern is repeated for each of the seven Gabor sub-features.
Figure 3: Progressive reduction of neuron-feature evaluation cases. Percentages indicate the proportion retained relative to the initial 105,000 evaluation cases.
Dataset
Patch
All Cases
Concept Found
Constraints Parsed
MW Test Completed
Significant
Validated
Cassava
BL
4200
3219
2356
2260
1200
362
BR
4200
3184
2318
2223
1213
387
C
4200
3155
2241
2170
1131
433
TL
4200
3227
2351
2296
1236
487
TR
4200
3242
2350
2302
1266
495
OpenSprayer
BL
4200
3005
2367
2003
791
468
Table 5: Dataset-patch-wise progression of neuron-feature evaluation cases. Patch locations are denoted as BL (Bottom-Left), BR (Bottom-Right), C (Center), TL (Top-Left), and TR (Top-Right).
Layer depth
Configurations
Mean validation rate (%)
SD
Min (%)
Max (%)
Mid
25
40.21
10.38
24.49
61.83
Mid-deep
25
26.91
12.89
14.43
54.02
Deep
25
11.99
5.65
4.41
29.07
Table 6: Summary of neuron-feature validation rates across the three analyzed CNN depth levels.
Figure 4: Validation rates of context-sensitive neuron-feature pairs across different layers for each dataset-patch configuration.
Figure 5: Dataset-wise validation rates of context-sensitive neuron-feature representations across the five predefined spatial patch locations.
Patch position
Observations
Mean validation rate (%)
SD
Min (%)
Max (%)
Top-left
15
27.39
16.25
8.87
57.32
Top-right
15
29.53
16.08
6.77
56.20
Center
15
25.61
14.85
6.96
52.43
Bottom-left
15
25.25
15.97
4.41
61.83
Bottom-right
15
24.06
14.77
6.23
50.37
Table 7: Patch-wise summary of neuron-feature validation rates across datasets and analyzed layer depths.
Figure 6: Overall validation outcomes for the analyzed UDAT feature families.
Figure 7: Feature-family validation rates grouped by dataset across the analyzed patch locations, architectures, and layer depths.
Figure 8: Feature-family validation rates grouped by spatial patch position across the analyzed datasets, architectures, and layer depths.
Figure 9: Validation rates of neuron-feature pairs across feature families and CNN layer depths.
Dataset
Bias category
Mean patch accuracy
Random chance
Above-chance patch accuracy
Tested
Validated
Validation rate
Cassava
High
33.62%
20.00%
13.62%
11,251
2,164
19.23%
OpenSprayer
High
70.18%
50.00%
20.18%
9,989
2,195
21.97%
Oxford
Low
1.70%
0.98%
0.72%
11,115
3,107
27.95%
Sugarcane
Moderate
16.92%
9.1%
7.83%
10,006
4,057
40.55%
WeedCrop
High
91.06%
50.00%
41.06%
5,652
1,641
29.03%
Table 8: Dataset-level bias and neuron-feature validation statistics.
Convolutional neural networks (CNNs) learn spatial and temporal patterns using shared filters, with applications from image recognition to scientific prediction.