Attribution methods are commonly validated through convergence across methods and consistency with domain knowledge, yet neither criterion establishes whether identified features are necessary for a task. This limitation is particularly relevant in multispectral remote sensing, where overlapping spectral information makes individual bands potentially substitutable. We investigate this gap in crop-type classification using GradientSHAP, Shapley sampling, and Occlusion on a U-TAE segmentation model trained on Sentinel-2 time series from the PASTIS benchmark. All three methods identify vegetation-sensitive spectral regions consistent with domain knowledge, while Shapley and Occlusion exhibit strong ordering sensitivity in deletion and insertion tests. However, these properties do not reliably predict retraining performance. Aggregate attribution consensus masks substantial per-class disagreement, with pairwise mean Spearman correlations ranging from 0.09 to 0.79. Models retrained from scratch using the five lowest-ranked bands perform comparably to or better than those using the five highest-ranked bands. Furthermore, attribution rankings from an independently trained model with identical configuration and seed identify a different top-five subset that yields better retraining performance. Analysis of all 252 five-band subsets reveals consistently low effective dimensionality (1.35 to 2.62), suggesting substantial spectral redundancy. These findings demonstrate that plausible and apparently faithful attribution rankings do not necessarily identify task-relevant bands for improved retrained performance, highlighting the limitations of band-level feature attribution in spectrally redundant input spaces.
Figures & tables
Figure 1: Two runs of one training configuration (same code and seed, equal accuracy) give Shapley rankings that differ by a single band swap (outlined). Both show order-sensitive frozen-model behaviour under the evaluated checks, yet retraining on the top-5 is significantly worse than on the bottom-5 for Run A and significantly better for Run B (mean and 95% CI over five seeds). Orange: top-5; blue: bottom-5.
Model
mIoU
Macro F1
Weighted F1
Garnot & Landrieu (5-fold) ‡
.631
–
–
Ours (18-cl., 5-fold)
.680±.010
– †
.897±.004
Ours (18-cl., fixed split)
.674±.012
.789±.009
.890±.006
Ours (7-class)
.913
.954
.967
Table 1: Performance of our U-TAE reproduction on PASTIS. The 18-class setting uses all 18 crop classes and all 10 spectral bands. The 7-class setting uses the selected seven crop classes with all remaining classes relabelled as background. Metrics are computed over crop classes. † Not reported; see the fixed-split row. ‡ Includes background in mIoU, so not directly comparable.
Figure 2: Per-class band importance for the 7-class model from GradientSHAP (left), Occlusion (centre), and Shapley sampling values (right), each row normalised to its largest magnitude; all three are computed on Run B. The methods exhibit different class-specific importance patterns despite their aggregate agreement. A star marks the top band for each crop class.
Method
1st
2nd
3rd
4th
5th
GradientSHAP
B06
B04
B05
B11
B07
Occlusion
B07
B11
B05
B08
B06
Shapley sampling
B07
B11
B08
B04
B05
Table 2: Top-5 bands per attribution method for the 7-class model (Run B), ranked by descending importance.
Input
Method pair
Mean ρ
Top match
Raw
GradientSHAP vs. Occlusion
0.16
1/7
Raw
Shapley vs. Occlusion
0.78
4/7
Raw
GradientSHAP vs. Shapley
0.09
1/7
PCA-10
GradientSHAP vs. Occlusion
0.61
4/7
PCA-10
Shapley vs. Occlusion
0.79
6/7
PCA-10
GradientSHAP vs. Shapley
0.57
3/7
Table 3: Mean per-class Spearman rank correlation between attribution-method band rankings, and fraction of classes where compared methods agree on the single top band, for the 7-class setting. All rankings from Run B (for PCA-10, Run B of the PCA model).
Zero
Mean
Marginal
Ranking
Del
Ins
Del
Ins
Del
Ins
Run B
Shapley
.074
.641
.225
.472
.250
.386
Occlusion
.079
.556
.226
.458
.242
.411
GradientSHAP
.260
.176
.299
.330
.286
.324
Random
.245
.246
.343
.352
.324
.326
Table 4: Deletion ( ↓ ) and insertion ( ↑ ) AUC for the 7-class model under three removal baselines: zero-filling, the per-band mean, and a marginal sample (the band taken from another patch). Each run is compared with its own random control.
Figure 3: Deletion and insertion curves for the 7-class model (Run B), in the ranked order (left) and with the order reversed so the least important band is removed or inserted first (right). The reversed curves invert relative to the standard ordering, confirming order-sensitivity on the frozen model.
Band subset
mIoU
Macro F1
Weighted F1
Full ( all 10 )
.9106 ± .0024
.9528 ± .0013
.9662 ± .0016
Top-5
.8893 ± .0044
.9407 ± .0026
.9591 ± .0018
Bottom-5
.9047 ± .0027
.9496 ± .0015
.9634 ± .0013
Random-5
.9021 ± .0056
.9481 ± .0031
.9626 ± .0020
Top-5 ∗
.9014 ± .0023
.9477 ± .0013
.9627 ± .0010
Bottom-5 ∗
.8952 ± .0032
.9443 ± .0018
.9603 ± .0014
Table 5: Retraining U-TAE on 7-class band subsets. Top-5 and Bottom-5 are the five highest and lowest ranked bands of Run A’s Shapley ranking; B03 is the single lowest-ranked band; Random-5 is an arbitrary five-band subset. Results are mean ± standard deviation across five random seeds (50 epochs). PCA: a single model trained for 91 epochs, as the attribution model (0.9131 on raw bands). ∗ Run A’s values ranked by magnitude, three seeds.
Figure 4: Retraining U-TAE on ranked band subsets (7-class unless noted). (a) Test mIoU over five seeds per arm (dots; hollow for Run B), with mean (tick) and 95% CI (bar), for subsets chosen by Run A’s and Run B’s Shapley rankings. (b) Top- k minus bottom- k mIoU with 95% CI; rows use Run A’s ranking except the 18-class row (that model’s own) and the last (Run B’s). (c) Twenty random five-band subsets and the three named subsets at seed 42, ranked; the grey band is ±1 seed SD around the median.
Figure 5: Global Shapley rank of each band for Run A and Run B, which share configuration and seed and have the same accuracy. Highlighted bands cross the top-5 boundary.
Figure 6: (a) Participation ratio of every k -band subset (median and range) on cloud-free crop pixels and on unfiltered pixels; the dashed line is the ceiling for fully independent bands. (b) Macro-F1 of a pixel-level gradient-boosted classifier over all subsets of each size, with Run A’s Shapley top-5 and bottom-5 marked. (c) Rank of the 23 retrained five-band subsets under the pixel classifier and after U-TAE retraining.