InvarOrd: Self-Supervised Ordinal Knowledge Distillation with Rank-Invariant Alignment for Label-Efficient Cross-Species Hyperspectral Fish Freshness Assessment
Authors: Kazi Nabiul Alam, Omenuko Joseph Nwebonyi, Pooneh Bagheri Zadeh, Akbar Sheikh-Akbari
Organizations: The authors are with the School of Built Environment, Engineering and Computing, Leeds Beckett University, Leeds, United Kingdom.
Hyperspectral models of fish freshness rarely transfer between species, and labelling a new species is slow and destructive. Freshness is also ordinal, and standard domain alignment can erase that order. We present InvarOrd, which adapts to a new species from 30 labelled cubes (5 per rank) in three stages: self-supervised spectral pretraining on unlabelled source cubes, cross-species alignment by a shrinkage covariance discrepancy computed only within matched ranks, and ordinal distillation into a compact student. The alignment term is provably invariant to the rank means, so it gives no incentive to collapse the ordinal axis. On paired salmon and cod data (800 cubes each, 16 days), a source-trained teacher reaches 1.6 to 1.9 days mean absolute error (MAE) on the other species, with quadratic weighted kappa (QWK) below zero. The 423K-parameter student, 2.54 times smaller than its teacher, attains the lowest MAE (1.476 days) and highest within 1-day accuracy (56.1%) among 14 configurations, matching the best of six ImageNet-pretrained backbones (ConvNeXt-T, 1.503 days) at 66 times fewer parameters, 1.44ms per cube on a GPU and 5s re-adaptation. InvarOrd beats target-only training by 0.29 days (p = 0.031). Among alignment objectives, rank-invariant alignment retains the most ordinal structure (embedding rank correlation 0.183, QWK 0.112), whereas global discrepancy lowers the rank correlation to 0.049 and QWK to zero at only 0.014 days higher MAE.
Fig. 1: Overview of InvarOrd, three training stages and deployment: (1) self-supervised spectral pre-training on unlabelled source-species hyperspectral cubes (salmon or cod, depending on the transfer direction); (2) rank-structured cross-species alignment using k target anchors per class; (3) ordinal knowledge distillation of the teacher’s cumulative rank distribution into a compact student; and (4) freshness prediction on unseen target-species fillets.
Model
(w,d)
Params
MACs
Role
Teacher
(64,256)
1.07 M
90.3 M
Stages 1 and 2
Student (compressed)
(40,160)
423 K
36.5 M
Stage 3, deployed
Student (same capacity)
(64,256)
1.07 M
90.3 M
Stage 3, deployed
Compression: 2.54× parameters, 2.47× MACs.
TABLE II: Teacher and student configurations. Both share the encoder of Eq. ( 3 ) with L=4 blocks and a CORAL head.
Alignment
Statistic
Pooling
Invariant to rank means
Global MMD (RBF)
all moments
global
no
Adversarial (DANN-type)
domain label
global
no
Global CORAL
covariance
global
no
Class-cond. [ 46 ]
all moments
per class
no
Rank-pooled MMD
all moments
per rank
no
Rank CORAL (ours)
covariance
per rank
yes
TABLE III: Alignment objectives compared in this work.
Stage 1
Stage 2
Stage 3
Data
US
DS , DTk
DS , DTk
Target labels
0
kK
kK (same)
Trained model
encoder, decoder
teacher
student
Objective
Eq. ( 9 )
Eq. ( 13 )
Eq. ( 20 )
Schedule
40 epochs
30 epochs
200 steps
Output
θssl
θ⋆
fs
TABLE IV: Summary of the three training stages.
Variant
SSL
Stage 2
Rank align
KD
Linv
InvarOrd (full)
✓
✓
✓
✓
✓
w/o SSL (supervised init.)
–
✓
✓
✓
✓
w/o Stage 2
✓
–
–
✓
✓
Rank-pooled MMD + KD
✓
✓
R
✓
✓
Global MMD align + KD
✓
✓
G
✓
✓
Adversarial align + KD
✓
✓
adv.
✓
✓
TABLE V: Definition of the ablation variants. ✓: term active; –: removed; R: rank-pooled MMD; G: global MMD; adv.: adversarial.
Fig. 2: Pseudo-RGB of salmon and cod fillets across the six ranks.
Fig. 3: Mean spectral response over storage day for (a) salmon and (b) cod, and (c) the salmon–cod domain gap.
Item
Setting
Species
salmon, cod; 800 cubes and 50 packs each
Storage period
16 days, binned into K=6 ranks
Cube (raw → used)
462×1282→64×322 , 400 to 1000 nm
Split
pack-disjoint, 30/10/10 packs
Target budget
k=5 per rank ( 30 cubes); sweep k∈{1,3,5,10}
Directions, seeds
2 directions ×3 seeds, n=6 pairs
TABLE VI: Datasets and evaluation protocol.
Source
Target (zero-shot)
Direction
MAE
QWK
MAE
QWK
Δ MAE
salmon → cod
1.144
0.500
1.625
−0.008
+0.481
cod → salmon
1.100
0.598
1.906
−0.044
+0.806
TABLE VII: Teacher on source and zero-shot on target.
Fig. 4: Teacher source versus zero-shot target, both directions: (a) source validation MAE during training; (b) source and zero-shot target MAE.
Linear probe
Full fine-tuning
Method
Family
Params (M)
× Params
MACs (M)
× MACs
MAE ↓
Acc±1↑
QWK ↑
MAE ↓
Acc±1↑
QWK ↑
FT − LP
ConvNeXt-T
CNN
27.8
65.7
50
1.37
1.914
47.7
0.147
1.503
52.5
0.094
−0.411
EfficientNet-B0
CNN
4.0
9.5
32
0.88
1.533
54.0
0.144
1.668
49.7
0.049
+0.135
Swin-T
ViT
27.5
65.0
42
1.15
1.584
54.6
0.223
1.918
47.3
0.004
+0.334
ResNet-50
CNN
23.5
55.6
334
9.15
1.658
49.9
0.134
1.894
46.1
−0.019
+0.236
ViT-S/16
ViT
21.7
51.3
79
2.16
1.740
51.7
0.166
1.660
53.4
0.200
−0.080
TABLE VIII: Comparison with ImageNet-pretrained backbones at k=5 , mean over directions and seeds. × Params, × MACs: relative to the compressed student. Bold: best backbone value per metric; underline: stronger regime. s → c: salmon → cod; c → s: reverse.
Backbone
Regime
× Params
× MACs
Δ MAE
ΔAcc±1
Δ QWK
ConvNeXt-T
FT
65.7
1.37
+0.027
+3.6
+0.069
EfficientNet-B0
LP
9.5
0.88
+0.057
+2.1
+0.019
Swin-T
LP
65.0
1.15
+0.108
+1.5
−0.060
ResNet-50
LP
55.6
9.15
+0.182
+6.2
+0.029
ViT-S/16
FT
51.3
2.16
+0.184
+2.7
−0.037
ViT-B/16
FT
202.8
5.51
+0.266
+3.6
−0.023
TABLE IX: Each backbone at its stronger regime relative to the compressed student. Positive Δ favours InvarOrd.
Fig. 5: Compressed student: MAE against (a) Parameters and (b) MACs.
Fig. 6: Same-capacity student: MAE against (a) Parameters and (b) MACs.
Fig. 7: Per-direction MAE of each backbone at its stronger regime; InvarOrd rightmost. Error bars: SD over seeds.
Regime
Mean MAE
Best MAE
Acc±1
QWK
Wins
Linear probe
1.703
1.533
51.2
0.171
3 / 6
Full fine-tuning
1.731
1.503
50.2
0.086
3 / 6
InvarOrd (compr.)
1.476
1.476
56.1
0.163
–
TABLE X: LP against FT over six backbones (means). Wins: backbones for which the regime gives lower MAE.
Fig. 8: LP (circle) and FT (square) MAE per backbone; vertical lines: the two InvarOrd students.
Metric
Compressed
Same capacity
Mean MAE ↓
1.476
1.486
MAE, sal → cod ↓
1.356±0.011
1.369
MAE, cod → sal ↓
1.596±0.067
1.604
Acc±1 (%) ↑
56.1
54.7
Acc±2 (%) ↑
79.9
82.0
QWK, sal → cod ↑
0.312±0.033
0.171
TABLE XI: Compressed vs. same-capacity student, k=5 ; ± : SD over seeds.
Quantity
Teacher
Compressed
Same capacity
Width, embedding (w,d)
(64,256)
(40,160)
(64,256)
Parameters
1075 K
423 K
1075 K
MACs per cube
90.3 M
36.5 M
90.3 M
Weights, 32/8-bit (MB)
4.30/1.08
1.69/0.42
4.30/1.08
Latency (batch 1)
1.46 ms
1.44 ms
1.46 ms
Compression
1.00×
2.54×
1.00×
TABLE XII: Capacity and deployment cost. Latency and wall time are measured on one NVIDIA GPU; weight sizes follow from the parameter counts, and 8-bit sizes are prospective.