InvarOrd: Self-Supervised Ordinal Knowledge Distillation with Rank-Invariant Alignment for Label-Efficient Cross-Species Hyperspectral Fish Freshness Assessment
Authors: Kazi Nabiul Alam, Omenuko Joseph Nwebonyi, Pooneh Bagheri Zadeh, Akbar Sheikh-Akbari
Organizations: The authors are with the School of Built Environment, Engineering and Computing, Leeds Beckett University, Leeds, United Kingdom.
Hyperspectral models of fish freshness rarely transfer between species, and labelling a new species is slow and destructive. Freshness is also ordinal, and standard domain alignment can erase that order. We present InvarOrd, which adapts to a new species from 30 labelled cubes (5 per rank) in three stages: self-supervised spectral pretraining on unlabelled source cubes, cross-species alignment by a shrinkage covariance discrepancy computed only within matched ranks, and ordinal distillation into a compact student. The alignment term is provably invariant to the rank means, so it gives no incentive to collapse the ordinal axis. On paired salmon and cod data (800 cubes each, 16 days), a source-trained teacher reaches 1.6 to 1.9 days mean absolute error (MAE) on the other species, with quadratic weighted kappa (QWK) below zero. The 423K-parameter student, 2.54 times smaller than its teacher, attains the lowest MAE (1.476 days) and highest within 1-day accuracy (56.1%) among 14 configurations, matching the best of six ImageNet-pretrained backbones (ConvNeXt-T, 1.503 days) at 66 times fewer parameters, 1.44ms per cube on a GPU and 5s re-adaptation. InvarOrd beats target-only training by 0.29 days (p = 0.031). Among alignment objectives, rank-invariant alignment retains the most ordinal structure (embedding rank correlation 0.183, QWK 0.112), whereas global discrepancy lowers the rank correlation to 0.049 and QWK to zero at only 0.014 days higher MAE.
Fig. 1: Overview of InvarOrd, three training stages and deployment: (1) self-supervised spectral pre-training on unlabelled source-species hyperspectral cubes (salmon or cod, depending on the transfer direction); (2) rank-structured cross-species alignment using k target anchors per class; (3) ordinal knowledge distillation of the teacher’s cumulative rank distribution into a compact student; and (4) freshness prediction on unseen target-species fillets.
Model
(w,d)
Params
MACs
Role
Teacher
(64,256)
1.07 M
90.3 M
Stages 1 and 2
Student (compressed)
(40,160)
423 K
36.5 M
Stage 3, deployed
Student (same capacity)
(64,256)
1.07 M
90.3 M
Stage 3, deployed
Compression: 2.54× parameters, 2.47× MACs.
TABLE II: Teacher and student configurations. Both share the encoder of Eq. ( 3 ) with L=4 blocks and a CORAL head.
Alignment
Statistic
Pooling
Invariant to rank means
Global MMD (RBF)
all moments
global
no
Adversarial (DANN-type)
domain label
global
no
Global CORAL
covariance
global
no
Class-cond. [ 46 ]
all moments
per class
no
Rank-pooled MMD
all moments
per rank
no
Rank CORAL (ours)
covariance
per rank
yes
TABLE III: Alignment objectives compared in this work.
Stage 1
Stage 2
Stage 3
Data
US
DS , DTk
DS , DTk
Target labels
0
kK
kK (same)
Trained model
encoder, decoder
teacher
student
Objective
Eq. ( 9 )
Eq. ( 13 )
Eq. ( 20 )
Schedule
40 epochs
30 epochs
200 steps
Output
θssl
θ⋆
fs
TABLE IV: Summary of the three training stages.
Variant
SSL
Stage 2
Rank align
KD
Linv
InvarOrd (full)
✓
✓
✓
✓
✓
w/o SSL (supervised init.)
–
✓
✓
✓
✓
w/o Stage 2
✓
–
–
✓
✓
Rank-pooled MMD + KD
✓
✓
R
✓
✓
Global MMD align + KD
✓
✓
G
✓
✓
Adversarial align + KD
✓
✓
adv.
✓
✓
TABLE V: Definition of the ablation variants. ✓: term active; –: removed; R: rank-pooled MMD; G: global MMD; adv.: adversarial.
Fig. 2: Pseudo-RGB of salmon and cod fillets across the six ranks.
Fig. 3: Mean spectral response over storage day for (a) salmon and (b) cod, and (c) the salmon–cod domain gap.
Item
Setting
Species
salmon, cod; 800 cubes and 50 packs each
Storage period
16 days, binned into K=6 ranks
Cube (raw → used)
462×1282→64×322 , 400 to 1000 nm
Split
pack-disjoint, 30/10/10 packs
Target budget
k=5 per rank ( 30 cubes); sweep k∈{1,3,5,10}
Directions, seeds
2 directions ×3 seeds, n=6 pairs
TABLE VI: Datasets and evaluation protocol.
Source
Target (zero-shot)
Direction
MAE
QWK
MAE
QWK
Δ MAE
salmon → cod
1.144
0.500
1.625
−0.008
+0.481
cod → salmon
1.100
0.598
1.906
−0.044
+0.806
TABLE VII: Teacher on source and zero-shot on target.
Fig. 4: Teacher source versus zero-shot target, both directions: (a) source validation MAE during training; (b) source and zero-shot target MAE.
Linear probe
Full fine-tuning
Method
Family
Params (M)
× Params
MACs (M)
× MACs
MAE ↓
Acc±1↑
QWK ↑
MAE ↓
Acc±1↑
QWK ↑
FT − LP
ConvNeXt-T
CNN
27.8
65.7
50
1.37
1.914
47.7
0.147
1.503
52.5
0.094
−0.411
EfficientNet-B0
CNN
4.0
9.5
32
0.88
1.533
54.0
0.144
1.668
49.7
0.049
+0.135
Swin-T
ViT
27.5
65.0
42
1.15
1.584
54.6
0.223
1.918
47.3
0.004
+0.334
ResNet-50
CNN
23.5
55.6
334
9.15
1.658
49.9
0.134
1.894
46.1
−0.019
+0.236
ViT-S/16
ViT
21.7
51.3
79
2.16
1.740
51.7
0.166
1.660
53.4
0.200
−0.080
TABLE VIII: Comparison with ImageNet-pretrained backbones at k=5 , mean over directions and seeds. × Params, × MACs: relative to the compressed student. Bold: best backbone value per metric; underline: stronger regime. s → c: salmon → cod; c → s: reverse.
Backbone
Regime
× Params
× MACs
Δ MAE
ΔAcc±1
Δ QWK
ConvNeXt-T
FT
65.7
1.37
+0.027
+3.6
+0.069
EfficientNet-B0
LP
9.5
0.88
+0.057
+2.1
+0.019
Swin-T
LP
65.0
1.15
+0.108
+1.5
−0.060
ResNet-50
LP
55.6
9.15
+0.182
+6.2
+0.029
ViT-S/16
FT
51.3
2.16
+0.184
+2.7
−0.037
ViT-B/16
FT
202.8
5.51
+0.266
+3.6
−0.023
TABLE IX: Each backbone at its stronger regime relative to the compressed student. Positive Δ favours InvarOrd.
Fig. 5: Compressed student: MAE against (a) Parameters and (b) MACs.
Fig. 6: Same-capacity student: MAE against (a) Parameters and (b) MACs.
Fig. 7: Per-direction MAE of each backbone at its stronger regime; InvarOrd rightmost. Error bars: SD over seeds.
Regime
Mean MAE
Best MAE
Acc±1
QWK
Wins
Linear probe
1.703
1.533
51.2
0.171
3 / 6
Full fine-tuning
1.731
1.503
50.2
0.086
3 / 6
InvarOrd (compr.)
1.476
1.476
56.1
0.163
–
TABLE X: LP against FT over six backbones (means). Wins: backbones for which the regime gives lower MAE.
Fig. 8: LP (circle) and FT (square) MAE per backbone; vertical lines: the two InvarOrd students.
Metric
Compressed
Same capacity
Mean MAE ↓
1.476
1.486
MAE, sal → cod ↓
1.356±0.011
1.369
MAE, cod → sal ↓
1.596±0.067
1.604
Acc±1 (%) ↑
56.1
54.7
Acc±2 (%) ↑
79.9
82.0
QWK, sal → cod ↑
0.312±0.033
0.171
TABLE XI: Compressed vs. same-capacity student, k=5 ; ± : SD over seeds.
Quantity
Teacher
Compressed
Same capacity
Width, embedding (w,d)
(64,256)
(40,160)
(64,256)
Parameters
1075 K
423 K
1075 K
MACs per cube
90.3 M
36.5 M
90.3 M
Weights, 32/8-bit (MB)
4.30/1.08
1.69/0.42
4.30/1.08
Latency (batch 1)
1.46 ms
1.44 ms
1.46 ms
Compression
1.00×
2.54×
1.00×
TABLE XII: Capacity and deployment cost. Latency and wall time are measured on one NVIDIA GPU; weight sizes follow from the parameter counts, and 8-bit sizes are prospective.
Non-destructive food quality assessment has increasingly benefited from hyperspectral imaging (HSI), which captures spectral signatures linked to biochemical changes during storage. Estimating day-wise freshness, however, remains challenging owing to strong inter-fillet variability and scarce labelled data per product. All existing deep learning approaches for HSI-based freshness prediction operate under full supervision, requiring densely annotated training sets that are costly to obtain at the individual-product level. We introduce, to the best of our knowledge, the first few-shot learning framework for HSI-based food quality estimation. Each fillet defines a distinct episodic task, and a CORAL-style ordinal prediction head captures the ranked nature of freshness progression through cumulative threshold modelling. Biologically grounded monotonicity and embedding smoothness constraints further guide predictions toward plausible trajectories. On a 16-day salmon HSI dataset under a strict unseen-fillet protocol, our method achieves a mean absolute error of 1.58 days and 2-day accuracy of 72.3% with only three labelled days per fillet, substantially outperforming scalar regression and label-distribution baselines under an identical unseen-fillet protocol.
Kazi Nabiul Alam, Pooneh Bagheri Zadeh, Akbar Sheikh-Akbari
School of Built Environment, Engineering and Computing, Leeds Beckett University, Leeds, United Kingdom.
Hyperspectral imaging (HSI) offers nondestructive assessment of fish freshness by detecting biochemical alterations across spectral bands. However, conventional deep learning approaches do not fully address the particular characteristics of HSI data, such as spectral dominance over spatial textures, ordinal label structure, and a small number of training samples. We propose SGNet (Spectral-Grouped Network), a lightweight architecture that separates spectral and spatial feature extraction using grouped convolutions and a depthwise spatial pathway. A dual attention mechanism that couples channel-wise squeeze-and-excitation with spatial gating adaptively highlights informative features. SGNet achieves 97.8% classification accuracy and 0.64 days mean absolute error (MAE) with just 4.75M parameters when tested on our newly developed 16-day refrigerator-stored salmon fillet dataset. Ablation studies validate the contribution of each component, while comparisons demonstrate a five- to eighteen-fold parameter reduction relative to ResNet-50 and Vision Transformers. Our findings indicate that domain-aware design supports precise, real-time freshness prediction for industrial implementation.
Kazi Nabiul Alam, Pooneh Bagheri Zadeh, Akbar Sheikh-Akbari
School of Built Environment, Engineering, and Computing, Leeds Beckett University. Leeds LS6 3QS, United Kingdom.
We propose contrastive order learning (ConOrd), a contrastive learning framework for ordinal regression that integrates the strengths of contrastive learning and order learning. While contrastive learning effectively leverages all samples in a batch, it typically ignores the inherent ordering among rank labels. Conversely, order learning explicitly models label ordinality but often relies on local, margin-based comparisons, limiting its ability to capture global ordinal structure. ConOrd addresses these limitations by introducing a contrastive order loss with soft affinity and disparity weights based on rank differences, enabling fine-grained modeling of ordinal relationships across all sample pairs within a batch. Extensive experiments on a range of ordinal regression tasks, including facial age estimation, blind image quality assessment, and blind video quality assessment, demonstrate that ConOrd consistently achieves state-of-the-art performance and generalizes well across diverse ordinal regression scenarios. The source code is available at https://github.com/cwlee00/ConOrd.
Chaewon Lee, BeomJun Shim, Kwang Pyo Choi +1
School of Electrical Engineering, Korea University, Seoul, Korea · Samsung Electronics, Seoul, Korea