A Benchmark Study of Neural Network Compression Methods for Hyperspectral Image Classification

Sai Shi

cs.CVMar 5, 2026

A Benchmark Study of Neural Network Compression Methods for Hyperspectral Image Classification

Authors: Sai Shi

Organizations: Department of Computer and Information Science, Temple University, Philadelphia, USA

Abstract

Deep neural networks have achieved strong performance in image classification tasks due to their ability to learn complex patterns from high-dimensional data. However, their large computational and memory requirements often limit deployment on resource-constrained platforms such as remote sensing devices and edge systems. Network compression techniques have therefore been proposed to reduce model size and computational cost while maintaining predictive performance. In this study, we conduct a systematic evaluation of neural network compression methods for a remote sensing application, namely hyperspectral land cover classification. Specifically, we examine three widely used compression strategies for convolutional neural networks: pruning, quantization, and knowledge distillation. Experiments are conducted on two benchmark hyperspectral datasets, considering classification accuracy, memory consumption, and inference efficiency. Our results demonstrate that compressed models can significantly reduce model size and computational cost while maintaining competitive classification performance. These findings provide insights into the trade-offs between compression ratio, efficiency, and accuracy, and highlight the potential of compression techniques for enabling efficient deep learning deployment in remote sensing applications.

Figures & tables

Panel of Figure 1 — Figure 1: Random selection vs. spatially disjoint samples on the IP and UP dataset, considering the same number of samples per class

Figure 2: The implementation details of spectral model (CNN1D) , spatial model (CNN2D), and spatial-spectral model (CNN3D)

Figure 3: A typical three-stage network pruning pipeline

Figure 4: Three types of fine-tuning schemes: (a) One-shot pruning: prune and retrain the model only once (b) Iterative pruning: prune layer by layer, and retrain the model before pruning the next layer (c) Multi-pass pruning: prune and retrain the whole network, then repeat this process until satisfied

Approach	Paper	Network	Size Reduction	Datasets	Application
	[ 93 ]	MobileNet, ResNet	80%	OPTIMAL-31	Scene Classification
	[ 108 ]	2D-CNN	97%	Indian Pines, Salinas, Xiongan	Hyperspectral Classification
	[ 57 ]	VGG, ResNet	70%	AID, NWPU45, PatternNet, WHU19	Scene Classification
	[ 92 ]	2D-CNN	80%	Bijie	Landslides Recognition
Pruning	[ 32 ]	AlexNet, VGG	-	UC Merced, NWPU-RESISC45	Scene Classification
	[ 67 ]	3D-CNN	-	Indian Pines, Pavia University	Hyperspectral Classification

Table 1: Existing work of neural network compression in remote sensing applications

Quantization Modes	Data Requirements	Inference Latency	Inference Accuracy Loss
Dynamic Quantization	No Data	Fast	Smallest
Static Quantization	Unlabeled data	Fastest	Small
Quantization Aware Training	Labeled data	Fastest	Smallest

Table 2: Different quantization modes

Figure 5: Different distillations modes. Blue: networks are learned before distillation; Yellow: networks are learned during distillation

		INDIAN PINES		PAVIA UNIVERSITY
	# trainable parameters	Disjoint	Random	Disjoint	Random
RF	-	66.1	79.4	68.5	88.8
MLR	-	78.5	83.5	74.6	90.5
SVM	-	69.9	81.0	79.8	95.8
MLP	31,047	82.4	88.9	82.5	92.5
LSTM	102,416	80.7	88.4	78.1	96.1

Table 3: Top-1 accuracy for baseline models (c: number of channels after PCA)

	Conv1	Conv2	fc1	fc2	Total	Memory (MB)
MLP	-	-	-	-	31,047	0.13
CNN1D	-	-	-	-	72,416	0.29
CNN2D	50,050	125,100	250,100	909	426,866	1.71
Prune (90%)	15,015	11,280	22,530	496	49,321	0.60
Prune (95%)	10,010	5,020	10,020	336	25,386	0.31
Prune (98%)	5,005	1,260	2,510	176	8,951	0.12

Table 4: Number of parameters in each layer and memory consumption for original and pruned models

	Conv1	Conv2	fc1	fc2
	# of filters		# of neurons
CNN2D	50	100	2,500	100
Prune (90%)	15	30	750	30
Prune (95%)	10	20	500	20
Prune (98%)	5	10	250	10

Table 5: Number of remaining filters in Conv layers and neurons in FC layers after pruning

		INDIAN PINES				PAVIA UNIVERSITY
		Disjoint		Random		Disjoint		Random
Methods (Pruning ratio: 90%)		Top-1	Top-5	Top-1	Top-5	Top-1	Top-5	Top-1	Top-5
	MLP	82.4	98.6	88.9	99.7	82.5	99.5	92.5	99.9
	CNN-1D	82.5	97.9	89.4	99.5	82.3	99.7	93.5	99.9
Baselines	CNN-2D	86.3	98.7	99.5	99.9	83.2	99.9	99.3	100
Scratch	Pruned CNN2D	82.8	97.6	99.2	99.9	77.7	99.7	99.6	99.9

Table 6: Top-1 and Top-5 accuracy of model using pruning methods (90% reduction)

		INDIAN PINES				PAVIA UNIVERSITY
		Disjoint		Random		Disjoint		Random
Methods (Pruning ratio: 95%)		Top-1	Top-5	Top-1	Top-5	Top-1	Top-5	Top-1	Top-5
	MLP	82.4	98.6	88.9	99.7	82.5	99.5	92.5	99.9
	CNN-1D	82.5	97.9	89.4	99.5	82.3	99.7	93.5	99.9
Baselines	CNN-2D	86.3	98.7	99.4	99.9	83.2	99.9	99.3	100
Scratch	Pruned CNN2D	82.1	97.1	99.3	100	74.1	99.6	99.0	100

Table 7: Top-1 and Top-5 accuracy of model using pruning methods (95% reduction)

		INDIAN PINES				PAVIA UNIVERSITY
		Disjoint		Random		Disjoint		Random
Methods (Pruning ratio: 98%)		Top-1	Top-5	Top-1	Top-5	Top-1	Top-5	Top-1	Top-5
	MLP	82.4	98.6	88.9	99.7	82.5	99.5	92.5	99.9
	CNN-1D	82.5	97.9	89.4	99.5	82.3	99.7	93.5	99.9
Baselines	CNN-2D	86.3	98.7	99.4	99.9	83.2	99.9	99.3	100
Scratch	Pruned CNN2D	78.2	96.4	98.9	100	77.2	99.9	98.1	100

Table 8: Top-1 and Top-5 accuracy of model using pruning methods (98% reduction)

		INDIAN PINES				PAVIA UNIVERSITY
		Disjoint		Random		Disjoint		Random
Pruning Ratio	Fine-tuning Strategy	Top-1	Top-5	Top-1	Top-5	Top-1	Top-5	Top-1	Top-5
	I	84.6	98.1	99.3	99.9	82.0	99.6	99.4	100
	II	86.1	97.5	99.6	99.9	81.7	99.7	99.5	100
90%	III	81.5	97.9	99.2	99.9	81.3	98.9	98.7	99.9
	I	86.1	98.4	99.4	99.9	83.0	99.3	99.3	100

Table 9: Different fine-tuning strategy using L1-norm pruning method (Strategy I: prune once and retrain; Strategy II: prune and retrain iteratively; Strategy III: Multi-pass scheme)

			INDIAN PINES		PAVIA UNIVERSITY
	Memory (MB)	Inference Latency (ms/sample)	Disjoint	Random	Disjoint	Random
MLP	0.13	0.34	82.4	88.9	82.5	92.5
CNN1D	0.29	0.62	82.5	89.4	82.3	93.5
CNN2D	1.71	3.28	86.3	99.4	83.2	99.3
Dynamic Quantization	0.96	2.58	86.5	99.4	83.2	99.6
Static Quantization	0.44	0.70	86.7	99.4	83.3	99.6

Table 10: Neural network performance using different quantization modes (converted from float32 into int8)

		INDIAN PINES				PAVIA UNIVERSITY
		Disjoint		Random		Disjoint		Random
Methods (student network: 90%)		Top-1	Top-5	Top-1	Top-5	Top-1	Top-5	Top-1	Top-5
	MLP	82.4	98.6	88.9	99.7	82.5	99.5	92.5	99.9
	CNN-1D	82.5	97.9	89.4	99.5	82.3	99.7	93.5	99.9
Baselines	CNN-2D	86.3	98.7	99.5	99.9	83.2	99.9	99.3	100
Scratch	Pruned CNN2D	82.8	97.6	99.2	99.9	77.7	99.7	99.6	99.9

Table 11: Top-1 and Top-5 accuracy of student model using knowledge distillation methods (90% reduction)

		INDIAN PINES				PAVIA UNIVERSITY
		Disjoint		Random		Disjoint		Random
Methods (student network: 95%)		Top-1	Top-5	Top-1	Top-5	Top-1	Top-5	Top-1	Top-5
	MLP	82.4	98.6	88.9	99.7	82.5	99.5	92.5	99.9
	CNN-1D	82.5	97.9	89.4	99.5	82.3	99.7	93.5	99.9
Baselines	CNN-2D	86.3	98.7	99.5	99.9	83.2	99.9	99.3	100
Scratch	Pruned CNN2D	82.1	97.1	99.3	100	74.1	99.6	99.0	100

Table 12: Top-1 and Top-5 accuracy of student model using knowledge distillation methods (95% reduction)

		INDIAN PINES				PAVIA UNIVERSITY
		Disjoint		Random		Disjoint		Random
Methods (student network: 98%)		Top-1	Top-5	Top-1	Top-5	Top-1	Top-5	Top-1	Top-5
	MLP	82.4	98.6	88.9	99.7	82.5	99.5	92.5	99.9
	CNN-1D	82.5	97.9	89.4	99.5	82.3	99.7	93.5	99.9
Baselines	CNN-2D	86.3	98.7	99.5	99.9	83.2	99.9	99.3	100
Scratch	Pruned CNN2D	78.2	96.4	98.9	100	77.2	99.9	98.1	100

Table 13: Top-1 and Top-5 accuracy of student model using knowledge distillation methods (98% reduction)

Explore similar work

Hybrid Compression: Integrating Pruning and Quantization for Optimized Neural Networks
Jun 22, 2026Minh-Loi Nguyen, Long-Bao Nguyen, Van-Hieu Huynh +2Model Compression Efficient Neural Network Inference
University of Science, Ho Chi Minh city, Vietnam · Vietnam National University, Ho Chi Minh city, Vietnam
MSCM-net: A hyperspectral image classiffcation method based on multi-scale convolution and Mamba
Jul 30, 2026Jianjun Chen, Linlin Wang, Lifang Chang +4Channel Attention Multi-Scale Feature Fusion
Qingdao University of Technology, Qingdao, 266520, China · Nanyang Technological University, Singapore, 639798, Singapore
Vision Transformers and Convolutional Neural Networks for Land Use Scene Classification
May 20, 2026Arun D. KulkarniRemote Sensing Image Understanding Vision Transformer
Computer Science Department, University of Texas at Tyler, Tyler, TX 75799, USA

Jun 22, 2026J/K move · Enter open · S save

Hybrid Compression: Integrating Pruning and Quantization for Optimized Neural Networks

Minh-Loi Nguyen, Long-Bao Nguyen, Van-Hieu Huynh, Minh-Triet Tran+1

University of Science, Ho Chi Minh city, Vietnam · Vietnam National University, Ho Chi Minh city, Vietnam

Model Compression Efficient Neural Network Inference Neural Network Compression

Open paper AskPDF arXiv

Deep neural networks have witnessed remarkable advancements in recent years and have become integral to various applications. However, alongside these developments, training and deployment of neural network models on embedding and edge devices face significant challenges due to limited memory and computational resources. These problems can be addressed with deep neural network compression, which involves a trade-off between model size and performance. In this paper, we propose a novel method for model compression through two phases. First, we utilize model compression techniques, such as pruning and quantization, to significantly reduce the model size. Then, we use Mixture of Experts to route the previously compressed models to enhance performance while maintaining a balance in inference efficiency. MoEs consist of multiple expert models (i.e., compressed models) that are moderately sized and deliver stable performance. Experimental results on several benchmark datasets show that our method successfully compresses CNN models which achieves substantial reductions in FLOPs and parameters with a negligible accuracy drop.

Abstract

Explore similar work

Hybrid Compression: Integrating Pruning and Quantization for Optimized Neural Networks

MSCM-net: A hyperspectral image classiffcation method based on multi-scale convolution and Mamba

Vision Transformers and Convolutional Neural Networks for Land Use Scene Classification