Benchmarking Model Compression Techniques Across Heterogeneous Edge Platforms: A Predictive Maintenance Use Case for Rotating Machinery
Authors: Matheus Calheira Guimarães de Oliveira, Marco Aurélio da Silva Cruz, Regis Cardoso, Lucas Cambuim, Rafael Macieira, Arthur Paim, Helaine Neves, Alex Santos, Lilian Guarieiro, Jean Schmith, Prashant Kumar, Erick G. Sperandio Nascimento
Organizations: SENAI CIMATEC University, Salvador, BA, Brazil · Surrey Institute for People-Centred AI, School of Computer Science and Electronic Engineering, Faculty of Engineering and Physical Sciences, University of Surrey, Guildford, UK · SENAI Institute of Innovation in Embedded Systems, Florianópolis, SC, Brazil · UNISINOS University, São Leopoldo, RS, Brazil · Department of Statistics and Informatics (DEINFO), Federal Rural University of Pernambuco (UFRPE), Recife, PE, Brazil · Integrated Center for Advanced Security Systems, Center for Advanced Studies and Systems of Recife (CESAR), Recife, PE, Brazil · SENAI Institute for Innovation in Sensing Systems, São Leopoldo, RS, Brazil · Global Centre for Clean Air Research (GCARE), School of Sustainability, Civil and Environmental Engineering and Physical Sciences, University of Surrey, Guildford, UK · Institute for Sustainability, University of Surrey, Guildford, UK
To enable Industry 5.0, where autonomous systems are widely adopted and integrated with people to promote sustainable development in industries, it is imperative to adapt complex deep learning models to operate efficiently on resource-constrained hardware for edge inference. However, adapting these models remains challenging due to the hardware heterogeneity of edge computing in industrial settings, performance degradation caused by model compression, and the lack of clear guidance on which compression techniques are practical and effective for applications such as predictive maintenance. To address these challenges, this paper presents a comprehensive comparative analysis of four state-of-the-art model compression techniques applied to a high-performance convolutional neural network for predictive maintenance using a real-world dataset of rotating machinery, equipment typically used in industry. The compressed models were evaluated across a heterogeneous set of nine computing platforms, ranging from highly constrained edge computing devices to general purpose and high-performance processors, reflecting the diversity and fragmentation typical of industrial edge environments. By jointly prioritizing classification performance, inference speed, and compression ratio, we found that compression using knowledge distillation emerged as the most effective approach. The model had more than 90% speedup across all devices while maintaining a 91.22% balanced F-score, with a very small drop in predictive performance relative to the uncompressed model, along with a 98.9% size reduction. Through successful deployment across multiple edge platforms and a systematic comparison of compression strategies, we demonstrated a clear path to achieving Industry 5.0 targets, drastically cutting computational costs and, consequently, energy consumption without sacrificing predictive performance.
Figures & tables
Figure 1: Methodological summary for adapting Deep Neural Networks to diverse computing devices.
Figure 4: Student architecture of the distilled model (D-PdMCNN).
Device
Processor
Cores
Architecture
Frequency
Memory
Desktop
Intel Core i5-10400F
6
x86-64
2.90 GHz
16 GB
Laptop
Intel Core i5-11300H
4
x86-64
2.60 GHz
8 GB
HPC
Intel Xeon Gold 6148
20
x86-64
2.40 GHz
768 GB
STM32M
Dual Arm Cortex-A7
2
ARMv7-A
800 MHz
512 MB
Raspberry
Quad-core ARM Cortex-A72
4
ARMv8-A
1.5 GHz
8 GB
J. Nano
Quad-core ARM Cortex-A57
4
ARMv8-A
1.43 GHz
4 GB
Table 1: Devices used for the comparative study of DL model compression techniques.
Figure 5: Bar plot presenting the model size reduction achieved across different compression techniques relative to the baseline.
Device
Mode
Baseline
QDRI8
QF16
QI8
KD
Desktop
CPU
14.69
17.33 ↑ (18.0%)
15.13 ↑ (3.0%)
7.99 ↓ (45.6%)
0.34 ↓ (97.7%)
Laptop
CPU
7.77
10.98 ↑ (41.4%)
7.60 ↓ (2.1%)
5.81 ↓ (25.2%)
0.20 ↓ (97.5%)
HPC
CPU
7.34
12.44 ↑ (69.5%)
7.38 ↑ (0.6%)
7.47 ↑ (1.7%)
0.34 ↓ (95.3%)
STM32M
CPU
784.95
499.38 ↓ (36.4%)
772.23 ↑ (1.6%)
393.08 ↓ (49.9%)
9.96 ↓ (98.7%)
Raspberry
CPU
81.77
57.22 ↓ (30.0%)
83.68 ↑ (2.3%)
54.97 ↓ (32.8%)
1.85 ↓ (97.7%)
J. Nano
CPU
148.52
98.55 ↓ (33.6%)
150.65 ↑ (1.4%)
93.05 ↓ (37.4%)
2.58 ↓ (98.3%)
Table 2: Average inference time (ms) for each device under the different compression techniques. ↑ indicates percentage growth and ↓ percentage reduction relative to the baseline.
Model
F1
Prec.
Recall
Spec.
Acc.
PdM-CNN ∗
0.9481
0.9590
0.9391
0.9903
0.9852
QDRI8
0.9485 ↑ (0.04%)
0.9577 ↓ (0.13%)
0.9406 ↑ (0.16%)
0.9903
0.9852
QF16
0.9481
0.9590
0.9391
0.9903
0.9852
QI8
0.9495 ↑ (0.15%)
0.9612 ↑ (0.23%)
0.9401 ↑ (0.11%)
0.9906 ↑ (0.03%)
0.9856 ↑ (0.04%)
D-PdMCNN (KD)
0.9122 ↓ (3.79%)
0.9176 ↓ (4.32%)
0.9102 ↓ (3.08%)
0.9822 ↓ (0.82%)
0.9717 ↓ (1.37%)
D-PdMCNN ∗∗
0.7298 ↓ (23.03%)
0.8328 ↓ (13.16%)
0.7313 ↓ (22.12%)
0.9588 ↓ (3.18%)
0.9364 ↓ (4.95%)
Table 3: Comparative performance metrics of models across multiple hardware devices. Results were identical across all devices (Desktop, Laptop, HPC, STM32M, Raspberry, J. Nano, J. Orin Nano, Arduino and STM32H); Arduino and STM32H ran only the KD model. ∗ Baseline model has no compression. ∗∗ D-PdMCNN trained from scratch with randomly initialized weights. Prec. = precision, Spec. = specificity, Acc. = accuracy.
Hardware Heterogeneity
Compression Techniques
Models and Performance
Ref.
#Dev
MCU
SBC (Acc.)
Proc
#Tech
KD
QI8
QF16
QDRI8
Prun
Clust
Compr%
PerfLoss%
DL Model(s)
[ 13 ]
6
4
1(0)
1
3 ∧
–
✓
–
–
✓
–
75–87.5%
5–24% acc
2 LSTMs
[ 14 ]
2
2
0(0)
0
1
–
–
–
–
✓
–
55.5–99.7%
0.72–3.25% acc
6 models
[ 15 ]
2
1
1(0)
0
1
–
–
–
–
–
✓
61.2–85.7%
0–89.8% acc
7 models
[ 16 ]
2
0
1(1)
1
1
✓
–
–
–
–
–
96.6%
0.44–1.61% acc
6 models
[ 17 ]
1
1
0(0)
0
3
–
✓
–
–
✓
✓
14.2–74.6%
2.4–16.6% RMSE
LSTM, CNN
Table 4: Comparison with related works: hardware diversity, compression technique coverage, and the trade-off between performance and compression rate achieved by the proposed model, D-PdMCNN. #Dev = number of devices; MCU = microcontroller; SBC (Acc. Edge) = single-board computers (number of which are accelerated edge devices); Proc = consumer and high-performance processors; #Tech = number of techniques; KD = knowledge distillation; QI8 = 8-bit integer quantization; QF16 = Float16 quantization; QDRI8 = dynamic range quantization; Prun = weight pruning; Clust = weight clustering; Compr% = compression rate; PerfLoss% = performance loss vs. baseline; Acc = accuracy. ∧ Used hard sigmoid activation functions. ∧∧ Used depthwise separable convolution.
Deploying large language models (LLMs) on Industrial Internet of Things (IIoT) edge devices demands extreme compression, yet existing structured pruning methods collapse at high compression ratios due to one-shot importance estimation, and their cross-architecture behavior remains unpredictable. This article presents a cascaded multi-granularity pruning framework that removes layers, attention heads, and feed-forward channels in coarse-to-fine order, with lightweight low-rank recovery between stages to re-estimate component importance. An information-theoretic analysis motivates this ordering, and the Structural Independence Assumption (SIA) is formalized as a checkable condition predicting whether per-component pruning criteria are reliable for a given architecture: Multi-Head Attention (MHA)+GELU designs satisfy the SIA, whereas Grouped Query Attention (GQA)+SwiGLU designs violate it. On bearing fault diagnosis spanning 88M to 6.25B-parameter models, the framework extends achievable compression to 13.8 times on MHA+GELU architectures with 83.82% accuracy (+3.70 percentage points (pp) over the strongest baseline), while exposing a ~74pp accuracy collapse on GQA+SwiGLU architectures that violate the SIA. Deployed on an industrial slewing bearing fault diagnosis platform with NVIDIA DGX Spark, compressed models reduce inference latency by up to 67.2% and peak memory by 62.5%, demonstrating viability for IIoT edge inference.
Jinghan Wang, Yanjun Chen, Wei Zhang +3
School of Mechanical and Electrical Engineering, Harbin Institute of Technology, Harbin 150001, China · School of Information Science and Technology, Eastern Institute of Technology, Ningbo, China
Edge devices are increasingly utilized for deploying deep learning applications on embedded systems. The real-time nature of many applications and the limited resources of edge devices necessitate latency-targeted neural network compression. However, measuring latency on real devices is challenging and expensive. Therefore, this letter presents a novel and efficient framework, named EvoLP, to accurately predict the inference latency of models on edge devices. This predictor can evolve to achieve higher latency prediction precision during the network compression process. Experimental results demonstrate that EvoLP outperforms previous state-of-the-art approaches by being evaluated on three edge devices and four model variants. Moreover, when incorporated into a model compression framework, it effectively guides the compression process for higher model accuracy while satisfying strict latency constraints. We open source EvoLP at https://github.com/ntuliuteam/EvoLP.
Shuo Huai, Hao Kong, Shiqing Li +5
School of Computer Science and Engineering, Nanyang Technological University, Singapore · HP-NTU Digital Manufacturing Corporate Lab, Nanyang Technological University, Singapore · HP Inc., Palo Alto, California, USA
Edge deployment is often the preferred solution for industrial machine vision systems when low latency, data security, or limited connectivity are critical requirements. Several frameworks are available to optimise inference on edge devices; however, relatively few studies have systematically compared their inference-time performance under industrial deployment conditions. In this work, we present a comparative study of four widely used approaches for machine vision inference in industrial settings: plain PyTorch, ONNX Runtime, OpenVINO, and TensorRT. The evaluation focuses on inference time, covers several CPU- and GPU-based hardware platforms, and includes both conventional convolutional neural networks and a transformer-based vision model. For the hardware platforms and models evaluated, the results show that OpenVINO achieves the lowest inference time on CPUs, while TensorRT achieves the lowest inference time on GPUs. However, TensorRT does not outperform plain PyTorch for the transformer-based model considered in this study.
Miguel Gomez Fernandez, David Castro Boga, Roi Mendez-Rial +1
Smart Systems and Smart Manufacturing (S3M) AIMEN O Porri˜no, Spain