Deep Hierarchical Knowledge Loss for Fault Intensity Diagnosis

Yu Sha; Shuiping Gou; Bo Liu; Haofan Lu; Ningtao Liu; Jiahui Fu; Horst Stoecker; Domagoj Vnucec

eess.ASApr 8, 2026

Deep Hierarchical Knowledge Loss for Fault Intensity Diagnosis

Authors: Yu Sha, Shuiping Gou, Bo Liu, Haofan Lu, Ningtao Liu, Jiahui Fu, Horst Stoecker, Domagoj Vnucec,

Organizations: School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen 518172, China · Frankfurt Institute for Advanced Studies, Frankfurt am Main 60438, Germany · School of Artificial Intelligence, Xidian University, Xi’an 710126, China · School of Computer, Luoyang Institute of Science and Technology, Luoyang 471023, China ·

Abstract

Fault intensity diagnosis (FID) plays a pivotal role in intelligent manufacturing while neglecting dependencies among target classes hinders its practical deployment. This paper introduces a novel and general framework with deep hierarchical knowledge loss (DHK) to achieve hierarchical consistent representation and prediction. We develop a novel hierarchical tree loss to enable a holistic mapping of same-attribute classes, leveraging tree-based positive and negative hierarchical knowledge constraints. We further design a focal hierarchical tree loss to enhance its extensibility and devise two adaptive weighting schemes based on tree height. In addition, we propose a group tree triplet loss with hierarchical dynamic margin by incorporating hierarchical group concepts and tree distance to model boundary structural knowledge across classes. The joint two losses significantly improve the recognition of subtle faults. Extensive experiments are performed on four real-world datasets from various industrial domains (three cavitation datasets from SAMSON AG and one publicly available dataset) for FID, all showing superior results and outperforming recent state-of-the-art FID methods.

Figures & tables

Figure 1. Schematic diagrams for visualising different mappings. Each rectangular box represents an operation, which is a filter in the CNN. Different coloured ellipse boxes of the same colour scheme indicate knowledge associations.

Figure 2. Evolution schematic of deep hierarchical knowledge loss (DHK), comprising hierarchical tree loss

{\mathcal{L}^{HT}}

with two adaptive weighting schemes and group tree triplet loss

{\mathcal{L}^{GTT}}

with hierarchical dynamic margin.

Panel of Figure 3 — Figure 3. Visualisation of the learned deep feature distribution of vanilla ResNet34, HKG+ResNet34 and DHK+ResNet34 via t-SNE on Cavitation-Short.

Dataset		Cavitation-Short				Cavitation-Long				Cavitation-Noise
Methods	Image size	Acc	Pre	Rec	F1	Acc	Pre	Rec	F1	Acc	Pre	Rec	F1
ResNet18	${256^{2}}$	87.57	90.25	77.16	77.95	90.83	86.31	90.75	88.21	98.94	98.94	98.94	98.94
ResNet34	${256^{2}}$	88.57	91.47	76.22	74.00	91.88	87.60	91.86	89.42	98.81	98.82	98.81	98.81
ResNet50	${256^{2}}$	84.43	86.97	71.70	66.83	91.45	87.09	91.25	88.89	98.19	98.20	98.19	98.19
VGG11	${256^{2}}$	84.29	62.31	70.10	65.95	80.78	74.46	81.11	76.69	97.81	97.82	97.81	97.81
VGG13	${256^{2}}$	80.71	58.96	66.85	62.60	81.56	75.21	82.00	77.50	98.19	98.19	98.19	98.19

Table 1. Results of different backbones on three real-world cavitation datasets averaged over three runs. We compare different representation learning methods including CNNs, LNNs and Transformers. According to our previous research ( Sha et al., 2022 ) , the sliding window with a window size of 466944 and a step size of 466944.

Dataset		Cavitation-Short				Cavitation-Long				Cavitation-Noise
Methods	Image size	Acc	Pre	Rec	F1	Acc	Pre	Rec	F1	Acc	Pre	Rec	F1
LiftingNet	${256^{2}}$	85.29	82.75	75.89	73.95	88.43	86.31	87.97	87.05	95.86	94.62	95.84	95.20
MIPLCNet	${256^{2}}$	86.57	83.38	77.71	75.00	89.14	87.06	88.18	87.55	96.57	95.78	96.61	96.18
ResNet-APReLU	${256^{2}}$	86.86	84.04	77.89	75.56	90.71	88.83	90.91	89.70	97.29	96.77	97.49	97.12
LSTM-RDRN	${256^{2}}$	87.71	85.42	78.50	76.72	91.14	89.36	91.55	90.32	98.71	98.40	98.53	98.46
BCNN	${256^{2}}$	81.71	78.63	72.25	70.03	85.71	82.42	85.09	83.56	92.14	89.75	92.71	91.03

Table 2. Results of different comparison methods on three real-world cavitation datasets averaged over three runs. Non-hierarchical and hierarchical methods from fault intensity diagnosis are used as comparative methods.

Dataset	Cavitation-Short				Cavitation-Long				Cavitation-Noise
Methods	cho	con	inc	non	cho	con	inc	non	cho	con	inc	non
ResNet34	100.0	96.11	8.75	100.0	92.21	92.24	92.17	90.84	98.50	98.75	99.00	99.00
VGG11	85.71	95.00	0.00	99.67	81.04	80.33	81.93	81.14	97.75	98.00	97.25	98.25
DenseNet169	92.14	100.0	0.00	100.0	92.45	91.96	92.17	91.99	98.50	98.25	98.50	99.50
ConvNeXt-B	89.29	90.56	86.25	87.33	92.33	93.14	91.06	93.80	98.25	98.00	98.00	98.25
HKG-ResNet34	100.0	98.33	13.75	100.0	91.81	92.82	91.87	92.35	99.50	98.50	99.50	98.75

Table 3. Accuracy of various fine states on three real-world cavitation datasets, encompassing non-cavitation (non), choked flow cavitation (cho), constant cavitation (con), and incipient cavitation (inc). We show different representation learning methods including CNNs, LNNs and Transformers. All results are averaged over three runs.

Methods	Backbones	Image size	Accuracy of fine states						Overall
Methods	Backbones	Image size	Health	IR-1	IR-2	IR-3	OR-1	OR-2	Acc	Pre	Rec	F1
DATMMD	LeNet	${256^{2}}$	90.63	93.75	93.75	100.0	93.75	94.94	93.52	92.46	94.47	93.38
LiftingNet	AlexNet	${256^{2}}$	95.83	90.18	93.75	93.75	94.64	92.41	93.30	91.18	93.43	92.09
MIP-LCNet	AlexNet	${256^{2}}$	97.92	89.29	95.83	93.75	94.64	96.20	93.48	91.68	94.61	92.96
ResNet-APReLU	ResNet18	${256^{2}}$	97.92	98.21	91.67	100.0	96.43	96.20	96.76	95.07	96.74	95.78
ADMTL	ResNet18	${256^{2}}$	95.83	95.54	97.92	93.75	98.21	96.20	96.54	95.13	96.24	95.66

Table 4. Results of different compared methods on PUB dataset averaged over three runs.

Backbones	Loss	Accuracy	Precision	Recall	F1-score
ResNet-34	CCE	88.57	91.47	76.22	74.00
	Focal	90.43	88.81	89.91	89.32
	SCE	89.86	87.57	89.95	88.63
	CB	89.14	86.77	87.88	87.29
	SL	91.14	88.33	91.03	89.46
	HS	92.71	90.78	92.84	91.71

Table 5. Comparison of loss functions on Cavitation-Short. All results are averaged over three runs.

Loss	Weight	Accuracy	Precision	Recall	F1-score
${{\mathcal{L}}^{{\rm{HT}}}}$	-	90.43	88.46	90.60	89.38
	NHW	90.86	89.61	91.18	90.34
	PHW	91.29	89.02	90.69	89.79
${{\mathcal{L}}^{{\rm{FHT}}}}$	-	91.57	89.62	92.05	90.69
	NHW	91.86	89.87	92.20	90.93
	PHW	92.29	90.09	92.38	91.12

Table 6. Analysis of weight schemes for

{{\mathcal{L}}^{{\rm{HT}}}}

and

{{\mathcal{L}}^{{\rm{FHT}}}}

on Cavitation-Short. All results are averaged over three runs.

Loss	Weight			${{\mathcal{L}}^{{\rm{GTT}}}}$	Accuracy	Precision	Recall	F1-score
Loss	-	NHW	PHW	${{\mathcal{L}}^{{\rm{GTT}}}}$	Accuracy	Precision	Recall	F1-score
${{\mathcal{L}}^{{\rm{HT}}}}$	✓	✗	✗	✓	90.86	88.83	91.16	89.87
	✗	✓	✗	✓	91.14	89.36	91.55	90.32
	✗	✗	✓	✓	91.71	89.46	92.43	90.72
${{\mathcal{L}}^{{\rm{FHT}}}}$	✓	✗	✗	✓	92.43	90.28	92.18	91.13
	✗	✓	✗	✓	92.86	90.97	93.00	91.88

Table 7. Analysis of essential components for DHK on Cavitation-Short. All results are averaged over three runs.

Triplet Loss	Margin	Accuracy	Precision	Recall	F1-score
Vanilla	Constant	92.71	90.78	92.65	91.62
${{\mathcal{L}}^{{\rm{GTT}}}}$	Constant	93.14	92.05	93.73	92.83
${{\mathcal{L}}^{{\rm{GTT}}}}$	Hierarchy	93.57	92.27	93.79	92.94

Table 8. Analysis of variants of

{{\mathcal{L}}^{{\rm{GTT}}}}

under

{{\mathcal{L}}^{{\rm{FHT}}}}

(w/ PHW) on Cavitation-Short. All results are averaged over three runs.

Triplet Loss	Distance	Accuracy	Precision	Recall	F1-score
${{\mathcal{L}}^{{\rm{GTT}}}}$	Euclidean	93.29	91.80	93.67	92.79
${{\mathcal{L}}^{{\rm{GTT}}}}$	Cosine	93.57	92.27	93.79	92.94

Table 9. Effects of distance measure for

{{\mathcal{L}}^{{\rm{GTT}}}}

on Cavitation-Short. All results are averaged over three runs.

Panel of Figure 4 — Figure 4. Results of different evaluation metrics from various ablation experiments on Cavitation-Short. (a)-(b) and (d) are all performed with a window size of 466944 and a step size of 466944. All results are the average of three runs.

Backbone	Loss	ave_epoch (min)	total (h)
ResNet18	CCE	2.79	4.64
ResNet18	DHK	2.87	4.87

Table 10. The training time on Cavitation-Short. All results are averaged over three runs.

Backbone	Loss	ave_batch (s)	total (s)
ResNet18	CCE	0.0076	3.05
ResNet18	DHK	0.0078	3.13

Table 11. The inference time on Cavitation-Short. All results are averaged over three runs.

Noise Ratios	0%				5%				10%
Method	Acc	Pre	Rec	F1	Acc	Pre	Rec	F1	Acc	Pre	Rec	F1
ResNet18	87.57	90.25	77.16	77.95	85.43	82.67	84.87	83.62	81.29	78.17	82.90	79.66
ResNet18+DHK	92.57	90.67	91.95	91.27	88.85	86.84	88.48	87.55	83.86	80.57	83.44	81.59

Table 12. Results of ResNet18 and ResNet18+DHK with different label noise ratios on Cavitation-Short.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Figure A1. Pressure flow model in a valve. The solid black, dotted orange, dotted green and dotted blue lines represent the upstream pressure

{p}_{u}

, the downstream pressure

{p}_{d}

, the minimum pressure

{p}_{min}

and the vapor pressure

{p}_{v}

, respectively.

Figure A2. Schematic diagram of the maximum and minimum boundaries for anchor sample nodes, positive sample nodes and negative sample nodes. The left part shows a given hierarchical tree and the right part provides examples of maximum and minimum distances.

Figure A3. Overall framework of the DHK. The T-F domain spectrograms are fed into feature representation learning module to extract deep learned features. Meanwhile, the features are jointly trained and optimized by FHT and GTT loss functions for hierarchical fault intensity recognition.

Figure A4. Schematic view of the test rack at SAMSON AG.

Dataset	Cavitation			Non-cavitation
Dataset	choked flow	constant	incipient	turbulent	no flow
Cavitation-Short	72	93	40	118	33
Cavitation-Long	148	396	64	183	15
Cavitation-Noise	40	40	40	40	0

Appendix

Table A1. The content details of the three real-world cavitation datasets for each flow state.

Dataset	Operation parameters
Dataset	Valve stroke ( $\text{\,}\mathrm{mm}$ )	Upstream pressure ( $\text{\,}\mathrm{bar}$ )	Temperature ( $\text{\,}\mathrm{\SIUnitSymbolCelsius}$ )
Cavitation-Short	[15,13.5,11.25,7.5,3.75,1.5,0.75]	[10,9,6,4]	25-50
Cavitation-Long	[60,55,45,30,25,15,6]	[10,6,4]	23-52
Cavitation-Noise	15	10	32-39

Appendix

Table A2. Details of three real-world cavitation datasets for valve operation with various upstream pressures.

Dataset	Training set				Testing set
	Cavitation			Non	Cavitation			Non
	Choked flow	Constant	Incipient	Non	Choked flow	Constant	Incipient	Non
Cavitation-Short	58( $\times$ 10)	75( $\times$ 10)	32( $\times$ 10)	121( $\times$ 10)	14( $\times$ 10)	18( $\times$ 10)	8( $\times$ 10)	30( $\times$ 10)
Cavitation-Long	118( $\times$ 83)	317( $\times$ 83)	52( $\times$ 83)	158( $\times$ 83)	30( $\times$ 83)	79( $\times$ 83)	12( $\times$ 83)	40( $\times$ 83)
Cavitation-Noise	32( $\times$ 83)	32( $\times$ 83)	32( $\times$ 83)	32( $\times$ 83)	8( $\times$ 83)	8( $\times$ 83)	8( $\times$ 83)	8( $\times$ 83)

Appendix

Table A3. Details of the training and test sets.

(\cdot)

denotes the number after the sliding window (window size is 466944).

Damage level	Percentage values	Bearing limitations
1	0-2 $\%$	$\leq$ $2\text{\,}\mathrm{m}\mathrm{m}$
2	2-5 $\%$	$>$ $2\text{\,}\mathrm{m}\mathrm{m}$
3	5-15 $\%$	$>$ $4.5\text{\,}\mathrm{m}\mathrm{m}$

Appendix

Table A4. The bearing fault damage levels in the PUB.

Fault type	Healthy	OR damage		IR damage
Fault type	Healthy	OR-1	OR-2	IR-1	IR-2	IR-3
File code	K001	KA01	KA03	KI01	KI07	KI16
	K002	KA05	KA06	KI03	KI08	-
	K003	KA04	KA08	KI04	KI18	-
	K004	KA07	KA09	KI05	-	-
	K005	KA15	KA16	KI14	-	-

Appendix

Table A5. The bearing fault types and file codes in the PUB.

Figure A7. Statistical p-value comparison of different methods under six runs. The baselines include convolutional neural network (ResNet, VGG, DenseNet and ConvNeXt), lightweight neural network (MobileNet-v2 and ShuffleNet-v2) and Transformer (ViT, Swin-Transformer, PerViT and UniFormer).

Panel of Figure 28 — Figure A8. The confusion matrix of DHK with different backbone networks on various datasets. (a)-(c) and (e)-(g) denote the confusion matrix on Cavitation-Short (CS), Cavitation-Long (CL) and Cavitation-Noise (CN), respectively. (d) and (h) represent the confusion matrix on the PUB dataset.

Fault Diagnosis

Fault diagnosis identifies, localizes, and explains failures in engineered and computational systems using sensor data, logs, models, and domain knowledge.

See all 143 papers

Mar 6, 2026

CLAIRE: Compressed Latent Autoencoder for Industrial Representation and Evaluation -- A Deep Learning Framework for Smart Manufacturing

Accurate fault detection in high-dimensional industrial environments remains a major challenge due to the inherent complexity, noise, and redundancy in sensor data. This paper introduces CLAIRE, i.e., a hybrid end-to-end learning framework that integrates unsupervised deep representation learning with supervised classification for intelligent quality control in smart manufacturing systems. It employs an optimized deep autoencoder to transform raw input into a compact latent space, effectively capturing the intrinsic data structure while suppressing irrelevant or noisy features. The learned representations are then fed into a downstream classifier to perform binary fault prediction. Experimental results on a high-dimensional dataset demonstrate that CLAIRE significantly outperforms conventional classifiers trained directly on raw features. Moreover, the framework incorporates a post hoc phase, using a game-theory-based interpretability technique, to analyze the latent space and identify the most informative input features contributing to fault predictions. The proposed framework highlights the potential of integrating explainable AI with feature-aware regularization for robust fault detection. The modular and interpretable nature of the proposed framework makes it highly adaptable, offering promising applications in other domains characterized by complex, high-dimensional data, such as healthcare, finance, and environmental monitoring.

Mohammadhossein Ghahramani, Mengchu Zhou

Birmingham City University, UK · the Helen and John C. Hartmann Department of Electrical and Computer Engineering, New Jersey Institute of Technology, Newark, NJ 07102, USA

Industrial Fault Diagnosis Industrial Anomaly Detection Fault Diagnosis Manufacturing Autoencoders

Ask PDF

Jun 18, 2026

Leveraging systems' non-linearity to tackle the scarcity of data in the design of Intelligent Fault Diagnosis Systems

Deep Transfer Learning (DTL) allows for the efficient building of Intelligent Fault Diagnosis Systems (IFDS). On the other hand, DTL methods still heavily rely on large amounts of labelled data. Obtaining such an amount of data can be challenging when dealing with machines or structures faults. This document proposes a novel approach to the design of vibration-based IFDS using DTL in condition of strong data scarcity. A periodic multi-excitation level procedure leveraging intrinsic non-linearities of real-world systems is used to produce images that can be conveniently analysed by pre-trained Convolutional Neural Networks (CNNs) to diagnose faults. A new data visualization method and its augmentation technique are proposed in this paper to tackle the typical lack of data encountered during the design of IFDS. Experimental validation on a railway pantograph structure provides effective support for the proposed method.

Giancarlo Santamato, Andrea Mattia Garavagno, Massimiliano Solazzi +1

1*Institute of Mechanical Intelligence, Scuola Superiore Sant’Anna, via Alamanni 13b, Ghezzano, 56010, Pisa, Italy. · Department of Excellence in Robotics & AI, Scuola Superiore Sant’Anna, Pisa, Italy.

Data Augmentation Predictive Maintenance Fault Diagnosis Data-Efficient Learning

Ask PDF

Apr 22, 2026

Domain-Aware Hierarchical Contrastive Learning for Semi-Supervised Generalization Fault Diagnosis

Fault diagnosis under unseen operating conditions remains highly challenging when labeled data are scarce. Semi-supervised domain generalization fault diagnosis (SSDGFD) provides a practical solution by jointly exploiting labeled and unlabeled source domains. However, existing methods still suffer from two coupled limitations. First, pseudo-labels for unlabeled domains are typically generated primarily from knowledge learned on the labeled source domain, which neglects domain-specific geometric discrepancies and thus induces systematic cross-domain pseudo-label bias. Second, unlabeled samples are commonly handled with a hard accept-or-discard strategy, where rigid thresholding causes imbalanced sample utilization across domains, while hard-label assignment for uncertain samples can easily introduce additional noise. To address these issues, we propose a unified framework termed domain-aware hierarchical contrastive learning (DAHCL) for SSDGFD. Specifically, DAHCL introduces a domain-aware learning (DAL) module to explicitly capture source-domain geometric characteristics and calibrate pseudo-label predictions across heterogeneous source domains, thereby mitigating cross-domain bias in pseudo-label generation. In addition, DAHCL develops a hierarchical contrastive learning (HCL) module that combines dynamic confidence stratification with fuzzy contrastive supervision, enabling uncertain samples to contribute to representation learning without relying on unreliable hard labels. In this way, DAHCL jointly improves the quality of supervision and the utilization of unlabeled samples. Furthermore, to better reflect practical industrial scenarios, we incorporate engineering noise into the SSDGFD evaluation protocol. Extensive experiments on three benchmark datasets demonstrate that...

Junyu Ren, Wensheng Gan, Philip S Yu

College of Cyber Security, Jinan University, Guangzhou 510632, China · Department of Computer Science, University of Illinois Chicago, Chicago, USA

Contrastive Learning Cross-Domain Generalization Bearing Fault Diagnosis Fault Diagnosis

Ask PDF

Abstract

Fault Diagnosis

Explore similar work

CLAIRE: Compressed Latent Autoencoder for Industrial Representation and Evaluation -- A Deep Learning Framework for Smart Manufacturing

Leveraging systems' non-linearity to tackle the scarcity of data in the design of Intelligent Fault Diagnosis Systems

Domain-Aware Hierarchical Contrastive Learning for Semi-Supervised Generalization Fault Diagnosis