Deep Hierarchical Knowledge Loss for Fault Intensity Diagnosis

Yu Sha; Shuiping Gou; Bo Liu; Haofan Lu; Ningtao Liu; Jiahui Fu; Horst Stoecker; Domagoj Vnucec

eess.ASApr 8, 2026

Deep Hierarchical Knowledge Loss for Fault Intensity Diagnosis

Authors: Yu Sha, Shuiping Gou, Bo Liu, Haofan Lu, Ningtao Liu, Jiahui Fu, Horst Stoecker, Domagoj Vnucec,

Organizations: School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen 518172, China · Frankfurt Institute for Advanced Studies, Frankfurt am Main 60438, Germany · School of Artificial Intelligence, Xidian University, Xi’an 710126, China · School of Computer, Luoyang Institute of Science and Technology, Luoyang 471023, China ·

Abstract

Fault intensity diagnosis (FID) plays a pivotal role in intelligent manufacturing while neglecting dependencies among target classes hinders its practical deployment. This paper introduces a novel and general framework with deep hierarchical knowledge loss (DHK) to achieve hierarchical consistent representation and prediction. We develop a novel hierarchical tree loss to enable a holistic mapping of same-attribute classes, leveraging tree-based positive and negative hierarchical knowledge constraints. We further design a focal hierarchical tree loss to enhance its extensibility and devise two adaptive weighting schemes based on tree height. In addition, we propose a group tree triplet loss with hierarchical dynamic margin by incorporating hierarchical group concepts and tree distance to model boundary structural knowledge across classes. The joint two losses significantly improve the recognition of subtle faults. Extensive experiments are performed on four real-world datasets from various industrial domains (three cavitation datasets from SAMSON AG and one publicly available dataset) for FID, all showing superior results and outperforming recent state-of-the-art FID methods.

Figures & tables

Figure 1. Schematic diagrams for visualising different mappings. Each rectangular box represents an operation, which is a filter in the CNN. Different coloured ellipse boxes of the same colour scheme indicate knowledge associations.

Figure 2. Evolution schematic of deep hierarchical knowledge loss (DHK), comprising hierarchical tree loss

{\mathcal{L}^{HT}}

with two adaptive weighting schemes and group tree triplet loss

{\mathcal{L}^{GTT}}

with hierarchical dynamic margin.

Panel of Figure 3 — Figure 3. Visualisation of the learned deep feature distribution of vanilla ResNet34, HKG+ResNet34 and DHK+ResNet34 via t-SNE on Cavitation-Short.

Dataset		Cavitation-Short				Cavitation-Long				Cavitation-Noise
Methods	Image size	Acc	Pre	Rec	F1	Acc	Pre	Rec	F1	Acc	Pre	Rec	F1
ResNet18	${256^{2}}$	87.57	90.25	77.16	77.95	90.83	86.31	90.75	88.21	98.94	98.94	98.94	98.94
ResNet34	${256^{2}}$	88.57	91.47	76.22	74.00	91.88	87.60	91.86	89.42	98.81	98.82	98.81	98.81
ResNet50	${256^{2}}$	84.43	86.97	71.70	66.83	91.45	87.09	91.25	88.89	98.19	98.20	98.19	98.19
VGG11	${256^{2}}$	84.29	62.31	70.10	65.95	80.78	74.46	81.11	76.69	97.81	97.82	97.81	97.81
VGG13	${256^{2}}$	80.71	58.96	66.85	62.60	81.56	75.21	82.00	77.50	98.19	98.19	98.19	98.19

Table 1. Results of different backbones on three real-world cavitation datasets averaged over three runs. We compare different representation learning methods including CNNs, LNNs and Transformers. According to our previous research ( Sha et al., 2022 ) , the sliding window with a window size of 466944 and a step size of 466944.

Dataset		Cavitation-Short				Cavitation-Long				Cavitation-Noise
Methods	Image size	Acc	Pre	Rec	F1	Acc	Pre	Rec	F1	Acc	Pre	Rec	F1
LiftingNet	${256^{2}}$	85.29	82.75	75.89	73.95	88.43	86.31	87.97	87.05	95.86	94.62	95.84	95.20
MIPLCNet	${256^{2}}$	86.57	83.38	77.71	75.00	89.14	87.06	88.18	87.55	96.57	95.78	96.61	96.18
ResNet-APReLU	${256^{2}}$	86.86	84.04	77.89	75.56	90.71	88.83	90.91	89.70	97.29	96.77	97.49	97.12
LSTM-RDRN	${256^{2}}$	87.71	85.42	78.50	76.72	91.14	89.36	91.55	90.32	98.71	98.40	98.53	98.46
BCNN	${256^{2}}$	81.71	78.63	72.25	70.03	85.71	82.42	85.09	83.56	92.14	89.75	92.71	91.03

Table 2. Results of different comparison methods on three real-world cavitation datasets averaged over three runs. Non-hierarchical and hierarchical methods from fault intensity diagnosis are used as comparative methods.

Dataset	Cavitation-Short				Cavitation-Long				Cavitation-Noise
Methods	cho	con	inc	non	cho	con	inc	non	cho	con	inc	non
ResNet34	100.0	96.11	8.75	100.0	92.21	92.24	92.17	90.84	98.50	98.75	99.00	99.00
VGG11	85.71	95.00	0.00	99.67	81.04	80.33	81.93	81.14	97.75	98.00	97.25	98.25
DenseNet169	92.14	100.0	0.00	100.0	92.45	91.96	92.17	91.99	98.50	98.25	98.50	99.50
ConvNeXt-B	89.29	90.56	86.25	87.33	92.33	93.14	91.06	93.80	98.25	98.00	98.00	98.25
HKG-ResNet34	100.0	98.33	13.75	100.0	91.81	92.82	91.87	92.35	99.50	98.50	99.50	98.75

Table 3. Accuracy of various fine states on three real-world cavitation datasets, encompassing non-cavitation (non), choked flow cavitation (cho), constant cavitation (con), and incipient cavitation (inc). We show different representation learning methods including CNNs, LNNs and Transformers. All results are averaged over three runs.

Methods	Backbones	Image size	Accuracy of fine states						Overall
Methods	Backbones	Image size	Health	IR-1	IR-2	IR-3	OR-1	OR-2	Acc	Pre	Rec	F1
DATMMD	LeNet	${256^{2}}$	90.63	93.75	93.75	100.0	93.75	94.94	93.52	92.46	94.47	93.38
LiftingNet	AlexNet	${256^{2}}$	95.83	90.18	93.75	93.75	94.64	92.41	93.30	91.18	93.43	92.09
MIP-LCNet	AlexNet	${256^{2}}$	97.92	89.29	95.83	93.75	94.64	96.20	93.48	91.68	94.61	92.96
ResNet-APReLU	ResNet18	${256^{2}}$	97.92	98.21	91.67	100.0	96.43	96.20	96.76	95.07	96.74	95.78
ADMTL	ResNet18	${256^{2}}$	95.83	95.54	97.92	93.75	98.21	96.20	96.54	95.13	96.24	95.66

Table 4. Results of different compared methods on PUB dataset averaged over three runs.

Backbones	Loss	Accuracy	Precision	Recall	F1-score
ResNet-34	CCE	88.57	91.47	76.22	74.00
	Focal	90.43	88.81	89.91	89.32
	SCE	89.86	87.57	89.95	88.63
	CB	89.14	86.77	87.88	87.29
	SL	91.14	88.33	91.03	89.46
	HS	92.71	90.78	92.84	91.71

Table 5. Comparison of loss functions on Cavitation-Short. All results are averaged over three runs.

Loss	Weight	Accuracy	Precision	Recall	F1-score
${{\mathcal{L}}^{{\rm{HT}}}}$	-	90.43	88.46	90.60	89.38
	NHW	90.86	89.61	91.18	90.34
	PHW	91.29	89.02	90.69	89.79
${{\mathcal{L}}^{{\rm{FHT}}}}$	-	91.57	89.62	92.05	90.69
	NHW	91.86	89.87	92.20	90.93
	PHW	92.29	90.09	92.38	91.12

Table 6. Analysis of weight schemes for

{{\mathcal{L}}^{{\rm{HT}}}}

and

{{\mathcal{L}}^{{\rm{FHT}}}}

on Cavitation-Short. All results are averaged over three runs.

Loss	Weight			${{\mathcal{L}}^{{\rm{GTT}}}}$	Accuracy	Precision	Recall	F1-score
Loss	-	NHW	PHW	${{\mathcal{L}}^{{\rm{GTT}}}}$	Accuracy	Precision	Recall	F1-score
${{\mathcal{L}}^{{\rm{HT}}}}$	✓	✗	✗	✓	90.86	88.83	91.16	89.87
	✗	✓	✗	✓	91.14	89.36	91.55	90.32
	✗	✗	✓	✓	91.71	89.46	92.43	90.72
${{\mathcal{L}}^{{\rm{FHT}}}}$	✓	✗	✗	✓	92.43	90.28	92.18	91.13
	✗	✓	✗	✓	92.86	90.97	93.00	91.88

Table 7. Analysis of essential components for DHK on Cavitation-Short. All results are averaged over three runs.

Triplet Loss	Margin	Accuracy	Precision	Recall	F1-score
Vanilla	Constant	92.71	90.78	92.65	91.62
${{\mathcal{L}}^{{\rm{GTT}}}}$	Constant	93.14	92.05	93.73	92.83
${{\mathcal{L}}^{{\rm{GTT}}}}$	Hierarchy	93.57	92.27	93.79	92.94

Table 8. Analysis of variants of

{{\mathcal{L}}^{{\rm{GTT}}}}

under

{{\mathcal{L}}^{{\rm{FHT}}}}

(w/ PHW) on Cavitation-Short. All results are averaged over three runs.

Triplet Loss	Distance	Accuracy	Precision	Recall	F1-score
${{\mathcal{L}}^{{\rm{GTT}}}}$	Euclidean	93.29	91.80	93.67	92.79
${{\mathcal{L}}^{{\rm{GTT}}}}$	Cosine	93.57	92.27	93.79	92.94

Table 9. Effects of distance measure for

{{\mathcal{L}}^{{\rm{GTT}}}}

on Cavitation-Short. All results are averaged over three runs.

Panel of Figure 4 — Figure 4. Results of different evaluation metrics from various ablation experiments on Cavitation-Short. (a)-(b) and (d) are all performed with a window size of 466944 and a step size of 466944. All results are the average of three runs.

Backbone	Loss	ave_epoch (min)	total (h)
ResNet18	CCE	2.79	4.64
ResNet18	DHK	2.87	4.87

Table 10. The training time on Cavitation-Short. All results are averaged over three runs.

Backbone	Loss	ave_batch (s)	total (s)
ResNet18	CCE	0.0076	3.05
ResNet18	DHK	0.0078	3.13

Table 11. The inference time on Cavitation-Short. All results are averaged over three runs.

Noise Ratios	0%				5%				10%
Method	Acc	Pre	Rec	F1	Acc	Pre	Rec	F1	Acc	Pre	Rec	F1
ResNet18	87.57	90.25	77.16	77.95	85.43	82.67	84.87	83.62	81.29	78.17	82.90	79.66
ResNet18+DHK	92.57	90.67	91.95	91.27	88.85	86.84	88.48	87.55	83.86	80.57	83.44	81.59

Table 12. Results of ResNet18 and ResNet18+DHK with different label noise ratios on Cavitation-Short.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Figure A1. Pressure flow model in a valve. The solid black, dotted orange, dotted green and dotted blue lines represent the upstream pressure

{p}_{u}

, the downstream pressure

{p}_{d}

, the minimum pressure

{p}_{min}

and the vapor pressure

{p}_{v}

, respectively.

Figure A2. Schematic diagram of the maximum and minimum boundaries for anchor sample nodes, positive sample nodes and negative sample nodes. The left part shows a given hierarchical tree and the right part provides examples of maximum and minimum distances.

Figure A3. Overall framework of the DHK. The T-F domain spectrograms are fed into feature representation learning module to extract deep learned features. Meanwhile, the features are jointly trained and optimized by FHT and GTT loss functions for hierarchical fault intensity recognition.

Figure A4. Schematic view of the test rack at SAMSON AG.

Dataset	Cavitation			Non-cavitation
Dataset	choked flow	constant	incipient	turbulent	no flow
Cavitation-Short	72	93	40	118	33
Cavitation-Long	148	396	64	183	15
Cavitation-Noise	40	40	40	40	0

Appendix

Table A1. The content details of the three real-world cavitation datasets for each flow state.

Dataset	Operation parameters
Dataset	Valve stroke ( $\text{\,}\mathrm{mm}$ )	Upstream pressure ( $\text{\,}\mathrm{bar}$ )	Temperature ( $\text{\,}\mathrm{\SIUnitSymbolCelsius}$ )
Cavitation-Short	[15,13.5,11.25,7.5,3.75,1.5,0.75]	[10,9,6,4]	25-50
Cavitation-Long	[60,55,45,30,25,15,6]	[10,6,4]	23-52
Cavitation-Noise	15	10	32-39

Appendix

Table A2. Details of three real-world cavitation datasets for valve operation with various upstream pressures.

Dataset	Training set				Testing set
	Cavitation			Non	Cavitation			Non
	Choked flow	Constant	Incipient	Non	Choked flow	Constant	Incipient	Non
Cavitation-Short	58( $\times$ 10)	75( $\times$ 10)	32( $\times$ 10)	121( $\times$ 10)	14( $\times$ 10)	18( $\times$ 10)	8( $\times$ 10)	30( $\times$ 10)
Cavitation-Long	118( $\times$ 83)	317( $\times$ 83)	52( $\times$ 83)	158( $\times$ 83)	30( $\times$ 83)	79( $\times$ 83)	12( $\times$ 83)	40( $\times$ 83)
Cavitation-Noise	32( $\times$ 83)	32( $\times$ 83)	32( $\times$ 83)	32( $\times$ 83)	8( $\times$ 83)	8( $\times$ 83)	8( $\times$ 83)	8( $\times$ 83)

Appendix

Table A3. Details of the training and test sets.

(\cdot)

denotes the number after the sliding window (window size is 466944).

Damage level	Percentage values	Bearing limitations
1	0-2 $\%$	$\leq$ $2\text{\,}\mathrm{m}\mathrm{m}$
2	2-5 $\%$	$>$ $2\text{\,}\mathrm{m}\mathrm{m}$
3	5-15 $\%$	$>$ $4.5\text{\,}\mathrm{m}\mathrm{m}$

Appendix

Table A4. The bearing fault damage levels in the PUB.

Fault type	Healthy	OR damage		IR damage
Fault type	Healthy	OR-1	OR-2	IR-1	IR-2	IR-3
File code	K001	KA01	KA03	KI01	KI07	KI16
	K002	KA05	KA06	KI03	KI08	-
	K003	KA04	KA08	KI04	KI18	-
	K004	KA07	KA09	KI05	-	-
	K005	KA15	KA16	KI14	-	-

Appendix

Table A5. The bearing fault types and file codes in the PUB.

Figure A7. Statistical p-value comparison of different methods under six runs. The baselines include convolutional neural network (ResNet, VGG, DenseNet and ConvNeXt), lightweight neural network (MobileNet-v2 and ShuffleNet-v2) and Transformer (ViT, Swin-Transformer, PerViT and UniFormer).

Panel of Figure 28 — Figure A8. The confusion matrix of DHK with different backbone networks on various datasets. (a)-(c) and (e)-(g) denote the confusion matrix on Cavitation-Short (CS), Cavitation-Long (CL) and Cavitation-Noise (CN), respectively. (d) and (h) represent the confusion matrix on the PUB dataset.

Fault Diagnosis

Fault diagnosis identifies, localizes, and explains failures in engineered and computational systems using sensor data, logs, models, and domain knowledge.

See all 143 papers

CLAIRE: Compressed Latent Autoencoder for Industrial Representation and Evaluation -- A Deep Learning Framework for Smart Manufacturing
Mar 6, 2026Mohammadhossein Ghahramani, Mengchu ZhouIndustrial Fault Diagnosis Industrial Anomaly Detection
Birmingham City University, UK · the Helen and John C. Hartmann Department of Electrical and Computer Engineering, New Jersey Institute of Technology, Newark, NJ 07102, USA
Leveraging systems' non-linearity to tackle the scarcity of data in the design of Intelligent Fault Diagnosis Systems
Jun 18, 2026Giancarlo Santamato, Andrea Mattia Garavagno, Massimiliano Solazzi +1Data Augmentation Predictive Maintenance
1*Institute of Mechanical Intelligence, Scuola Superiore Sant’Anna, via Alamanni 13b, Ghezzano, 56010, Pisa, Italy. · Department of Excellence in Robotics & AI, Scuola Superiore Sant’Anna, Pisa, Italy.
Domain-Aware Hierarchical Contrastive Learning for Semi-Supervised Generalization Fault Diagnosis
Apr 22, 2026Junyu Ren, Wensheng Gan, Philip S YuContrastive Learning Cross-Domain Generalization
College of Cyber Security, Jinan University, Guangzhou 510632, China · Department of Computer Science, University of Illinois Chicago, Chicago, USA

Mar 6, 2026J/K move · Enter open · S save

CLAIRE: Compressed Latent Autoencoder for Industrial Representation and Evaluation -- A Deep Learning Framework for Smart Manufacturing

Mohammadhossein Ghahramani, Mengchu Zhou

Birmingham City University, UK · the Helen and John C. Hartmann Department of Electrical and Computer Engineering, New Jersey Institute of Technology, Newark, NJ 07102, USA

Industrial Fault Diagnosis Industrial Anomaly Detection Fault Diagnosis Manufacturing Autoencoders

Open paper AskPDF arXiv

Accurate fault detection in high-dimensional industrial environments remains a major challenge due to the inherent complexity, noise, and redundancy in sensor data. This paper introduces CLAIRE, i.e., a hybrid end-to-end learning framework that integrates unsupervised deep representation learning with supervised classification for intelligent quality control in smart manufacturing systems. It employs an optimized deep autoencoder to transform raw input into a compact latent space, effectively capturing the intrinsic data structure while suppressing irrelevant or noisy features. The learned representations are then fed into a downstream classifier to perform binary fault prediction. Experimental results on a high-dimensional dataset demonstrate that CLAIRE significantly outperforms conventional classifiers trained directly on raw features. Moreover, the framework incorporates a post hoc phase, using a game-theory-based interpretability technique, to analyze the latent space and identify the most informative input features contributing to fault predictions. The proposed framework highlights the potential of integrating explainable AI with feature-aware regularization for robust fault detection. The modular and interpretable nature of the proposed framework makes it highly adaptable, offering promising applications in other domains characterized by complex, high-dimensional data, such as healthcare, finance, and environmental monitoring.

Abstract

Fault Diagnosis

Explore similar work

CLAIRE: Compressed Latent Autoencoder for Industrial Representation and Evaluation -- A Deep Learning Framework for Smart Manufacturing

Leveraging systems' non-linearity to tackle the scarcity of data in the design of Intelligent Fault Diagnosis Systems

Domain-Aware Hierarchical Contrastive Learning for Semi-Supervised Generalization Fault Diagnosis