HARMONY: Hierarchical Anchor Retrieval on Manifold for Oblivious-source Machinery Acoustic Anomaly Detection
Authors: Yuhang Zhang, Yang Li, Qiang Wu, Mingyuan Li
Organizations: School of Software and Microelectronics, Peking University, Beijing, China · Independent Researcher · Collaborative Innovation Center for Western Ecological Safety, Lanzhou University, Lanzhou, China · ELLIS Institute of Finland · University of Turku
In standard acoustic anomaly detection (ASD), models are usually trained separately for each machine or domain using explicit metadata. However, in realistic deployments, machine identifiers are incomplete, unreliable, or simply unavailable. Thus, metadata-free machinery ASD asks a single model to monitor mixed machine populations without machine IDs. In this paper, we argue that the main failure mode is not a weak detector family, but representation entanglement. The heterogeneous normal modes overlap in frozen pre-trained feature spaces, making anomaly scoring and approximate retrieval unreliable. We study this problem directly and propose HARMONY (Hierarchical Anchor Retrieval on Manifold for Oblivious-source acoustic aNomalY detection), an anchor-guided geometric repartitioning framework that reorganizes mixed-source features into compact Voronoi regions and reuses this structure for hierarchical retrieval. On DCASE 2020, HARMONY improves mean AUC from 75.40% to 90.00% (14.60 percentage points) over the strongest unified baseline and from 83.90% to 90.00% over a strong foundation-feature baseline. On a single CPU, its hierarchical retrieval achieves a 7.8x speedup over exhaustive search with only a 0.20 percentage-point AUC drop. These results suggest that incorporating explicit geometric partitioning can improve both detection accuracy and retrieval efficiency for ASD. Code is available at https://github.com/Zzyh0208/HARMONY.
Figures & tables
Figure 1: Overview of HARMONY. Spectral anchors define the coarse topology, anchor-guided projection compacts mixed-source features, and the same geometry is reused for hierarchical retrieval.
Method
Unified Setting
Per-Region Setting
Cost Transition (Uni → Reg)
AUC
pAUC
AUC ( Δ )
pAUC
Train Time (h)
Latency (ms)
PatchCore
62.10
56.20
91.80 +29.70
78.90
0 → 0
41.8 → 30.6
Audio-PC
83.90
71.30
93.10 +9.20
80.40
0 → 0
67.5 → 52.1
EfficientAD
66.30
58.70
88.90 +22.60
76.10
2.4 → 18.5
6.3 → 6.3
Audio-MAE
86.10
72.80
92.40 +6.30
79.60
14.2 → 108.0
82.4 → 82.4
MCDD
78.30
66.90
89.30 +11.00
76.80
7.1 → 55.6
54.7 → 44.9
Table 1: Unified vs. per-region training (%). Baseline AUC gains ( Δ ) under per-region training expose mixed-source geometry as the core bottleneck, yet partitioning severely inflates costs. HARMONY natively achieves the optimal accuracy-efficiency balance.
Methods
Machine Types
Average
ToyCar
ToyConv
Fan
Pump
Slider
Valve
PatchCore
62.80
60.60
62.20
57.20
67.80
61.50
62.10
Classic Adaptation
SimpleNet
63.20 +0.40
62.10 +1.50
56.70 -5.50
58.00 +0.80
66.30 -1.50
62.20 +0.70
61.40 (-1.1%)
EfficientAD
65.80 +3.00
65.60 +5.00
65.70 +3.50
68.50 +11.30
66.50 -1.30
65.60 +4.10
66.30 (+6.8%)
Foundation Embeddings
Table 2: Metadata-free machinery anomaly detection accuracy (AUC, %) on DCASE 2020. Absolute sub-metric changes ( + / - ) and relative average improvements (%) are computed against the PatchCore baseline. All methods share the same pre-trained features.
Method
MIMII
DCASE 2022
Target
Audio-PC
79.73±0.44
77.84±0.46
73.72±0.59
UniFlow
71.24±0.79
72.69±0.81
68.43±0.98
Dinomaly
72.06±0.71
73.21±0.73
68.97±0.91
HARMONY
84.57±0.58
82.36±0.52
77.99±0.68
Table 3: Cross-dataset AUC (%, mean ± standard deviation over five seeds). MIMII uses the −6 dB setting. DCASE 2022 is evaluated with machine metadata withheld from the model.
Variant
Machine Types
Average
Δ
ToyCar
ToyConv
Fan
Pump
Slider
Valve
Full Model
98.00
89.70
88.70
89.70
87.60
90.00
90.00
-
Ablated Components
w/o Soft Assignment
96.50
87.00
85.00
87.50
85.20
84.40
87.60
-2.40
w/o Spectral Init
89.50
76.50
78.20
82.50
79.00
83.10
81.40
-8.60
w/o Projection
64.10
56.50
58.80
69.20
62.30
63.50
62.40
-27.60
Table 4: Component ablation study on DCASE 2020 (%). The full flat-search model serves as the baseline. Performance drops ( Δ ) are highlighted in red . The final deployed HARMONY (Hierarchical) maintains near-lossless accuracy.
Figure 2: Robustness and stability. Left: HARMONY remains stable as SNR drops to −6 dB, unlike degrading baselines. Right: Silhouette Score tracks AUC, enabling unsupervised model selection.
Feature Space
QE ( ↓ )
Margin ( ↑ )
Ratio ( ↑ )
Silh. Score ( ↑ )
AUC ( ↑ , %)
Raw BYOL-A
0.842
0.418
0.496
0.331
62.40
HARMONY
0.181
0.852
4.707
0.824
90.00
Table 5: Empirical validation of geometric metrics (measured via aligned cosine distance).
Groups
Selected K
Selected AUC
Oracle gap
2 types
4 / 2–6
92.84
0.27
3 types
6 / 4–8
91.97
0.32
4 types
8 / 6–12
91.31
0.27
5 types
8 / 8–12
90.64
0.37
6 types
8 / 8–12
90.07
0.19
Table 6: Normal-only anchor selection across mixtures of C machine types. Counts show mode / range over five seeds; AUC entries are five-seed means (%). The test oracle is used only for retrospective comparison.
Retrieval Method
Accuracy (%)
Efficiency (CPU)
AUC ( ↑ )
pAUC ( ↑ )
Latency ( ↓ )
Speedup ( ↑ )
HARMONY (Flat)
90.00
74.90
42.3 ms
1.0 ×
HARMONY (IVF)
89.80 -0.20
74.70 -0.20
5.4 ms (-87.2%)
7.8 ×
Table 7: Efficiency versus accuracy trade-off on CPU. Hierarchical retrieval (IVF) offers substantial latency reduction ( green ) with negligible accuracy drop ( red ) compared to exhaustive search (Flat).
Appendix figures & tables31 assets
Supplementary material from the paper’s appendix.
Appendix
Symbol
Definition
ϕ:X→Z
Pre-trained frozen encoder mapping audio to feature space.
Ambient dimension of the full input feature space Z .
ϕ#P
Pushforward measure of probability distribution P under mapping ϕ .
gθ:Z→H
Trainable projection network mapping features to hypersphere Sd−1 .
d′ / η
Intrinsic dimensionality / Cumulative energy threshold for spectral filtering.
Appendix
Table 8: Summary of key mathematical notations.
Figure 3: Anchor occupancy statistics across varying machine complexities. (Left) The Entropy Ratio remains near 1.0, indicating uniform utilization. (Right) The maximum occupancy percentage tightly tracks the ideal uniform distribution boundary, showing nonempty anchors in these evaluated configurations.
Figure 4: Smooth optimization trajectory of the alignment loss Lalign over 20 epochs, showing the observed loss trajectory.
Group
Count
Mean
Median
95th Pctl
Min
Normal
5,399
0.013
0.011
0.022
0.003
Anomalous
5,469
0.187
0.164
0.361
0.021
Appendix
Table 9: Distribution of Anomaly Scores s(q) for normal vs. anomalous test queries.
Figure 5: t-SNE [ 19 ] visualization of manifold disentanglement. Projections transition from entangled raw features (Left) to separated Voronoi regions (Right), illustrating the learned feature organization.
Dataset / method
NMI
Purity
Hom./Comp.
DCASE / Raw KMeans
0.42±0.03
0.63±0.03
0.49 / 0.38
DCASE / MCDD
0.66±0.03
0.81±0.02
0.74 / 0.57
DCASE / HARMONY
0.78±0.02
0.89±0.01
0.85 / 0.65
MIMII / MCDD
0.61±0.03
0.77±0.03
0.69 / 0.51
MIMII / HARMONY
0.72±0.03
0.85±0.02
0.80 / 0.59
Appendix
Table 10: Post-hoc alignment with machine types. Hom./Comp. lists homogeneity/completeness; NMI and purity are mean ± standard deviation over five seeds.
Backbone
Model Type
Mean AUC (%)
Audio-MAE
Masked SSL
92.60
CLAP
Contrastive
87.40
PANNS
Supervised Semantic
84.40
BYOL-A (Default)
Audio SSL
90.00
Appendix
Table 11: Performance verification across different pre-trained backbones.
Initialization Method
Assignment Method
Mean AUC (%)
Spectral + GMM
Boltzmann
89.40
Spectral + vMF Mixture
Boltzmann
90.70
Spectral + K-Means++ (Default)
Boltzmann
90.00
Appendix
Table 12: Robustness test replacing the topological initialization method.
Assignment Method
Mean AUC (%)
Δ AUC
Boltzmann (Ours)
90.00
-
Hard Nearest
87.60
-2.40
Appendix
Table 13: Ablation of the assignment mechanism (Soft vs. Hard). Metrics are in percentages (%).
Retrieval Strategy
Mean AUC (%)
Latency (CPU)
Speedup
Flat Search (Exact)
90.00
42.3 ms
1.0 ×
Hierarchical Index (IVF)
89.80
5.4 ms
7.8 ×
Appendix
Table 14: Efficiency vs. accuracy trade-off between retrieval strategies (measured on CPU).
Methods
Machine Types
Average
ToyCar
ToyConv
Fan
Pump
Slider
Valve
PatchCore
62.80 ± 0.1
60.60 ± 0.2
62.20 ± 0.1
57.20 ± 0.2
67.80 ± 0.2
61.50 ± 0.1
62.10 ± 0.2
Classic Adaptation
SimpleNet
63.20 ± 0.7
62.10 ± 0.9
56.70 ± 0.6
58.00 ± 0.8
66.30 ± 0.7
62.20 ± 0.6
61.40 ± 0.7
EfficientAD
65.80 ± 0.4
65.60 ± 0.5
65.70 ± 0.3
68.50 ± 0.4
66.50 ± 0.3
65.60 ± 0.4
66.30 ± 0.4
Foundation Embeddings
Appendix
Table 15: Multi-seed stability for universal anomaly detection (AUC, %) on DCASE 2020. Metrics are reported as Mean ± Std over 5 random seeds.
Methods
Machine Types
Average
ToyCar
ToyConv
Fan
Pump
Slider
Valve
PatchCore
57.60 ± 0.1
56.00 ± 0.1
57.10 ± 0.2
52.30 ± 0.1
59.40 ± 0.1
54.90 ± 0.2
56.20 ± 0.1
Classic Adaptation
SimpleNet
55.00 ± 0.6
53.20 ± 0.5
49.60 ± 0.4
52.90 ± 0.7
56.80 ± 0.6
56.00 ± 0.5
53.90 ± 0.5
EfficientAD
58.20 ± 0.3
57.40 ± 0.4
57.60 ± 0.2
59.30 ± 0.3
60.80 ± 0.2
58.30 ± 0.3
58.70 ± 0.3
Foundation Embeddings
Appendix
Table 16: Multi-seed stability for universal anomaly detection (pAUC, %) on DCASE 2020. Metrics are reported as Mean ± Std over 5 random seeds.
Method Variant
Machine Types
Avg. AUC
ToyCar
ToyConv
Fan
Pump
Slider
Valve
Raw Feat + Exact kNN
64.10
56.50
58.80
69.20
62.30
63.50
62.40
PCA + KMeans + Ret.
66.50
58.20
61.00
71.50
64.80
66.20
64.70
Raw Feat + IVF/HNSW
63.50
56.10
58.40
68.50
61.90
63.10
61.90
Proj + Random Init
89.50
76.50
78.20
82.50
79.00
83.10
81.40
Euclidean Proj ( R128 )
86.20
72.10
74.50
80.10
76.80
79.50
78.20
Appendix
Table 17: Direct control experiments validating the necessity of individual architectural choices. Metrics are in percentages (%).
Method
Routed AUC
Recall
Latency
PCA+KMeans
64.71±0.29
87.94±0.83
6.81±0.28
MCDD+local kNN
81.47±0.87
94.18±0.62
7.26±0.33
DeepCluster+local kNN
84.62±0.72
95.73±0.54
6.32±0.27
vMF soft prototypes
85.71±0.61
96.21±0.49
6.19±0.26
HARMONY hard
87.34±0.49
96.77±0.43
5.57±0.23
HARMONY rep.+HNSW
89.71±0.32
97.26±0.34
5.16±0.22
Appendix
Table 18: Matched controls on DCASE 2020. AUC and recall are percentages; latency is milliseconds.
Figure 6: Performance degradation under severe manifold perturbations. (Left) Non-linear distortion eroding the geometric margin. (Right) PCA-aligned anisotropic noise violating orthogonal assumptions. HARMONY exhibits graceful decay compared to the baseline.
Method
Original
+6 dB
0 dB
−6 dB
Dynamic
Unseen
Audio-PC
83.90±0.20
80.82±0.34
75.47±0.51
66.11±0.76
71.84±0.64
69.52±0.71
UniFlow
75.40±0.60
72.17±0.74
66.32±0.91
58.41±1.18
63.18±1.02
61.02±1.11
Dinomaly
75.30±0.50
72.64±0.63
67.28±0.79
60.17±1.04
64.22±0.89
62.34±0.97
HARMONY
90.00±0.30
88.73±0.42
85.64±0.62
79.42±0.89
83.12±0.75
81.37±0.82
Appendix
Table 19: AUC under recorded environmental noise (%, mean ± standard deviation).
Figure 7: Hyperparameter sensitivity analysis. The framework demonstrates robust performance plateaus across broad ranges of anchor count K , intrinsic dimensionality d′ , and softmax temperature τ . The default configurations are situated within optimal stable regions.
Figure 8: Global correlation between the unsupervised Silhouette Score and test Average AUC across all hyperparameter variations and extreme stress tests. The strong linear alignment (Pearson ρ=0.973 ) validates the use of the Silhouette Score as a highly reliable surrogate metric for label-free model selection.
C
Selected K
Oracle K
Selected / oracle AUC
Δoracle
ρs
2
4 / 2–6
4 / 4–8
92.84 / 93.11
0.27
0.91
3
6 / 4–8
6 / 6–8
91.97 / 92.29
0.32
0.93
4
8 / 6–12
8 / 8–12
91.31 / 91.58
0.27
0.94
5
8 / 8–12
12 / 8–12
90.64 / 91.01
0.37
0.96
6
8 / 8–12
8 / 8–16
90.07 / 90.26
0.19
0.97
Appendix
Table 20: Normal-only selection for mixtures of C machine types (AUC, %).
Setting
K (mode / range)
AUC (%)
Δoracle
DCASE count transferred
8 / 8–8
84.21±0.63
0.63±0.28
Normal-only selection
12 / 8–16
84.57±0.58
0.27±0.19
Test oracle
12 / 12–16
84.84±0.51
0.00±0.00
Appendix
Table 21: MIMII selection and retrospective oracle comparison.
IR
K/τ
Macro AUC
Tail AUC
Head–Tail
Entropy
Δoracle
1
8 / 0.10
90.00±0.30
89.78±0.42
0.34±0.31
0.98±0.01
0.18±0.14
3
8 / 0.10
89.31±0.44
88.16±0.68
1.88±0.55
0.96±0.02
0.28±0.20
5
8 / 0.10
88.54±0.57
86.63±0.87
3.23±0.73
0.93±0.02
0.37±0.25
10
16 / 0.05
86.96±0.82
83.41±1.24
5.76±1.08
0.89±0.03
0.58±0.34
20
16 / 0.05
84.83±1.06
79.36±1.72
8.97±1.46
0.83±0.04
0.84±0.47
Appendix
Table 22: Joint selection under long-tailed training mixtures. AUC and gaps use percentage-point units; entropy is dimensionless.
IR
τ=0.05
τ=0.10
Difference
1
89.34±0.35
90.00±0.30
−0.66
10
86.44±0.79
85.91±0.91
+0.53
20
84.11±1.04
82.86±1.27
+1.25
Appendix
Table 23: Fixed- K temperature comparison (AUC, %; K=8 ).
Method / IR
K
Rare-type anchors
Absorption
Rare-type AUC (%)
HARMONY / 1
8
2.67±0.31
0.05±0.02
89.78±0.42
HARMONY / 5
8
2.14±0.26
0.14±0.04
86.63±0.87
HARMONY / 10
16
1.87±0.24
0.18±0.04
83.41±1.24
HARMONY / 20
16
1.19±0.22
0.29±0.05
79.36±1.72
MCDD / 20
16
0.87±0.21
0.47±0.07
66.21±1.48
PCA-KMeans / 20
8
0.23±0.18
0.64±0.08
53.92±1.57
Appendix
Table 24: Rare-type behavior (five-seed mean ± standard deviation).
Category
Parameter
Symbol
Value
Data
Sample Rate
sr
16,000 Hz
FFT / Hop
-
1024 / 160
Mel Bins
Nmels
64
Model
Input Dim
D
3072
Latent Dim
d
128
Train
Epochs
E
20 (DCASE)
Appendix
Table 25: Default hyperparameter settings.
Machine Type
Machine IDs
Train (Normal)
Test (Normal)
Test (Anomaly)
ToyCar
01, 02, 03, 04
4,000
1,400
1,059
ToyConveyor
01, 02, 03
3,000
2,399
1,110
Fan
00, 02, 04, 06
3,675
400
1,475
Pump
00, 02, 04, 06
3,349
400
456
Slider
00, 02, 04, 06
2,804
400
890
Valve
00, 02, 04, 06
3,291
400
479
Appendix
Table 26: Detailed statistics of the DCASE 2020 Universal Dataset.
Machine Type
Machine IDs
Train (Normal)
Test (Normal)
Test (Anomaly)
Fan
00, 02, 04, 06
3,261
814
1,475
Pump
00, 02, 04, 06
3,000
749
456
Slider
00, 02, 04, 06
2,566
638
890
Valve
00, 02, 04, 06
2,954
737
479
Total
16 IDs
11,781
2,938
3,300
Appendix
Table 27: Detailed statistics of the MIMII -6dB Dataset.
Type
ID
AUC
pAUC
Time
Type
ID
AUC
pAUC
Time
ToyCar
01
97.78
88.89
5.39 ms
Pump
00
89.08
74.53
5.34 ms
02
97.97
90.51
5.12 ms
02
85.50
65.39
5.11 ms
03
97.77
88.80
5.46 ms
04
89.40
80.68
5.64 ms
04
98.42
92.42
5.33 ms
06
90.77
79.51
5.57 ms
ToyConveyor
01
92.15
75.47
5.06 ms
Slider
00
93.52
80.53
5.65 ms
02
88.94
71.41
5.16 ms
02
89.51
72.26
5.62 ms
Appendix
Table 28: Per-ID performance on DCASE 2020. Metrics are reported in percentages (%).
Type
ID
AUC
pAUC
Time
Type
ID
AUC
pAUC
Time
Fan
00
84.54
65.68
5.63 ms
Slider
00
90.72
78.32
5.24 ms
02
83.96
64.73
5.92 ms
02
86.24
67.76
5.39 ms
04
81.16
63.10
5.93 ms
04
84.23
69.59
5.38 ms
06
70.21
61.68
5.21 ms
06
82.13
66.08
5.79 ms
Pump
00
90.23
72.59
5.48 ms
Valve
00
80.72
62.17
5.18 ms
02
89.20
73.27
5.61 ms
02
83.95
67.34
5.10 ms
Appendix
Table 29: Per-ID performance on MIMII -6dB. Metrics are reported in percentages (%).
Method
Macro AUC
Macro pAUC
Worst-ID AUC
PatchCore
58.74±0.41
53.92±0.36
49.18±0.83
SimpleNet
57.93±0.87
52.84±0.72
47.62±1.34
EfficientAD
62.18±0.62
56.73±0.51
51.37±1.06
Gen-Rep
77.86±0.48
65.82±0.41
63.94±0.92
Audio-PC
79.73±0.44
67.11±0.38
65.72±0.87
CADA
68.41±0.92
59.28±0.76
54.13±1.42
Appendix
Table 30: MIMII −6 dB baseline comparison (%).
Method
Overall
Target
Worst section
Audio-PC
77.84±0.46
73.72±0.59
64.27±0.99
UniFlow
72.69±0.81
68.43±0.98
58.92±1.48
Dinomaly
73.21±0.73
68.97±0.91
59.41±1.39
HARMONY
82.36±0.52
77.99±0.68
68.42±1.12
Appendix
Table 31: DCASE 2022 comparison (AUC, %).
Method
0%
0.5%
1%
2%
5%
Raw+1NN
62.18±0.23
59.84±0.38
57.63±0.51
54.21±0.74
50.47±1.06
PatchCore
61.91±0.25
60.42±0.36
58.73±0.48
55.97±0.69
52.86±0.94
Audio-PC
83.74±0.27
82.16±0.39
80.34±0.53
76.88±0.77
69.41±1.12
MCDD
78.09±1.03
77.18±1.08
75.93±1.17
73.47±1.31
68.26±1.57
HARMONY
89.65±0.35
89.04±0.43
88.29±0.57
86.82±0.76
83.31±1.13
Appendix
Table 32: Training contamination on DCASE 2020. Columns are contamination rates; entries are AUC (%, mean ± standard deviation).
Anomalous sound detection (ASD) benchmarks typically assume that the identity of the monitored machine is known at test time and that recordings are evaluated in a machine-wise manner. However, in realistic monitoring scenarios with multiple known machines operating concurrently, test recordings may not be reliably attributable to a specific machine, and requiring machine identity imposes deployment constraints such as dedicated sensors per machine. To reveal performance degradations and method-specific differences in robustness that are hidden under standard machine-wise evaluation, we consider a minimal modification of the ASD evaluation protocol in which test recordings from multiple machines are merged and evaluated jointly without access to machine identity at inference time. Training data and evaluation metrics remain unchanged, and machine identity labels are used only for post hoc evaluation. Experiments with representative ASD methods show that relaxing this assumption reveals performance degradations and method-specific differences in robustness that are hidden under standard machine-wise evaluation, and that these degradations are strongly related to implicit machine identification accuracy.
Kevin Wilkinghoff, Keisuke Imoto, Zheng-Hua Tan
Aalborg University, Denmark · Pioneer Centre for Artificial Intelligence, Denmark · Kyoto University, Japan
Anomalous Sound Detection (ASD) aims to determine whether faults have occurred by monitoring sounds. Existing methods detect a limited range of anomalies, exhibit poor generalization, or train a separate model for each machine. Diffusion models possess strong generalization and can generate specific data with condition guidance. We propose a unified diffusion model only with a small module. The audio is first transformed into log-Mel spectrograms. The lightweight module embeds machine IDs into condition embeddings, guiding the model to reconstruct data for specific machines. Then diffusion model reconstructs data with condition, using Gaussian Mixture Models to fit the distributions of reconstruction errors. Our unified model could monitor multiple machine types and learn more fundamental feature spaces with cross-domain learning. Experiments on DCASE2022 Challenge Task 2 show that our model achieves 3.44% AUC and 2.52% pAUC improvements over baseline, validating its effectiveness.
Pengxiang Gao, Yu Qiu, Yanzhi Song
University of Science and Technology of China, China
Detecting subtle deviations in noisy acoustic environments is central to anomalous sound detection (ASD). A common training-free ASD pipeline temporally pools frame-level representations into a band-preserving feature vector and scores anomalies using a single nearest-neighbor match. However, this global matching can inflate normal-score variance through two effects. First, when normal sounds exhibit band-wise variability, a single global neighbor forces all bands to share the same reference, increasing band-level mismatch. Second, cosine-based matching is energy-coupled, allowing a few high-energy bands to dominate score computation under normal energy fluctuations and further increase variance. We propose BEAM, which stores temporally pooled sub-band vectors in a memory bank, retrieves neighbors per sub-band, and uniformly aggregates scores to reduce normal-score variability and improve discriminability. We further introduce a parameter-free adaptive fusion to better handle diverse temporal dynamics in sub-band responses. Experiments on multiple DCASE Task 2 benchmarks show strong performance without task-specific training, robustness to noise and domain shifts, and complementary gains when combined with encoder fine-tuning.
Phurich Saengthong, Takahiro Shinozaki
Department of Information and Communications Engineering, Institute of Science Tokyo, Japan