HARMONY: Hierarchical Anchor Retrieval on Manifold for Oblivious-source Machinery Acoustic Anomaly Detection
Organizations: School of Software and Microelectronics, Peking University, Beijing, China · Independent Researcher · Collaborative Innovation Center for Western Ecological Safety, Lanzhou University, Lanzhou, China · ELLIS Institute of Finland ·
Abstract
In standard acoustic anomaly detection (ASD), models are usually trained separately for each machine or domain using explicit metadata. However, in realistic deployments, machine identifiers are incomplete, unreliable, or simply unavailable. Thus, metadata-free machinery ASD asks a single model to monitor mixed machine populations without machine IDs. In this paper, we argue that the main failure mode is not a weak detector family, but representation entanglement. The heterogeneous normal modes overlap in frozen pre-trained feature spaces, making anomaly scoring and approximate retrieval unreliable. We study this problem directly and propose HARMONY (Hierarchical Anchor Retrieval on Manifold for Oblivious-source acoustic aNomalY detection), an anchor-guided geometric repartitioning framework that reorganizes mixed-source features into compact Voronoi regions and reuses this structure for hierarchical retrieval. On DCASE 2020, HARMONY improves mean AUC from 75.40% to 90.00% (14.60 percentage points) over the strongest unified baseline and from 83.90% to 90.00% over a strong foundation-feature baseline. On a single CPU, its hierarchical retrieval achieves a 7.8x speedup over exhaustive search with only a 0.20 percentage-point AUC drop. These results suggest that incorporating explicit geometric partitioning can improve both detection accuracy and retrieval efficiency for ASD. Code is available at https://github.com/Zzyh0208/HARMONY.
Figures & tables
| Method | Unified Setting | Per-Region Setting | Cost Transition (Uni Reg) | |||
|---|---|---|---|---|---|---|
| AUC | pAUC | AUC ( ) | pAUC | Train Time (h) | Latency (ms) | |
| PatchCore | 62.10 | 56.20 | 91.80 +29.70 | 78.90 | 0 0 | 41.8 30.6 |
| Audio-PC | 83.90 | 71.30 | 93.10 +9.20 | 80.40 | 0 0 | 67.5 52.1 |
| EfficientAD | 66.30 | 58.70 | 88.90 +22.60 | 76.10 | 2.4 18.5 | 6.3 6.3 |
| Audio-MAE | 86.10 | 72.80 | 92.40 +6.30 | 79.60 | 14.2 108.0 | 82.4 82.4 |
| MCDD | 78.30 | 66.90 | 89.30 +11.00 | 76.80 | 7.1 55.6 | 54.7 44.9 |
| Methods | Machine Types | Average | |||||
|---|---|---|---|---|---|---|---|
| ToyCar | ToyConv | Fan | Pump | Slider | Valve | ||
| PatchCore | 62.80 | 60.60 | 62.20 | 57.20 | 67.80 | 61.50 | 62.10 |
| Classic Adaptation | |||||||
| SimpleNet | 63.20 +0.40 | 62.10 +1.50 | 56.70 -5.50 | 58.00 +0.80 | 66.30 -1.50 | 62.20 +0.70 | 61.40 (-1.1%) |
| EfficientAD | 65.80 +3.00 | 65.60 +5.00 | 65.70 +3.50 | 68.50 +11.30 | 66.50 -1.30 | 65.60 +4.10 | 66.30 (+6.8%) |
| Foundation Embeddings | |||||||
| Method | MIMII | DCASE 2022 | Target |
|---|---|---|---|
| Audio-PC | |||
| UniFlow | |||
| Dinomaly | |||
| HARMONY |
| Variant | Machine Types | Average | ||||||
|---|---|---|---|---|---|---|---|---|
| ToyCar | ToyConv | Fan | Pump | Slider | Valve | |||
| Full Model | 98.00 | 89.70 | 88.70 | 89.70 | 87.60 | 90.00 | 90.00 | - |
| Ablated Components | ||||||||
| w/o Soft Assignment | 96.50 | 87.00 | 85.00 | 87.50 | 85.20 | 84.40 | 87.60 | -2.40 |
| w/o Spectral Init | 89.50 | 76.50 | 78.20 | 82.50 | 79.00 | 83.10 | 81.40 | -8.60 |
| w/o Projection | 64.10 | 56.50 | 58.80 | 69.20 | 62.30 | 63.50 | 62.40 | -27.60 |
| Feature Space | QE ( ) | Margin ( ) | Ratio ( ) | Silh. Score ( ) | AUC ( , %) |
|---|---|---|---|---|---|
| Raw BYOL-A | 0.842 | 0.418 | 0.496 | 0.331 | 62.40 |
| HARMONY | 0.181 | 0.852 | 4.707 | 0.824 | 90.00 |
| Groups | Selected | Selected AUC | Oracle gap |
|---|---|---|---|
| 2 types | 4 / 2–6 | 92.84 | 0.27 |
| 3 types | 6 / 4–8 | 91.97 | 0.32 |
| 4 types | 8 / 6–12 | 91.31 | 0.27 |
| 5 types | 8 / 8–12 | 90.64 | 0.37 |
| 6 types | 8 / 8–12 | 90.07 | 0.19 |
| Retrieval Method | Accuracy (%) | Efficiency (CPU) | ||
|---|---|---|---|---|
| AUC ( ) | pAUC ( ) | Latency ( ) | Speedup ( ) | |
| HARMONY (Flat) | 90.00 | 74.90 | 42.3 ms | 1.0 |
| HARMONY (IVF) | 89.80 -0.20 | 74.70 -0.20 | 5.4 ms (-87.2%) | 7.8 |
Appendix figures & tables31 assets
Supplementary material from the paper’s appendix.
Appendix
| Symbol | Definition |
|---|---|
| Pre-trained frozen encoder mapping audio to feature space. | |
| / | Latent normal-mode count / learned anchor count (not necessarily equal). |
| Ambient dimension of the full input feature space . | |
| Pushforward measure of probability distribution under mapping . | |
| Trainable projection network mapping features to hypersphere . | |
| / | Intrinsic dimensionality / Cumulative energy threshold for spectral filtering. |
| Group | Count | Mean | Median | 95th Pctl | Min |
|---|---|---|---|---|---|
| Normal | 5,399 | 0.013 | 0.011 | 0.022 | 0.003 |
| Anomalous | 5,469 | 0.187 | 0.164 | 0.361 | 0.021 |
| Dataset / method | NMI | Purity | Hom./Comp. |
|---|---|---|---|
| DCASE / Raw KMeans | 0.49 / 0.38 | ||
| DCASE / MCDD | 0.74 / 0.57 | ||
| DCASE / HARMONY | 0.85 / 0.65 | ||
| MIMII / MCDD | 0.69 / 0.51 | ||
| MIMII / HARMONY | 0.80 / 0.59 |
| Backbone | Model Type | Mean AUC (%) |
|---|---|---|
| Audio-MAE | Masked SSL | 92.60 |
| CLAP | Contrastive | 87.40 |
| PANNS | Supervised Semantic | 84.40 |
| BYOL-A (Default) | Audio SSL | 90.00 |
| Initialization Method | Assignment Method | Mean AUC (%) |
|---|---|---|
| Spectral + GMM | Boltzmann | 89.40 |
| Spectral + vMF Mixture | Boltzmann | 90.70 |
| Spectral + K-Means++ (Default) | Boltzmann | 90.00 |
| Assignment Method | Mean AUC (%) | AUC |
|---|---|---|
| Boltzmann (Ours) | 90.00 | - |
| Hard Nearest | 87.60 | -2.40 |
| Retrieval Strategy | Mean AUC (%) | Latency (CPU) | Speedup |
|---|---|---|---|
| Flat Search (Exact) | 90.00 | 42.3 ms | 1.0 |
| Hierarchical Index (IVF) | 89.80 | 5.4 ms | 7.8 |
| Methods | Machine Types | Average | |||||
|---|---|---|---|---|---|---|---|
| ToyCar | ToyConv | Fan | Pump | Slider | Valve | ||
| PatchCore | 62.80 0.1 | 60.60 0.2 | 62.20 0.1 | 57.20 0.2 | 67.80 0.2 | 61.50 0.1 | 62.10 0.2 |
| Classic Adaptation | |||||||
| SimpleNet | 63.20 0.7 | 62.10 0.9 | 56.70 0.6 | 58.00 0.8 | 66.30 0.7 | 62.20 0.6 | 61.40 0.7 |
| EfficientAD | 65.80 0.4 | 65.60 0.5 | 65.70 0.3 | 68.50 0.4 | 66.50 0.3 | 65.60 0.4 | 66.30 0.4 |
| Foundation Embeddings | |||||||
| Methods | Machine Types | Average | |||||
|---|---|---|---|---|---|---|---|
| ToyCar | ToyConv | Fan | Pump | Slider | Valve | ||
| PatchCore | 57.60 0.1 | 56.00 0.1 | 57.10 0.2 | 52.30 0.1 | 59.40 0.1 | 54.90 0.2 | 56.20 0.1 |
| Classic Adaptation | |||||||
| SimpleNet | 55.00 0.6 | 53.20 0.5 | 49.60 0.4 | 52.90 0.7 | 56.80 0.6 | 56.00 0.5 | 53.90 0.5 |
| EfficientAD | 58.20 0.3 | 57.40 0.4 | 57.60 0.2 | 59.30 0.3 | 60.80 0.2 | 58.30 0.3 | 58.70 0.3 |
| Foundation Embeddings | |||||||
| Method Variant | Machine Types | Avg. AUC | |||||
|---|---|---|---|---|---|---|---|
| ToyCar | ToyConv | Fan | Pump | Slider | Valve | ||
| Raw Feat + Exact kNN | 64.10 | 56.50 | 58.80 | 69.20 | 62.30 | 63.50 | 62.40 |
| PCA + KMeans + Ret. | 66.50 | 58.20 | 61.00 | 71.50 | 64.80 | 66.20 | 64.70 |
| Raw Feat + IVF/HNSW | 63.50 | 56.10 | 58.40 | 68.50 | 61.90 | 63.10 | 61.90 |
| Proj + Random Init | 89.50 | 76.50 | 78.20 | 82.50 | 79.00 | 83.10 | 81.40 |
| Euclidean Proj ( ) | 86.20 | 72.10 | 74.50 | 80.10 | 76.80 | 79.50 | 78.20 |
| Method | Routed AUC | Recall | Latency |
|---|---|---|---|
| PCA+KMeans | |||
| MCDD+local kNN | |||
| DeepCluster+local kNN | |||
| vMF soft prototypes | |||
| HARMONY hard | |||
| HARMONY rep.+HNSW |
| Method | Original | dB | 0 dB | dB | Dynamic | Unseen |
|---|---|---|---|---|---|---|
| Audio-PC | ||||||
| UniFlow | ||||||
| Dinomaly | ||||||
| HARMONY |
| Selected | Oracle | Selected / oracle AUC | |||
|---|---|---|---|---|---|
| 2 | 4 / 2–6 | 4 / 4–8 | 92.84 / 93.11 | 0.27 | 0.91 |
| 3 | 6 / 4–8 | 6 / 6–8 | 91.97 / 92.29 | 0.32 | 0.93 |
| 4 | 8 / 6–12 | 8 / 8–12 | 91.31 / 91.58 | 0.27 | 0.94 |
| 5 | 8 / 8–12 | 12 / 8–12 | 90.64 / 91.01 | 0.37 | 0.96 |
| 6 | 8 / 8–12 | 8 / 8–16 | 90.07 / 90.26 | 0.19 | 0.97 |
| Setting | (mode / range) | AUC (%) | |
|---|---|---|---|
| DCASE count transferred | 8 / 8–8 | ||
| Normal-only selection | 12 / 8–16 | ||
| Test oracle | 12 / 12–16 |
| Macro AUC | Tail AUC | Head–Tail | Entropy | |||
|---|---|---|---|---|---|---|
| 1 | 8 / 0.10 | |||||
| 3 | 8 / 0.10 | |||||
| 5 | 8 / 0.10 | |||||
| 10 | 16 / 0.05 | |||||
| 20 | 16 / 0.05 |
| Difference | |||
|---|---|---|---|
| 1 | |||
| 10 | |||
| 20 |
| Method / | Rare-type anchors | Absorption | Rare-type AUC (%) | |
|---|---|---|---|---|
| HARMONY / 1 | 8 | |||
| HARMONY / 5 | 8 | |||
| HARMONY / 10 | 16 | |||
| HARMONY / 20 | 16 | |||
| MCDD / 20 | 16 | |||
| PCA-KMeans / 20 | 8 |
| Category | Parameter | Symbol | Value |
| Data | Sample Rate | 16,000 Hz | |
| FFT / Hop | - | 1024 / 160 | |
| Mel Bins | 64 | ||
| Model | Input Dim | 3072 | |
| Latent Dim | 128 | ||
| Train | Epochs | 20 (DCASE) |
| Machine Type | Machine IDs | Train (Normal) | Test (Normal) | Test (Anomaly) |
| ToyCar | 01, 02, 03, 04 | 4,000 | 1,400 | 1,059 |
| ToyConveyor | 01, 02, 03 | 3,000 | 2,399 | 1,110 |
| Fan | 00, 02, 04, 06 | 3,675 | 400 | 1,475 |
| Pump | 00, 02, 04, 06 | 3,349 | 400 | 456 |
| Slider | 00, 02, 04, 06 | 2,804 | 400 | 890 |
| Valve | 00, 02, 04, 06 | 3,291 | 400 | 479 |
| Machine Type | Machine IDs | Train (Normal) | Test (Normal) | Test (Anomaly) |
|---|---|---|---|---|
| Fan | 00, 02, 04, 06 | 3,261 | 814 | 1,475 |
| Pump | 00, 02, 04, 06 | 3,000 | 749 | 456 |
| Slider | 00, 02, 04, 06 | 2,566 | 638 | 890 |
| Valve | 00, 02, 04, 06 | 2,954 | 737 | 479 |
| Total | 16 IDs | 11,781 | 2,938 | 3,300 |
| Type | ID | AUC | pAUC | Time | Type | ID | AUC | pAUC | Time |
|---|---|---|---|---|---|---|---|---|---|
| ToyCar | 01 | 97.78 | 88.89 | 5.39 ms | Pump | 00 | 89.08 | 74.53 | 5.34 ms |
| 02 | 97.97 | 90.51 | 5.12 ms | 02 | 85.50 | 65.39 | 5.11 ms | ||
| 03 | 97.77 | 88.80 | 5.46 ms | 04 | 89.40 | 80.68 | 5.64 ms | ||
| 04 | 98.42 | 92.42 | 5.33 ms | 06 | 90.77 | 79.51 | 5.57 ms | ||
| ToyConveyor | 01 | 92.15 | 75.47 | 5.06 ms | Slider | 00 | 93.52 | 80.53 | 5.65 ms |
| 02 | 88.94 | 71.41 | 5.16 ms | 02 | 89.51 | 72.26 | 5.62 ms |
| Type | ID | AUC | pAUC | Time | Type | ID | AUC | pAUC | Time |
|---|---|---|---|---|---|---|---|---|---|
| Fan | 00 | 84.54 | 65.68 | 5.63 ms | Slider | 00 | 90.72 | 78.32 | 5.24 ms |
| 02 | 83.96 | 64.73 | 5.92 ms | 02 | 86.24 | 67.76 | 5.39 ms | ||
| 04 | 81.16 | 63.10 | 5.93 ms | 04 | 84.23 | 69.59 | 5.38 ms | ||
| 06 | 70.21 | 61.68 | 5.21 ms | 06 | 82.13 | 66.08 | 5.79 ms | ||
| Pump | 00 | 90.23 | 72.59 | 5.48 ms | Valve | 00 | 80.72 | 62.17 | 5.18 ms |
| 02 | 89.20 | 73.27 | 5.61 ms | 02 | 83.95 | 67.34 | 5.10 ms |
| Method | Macro AUC | Macro pAUC | Worst-ID AUC |
|---|---|---|---|
| PatchCore | |||
| SimpleNet | |||
| EfficientAD | |||
| Gen-Rep | |||
| Audio-PC | |||
| CADA |
| Method | Overall | Target | Worst section |
|---|---|---|---|
| Audio-PC | |||
| UniFlow | |||
| Dinomaly | |||
| HARMONY |
| Method | 0% | 0.5% | 1% | 2% | 5% |
|---|---|---|---|---|---|
| Raw+1NN | |||||
| PatchCore | |||||
| Audio-PC | |||||
| MCDD | |||||
| HARMONY |
Anomaly Detection
Anomaly detection finds rare or unexpected patterns in data, from defective products and system faults to fraud and security threats.
See all 652 papers