FedDyMem: Efficient Federated Learning with Dynamic Memory and Memory-Reduce for Unsupervised Image Anomaly Detection
Authors: Silin Chen, Andy Liu, Kangjian Di, Yichu Xu, Han-Jia Ye, Wenhan Luo, Ningmu Zou
Organizations: School of Integrated Circuits, Nanjing University, Suzhou, 215163, Jiangsu, China. · National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, 210023, Jiangsu, China. · School of Artificial Intelligence, Nanjing University, Nanjing, 210023, Jiangsu, China. · Hong Kong University of Science and Technology, Hong Kong. · Interdisciplinary Research Center for Future Intelligent Chips (Chip-X), Nanjing University, Suzhou, 215163, Jiangsu, China.
Unsupervised image anomaly detection (UAD) has become a critical process in industrial and medical applications, where models are trained on normal samples without requiring labeled anomalies. However, even normal images may contain sensitive product designs, manufacturing information, or institution-specific population characteristics, making centralized data collection difficult in practical deployments. Federated UAD provides a promising decentralized solution, but it remains challenging because clients use only normal training samples, their feature distributions remain highly heterogeneous within the same task, and communication should be efficient while limiting the exposure of sensitive client information. To address these challenges, we propose FedDyMem, an efficient federated learning method with dynamic memory and memory-reduce for unsupervised image anomaly detection. Instead of exchanging full model parameters, FedDyMem facilitates knowledge sharing between the clients and server through reduced dynamic memory banks. In the local clients, a memory generator and a metric loss are employed to improve the consistency of the feature distribution for normal samples, leveraging the local model to update the memory bank dynamically. For efficient communication and data privacy, a memory-reduce method based on weighted averages is proposed to significantly decrease the scale of memory banks. This representation reduces communication overhead and empirically lowers reconstruction fidelity under inversion attacks. On the server, global memory is constructed and distributed to individual clients through k-means aggregation. Experiments on six federated UAD tasks constructed from eleven public industrial and medical datasets show that FedDyMem consistently improves anomaly detection and localization while reducing communication cost.
Figures & tables
Figure 1: Illustration of federated learning in UAD. As shown on the left, for the same inspection or screening task, an intra-class distribution bias exists between local models, caused by clients possessing different and incomplete subsets of normal data within the same inspection or screening task. As shown in the top right, federated UAD aims to construct a shared normal distribution for one-class anomaly detection across clients. Examples of type differences in anomaly detection are shown in the bottom right.
Figure 2: Overview of our proposed FedDyMem framework.
Figure 3: Illustration of the memory feature generation process in each client model.
Figure 4: Illustration of Memory-reduce.
Task
Partitioning attribute
#Groups
N
α
Brain
Data source
3
5
0.1
X-ray
Data source
3
5
0.1
PCB
PCB product type
6
5
0.1
Texture
Texture class
5
5
0.1
MPDD
Metal-part type
6
5
0.1
Wafer
Wafer product type
14
5
0.1
Table 1: Default non-IID client partition used in the main experiments.
Figure 5: Local metric loss curves of the first client over 200 communication rounds on four federated UAD tasks.
Figure 6: Illustration of I-AUROC over communication rounds on six federated UAD tasks.
Figure 7: Comparison of feature distributions on the texture task after 200 communication rounds. (a) t-SNE ( Maaten and Hinton (2008) ) visualization of local and global memory banks generated by five clients using FedAvg (top row) and FedDyMem (bottom row). (b) Comparison of global memory banks aggregated from FedDyMem and FedAvg clients, together with the corresponding I-AUROC results.
Methods
Communication
Wafer
PCB
Texture
MPDD
Brain
X-ray
Local
-
57.35( ± 0.71)
51.89( ± 0.76)
75.94( ± 0.39)
54.39( ± 0.77)
50.66( ± 0.87)
61.02( ± 0.58)
FedAvg
Model Parameters: ≈ 10.66M, (SCAFFOLD) ≈ 10.66M × 2
63.91( ± 0.36)
54.18( ± 0.33)
80.32( ± 0.81)
61.87( ± 0.76)
50.16( ± 0.63)
61.48( ± 0.56)
FedProx
63.98( ± 0.51)
54.11( ± 0.48)
80.22( ± 0.78)
61.83( ± 0.61)
50.47( ± 0.34)
61.45( ± 0.66)
FedPHP
63.94( ± 0.56)
54.13( ± 0.36)
80.19( ± 0.62)
61.80( ± 0.63)
50.38( ± 0.39)
61.45( ± 0.37)
APPLE
65.92( ± 0.33)
56.79( ± 0.65)
82.43( ± 0.51)
61.03( ± 0.77)
50.82( ± 0.45)
68.47( ± 0.51)
FedBN
63.89( ± 0.35)
54.09( ± 0.79)
80.13( ± 0.77)
61.90( ± 0.61)
50.04( ± 0.54)
61.45( ± 0.72)
Table 2: Comparison of Image-AUROC (%) results on six federated UAD tasks under the default non-IID protocol.
Methods
Pixel-AUROC (%)
PRO (%)
Wafer
PCB
Texture
MPDD
Wafer
PCB
Texture
MPDD
Local
58.27( ± 1.20)
68.12( ± 1.11)
72.63( ± 0.98)
87.02( ± 1.05)
47.76( ± 0.52)
45.86( ± 1.31)
56.97( ± 0.48)
51.31( ± 0.73)
FedAvg
66.56( ± 0.70)
69.30( ± 0.34)
76.16( ± 0.73)
91.67( ± 0.94)
67.00( ± 0.30)
54.04( ± 0.57)
53.35( ± 1.49)
55.07( ± 1.25)
FedProx
66.41( ± 0.74)
69.10( ± 1.28)
76.08( ± 1.28)
91.59( ± 1.02)
67.25( ± 0.33)
54.04( ± 1.28)
53.62( ± 0.70)
54.96( ± 1.16)
FedPHP
66.44( ± 1.38)
69.11( ± 0.42)
76.03( ± 0.57)
91.56( ± 0.73)
67.16( ± 0.38)
54.23( ± 1.16)
53.37( ± 0.47)
54.86( ± 1.42)
APPLE
69.88( ± 0.95)
71.34( ± 0.85)
78.06( ± 1.08)
92.12( ± 1.19)
69.02( ± 0.42)
60.11( ± 1.45)
54.14( ± 1.16)
55.84( ± 0.76)
Table 3: Comparison of Pixel-AUROC (%) and PRO (%) results on four federated UAD tasks under the default non-IID protocol.
Methods
Setting
Wafer
PCB
Texture
MPDD
Brain
X-ray
All normal training data are jointly accessible.
PatchCore
Centralized
75.56/77.92/71.80
91.25/96.12/77.97
99.04/97.46/88.76
85.00/98.88/71.31
95.69/-/-
78.10/-/-
SimpleNet
Centralized
69.92/72.16/70.51
84.35/96.17/70.47
99.32/96.43/85.59
86.91/98.92/74.65
92.79/-/-
77.96/-/-
UniAD
Centralized
74.05/83.23/76.56
83.79/96.74/79.59
98.92/94.90/89.63
69.28/87.65/70.06
87.51/-/-
84.89 /-/-
MambaAD
Centralized
76.34/84.69/84.67
94.59/ 99.11 / 91.52
98.41/97.99/88.15
88.11/96.74/90.58
97.73/-/-
82.67/-/-
InpFormer
Centralized
81.85 / 85.02 / 85.05
94.74 /98.87/88.36
99.82 / 98.67 / 95.56
91.09 / 99.55 / 94.56
98.73 /-/-
83.44/-/-
Table 4: Comparison with representative UAD methods under centralized and federated settings. Results are reported as the mean I-AUROC/P-AUROC/PRO (%).
Methods
Setting
Wafer ( N=14 )
Brain ( N=3 )
UniAD
FedAvg
67.48/76.06/61.75
74.81/-/-
MambaAD
FedAvg
65.91/69.32/55.66
86.42/-/-
InpFormer
FedAvg
64.15/71.16/53.45
83.31/-/-
Ours
FedDyMem
70.20 / 77.62 / 73.37
88.24 /-/-
Table 5: Comparison under the group-disjoint federated setting. Results are reported as the mean I-AUROC/P-AUROC/PRO (%).
Figure 8: Impact of different local epochs on clients.
Ratio
Methods
Brain
MPDD
Texture
20%
FedAvg
50.06/-/-
58.65/90.72/54.22
77.18/72.64/50.53
Ours
85.05/-/-
79.60/97.97/75.68
98.34/96.92/81.82
40%
FedAvg
50.28/-/-
58.27/90.41/54.11
78.50/74.16/50.20
Ours
86.49/-/-
81.23/97.97/77.54
99.09/96.89/82.53
60%
FedAvg
50.08/-/-
60.62/91.01/54.20
78.44/74.14/51.75
Ours
87.59/-/-
82.96/97.94/78.91
99.09/96.86/83.47
Table 6: Impact of per-round local training sample ratio. Results are reported as the mean I-AUROC/P-AUROC/PRO (%).
rEMA
DW
Brain
MPDD
Texture
-
-
84.47/-/-
78.63/95.72/74.17
97.91/95.32/83.13
✓
-
86.75/-/-
80.48/98.88/74.85
98.66/96.88/85.67
-
✓
87.62/-/-
81.88/97.85/77.73
98.99/96.99/85.88
✓
✓
89.27 /-/-
83.93 / 98.97 / 79.43
99.24 / 97.04 / 86.81
Table 7: Ablation study of rEMA and Dynamic Weighting (DW) in the memory-reduce.
Figure 9: Impact of parameter initialization.
Factor
Setting
PCB
Brain
Hg×Wg
2×2
79.02/93.38/78.89
86.27/-/-
4×4
82.24/95.74/80.96
88.73/-/-
8×8
83.34/ 96.03 / 81.09
89.27 /-/-
16×16
83.36 /95.87/80.99
88.93/-/-
knn
1
78.84/93.99/78.31
86.15/-/-
3
83.34 / 96.03 / 81.09
89.27 /-/-
Table 8: Impact of hyperparameters and backbones. Results are reported as I-AUROC/P-AUROC/PRO (%).
Configuration
N
Tˉagg
Texture
Average Aggregation
5
0.0241
92.76/96.10/84.42
Kclu=Q,Iclu=5
5
0.2373
97.94/95.77/85.12
Kclu=Q,Iclu=10
5
0.2885
99.09/97.07/86.01
Kclu=Q,Iclu=15
5
0.3429
99.24/97.04/ 86.81
Kclu=Q,Iclu=20
5
0.3987
99.22/97.05/86.81
Kclu=Q,Iclu=40
5
0.5375
99.27/97.03/86.77
Table 9: Effectiveness and scalability of K -means. Results are reported as the mean per-round server aggregation time Tˉagg (s) measured over 200 communication rounds and I-AUROC/P-AUROC/PRO (%).
+Proj
+MG
+KM
I-AUROC (%)
P-AUROC (%)
Wafer
PCB
Texture
MPDD
Brain
X-ray
Wafer
PCB
Texture
MPDD
-
-
-
67.69
69.58
85.86
70.41
69.03
68.92
70.23
83.23
85.22
94.19
✓
-
-
68.93
74.10
87.27
78.28
74.95
72.71
74.29
89.26
90.84
95.77
✓
✓
-
70.96
81.26
92.76
81.05
85.94
78.12
77.01
94.97
96.10
97.96
✓
✓
✓
71.96
83.34
99.24
83.93
89.27
80.30
78.40
96.03
97.04
98.97
Table 10: Ablation study of structure for FedDyMem.
Ratio
Methods
Brain
MPDD
Texture
0%
FedAvg
50.16/-/-
61.87/91.67/55.07
80.32/76.16/53.35
Ours
89.27/-/-
83.93/98.97/79.43
99.24/97.04/86.81
1%
FedAvg
50.14/-/-
59.43/89.91/54.91
77.97/73.93/49.74
Ours
88.90/-/-
83.86/98.76/79.40
98.97/96.95/85.85
5%
FedAvg
50.12/-/-
58.49/88.07/53.24
72.96/68.34/47.59
Ours
88.50/-/-
83.52/98.47/79.55
98.35/96.87/85.44
Table 11: Robustness to contaminated local training data. Ratio denotes the proportion of contaminated samples included in each client’s local training set. Results are reported as the mean I-AUROC/P-AUROC/PRO (%).
Figure 10: Robustness to dynamic client participation.
Dataset
Local-only (w/o B(T) )
Local Adaptation (w/ B(T) )
Wafer
57.35/58.27/47.76
71.66/75.96/73.11
PCB
51.89/68.12/45.86
82.88/95.67/79.96
Texture
75.94/72.63/56.97
99.11/96.80/85.91
MPDD
54.39/87.02/51.31
81.83/98.92/78.36
Brain
50.66/-/-
87.77/-/-
X-ray
61.02/-/-
79.71/-/-
Table 12: Local adaptation for a newly arriving client after federated training. Results are reported as the mean I-AUROC/P-AUROC/PRO (%).
Method
N
αDir
Texture
MPDD
FedAvg
5
0.01
61.22/56.09/52.94
55.36/86.95/50.45
Ours
5
0.01
99.12/96.85/85.30
83.00/98.90/79.15
FedAvg
5
0.5
89.42/79.24/63.50
63.07/91.31/67.34
Ours
5
0.5
99.14/96.85/85.48
84.80/98.99/79.59
FedAvg
5
1.0
97.26/86.67/77.35
63.81/92.37/70.89
Ours
5
1.0
99.13/96.87/87.07
84.58/98.96/79.81
Table 13: Performance under different client counts N and Dirichlet partition parameters αDir .
Injected carpet ratio ρ (# samples)
Carpet
Average
5% (14)
96.11/93.50/80.61
97.86/94.75/81.42
10% (28)
96.91/93.76/80.80
97.87/94.99/81.23
20% (56)
97.47/96.15/81.54
98.24/95.10/83.01
50% (140)
98.79/97.42/86.01
98.52/95.28/84.53
100% (279)
99.10 / 98.42 / 86.71
99.03 / 96.96 / 86.73
Table 14: Robustness to a rare normal mode. Results are reported as I-AUROC/P-AUROC/PRO (%).
Figure 11: Qualitative comparison of FedAvg and our proposed method. The results were obtained from a randomly selected client using a global memory bank.
Figure 12: Qualitative reconstruction results before and after applying the proposed memory-reduce operation on Texture, PCB, and Brain.
Attack
Dataset
+Memory-
SSIM ↓
PSNR ↓
FID ↑
Methods
Reduce
Deconvolution Inverting Method
Texture
-
0.6350
25.9003
246.9916
✓
0.2088
17.4787
415.9250
PCB
-
0.5213
25.8625
281.5855
✓
0.3379
12.1827
358.7053
Brain
-
0.2273
11.0985
300.5543
Table 15: Quantitative reconstruction attack results before and after memory-reduce.
Method
Server-visible Message
ROC-AUC ↓
TPR@1% FPR ↓
Advantage ↓
FedAvg
Δθn(T)
0.6024±0.0206
0.0619±0.0104
0.1687±0.0420
Raw-memory Sharing
{Mn,i(T)}i
0.7537±0.0403
0.3936±0.0125
0.6583±0.0428
FedDyMem
Rn(T)
0.5310±0.0223
0.0288±0.0122
0.0960±0.0390
Table 16: Membership inference on the texture task under the passive honest-but-curious server threat model. Values are reported as mean ±95% normal-approximation confidence intervals over five held-out federations.
Method
Server-visible Message
Client Domain Inference
Client Property Inference
Balanced Accuracy(%) ↓
Macro F1(%) ↓
AUROC ↓
Balanced Accuracy(%) ↓
F1 Score (%) ↓
FedAvg
Δθn(T)
66.8±5.9
64.5±6.2
0.840±0.050
76.1±5.1
75.0±5.5
Raw-memory Sharing
{Mn,i(T)}i
94.7±2.7
94.1±2.9
0.985±0.010
95.2±2.5
95.0±2.7
FedDyMem
Rn(T)
54.9±6.4
51.7±7.0
0.720±0.060
65.7±6.2
64.9±6.5
Table 17: Client domain and property inference on the texture task under the passive honest but curious server threat model. Values are reported as mean ± standard deviation across five partition level attacker train and test splits constructed from 20 independently generated federation partitions.
Attention layers are the backbone of today's most powerful and impactful models. Models with multi-million and billion parameters rely on contextual knowledge provided by attention layers. However, their use goes well beyond just being the core component of large language models. One particularly interesting application is in Memory Augmented Autoencoders (MemAE), specifically for unsupervised representation learning in outlier detection tasks. It was shown that attention helps these models be more effective in centralized learning scenarios. Our work aims to address the lack of specialized aggregation techniques in Federated Learning (FL) when it comes to MemAE models. In this paper we analyze the intricacies of the architecture behind Memory Augmented Autoencoders, and propose novel, guided approaches to effectively aggregate these models in federated scenarios. We demonstrate our approach on non-IID datasets and show that these novel aggregation schemes are more robust when dealing with numerous edge nodes in environments with unbalanced datasets, specifically for unsupervised anomaly detection scenarios. This approach improves the performance of even very shallow autoencoders, allowing them to be used in resource constrained environments.
Mihailo Ilić, Miloš Savić, Vladimir Kurbalija +3
Department of Mathematics and Informatics, Faculty of Sciences, University of Novi Sad, Serbia · Department of Informatics, Modeling, Electronics, and Systems, University of Calabria, Italy
In real-world scenarios, industrial processes can span multiple factories and goods are often produced at multiple sites, leading to a shared yet non-identical distribution of data across locations. To address this, we propose two federated approaches for i) combining existing coresets (Double Subsampling) and ii) creating new coresets (Federated Subsampling) in a distributed anomaly detection setting. In particular, we adapt the memory bank based PatchCore algorithm to a federated setting, resulting in increased cold-start performance, resilience to domain- and sensor-shift, and resilience to unreliable actor participation, a common problem in classical, network based federated learning strategies.
Jason Witzemann, Daniel Pröll, Sebastian Hegenbart
Digital Factory Vorarlberg GmbH, Dornbirn, Austria · University of Salzburg, Salzburg, Austria · Vorarlberg University of Applied Sciences, Dornbirn, Austria
Unsupervised federated learning (UFL) enables privacy-preserving distributed training without data labeling, yet practical deployment remains challenging due to non-IID data, high computational and communication costs at edge devices, and sensitivity to communication noise. We propose FedUHD, the first UFL framework based on Hyperdimensional Computing (HDC). On the client side, FedUHD employs kNN-based cluster hypervector removal to mitigate non-IID effects by filtering detrimental local outliers. On the server side, cluster-aware HDC aggregation leverages cluster-level statistics to stabilize learning across heterogeneous clients. To further improve efficiency, we design a compute-in-memory (CIM) accelerator based on a novel phase-change memory (PCM) device, integrated with lightweight ASIC digital modules to execute the client-side HDC pipeline within the accelerator. The intrinsic robustness of HDC to low precision and device variations enables efficient mapping onto analog PCM crossbars, exploiting massive parallelism while minimizing data movement. Experimental results show that FedUHD achieves comparable accuracy to state-of-the-art neural network-based UFL methods across all datasets. On HAR and CIFAR10/100, FedUHD delivers an average 2,239x speedup and 1,542x higher energy efficiency on GPU. In addition, FedUHD reduces communication cost by up to 176x on HAR and CIFAR10/100 and demonstrates greater robustness than Orchestra under communication noise. Compared to GPU implementation of FedUHD, the proposed PCM-based accelerator provides an additional 4.07x speedup and three orders of magnitude higher energy efficiency on average. Furthermore, the results demonstrate the benefit of PCM over RRAM as a CIM substrate.