From Selection to Scheduling: Federated Geometry-Aware Correction Makes Exemplar Replay Work Better under Continual Dynamic Heterogeneity
Authors: Zhuang Qi, Ying-Peng Tang, Lei Meng, Guoqing Chao, Lei Wu, Han Yu, Xiangxu Meng
Organizations: School of Software, Shandong University, China · College of Computing and Data Science, Nanyang Technological University, Singapore · School of Computer Science and Technology, Harbin Institute of Technology, China
Exemplar replay has become an effective strategy for mitigating catastrophic forgetting in federated continual learning (FCL) by retaining representative samples from past tasks. Existing studies focus on designing sample-importance estimation mechanisms to identify information-rich samples. However, they typically overlook strategies for effectively utilizing the selected exemplars, which limits their performance under continual dynamic heterogeneity across clients and tasks. To address this issue, this paper proposes a Federated gEometry-Aware correcTion method, termed FEAT, which alleviates imbalance-induced representation collapse that drags rare-class features toward frequent classes across clients. Specifically, it consists of two key modules: 1) the Geometric Structure Alignment module performs structural knowledge distillation by aligning the pairwise angular similarities between feature representations and their corresponding Equiangular Tight Frame prototypes, which are fixed and shared across clients to serve as a class-discriminative reference structure. This encourages geometric consistency across tasks and helps mitigate representation drift; 2) the Energy-based Geometric Correction module removes task-irrelevant directional components from feature embeddings, which reduces prediction bias toward majority classes. This improves sensitivity to minority classes and enhances the model's robustness under class-imbalanced distributions.
Figures & tables
Figure 1: (a) Continual dynamic heterogeneity: clients see different class distributions over time, and each new task is learned with limited replay of past data. (b) Federated geometry-aware calibration: FEAT aligns client feature geometry and applies an energy-based correction at inference to calibrate prediction.
Figure 2: Illustration of the proposed FEAT framework. It contains two main modules: 1) the Geometry Structure Alignment module aligns local representations with global ETF prototypes to alleviate inter-client heterogeneity; and 2) the Energy-based Geometric Correction module removes biased components from the feature space during inference.
Figure 3: Cross-client feature alignment in an incremental 5-task scenario. At Task 3 (left) and Task 5 (right), cross-client alignment for earlier classes is clearly weaker than for current classes.
Figure 4: The GSA module reduces the tendency of tail classes to drift toward head-class directions, but tail-class features still show high similarity to head-class prototypes. Moreover, a notable fraction of tail-class samples are aligned with the head-class subspace.
Method
CIFAR10
CIFAR100
TinyImageNet-Subset
3 Tasks
5 Tasks
5 Tasks
10 Tasks
5 Tasks
10 Tasks
β=0.5
β=1.0
β=0.5
β=1.0
β=0.1
β=0.5
β=0.1
β=0.5
β=0.1
β=0.5
β=0.1
β=0.5
K=5 Clients
FedEWC
40.48 ±1.6
43.11 ±3.1
22.16 ±2.1
29.38 ±2.3
18.66 ±2.6
22.64 ±4.4
10.12 ±0.6
11.34 ±0.6
11.52 ±1.4
14.27 ±2.2
7.92 ±0.7
8.85 ±0.9
FedLwF
56.84 ±1.2
58.97 ±3.1
39.21 ±1.9
52.32 ±2.7
34.45 ±2.0
36.34 ±2.6
19.36 ±1.4
24.91 ±0.9
16.78 ±0.8
21.87 ±0.7
12.16 ±0.6
13.11 ±0.3
TARGET
43.53 ±3.8
54.87 ±4.2
36.02 ±2.4
48.91 ±1.6
30.62 ±2.9
32.89 ±3.7
13.04 ±1.1
20.57 ±0.7
17.69 ±1.5
21.11 ±1.3
11.98 ±0.5
15.28 ±0.7
Table 1: Performance comparison between FEAT and baselines across three datasets under varying levels of data heterogeneity ( β ) and different numbers of clients ( K ). Results are averaged over three random seeds, with standard deviations reported.
Method
CIFAR10
CIFAR100
β=0.5
β=1.0
β=0.1
β=0.5
FedCBDR
63.32 ±1.6
65.88 ±2.1
45.84 ±2.0
50.33 ±2.3
+ETF
62.17 ±1.6
63.66 ±2.0
44.58 ±1.9
48.69 ±1.2
+ETF+GSA
68.54 ±1.6
70.42 ±1.7
47.77 ±1.2
52.23 ±1.6
+ETF+EGC
69.12 ±1.8
70.83 ±2.1
47.16 ±1.4
51.72 ±1.5
+ETF+GSA+EGC
72.67 ±1.5
74.21 ±2.6
50.14 ±1.2
53.31 ±0.9
Table 2: Ablation studies are conducted on CIFAR-10 with 3 tasks under β∈{0.5,1.0} and on CIFAR-100 with 5 tasks under β∈{0.1,0.5} . The number of clients is 5.
Figure 5: Evaluation on CIFAR-10 (5 tasks), CIFAR-100 (10 tasks), and TinyImageNet-Subset (5 tasks) with 5 clients and heterogeneity β=0.5 . FEAT F and FEAT R consistently outperform their baselines on all cases.
Figure 6: Impact of per-task memory capacity M on model performance across different datasets, where FEAT F shows the best overall performance.
Figure 7: Sensitivity analysis of hyperparameters λ , ρ , and τ , adjusted over the ranges {0.05,0.1,0.5} , {0.5,0.7,0.9} , and {0.07,0.5} , respectively. For each parameter, the default value is fixed while the other two are varied. It achieves robust performance under a wide range of these parameters.
Methods
CIFAR10
CIFAR100
Tiny-Subset
β=0.5
β=1.0
β=0.1
β=0.5
β=0.1
β=0.5
FedCBDR
61.77
64.79
45.84
50.33
25.22
26.38
+ MOON
62.56
66.32
45.14
49.32
24.73
26.44
+ FedRCL
58.74
60.21
42.19
46.46
20.27
22.46
+ CLIP2FL
62.33
67.97
46.19
51.11
24.18
26.67
+ FedFSA
64.42
68.32
46.88
51.38
24.61
26.33
Table 3: Performance analysis of alternative optimization strategies, including MOON, FedRCL, CLIP2FL, FedFSA, and FEAT , applied on top of the FedCBDR baseline. Results are reported as the mean over three runs on three datasets.
CIFAR10
CIFAR100
Task 1
Task 2
Task 3
Task 1
Task 2
Task 3
Task 4
Task 5
FedCBDR
43.37
33.90
96.80
43.80
34.25
32.85
45.00
83.55
+ GSA
47.47
51.60
97.92
48.75
36.25
36.10
52.15
84.40
+ EGC
53.53
53.57
97.65
50.15
38.70
39.10
51.40
82.45
+ EGC + GSA
60.83
69.97
94.28
50.70
40.65
39.70
53.85
81.95
Table 4: Per-task and average accuracy (%) of different methods on CIFAR10 and CIFAR100.
Figure 8: Performance of Re-Fed+ and FedCBDR, with and without FEAT , across local epochs {1,2,5,10} on three datasets. And FEAT brings performance gains to all baselines.
Figure 9: Visualization of visual attention. The GSA module corrects the attention bias of tail classes and strengthens the confidence of attention on head classes.
Figure 10: Geometric correction effect of the EGC module on the CIFAR-10 (first two tasks). (a–b) EGC corrects the prediction bias; (c) EGC corrects the prediction bias; (d) it struggles when features deviate too far from the ground truth; (e) The removed component aligns more with the head subspace than the tail subspace for both Class 0 and Class 1.