Local Epochs as Optimal Stopping: Theory and Evidence in Federated Learning

Ke Xiao; Qiyuan Wang; Christos Anagnostopoulos

cs.LGOct 15, 2026

Local Epochs as Optimal Stopping: Theory and Evidence in Federated Learning

Authors: Ke Xiao, Qiyuan Wang, Christos Anagnostopoulos

Organizations: School of Computing Science, University of Glasgow

Abstract

How long should each client train before communicating? Federated learning usually answers with a fixed, hand-tuned number of local epochs, yet an extra epoch helps only while the loss it removes outweighs the client drift it adds. We cast local training as finite-horizon optimal stopping: a client trains while the next epoch's expected loss reduction exceeds its priced squared drift. We prove that this one-stage look-ahead threshold is optimal whenever stopping decisions are nested, derive closed forms for geometric and power-law returns, and show that an estimated rule loses utility only through wrong-signed decisions. Epoch-level evidence supports the model's structure (geometric diminishing returns and additive drift) and reveals a simple regularity: the stage of training, not the client, sets the budget. The resulting rule, FedStop, is never more than one point below tuned FedAvg in mean accuracy. Where its budgets vary, it raises worst-decile accuracy by 4-8 points over tuned FedAvg and an adaptive-interval baseline, and every one of these gains remains significant after a Holm correction. Against the client-side stopping rules ALT and FLASH, it has the highest CIFAR-10 worst decile on every seed and is level elsewhere. It stays stable where per-client early stopping collapses, and each stopping decision takes constant time.

Figures & tables

Figure 1: How FedStop decides. (a, b) Two client-rounds from the CIFAR-10 logs (geometric profile, seed 6): the predicted gain of the next epoch,

\hat{\bar{\ell}}_{i}\hat{\psi}_{i}(k+1)

, against its priced drift

\lambda\hat{\bar{\delta}}_{i}

. The client stops where the gain falls below the drift. (c) Optimal budgets from Corollaries 1 and 2 as functions of the exchange ratio

\kappa

Setting	Partition	Model	$\eta$	$T$	$E^{\star}$
CIFAR-10	Dirichlet $\alpha=0.1$	CNN	0.03	1000	2
CIFAR-10	Dirichlet $\alpha=0.5$	CNN	0.03	1000	1
CIFAR-10 †	Dirichlet $\alpha=1.0$	CNN	0.03	1000	1
CIFAR-10 †	two-class shards	CNN	0.03	1000	2
CIFAR-100	Dirichlet $\alpha=0.1$	CNN	0.03	500	2
FEMNIST	natural writers	LEAF CNN	0.03	1000	2

Table 1: Experimental settings (

\eta

: client learning rate,

E^{\star}

: tuned FedAvg epochs)

	Geometric			Power-law
Setting	Int.	$\bar{\tau}$	Var	Int.	$\bar{\tau}$	Var
CIFAR-10, $\alpha=0.1$	0.98	2.5	0.57	0.31	1.3	0.20
CIFAR-10, $\alpha=0.5$	0.00	1.0	0.00	0.04	1.0	0.03
CIFAR-100	0.00	1.0	0.00	0.00	1.0	0.00
FEMNIST	0.94	5.4	8.17	0.85	2.9	3.05
Shakespeare	0.00	1.0	0.00	0.25	1.4	1.28

Table 2: Stopping behaviour of the two profiles

Figure 2: (a) Mean drift increment

\delta(k)

by local epoch within the same client-rounds of the geometric random-swap runs (budgets

\geq 4

on CIFAR-10 and

\geq 6

on FEMNIST, with 95% intervals). (b) Mean geometric FedStop budget per round (five seeds, 20-round moving average).

Model element	Used in	Evidence (CIFAR-10, FEMNIST)
Geometric returns (P1)	Cor. 1	preferred for 63–79% of clients
Additive drift, stationary or slowly declining (P2)	( 7 ), Cor. 1 , 3	ratio $0.94$ – $0.96$ , stationary on CIFAR-10, $\rho\leq\sigma$ in $86\%$ of FEMNIST epochs
Budgets fall in training, and annealing matters (P3)	( 12 )	$3.8\!\to\!2.0$ and $7.4\!\to\!4.7$ epochs, and fixing budgets per client costs $3.9$ tail points
Round, not client, sets the budget (P3)	( 12 )	random swaps match FedStop (Table 6 )
Over-pricing is safe	Cor. 1	no failure at the two largest $\lambda$
One-stage rule loses little to correlation	Prop. 2	exact AR(1) rule, same scale: $+0.07$ points, $[-0.57,0.72]$

Table 3: Model assumptions and their empirical support

	CIFAR-10	CIFAR-10	CIFAR-100
Method	$\alpha=0.1$	$\alpha=0.5$	$\alpha=0.1$	FEMNIST	Shakespeare
FedAvg, $E=1$	60.7 $\pm$ 1.3	67.8 $\pm$ 1.0	29.0 $\pm$ 1.1	81.7 $\pm$ 1.9	50.0 $\pm$ 0.4
FedAvg, $E=10$	60.5 $\pm$ 0.7	51.2 $\pm$ 23.1	14.2 $\pm$ 12.0	81.3 $\pm$ 1.3	47.5 $\pm$ 0.4
FedAvg, tuned $E^{\star}$	60.2 $\pm$ 4.0	67.8 $\pm$ 1.0	30.4 $\pm$ 1.8	81.8 $\pm$ 1.4	50.0 $\pm$ 0.4
Wang et al. (2019)	62.0 $\pm$ 1.1	67.8 $\pm$ 1.2	30.1 $\pm$ 1.8	82.0 $\pm$ 1.5	50.0 $\pm$ 0.4
FedNova ( Wang et al., 2020 )	63.1 $\pm$ 2.2	68.3 $\pm$ 0.6	30.7 $\pm$ 0.9	82.1 $\pm$ 1.3	50.1 $\pm$ 0.5

Table 4: Top-1 accuracy (%, five seeds)

Figure 3: Validation accuracy (%) of fixed-budget FedAvg over epochs

E

and exposure

\eta E

, three seeds per cell. Hatched cells diverged on the stated number of seeds, and the red box marks the best configuration.

	CIFAR-10	FEMNIST		Shakespeare
Method	$\alpha=0.1$	Cross-fit.	Shared	Cross-fit.	Shared
FedAvg, $E=1$	28.7 $\pm$ 3.7	67.6 $\pm$ 6.3	75.1 $\pm$ 3.1	49.4 $\pm$ 1.7	50.7 $\pm$ 0.5
FedAvg, $E=10$	35.6 $\pm$ 4.1	69.6 $\pm$ 5.8	76.3 $\pm$ 2.6	47.2 $\pm$ 1.6	47.9 $\pm$ 0.8
FedAvg, tuned $E^{\star}$	30.7 $\pm$ 2.0	68.3 $\pm$ 5.3	75.5 $\pm$ 3.9	49.4 $\pm$ 1.7	50.7 $\pm$ 0.5
Wang et al. (2019)	32.7 $\pm$ 3.0	68.8 $\pm$ 5.8	75.2 $\pm$ 3.9	50.0 $\pm$ 1.4	50.5 $\pm$ 0.6
FedNova	34.8 $\pm$ 3.4	71.6 $\pm$ 6.3	77.0 $\pm$ 3.7	50.4 $\pm$ 1.9	50.7 $\pm$ 0.5

Table 5: Worst-decile accuracy (%, five seeds)

Profile	Setting	Control	FedStop	Control	$\Delta$	FedStop	Control	$\Delta$
			Top-1 accuracy			Worst-decile accuracy
Geometric	CIFAR-10, $\alpha=0.1$	Random swap	63.5	63.6	$-0.06$ [ $-1.33$ , $1.22$ ]	39.0	39.6	$-0.61$ [ $-2.46$ , $1.24$ ]
Geometric	CIFAR-10, $\alpha=0.1$	Equal split	63.5	63.5	$+0.01$ [ $-0.26$ , $0.28$ ]	39.0	39.3	$-0.35$ [ $-2.09$ , $1.39$ ]
Geometric	FEMNIST	Random swap	81.1	81.5	$-0.46$ [ $-1.50$ , $0.58$ ]	73.1	76.5	$-3.33$ [ $-7.87$ , $1.22$ ]
Geometric	FEMNIST	Equal split	81.1	81.7	$-0.64$ [ $-2.49$ , $1.21$ ]	73.1	74.4	$-1.21$ [ $-3.76$ , $1.34$ ]
Power-law	CIFAR-10, $\alpha=0.1$	Random swap	62.8	63.9	$-1.11$ [ $-2.24$ , $0.03$ ]	33.8	36.3	$-2.54$ [ $-5.19$ , $0.11$ ]

Table 6: FedStop against controls built from its own budgets: every difference is within noise (%, five seeds,

\Delta

: FedStop minus control with 95% CI)

Figure 4: Accuracy over training on seeds 6–10 (mean

\pm

one standard deviation, 25-round moving average). The tables report the final round. On CIFAR-10, early stopping finishes at chance accuracy.

Figure 5: Worst-decile accuracy against total local epochs (means over the five seeds). Lines trace fixed-budget sweeps of FedAvg and FedNova, and FedStop reaches a higher tail with fewer epochs. Per-client early stopping on CIFAR-10 ends at chance and lies below the axis.

	CIFAR-10, $\alpha=0.1$		FEMNIST			Shakespeare
Method	Top-1	Worst decile	Top-1	Cross-fit.	Shared	Top-1	Cross-fit.	Shared
ALT ( Shenaj et al., 2025 )	61.4 $\pm$ 2.1	30.0 $\pm$ 7.2	81.6 $\pm$ 1.2	73.2 $\pm$ 3.2	78.8 $\pm$ 3.0	50.0 $\pm$ 0.5	49.7 $\pm$ 1.0	50.4 $\pm$ 0.5
FLASH ( Panchal et al., 2023 )	63.2 $\pm$ 1.4	35.3 $\pm$ 3.2	81.8 $\pm$ 1.7	71.5 $\pm$ 4.2	78.1 $\pm$ 3.4	50.0 $\pm$ 0.5	49.9 $\pm$ 1.1	50.5 $\pm$ 0.5
FedStop (geometric)	63.5 $\pm$ 0.7	39.0 $\pm$ 2.3	81.1 $\pm$ 1.9	73.1 $\pm$ 5.2	77.0 $\pm$ 3.6	50.1 $\pm$ 0.4	49.5 $\pm$ 1.4	50.8 $\pm$ 0.7

Table 7: Client-side stopping rules (%, five seeds)

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Setting	Accuracy	Tail	Collapsed	Per-seed accuracy
CIFAR-10, $\alpha=0.1$	10.0 $\pm$ 0.0	10.0 $\pm$ 0.0	5/5	10.0, 10.0, 10.0, 10.0, 10.0
FEMNIST	17.6 $\pm$ 35.7	15.0 $\pm$ 30.0	4/5	1.6, 1.6, 1.6, 1.6, 81.4
Shakespeare	54.9 $\pm$ 0.5	52.2 $\pm$ 1.3	0/5	55.1, 54.6, 55.4, 54.3, 55.3

Appendix

Table S1: SCAFFOLD, untuned, at the shared learning rate (%, five seeds)

Configuration	Accuracy	Tail	Epochs ( $10^{3}$ )	$\Delta$ accuracy	$\Delta$ tail
FedNova, $E=1$	61.4 $\pm$ 2.3	30.4 $\pm$ 6.3	10.0	$-2.17$ [ $-4.35$ , $+0.01$ ]	$-8.56$ [ $-14.81$ , $-2.31$ ]
FedNova, $E^{\star}=2$	63.1 $\pm$ 2.2	34.8 $\pm$ 3.4	20.0	$-0.46$ [ $-2.44$ , $+1.51$ ]	$-4.22$ [ $-8.29$ , $-0.15$ ]
FedNova, $E=5$	62.7 $\pm$ 0.6	38.9 $\pm$ 2.8	50.0	$-0.89$ [ $-1.52$ , $-0.26$ ]	$-0.03$ [ $-2.05$ , $+2.00$ ]
FedNova, $E=10$	60.9 $\pm$ 0.9	39.0 $\pm$ 3.0	100.0	$-2.61$ [ $-3.47$ , $-1.75$ ]	$-0.02$ [ $-4.62$ , $+4.57$ ]
FedNova, $E=20$	59.2 $\pm$ 1.1	38.9 $\pm$ 3.3	200.0	$-4.33$ [ $-5.38$ , $-3.28$ ]	$-0.07$ [ $-5.72$ , $+5.58$ ]
Wang et al. (A)	62.0 $\pm$ 1.1	32.7 $\pm$ 3.0	20.5	$-1.58$ [ $-2.58$ , $-0.57$ ]	$-6.29$ [ $-8.18$ , $-4.39$ ]

Appendix

Table S2: Baseline configurations on CIFAR-10,

\alpha=0.1

(

\Delta

: configuration minus geometric FedStop )

Configuration	Accuracy	Tail	Epochs ( $10^{3}$ )	Coll.	$\Delta$ accuracy	$\Delta$ tail
AR(1) rule	62.5 $\pm$ 2.1	33.5 $\pm$ 2.3	13.3	0	$-0.23$ [ $-1.01$ , $+0.56$ ]	$-0.28$ [ $-3.61$ , $+3.05$ ]
Sequential probability ratio test	10.0 $\pm$ 0.0	6.0 $\pm$ 5.5	130.2	5	–	–
Upper-confidence-bound rule	10.4 $\pm$ 0.8	8.0 $\pm$ 4.5	48.9	4	–	–
Linear multi-step look-ahead	61.2 $\pm$ 2.6	29.9 $\pm$ 6.8	10.0	0	$-1.58$ [ $-3.71$ , $+0.55$ ]	$-3.89$ [ $-7.67$ , $-0.10$ ]
Minimum of two epochs	63.3 $\pm$ 1.2	35.6 $\pm$ 2.4	20.1	0	$+0.53$ [ $-0.16$ , $+1.21$ ]	$+1.83$ [ $-1.37$ , $+5.04$ ]
Shrunk loss scale	62.5 $\pm$ 1.8	33.4 $\pm$ 3.6	13.3	0	$-0.30$ [ $-0.85$ , $+0.24$ ]	$-0.42$ [ $-2.34$ , $+1.49$ ]

Appendix

Table S3: Prespecified ablations of the power-law rule on CIFAR-10,

\alpha=0.1

(

\Delta

: variant minus the power-law rule)

Configuration	Accuracy	Tail	Epochs ( $10^{3}$ )	$\Delta$ accuracy	$\Delta$ tail
Random swap within round	63.9 $\pm$ 1.5	36.3 $\pm$ 2.7	13.3	$+1.11$ [ $-0.03$ , $+2.24$ ]	$+2.54$ [ $-0.11$ , $+5.19$ ]
Random swap within size strata	62.8 $\pm$ 1.4	32.7 $\pm$ 5.6	13.3	$-0.02$ [ $-1.04$ , $+1.00$ ]	$-1.08$ [ $-2.72$ , $+0.57$ ]
Permuted across rounds	63.3 $\pm$ 0.9	32.2 $\pm$ 8.5	13.3	$+0.52$ [ $-2.09$ , $+3.14$ ]	$-1.61$ [ $-6.91$ , $+3.68$ ]
De-annealed	61.8 $\pm$ 2.4	29.9 $\pm$ 4.7	13.3	$-0.99$ [ $-2.43$ , $+0.44$ ]	$-3.91$ [ $-5.25$ , $-2.58$ ]
Hindsight oracle (ref.)	60.6 $\pm$ 2.3	29.8 $\pm$ 6.0	200.0	$-2.21$ [ $-3.93$ , $-0.49$ ]	$-3.96$ [ $-6.23$ , $-1.69$ ]
Round-wise validation oracle (ref.)	64.0 $\pm$ 0.5	37.8 $\pm$ 6.5	200.0	$+1.22$ [ $-0.84$ , $+3.27$ ]	$+4.00$ [ $-4.10$ , $+12.09$ ]

Appendix

Table S4: Controls built from power-law budgets on CIFAR-10,

\alpha=0.1

(

\Delta

: control minus the power-law rule)

Figure S1: Epochs per participation chosen by geometric FedStop for every client and 25-round window (reporting seed 6), with clients sorted by local data size. White cells are windows in which a client was not sampled.

Configuration	Accuracy	Tail	Epochs ( $10^{3}$ )	$\Delta$ accuracy	$\Delta$ tail
FEMNIST: Minimum of two epochs	81.6 $\pm$ 1.6	72.4 $\pm$ 5.1	35.0	$+0.50$ [ $-1.38$ , $+2.37$ ]	$-0.71$ [ $-4.00$ , $+2.58$ ]
FEMNIST: Drift price transferred from CIFAR-10, $\alpha=0.5$	81.8 $\pm$ 1.7	71.2 $\pm$ 4.9	10.3	$+0.78$ [ $-1.46$ , $+3.01$ ]	$-1.97$ [ $-6.23$ , $+2.29$ ]
Shakespeare: Minimum of two epochs	48.4 $\pm$ 0.4	47.5 $\pm$ 1.3	10.6	$-1.76$ [ $-1.89$ , $-1.64$ ]	$-2.04$ [ $-3.14$ , $-0.94$ ]
Shakespeare: Drift price transferred from CIFAR-10, $\alpha=0.5$	50.1 $\pm$ 0.5	50.4 $\pm$ 1.9	5.0	$-0.07$ [ $-0.19$ , $+0.05$ ]	$+0.86$ [ $-1.69$ , $+3.40$ ]

Appendix

Table S5: Variants on FEMNIST and Shakespeare (

\Delta

: variant minus geometric FedStop )

Setting, profile	$\theta_{0}$	$s_{0}$	$\delta^{\mathrm{prior}}$	$\lambda$	Shape at prior
CIFAR-10 ( $\alpha=0.1$ ), geometric	0.302	0.160	0.118	0.029	5.3%
CIFAR-10 ( $\alpha=0.1$ ), power-law	2.426	0.561	0.118	1.39	69.1%
CIFAR-10 ( $\alpha=0.5$ ), geometric	0.302	0.160	0.118	$1.27\!\times\!10^{4}$	–
CIFAR-10 ( $\alpha=0.5$ ), power-law	2.426	0.561	0.118	1.39	–
CIFAR-100 ( $\alpha=0.1$ ), geometric	0.221	0.116	1.52	798	–
CIFAR-100 ( $\alpha=0.1$ ), power-law	2.742	0.616	1.52	1.34	–

Appendix

Table S6: Shape-estimator settings and pilot priors

Setting	$\tau$	Ext.	At edge	$\gamma$	Ext.	At edge
	ALT			FLASH
CIFAR-10	0.9889	2	yes	0.160	1	no
FEMNIST	0.8928	0	no	0.645	1	no
Shakespeare	0.9947	2	yes	0.331	2	yes

Appendix

Table S7: Selected thresholds of the client-side rules

Setting	Rule	$\Delta$ top-1	$\Delta$ worst decile	Holm $p$	$\Delta$ fixed cohort	Epochs	Work ( $10^{3}$ )
CIFAR-10	ALT	$+2.16$ [ $-0.06$ , $4.38$ ]	$+8.95$ [ $1.40$ , $16.50$ ]	0.151	–	1.19	14.1
CIFAR-10	FLASH	$+0.38$ [ $-0.62$ , $1.38$ ]	$+3.68$ [ $1.52$ , $5.83$ ]	0.055	–	2.06	21.3
FEMNIST	ALT	$-0.50$ [ $-2.89$ , $1.88$ ]	$-0.03$ [ $-4.39$ , $4.34$ ]	1.000	$-1.73$ [ $-4.92$ , $1.45$ ]	19.07	214.0
FEMNIST	FLASH	$-0.73$ [ $-2.95$ , $1.49$ ]	$+1.68$ [ $-1.84$ , $5.19$ ]	1.000	$-1.11$ [ $-3.15$ , $0.93$ ]	1.06	10.9
Shakespeare	ALT	$+0.15$ [ $0.01$ , $0.30$ ]	$-0.16$ [ $-1.85$ , $1.54$ ]	1.000	$+0.40$ [ $0.07$ , $0.74$ ]	1.00	5.36
Shakespeare	FLASH	$+0.11$ [ $-0.15$ , $0.37$ ]	$-0.36$ [ $-2.07$ , $1.35$ ]	1.000	$+0.28$ [ $-0.35$ , $0.91$ ]	1.01	5.32

Appendix

Table S8: Client-side rules: secondary results (

\Delta

: geometric FedStop minus the rule, 95% CI, unadjusted, with the rule’s epochs per participation and work)

Setting, profile	Full	Subset	$\Delta$ (points)	Full	Subset	$\Delta$ (points)	$\Delta$ top-1	$\Delta$ worst decile
	Work to reach target			Work over full horizon
CIFAR-10, geometric	161.3	148.6	$-12.8$ [ $-25.3$ , $-0.3$ ]	157.1	140.8	$-16.4$ [ $-17.1$ , $-15.6$ ]	$-0.31$ [ $-0.88$ , $0.27$ ]	$-1.34$ [ $-2.57$ , $-0.10$ ]
CIFAR-10, power-law	98.5	89.2	$-9.4$ [ $-14.4$ , $-4.3$ ]	88.3	77.3	$-10.9$ [ $-11.4$ , $-10.5$ ]	$-0.37$ [ $-1.62$ , $0.88$ ]	$-1.22$ [ $-3.43$ , $0.99$ ]
CIFAR-100, geometric	111.6	91.0	$-20.7$ [ $-23.2$ , $-18.2$ ]	67.7	55.2	$-12.5$ [ $-12.6$ , $-12.4$ ]	$+0.15$ [ $-0.20$ , $0.50$ ]	$+0.24$ [ $-0.00$ , $0.47$ ]
FEMNIST, geometric	222.4	203.3	$-19.0$ [ $-57.1$ , $19.1$ ]	307.2	298.2	$-9.0$ [ $-12.4$ , $-5.6$ ]	$-0.42$ [ $-0.89$ , $0.04$ ]	$-0.56$ [ $-4.44$ , $3.31$ ]
CIFAR-10, geometric, same- $\lambda$ repeat	161.3	138.1	$-23.2$ [ $-35.8$ , $-10.6$ ]	157.1	140.9	$-16.3$ [ $-17.0$ , $-15.5$ ]	$+0.02$ [ $-0.94$ , $0.98$ ]	$-0.20$ [ $-2.51$ , $2.12$ ]

Appendix

Table S9: Monitoring on a fixed 128-example subset (work in % of two-epoch FedAvg,

\Delta

: subset minus full monitoring, 95% CI)

Figure S2: Subset monitoring. (a) Measured work to reach

95\%

of tuned FedAvg’s final accuracy, as a percentage of two-epoch FedAvg’s work (mean

\pm

one standard deviation over five seeds). (b) Share of monitoring passes kept by a fixed subset of

\min(s,n_{i})

examples, computed from the client sizes and epoch counts of the reporting runs.

Method	Accuracy	Worst decile	Epochs	Compute
FedAvg, $E=1$	57.0 $\pm$ 1.4	34.6 $\pm$ 4.5	1.0	1.2
FLASH client rule	61.5 $\pm$ 2.3	41.4 $\pm$ 9.4	2.3	2.9
ALT	59.4 $\pm$ 1.1	41.1 $\pm$ 3.7	18.2	29.1
FedStop (geometric)	60.4 $\pm$ 1.2	39.7 $\pm$ 4.4	3.8	6.4
Equal split (control)	61.4 $\pm$ 1.2	40.8 $\pm$ 6.5	3.8	4.5

Appendix

Table S10: Synthetic benchmark (%, five seeds, compute in

10^{3}

client-epoch equivalents)

Quantity	CIFAR-10	FEMNIST
First-decision drift price / later realised drift rate (median)	16.2	6.9
Client-rounds stopped after one epoch	2.1%	5.1%
Fitted $\hat{\rho}$ : median [IQR]	0.05 [0.05, 0.12]	0.18 [0.08, 0.36]
$\hat{\rho}$ at a search bound	65.1%	15.1%
Pooled lag-one correlation $\phi$ of progress surprises	$-0.40$	$-0.32$
Client-rounds ( $\geq 4$ epochs) meeting the margin of Corollary 5	0.0%	0.0%

Appendix

Table S11: Diagnostics computed from the epoch logs of the reporting runs

Federated Learning

Federated learning trains shared models across data holders without requiring them to pool their raw data.

See all 961 papers

Beyond Aggregate Drift: Optimal Client-Specific Windows in Federated Learning
Oct 11, 2026Yuriy Dorn, Dmitrii Vishovan, Kirill Novoselov +3Federated Learning Online Federated Learning
Lomonosov Moscow State University · Innopolis University · Moscow Independent Research Institute of Artificial Intelligence +3
Rethinking the Personalized Relaxed Initialization in the Federated Learning: Consistency and Generalization
Apr 14, 2026Li Shen, Yan Sun, Dacheng TaoFederated Learning Non-IID Federated Learning
School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-sen University · School of Computer Science, Faculty of Engineering, The University of Sydney · Generative AI Lab, College of Computing and Data Science, Nanyang Technological University
FedeRage: Provably Convergent Agnostic Federated Learning under General Client Drift
Sep 17, 2026Herlock Rahimi, Dionysis KalogeriasFederated Learning Federated Learning Aggregation

Oct 11, 2026J/K move · Enter open · S save

Beyond Aggregate Drift: Optimal Client-Specific Windows in Federated Learning

Yuriy Dorn, Dmitrii Vishovan, Kirill Novoselov, Ilya Kuruzov+2

Lomonosov Moscow State University · Innopolis University · Moscow Independent Research Institute of Artificial Intelligence+3

Federated Learning Online Federated Learning Federated Continual Learning Distributed Optimization Non-IID Federated Learning

Open paper AskPDF arXiv

We study federated learning from continuously arriving data: each client samples from its own distribution, these distributions drift over time, and the server must track the minimizer of the clients' weighted average loss. Clients are not alike as they collect data at different rates, with different noise, and change at different speeds. The main question is how much history should each one keep? A single shared window must serve every client at once, and so makes a fast-changing client carry the sampling noise of a slow, nearly stationary one. We give each client a memory of its own, sized to that client's noise, arrival rate, and drift, and refresh it by restarted accelerated method that starts from a common point. The tracking error is then bounded by

R\left(\sum_m w_m^{3/2}\,σ_m\sqrt{γ_m/λ_m}\right)^{2/3},

for clients with weight

w_m

, noise

σ_m

, arrival rate

λ_m

, and drift rate

γ_m

on a domain of diameter

R

. This is never worse than what any common window can achieve, and strictly better whenever noise and drift are spread unevenly across clients. A lower bound that puts different groups of clients on different drift clocks shows this quantity is the right one. The two bounds agree up to a constant whenever the clients occupy boundedly many refresh scales, and in general differ only by the sixth root of the number of scales. Experiments show the gain comes from the memory rather than from the solver: it persists across accelerated, stochastic-gradient, and one-step methods, and against a separately tuned common-memory baseline.

Abstract

Federated Learning

Explore similar work

Beyond Aggregate Drift: Optimal Client-Specific Windows in Federated Learning

Rethinking the Personalized Relaxed Initialization in the Federated Learning: Consistency and Generalization

FedeRage: Provably Convergent Agnostic Federated Learning under General Client Drift