cs.LGOct 20, 2026

Discount Yourself: Self-inclusive Freshness-weighted Gossip for Decentralised Learning under Client Outages

Authors: Qiyuan Wang, Ke Xiao, Christos Anagnostopoulos

Organizations: University of Glasgow

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Decentralised federated learning (DFL) trains personalised models on devices that average with their neighbours rather than with a central server. Real devices, however, are often offline for hours or days, and a client returning from an outage can recover only by aggregating the models its neighbourhood holds. We observe that the stalest of these models is usually the client's own, a case that existing staleness-aware rules, which discount only the models a client receives, do not address. We formalise aggregation after an outage as estimating the state of a fully available network and derive three principles. First, the receiver must be discounted like any neighbour. Every rule that weights it at least as much as each neighbour has an error that grows without bound with the outage, whereas discounting all candidates by relative staleness keeps the error bounded. Second, staleness must be measured as inherited lag rather than age, since local training resets a model's age but not the lag it absorbed from stale neighbours. We introduce the effective lag, which tracks this quantity at the cost of one scalar per message. Third, the discount must be steep, because absorbed lag persists in every later round. These principles define Self-inclusive Freshness-weighted Gossip (SFG), a family of drop-in aggregation rules with the communication and computation of decentralised federated averaging, whose guarantees hold for the stale receiver itself. On three image benchmarks and eight outage processes, including real smartphone availability traces, the lag-based members SFG-L and SFG-Cut consistently outperform decentralised federated averaging and recent age-of-information rules, and ablations show that each principle is necessary. The analysis also explains why tuned limited merging from gossip learning is effective.

Figures & tables

Figure 1: The stalest model in the room is your own. A client ii returns from an outage of τ\tau rounds to neighbours that kept training. (a) Decentralised federated averaging (DFedAvg) and neighbour-only age-of-information (AoI) rules keep the receiver’s own weight at least as large as each neighbour’s. Its stale model therefore always enters the aggregate, and the error grows without bound in τ\tau ( Theorem 2 ). (b) Self-inclusive Freshness-weighted Gossip (SFG), the rule family we propose, treats the receiver as one more candidate and discounts it by its staleness relative to the freshest candidate. This keeps every receiver’s error bounded ( Theorem 3 ).
RuleSelf-inclusiveInheritedSteepBound
DFedAvg ( Sun et al., 2023 )––––
Neighbour-only AoI––✓–
DFL-AA ( Hewa Kaluannakkage and Buyya, 2026 )implicit ∗–✓–
Gossip-learning model age ( Hegedűs et al., 2019 )✓max––
Limited merging ( Danner et al., 2023 )✓max✓ †–
SFG (ours)✓–✓✓ ( Theorem 3 )
Table 1: Which rules satisfy which principle. Self-inclusive: the receiver’s own model is discounted by its staleness. Inherited: staleness survives local training. Steep: a geometric discount with a small γ\gamma , or a tight margin. Bound: the aggregation error of a returning receiver is bounded uniformly in its outage under our staleness models. ∗ IPW raises the weight of rarely live links, which acts as an implicit self-discount for rarely available receivers ( Section 6.4 ). † Steep only with a tight threshold.
RuleStaleness sjs_{j}Unnormalised weight hjh_{j}PrinciplesGuarantee
SFGage τj\tau_{j}γ sj−smin⁡\gamma^{\,s_{j}-s_{\min}}1, 3Theorem 3
SFG-Leffective lag ℓj\ell_{j}γ sj−smin⁡\gamma^{\,s_{j}-s_{\min}}1, 2, 3Proposition 5 (iv)
SFG-Cuteffective lag ℓj\ell_{j}(1+t−sj) 1[sj−smin⁡≤L](1+t-s_{j})\,\mathbf{1}[s_{j}-s_{\min}\leq L]1, 2, 3Proposition 7 (iii)
Table 2: The SFG family. All members discount the receiver like any neighbour and use staleness relative to the freshest candidate. Principles refer to Principles 1 , 2 and 3 .
Figure 2: Effectiveness, efficiency and necessity. (a) Gain in mean personalised accuracy over DFedAvg, with 95% stratified tt -intervals over 120 paired runs ( Table 3 ). Orange: age-of-information rules. Grey: gossip learning and catch-up training. Blue: the SFG family. Hatched rules use 1.71×1.71\times the local computation of DFedAvg, and all others use exactly as much. (b) Gain per dataset, averaged over the eight outage processes. The dashed line marks outage-free DFedAvg, so the height of each bar relative to it is the share of the outage gap that the rule recovers. (c) Removing one principle from SFG-Cut ( L=1L=1 ): never dropping the receiver, cutting on ages instead of lags, or a loose margin ( L=10L=10 ). A random cut that keeps the same number of candidates gains nothing ( Tables 20 and 31 ).
Figure 3: Why self-inclusive discounting works. (a) Aggregation error of a returning client relative to a fresh one ( Corollary 4 , six fresh neighbours). (b) Accuracy of returning clients after the local training of their return round, against their outage length (27 runs per rule). (c) Decomposition on bursty u=0.8u{=}0.8 , heterogeneous u≈0.77u{\approx}0.77 , and FLASH relaxed and strict (60 paired runs). Discounting neighbours or oneself alone gives 40–42% of SFG’s gain. (d) Gain by availability quartile under heterogeneous rates (30 paired runs, s.e. 0.12–0.32 pp, Table 34 ).
Figure 4: The advantage holds across protocols, backbones, models and networks. Gain over the same backbone with DFedAvg weights, for the strongest AoI baseline and the two lag-based SFG rules. The label gives the ratio of the best SFG gain to the AoI + IPW gain. Backbone rows use bursty u=0.8u{=}0.8 and FLASH strict. Link rows use bursty u=0.8u{=}0.8 . ResNet-18-GN is the mean over the six cells of Table 14 . SFG-Cut was not run in the learning-rate sweep. Sources: Tables 3 , 35 , 32 , 13 , 14 and 17 .
Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Fine-tuned geometry (DFedAvg, bursty u=0.8u{=}0.8 , 3 seeds). (a) Squared distance between a fresh receiver’s model and a neighbour’s, relative to fresh neighbours, against the neighbour’s age. Dashed: before the receiver’s K=10K=10 local steps from both models. Solid: after. (b) Rate fitted to the raw and post-fine-tuning distance laws for each KK .
Figure 6: Real availability traces (10-minute rounds, 50 hours from local midnight). (a) Availability by hour of day. The legend gives the number of devices and the mean availability. (b) Distribution of outage lengths. (c) Per-device availability.
Figure 7: Ages versus lags. (a) Squared distance between live models, relative to the zero-difference baseline, against their age difference (dashed) and their effective-lag difference (solid). DFedAvg trajectories under bursty u=0.8u{=}0.8 , heterogeneous u≈0.77u{\approx}0.77 , FLASH strict and i.i.d. u=0.6u{=}0.6 , 3 seeds. (b) The online rate estimate of PWG is 1.9–4.3 ×\times larger on lags. (c) Gain over DFedAvg of the same rules on ages (light) and on lags (dark), 3 datasets ×\times 5 seeds. (d) SFG’s response to γ\gamma has the same shape for every number of local steps KK (bursty u=0.8u{=}0.8 , gain normalised by its best value and averaged over datasets).
Figure 8: Anatomy of outages ( u=0.8u=0.8 ). Markers: CIFAR-10 ∘\circ , CIFAR-100 □\square , FEMNIST △\triangle . (a) At a matched rate, outage structure does not change DFedAvg’s loss. Correlated outages, however, leave SFG little to work with. (b) Sharing a model one cannot train hurts more than going silent. SFG neutralises this cost.
Figure 9: Gain over DFedAvg by outage process (3 datasets ×\times 5 seeds, bars: s.e. of the paired gains). Lags help where many neighbours are stale at once (bursty u=0.8u{=}0.8 , FLASH strict, STUNner). Limited merging with L=2L=2 on max-inherited counts (no IPW) tracks SFG-L everywhere, slightly ahead under i.i.d. and heterogeneous rates. The link correction (IPW) adds most under heterogeneous rates. SFG-Cut with L=1L=1 leads under i.i.d. outages and trails under heterogeneous rates. L=2L=2 is the most even.
More in this topic

Federated Learning Aggregation

Federated learning aggregation combines client updates into shared or personalized models while balancing data heterogeneity, robustness, privacy, and communication.

See all 266 papers

Explore similar work

CardsList
Jun 9, 2026

Asynchronous Decentralized Federated Learning over Lossy Wireless Links via Reception- and Age-Aware Aggregation

Decentralized Federated Learning(DFL) enables collaborative model training across wireless edge nodes, including IoT deployments, autonomous vehicles, UAV swarms, and satellite constellations. Operating over lossy wireless links under constraints, these systems cannot rely on retransmissions, so model parameters must be accepted as partial chunks, leading to two key failure modes, which are selection bias, where poor-quality links are systematically under-represented in gossip aggregation, and update staleness, where asynchronous nodes contribute outdated models. We prove that classical gossip aggregation introduces irreducible selection bias proportional to the link-loss rate. We propose DFL-AA (Decentralized Federated Learning with Adaptive AoI-weighted Aggregation), which corrects selection bias using Inverse Probability Weighting (IPW) with online channel estimation and mitigates staleness via Age-of-Information (AoI) decay without requiring a global clock. We prove that DFL-AA removes link-quality distortion in expectation and consistently outperforms state-of-the-art baselines across varying loss rates and heterogeneous channel conditions on fixed directed topologies.

Chanuka A. S. Hewa Kaluannakkage, Rajkumar Buyya

Quantum Cloud and Distributed Systems (qCLOUDS) Lab School of Computing and Information Systems The University of Melbourne, Australia

Federated Learning AggregationAsynchronous Federated LearningAge of InformationFederated Learning
AskPDF
Jul 3, 2026

Decentralised Federated Learning over Temporal Networks: The Role of Heterogeneities

Decentralised federated learning, based on peer-to-peer communication, is increasingly proposed for on-device training of machine learning models, promising a privacy-preserving, communication-efficient training process with no risk of single-point failure. However, the role of structural and temporal inhomogeneities in such fully decentralised settings remains poorly understood. Here, we investigate their effects when model parameters are locally averaged during aggregation. We show that the decentralised federated learning process is governed, both in the early phase and the late, stationary limit, by the same dynamics as a lazy random-walk diffusion process on temporal networks. Based on this mapping, we demonstrate that the typical experimental scenario used in decentralised federated learning leads to unrealistically rapid convergence because of ignoring the temporal and structural inhomogeneities inherent in the communication network. We analyse real-world temporal networks and find that inhomogeneities most often dramatically slow down diffusion, hence the convergence process.

Arash Badie-Modiri, Chiara Boldrini, Lorenzo Valerio +2

National Research Council, Pisa · Central European University, Vienna · Aalto University, Espoo +1

Federated Learning AggregationDecentralized LearningFederated Learning
AskPDF
Sep 17, 2026

FedeRage: Provably Convergent Agnostic Federated Learning under General Client Drift

Federated learning (FL) enables collaborative model training without sharing raw data, but its performance degrades under non-IID data and stochastic client participation. Remedies built on classical Federated Averaging (FedAvg) typically presuppose that client participation probabilities are known to the server, which is rarely the case in deployed systems. We first discuss and then characterize the optimization problem that distributionally agnostic FedAvg actually solves when participation is entirely unknown, possibly highly skewed, and of variable size across rounds: uniform aggregation is shown to minimize a well-defined stochastic objective, weighted by the participation-induced marginal, at a standard O(1/T)\mathcal{O}(1/\sqrt{T}) rate for convex and possibly nonsmooth losses. Building on this characterization, we propose Federated Risk-Averse Averaging (FedeRage), a risk-averse extension of FedAvg that embeds the Conditional Value-at-Risk (CVaR) into the local objective within a natural distributionally robust optimization (DRO) framework. FedeRage implicitly upweights high-loss and infrequently participating clients while adding only a single scalar per-client, and admits an O(κ/T)\mathcal{O}(κ/\sqrt{T}) rate in which the factor κκ is the upper bound on the ``price" of risk aversion. In contrast with aggregation-alignment schemes based on optimal transport, which require the availability distribution as an input, FedeRage remains agnostic to it. Several experiments on three heterogeneous benchmarks indicate consistent improvements over state-of-the-art methods in accuracy, fairness, and convergence speed.

Herlock Rahimi, Dionysis Kalogerias

Federated LearningFederated Learning AggregationNon-IID Federated LearningDistributionally Robust Optimization
AskPDF