Communication-Efficient Distributionally Robust Federated Averaging with Multi-Source Data
Organizations: Department of Control Science and Systems Engineering, Peking University · Engineering and Technology Institute Groningen, University of Groningen
Abstract
Federated learning trains a shared model from distributed client data. The deployment risk is typically specified through two approximations: each unknown local distribution is replaced by its empirical distribution, and the unknown deployment mixture across clients is replaced by empirical or prescribed training weights. Existing robust methods usually address one approximation at a time through local distributional ambiguity sets or worst-case client mixtures. However, joint treatment of both approximations remains underexplored. To address this gap, we construct a global ambiguity set formed by admissible mixtures of client-wise Wasserstein ambiguity sets, yielding a distributionally robust optimization formulation. The construction allows client-dependent ambiguity radii, preserves client-wise separability of the objective function, and yields a high-probability out-of-sample guarantee for the resulting robust problem. We then introduce a penalty-based approximation of this formulation and develop a communication-efficient algorithm called BiDRO-FedAvg, for which we establish a convergence guarantee. We validate the proposed framework through a market-demand prediction simulation.
Figures & tables
| Ambiguity modeling | Federated implementation | ||||
| Method | Local ambiguity | Mixture uncertainty | Client-dependent radius | Partial client participation | Multiple local steps |
| FedAvg ( McMahan et al., 2017 ) | |||||
| AFL ( Mohri et al., 2019 ) | |||||
| DRFA ( Deng et al., 2020 ) | |||||
| DRO-FL ( Wang et al., 2025 ) | |||||
| Group DRO with distributional uncertainty ( Konti et al., 2025 ) | – | – | |||
| Market stage | Maturity interval | ||||||
| 1 | Emerging | 600 | 250 | ||||
| 2 | Early growth | 150 | 500 | ||||
| 3 | Growth | 100 | 750 | ||||
| 4 | Mature | 75 | 1000 | ||||
| 5 | Saturated | 50 | 1500 | ||||
| 6 | Volatile late | 25 | 6000 |
| 1 | ||
| 2 | ||
| 3 | ||
| 4 | ||
| 5 | ||
| 6 |
| Method | Full participation | Matched rounds |
| FedAvg | ||
| AFL | ||
| DRFA | ||
| DRO-FL | ||
| BiDRO-FedAvg |
Federated Learning
Federated learning trains shared models across data holders without requiring them to pool their raw data.
See all 1,027 papers