cs.LGOct 16, 2026

Bridging Compressed Communication and Local Steps via Spectral Method in Federated Learning

Authors: Dmitriy Bystrov, George Chirkov, Daniil Medyakov, Dmitry Bylinkin, Aleksandr Beznosikov

Organizations: Basic Research of Artificial Intelligence Laboratory (BRAIn Lab) · Federated Learning Problems Laboratory · Innopolis University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Federated Learning (FL) enables collaborative training of complex models across large-scale networks of edge devices while keeping data localized. However, data heterogeneity and high communication overhead remain fundamental bottlenecks. Recently, the spectral optimizer Muon has demonstrated remarkable advantages over Adam-like methods in centralized training. While recent efforts have attempted to transition Muon to FL, naive integrations into traditional architectures fail short. Crucially, existing approaches do not support the the communication compression and local steps --- the two foundational techniques of efficient FL. To address this limitation, we introduce a novel spectral FL method that utilizes both local updates and compression. We establish rigorous convergence guarantees under mild assumptions, notably eliminating the restrictive bounded heterogeneity condition. Empirical evaluation on standard image classification benchmarks validates the efficiency and robustness of our approach.

Figures & tables

MethodCompressionModelAccuracyAccuracyAccuracy
size( CIFAR-10, Het. )( CIFAR-10, Path. )( SST-2 )
FedCALMTop1%×\times 0.020.8140.9560.930
Top10%×\times 0.20.8250.9590.928
SCAFCOMTop1%×\times 0.010.4600.9040.933
Top10%×\times 0.10.8010.9230.927
EF21-MuonTop1%×\times 0.010.3740.5560.900
Table 1: Comparison of the methods performance. Model size designates the amount of transmitted information compared to the size of a single model. Last columns report best accuracy across training for all methods in three setups: CIFAR-10 with Heterogeneous and Pathological distributions, and SST-2 with IID distribution.
Figure 1: The test accuracy accross the training on Heterogeneous distibution (left) and Pathological distribution (right) for all methods. The horizontal axis designates the amount of communicated bits.
Figure 2: The test accuracy accross the training on Pathological distibution for all methods over the communication rounds. That is, compression rates are not taken into account.
Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

MethodCompressionAccuracy
FedCALMTop1%0.930
Top10%0.928
SCAFCOMTop1%0.933
Top10%0.927
EF21-MuonTop1%0.900
Top10%0.912
Appendix
Table 2: The test accuracy accross the training on IID distibution for all methods over 3030 communication rounds in SST-2 experiments.
MethodClient GPUCompressionUplinkAggr. time
FedCALM208.9 MBTop1%8.94 MB171 ms
Top10%89.4 MB815 ms
SCAFCOM235.9 MBTop1%4.47 MB85 ms
Top10%44.7 MB407 ms
EF21-Muon181.3 MBTop1%4.47 MB85 ms
Top10%44.7 MB407 ms
Appendix
Table 3: Network statistics for CIFAR-10 experiments.
MethodCompressionUplinkTimeAverage memory
FedCALMTop1%143 MB2.73 s7.87 MB
Top10%1072 MB9.78 s5.90 MB
SCAFCOMTop1%522 MB9.94 s57.6 MB
Top10%11803 MB11.8 s14.2 MB
FedMuonTop100%4917 MB39.2 s4.39 MB
LocalSGDTop100%13857 MB110 s13.8 MB
Appendix
Table 4: Memory usage and total aggregation time to reach 90% accuracy for CIFAR-10 experiments.
MethodClient memoryCompressionUplinkAggr. time
FedCALM2952.98 MBTop1%0.74 MB109 ms
Top10%7.39 MB641 ms
SCAFCOM2947.30 MBTop1%0.37 MB80 ms
Top10%3.70 MB346 ms
EF21-Muon2937.51 MBTop1%0.37 MB80 ms
Top10%3.70 MB346 ms
Appendix
Table 5: Network statistics for SST-2 experiments.
MethodCompressionUplinkTimeAverage memory
FedCALMTop1%4 MB546 ms615.20 MB
Top10%15 MB1.28 s250.48 MB
SCAFCOMTop1%7 MB1.43 s2210.48 MB
Top10%22 MB2.07 s736.83 MB
EF21-MuonTop1%–––
Top10%89 MB8.30 s2937.51 MB
Appendix
Table 6: Memory usage and total aggregation time to reach 91%91\% accuracy for SST-2 experiments.
Figure 3: The ablation of FedCALM over the number of Newton-Schulz steps qq with CIFAR-10 experiments. The setup is the Pathological distribution with Top10% compression rate.
qqAccuracy
10.912
20.926
30.927
50.928
80.928
100.929
Appendix
Table 7: The ablation of FedCALM over the number of Newton-Schulz steps qq with SST-2 experiments. The setup is the IID distribution with Top10% compression rate during 2020 rounds.
Figure 4: The empirical lower bounds for β1\beta_{1} (green), β2\beta_{2} (blue) for Heterogeneous distribution. The red line marks the applied compression rates (Top1% on the left and Top10% on the right).
Figure 5: The empirical lower bounds for β1\beta_{1} (green), β2\beta_{2} (blue) for Pathological distribution. The red line marks the applied compression rates (Top1% on the left and Top10% on the right).
MethodCompressionHeter. distr.Path. distr.
FedCALMTop1%0.010.01
Top10%0.010.01
SCAFCOMTop1%0.00050.0005
Top10%0.00050.0005
EF21-MuonTop1%0.00050.0001
Top10%0.00050.0001
Appendix
Table 8: Stepsizes for the final runs of CIFAR-10 experiments.
MethodCompressionStepsize
FedCALMTop1%0.005
Top10%0.005
SCAFCOMTop1%0.0025
Top10%0.0025
EF21-MuonTop1%0.0005
Top10%0.0005
Appendix
Table 9: Stepsizes for the final runs of SST-2 experiments.
 Client 1Client 2Client 3Client 4Client 5
Class 011841184118411842
Class 111851185118511852
Class 211851185118511852
Class 311841184118411842
Class 42222790
Class 52222791
Appendix
Table 10: Number of objects in client datasets for Pathological distibution (clients 1-5).
 Client 6Client 7Client 8Client 9Client 10
Class 022222
Class 122222
Class 222222
Class 322222
Class 4790790790790790
Class 5791791791791791
Appendix
Table 11: Number of objects in client datasets for Pathological distibution (clients 6-10).
 Client 1Client 2Client 3Client 4Client 5
Class 0440448400428450
Class 1501481488488502
Class 2465479509518513
Class 3470428476495437
Class 4449482477486478
Class 5495512547507524
Appendix
Table 12: Number of objects in client datasets for Heterogeneous distibution .
 Client 6Client 7Client 8Client 9Client 10
Class 0421448412430482
Class 1510479523535503
Class 2469496446504497
Class 3447527490435486
Class 4443506480460425
Class 5497501500525482
Appendix
Table 13: Number of objects in client datasets for Pathological distibution .
More in this topic

Federated Learning

Federated learning trains shared models across data holders without requiring them to pool their raw data.

See all 921 papers

Explore similar work

CardsList

  1. A Full Compression Pipeline for Green Federated Learning in Communication-Constrained Environments

    Apr 13, 2026Elouan Colybes, Shirin Salehi, Anke SchmeinkFederated LearningCommunication-Efficient Distributed Training

    Chair of Information Theory and Data Analytics (INDA), RWTH Aachen University, Aachen, Germany

  2. SpecGradFilter: A Spectral Gradient Filtering Framework for Taming Federated Heterogeneity

    Jul 5, 2026Liyang Yuan, Yibo Yang, Dandan Guo +2Non-IID Federated LearningFederated Learning

    School of Artificial Intelligence, Jilin University, Changchun 130012, China · King Abdullah University of Science and Technology (KAUST), Thuwal, Saudi Arabia · School of Intelligence Science and Technology, Peking University, China

  3. Communication-Efficient Agnostic Federated Learning via Faster Convergence and Compression

    Sep 29, 2026Haomin Bai, Junyan Sun, Sifan Yang +2Error FeedbackOnline Federated Learning

    State Key Laboratory of Novel Software Technology, Nanjing University, Nanjing, China · School of Artificial Intelligence, Nanjing University, Nanjing, China · Department of Computer Science, City University of Hong Kong, Hong Kong, China