Beyond Uniform Compression: Budgeted Transmission Allocation for Extreme Federated Learning

Pengfei Li; Mohammad Khalil

cs.LGSep 30, 2026

Beyond Uniform Compression: Budgeted Transmission Allocation for Extreme Federated Learning

Authors: Pengfei Li, Mohammad Khalil

Organizations: Centre for the Science of Learning & Technology (SLATE), University of Bergen, Bergen, Norway

Abstract

Federated learning faces severe communication bottlenecks when clients upload high-dimensional model updates. Existing methods often compress these updates uniformly across all layers. This uniform approach ignores the heterogeneous value of different parameter blocks and wastes limited bandwidth on insensitive layers. To address this issue, we propose Layer-wise Budgeted Adaptive Transmission (LBAT). LBAT reframes federated communication under extreme uplink budgets as a resource allocation problem. Our framework dynamically estimates the transmission value of different layers utilising local training signals. It then employs an exact byte dynamic programming allocator to determine optimal rank and bit configurations under strict budgets. We validate LBAT on highly heterogeneous federated tabular prediction and data generation tasks. Extensive experiments demonstrate that LBAT consistently outperforms uniform rank, uniform quantisation, and fixed compression baselines across various extreme budget regimes. Furthermore, it achieves significantly better communication and utility tradeoffs while preserving essential distributional fidelity.

Figures & tables

Figure 1: Overview of the Layer-wise Budgeted Adaptive Transmission (LBAT) framework. LBAT evaluates layer-wise sensitivity locally and employs an exact-byte dynamic programming allocator to determine the optimal rank and bit configuration

(r_{l}^{*},b_{l}^{*})

under a strict communication budget

B

Figure 2: Communication–utility frontier. The horizontal axis is upload budget normalised by full update, and the vertical axis is TRTR-normalized utility. The margin strip below each panel shows the average utility gain of FedLBAT over the strongest communication-constrained peer. FedLBAT achieves a stronger frontier across all three datasets, particularly in the

0.05\times

–

0.25\times

extreme-budget range.

Budget	Adult $\Delta$	Adult W/T/L	Credit $\Delta$	Credit W/T/L	Cardio $\Delta$	Cardio W/T/L	Mean $\Delta$
$0.05\times$	+0.273 (+17.1%)	3/0/0	+0.334 (+15.8%)	3/0/0	+0.133 (+6.9%)	2/0/1	+0.247 (+13.3%)
$0.10\times$	+0.182 (+10.6%)	3/0/0	+0.325 (+15.0%)	3/0/0	+0.089 (+4.4%)	3/0/0	+0.199 (+10.0%)
$0.25\times$	+0.231 (+14.4%)	3/0/0	+0.120 (+4.9%)	3/0/0	-0.012 (-0.6%)	2/0/1	+0.113 (+6.2%)
$0.50\times$	+0.032 (+1.6%)	0/3/0	+0.117 (+4.9%)	3/0/0	+0.073 (+3.8%)	2/0/1	+0.074 (+3.5%)
$1.00\times$	+0.061 (+3.5%)	3/0/0	+0.363 (+16.8%)	2/1/0	+0.124 (+6.6%)	2/1/0	+0.183 (+9.0%)
All	–	12/3/0	–	14/1/0	–	11/1/3	+0.163 (+8.4%)

Table 1: Detailed dominance summary across all upload budgets. For each dataset–budget cell,

\Delta

is FedLBAT minus the strongest non-FedLBAT peer in TRTR-normalized utility; parenthesised values report the relative TSTR-AUC gain,

100(\overline{A}_{\mathrm{FedLBAT}}-\overline{A}_{\mathrm{peer}})/\overline{A}_{\mathrm{peer}}

, computed from unrounded three-seed mean AUCs. W/T/L reselects the strongest peer within each matched seed (

|\Delta_{s}|\leq 10^{-6}

is a tie), while Mean percentages macro-average the corresponding cellwise AUC gains.

Figure 3: Dominance atlas across datasets and budgets. Cell colors denote FedLBAT’s average utility margin against the strongest non-FedLBAT communication baseline. Cell text reports seed-level win/tie/loss counts. FedLBAT obtains positive mean margins in all $14$ dataset–budget cells.

Figure 4: LBAT allocation anatomy. A: adapter-budget mix across blocks; B: selected rank–bit schedule; C: correlation between sensitivity and normalised layer share. LBAT allocates more communication to sensitive blocks and increases rank/bit adaptively as the budget grows.

Figure 5: Utility and data-quality diagnostics. Left: utility–fidelity tradeoff. Right: relative quality scorecard. FedLBAT achieves the strongest utility while keeping distributional fidelity and categorical support competitive with or better than uniform baselines.

Track	Budget Group	FedLBAT	Best Peer	$\Delta$ AUC	Cells Won
Syn.	$0.05/0.10\times$	0.686 $\pm$ 0.081	0.608 $\pm$ 0.075	+0.0778	6/6
Syn.	$0.25/0.50\times$	0.685 $\pm$ 0.096	0.636 $\pm$ 0.095	+0.0489	6/6
Syn.	$1.00\times$	0.670 $\pm$ 0.080	0.616 $\pm$ 0.118	+0.0534	3/3
Real	$0.05/0.10\times$	0.811 $\pm$ 0.068	0.811 $\pm$ 0.069	-0.0005	0/6
Real	$0.25/0.50\times$	0.810 $\pm$ 0.070	0.812 $\pm$ 0.068	-0.0021	2/6
Real	$1.00\times$	0.810 $\pm$ 0.073	0.811 $\pm$ 0.071	-0.0010	1/3

Table 2: Cross-track AUC summary. Syn. denotes the synthetic TSTR track; Real denotes the real tabular prediction track. Best Peer is the strongest non-FedLBAT communication-constrained baseline within each budget group. Boldface indicates the higher mean AUC in each row, determined from the unrounded results.

Variant	Ablated Component	Budget	Adult	Credit	Cardio	Avg. AUC	$\Delta$ vs. Full
Full FedLBAT	None	$0.05\times$	0.755 $\pm$ 0.056	0.703 $\pm$ 0.037	0.642 $\pm$ 0.050	0.700	–
w/o layer sensitivity	Layer sensitivity	$0.05\times$	0.755 $\pm$ 0.058	0.655 $\pm$ 0.035	0.598 $\pm$ 0.048	0.669	-0.031
w/o drop-loss calib.	Drop-loss calibration	$0.05\times$	0.755 $\pm$ 0.052	0.646 $\pm$ 0.041	0.601 $\pm$ 0.055	0.667	-0.033
w/o adaptive alloc.	Budgeted allocation	$0.05\times$	0.755 $\pm$ 0.061	0.540 $\pm$ 0.039	0.588 $\pm$ 0.047	0.628	-0.072
Random allocation	Value-guided allocation	$0.05\times$	0.689 $\pm$ 0.054	0.609 $\pm$ 0.034	0.568 $\pm$ 0.052	0.622	-0.078

Table 3: Ablation study of FedLBAT under the strict

0.05\times

communication budget. Removing adaptive allocation, layer sensitivity, or drop-loss calibration consistently degrades performance, validating the value-guided dynamic-programming design. Boldface marks the best mean in each column, with ties retained at the displayed precision.

Explore similar work

Entropy-Guided Tensor Compression for Multimodal Federated Learning on Edge Devices
Jul 7, 2026Quoc Bao Phan, Tuy Tan NguyenFederated Learning Edge Devices
Department of Electrical and Computer Engineering, Center for Advanced Power Systems, FAMU-FSU College of Engineering, Florida State University, Tallahassee, FL 32310, USA
Exploiting Correlations in Federated Learning: Opportunities and Practical Limitations
Apr 16, 2026Adrian Edin, Michel Kieffer, Mikael Johansson +1Federated Learning Generative Image Compression
Department of Electrical Engineering (ISY), Linköping University, SE-581 83 Linköping, Sweden · CentraleSupélec, CNRS, L2S, Université Paris-Saclay, Gif-sur-Yvette, FR-91192, France · Division of Decision and Control Systems, School of EECS, KTH Royal Institute of Technology, SE-100 44 Stockholm, Sweden
FEAST: Federated Shared-Space Training for Resource-Heterogeneous Clients
Aug 10, 2026Bostan Khan, Masoud DaneshtalabBalanced Learning Taco
Mälardalen University, Västerås, Sweden

Jul 7, 2026, cs.LGJ/K move · Enter open · S save

Entropy-Guided Tensor Compression for Multimodal Federated Learning on Edge Devices

Quoc Bao Phan, Tuy Tan Nguyen

Department of Electrical and Computer Engineering, Center for Advanced Power Systems, FAMU-FSU College of Engineering, Florida State University, Tallahassee, FL 32310, USA

Federated Learning Edge Devices Data Compression Methods

Open paper AskPDF arXiv

Federated learning (FL) over mobile and edge devices increasingly involves multimodal models in which clients differ in both sensing capability and computational capacity. Existing update compression schemes typically apply uniform policies across layers and devices, without accounting for modality-specific differences in spectral structure and compressibility. We propose MESH-FL, an entropy-guided matrix product state (MPS) update-compression framework for modality-heterogeneous FL on resource-constrained devices. MESH-FL estimates the spectral entropy of each layer-wise update via truncated singular value decomposition and allocates MPS compression ranks adaptively across layers, modalities, and devices under per-client payload budgets. We show that higher spectral entropy necessitates a higher reconstruction rank under the majorization order on singular-value energy distributions. Building on this result, we prove that the proposed entropy-guided allocation solves a convex surrogate rank-allocation problem, preserves monotonicity under the exact payload model, and achieves convergence with an explicit compression-dependent error term. Experiments on a 15-node heterogeneous Raspberry Pi~4/5 cluster with modality-heterogeneous clients show that MESH-FL achieves up to

56.8\times

compression while surpassing the uncompressed FedAvg baseline in final accuracy by up to 2.01%, and reduces total transmitted data to reach convergence by up to

66\times

Abstract

Explore similar work

Entropy-Guided Tensor Compression for Multimodal Federated Learning on Edge Devices

Exploiting Correlations in Federated Learning: Opportunities and Practical Limitations

FEAST: Federated Shared-Space Training for Resource-Heterogeneous Clients