TALLOP: Joint Optimization of Local Training and Communication Compression for Time-Constrained Federated Learning

Honoka Anada; Yafei Ou; Shinya Takamaeda-Yamazaki

cs.LGOct 16, 2026

TALLOP: Joint Optimization of Local Training and Communication Compression for Time-Constrained Federated Learning

Authors: Honoka Anada, Yafei Ou, Shinya Takamaeda-Yamazaki

Organizations: The University of Tokyo / RIKEN · RIKEN

Abstract

Minimizing the wall-clock time required to reach a target accuracy is crucial in federated learning (FL), where many applications require frequent model updates as data distributions evolve over time. Nevertheless, in heterogeneous environments, slow clients with limited computational and communication capabilities can delay global synchronization, forcing faster clients to wait and increasing the overall training time. To mitigate this straggler effect, recent studies have introduced per-round time constraints and adapted client workloads within the available time. However, these approaches primarily focus on local computation, whereas communication can become a major bottleneck in communication-limited environments, motivating the joint optimization of training workload and communication compression. Although previous studies have explored such joint optimization, determining an effective training--communication balance under a fixed per-round time budget remains challenging, as the resulting model performance depends on both training progress and compression-induced degradation. We therefore propose TALLOP, which jointly optimizes the local training workload and communication compression under per-round time constraints. TALLOP predicts the training loss achievable under different computation budgets from the early-stage mini-batch loss trajectory (LENS), and estimates the degradation caused by communication compression using a compression-error model (CLEN). TALLOP combines these predictions to select the training-communication allocation that minimizes the post-compression loss under the given round time constraint. Experiments in heterogeneous FL environments show that TALLOP generally achieves competitive or improved time-to-accuracy compared with approaches that optimize only local training or neither, although the benefit over training-only adaptation depends on the setting.

Figures & tables

Fig. 1 : Given a fixed per-round time budget $T$ , the time allocated to local training and communication determines the local training steps $s$ and the communication compression ratio $p$ . While the optimal training-communication time allocation is unknown in advance, TALLOP predicts the post-compression model loss for different allocations and selects the one expected to achieve the best training outcome.

Fig. 2 : Overview of TALLOP. TALLOP predicts the post-compression loss by combining LENS and CLEN, and determines the optimal time allocation factor $\rho$ by minimizing the predicted loss.

Panel of Figure 3 — Fig. 3 : Actual and predicted losses of sparsified models for various non-zero ratios $p$ . The proposed CLEN accurately estimates the loss after compression.

	Computation [GFLOPS]				Communication [Mbps]
	$m$	$\sigma$	min	max	$m$	$\sigma$	min	max
Config. 1	20	0.5	5.48	61.64	80	1.0	3.83	491.25
Config. 2	200	0.5	54.8	616.4	80	1.0	3.83	491.25

TABLE I : Distribution of client computation speeds and communication bandwidths in the experiments.

m

and

\sigma

represent the mean and log-standard deviation of the log-normal distribution, respectively.

Panel of Figure 5 — Fig. 4 : Test accuracy as a function of wall-clock time for MobileNetV2 on CIFAR-10 under the two system configurations.

Dataset/model	Method	Config. 1 (slow comput.)					Config. 2 (fast comput.)
Dataset/model	Method	FA	TTA at targets				FA	TTA at targets
			30%	40%	50%	60%		30%	40%	50%	60%
CIFAR-10 MobileNetV2	Vanilla w/o deadline	56.83	3239	7140	12532	–	37.10	1410	–	–	–
	Vanilla w/ deadline	64.75	1600	3350	6400	11900	46.25	675	1450	–	–
	Training-only optim.	66.85	1000	2000	3600	8150	60.53	175	350	725	2325
	FFL	62.79	2214	3886	7718	14011	59.14	397	769	1531	–

TABLE II : Final accuracy (FA, %) and time-to-accuracy (TTA, seconds), rounded to the nearest second. “–” indicates that the target was not reached. FFL, TimelyFL, and FedBalancer are evaluated for MobileNetV2/CIFAR-10.

Federated Learning

Federated learning trains shared models across data holders without requiring them to pool their raw data.

See all 961 papers

Apr 13, 2026

A Full Compression Pipeline for Green Federated Learning in Communication-Constrained Environments

Federated Learning (FL) enables collaborative model training across distributed clients without sharing raw data, thereby preserving privacy. However, FL often suffers from significant communication and computational overhead, limiting its scalability and sustainability. In this work, we introduce a Full Compression Pipeline (FCP) for FL in communication-constrained environments. FCP integrates three complementary deep compression techniques (pruning, quantization, and Huffman encoding) into a unified end-to-end framework. By compressing local models and communication payloads, FCP substantially reduces transmission costs and resource consumption while maintaining competitive accuracy. To quantify its impact, we develop an evaluation framework that captures both communication and computation overheads as a unified model cost, allowing a holistic assessment of efficiency trade-offs. The pipeline is evaluated in an independent and identically distributed (IID) and non-IID data setting. In one representative scenario, training a ResNet-12 model on the CIFAR-10 dataset with ten clients and a 2 Mbps bandwidth, the FCP achieves more than 11

\times

reduction in model size, with only a 2% drop in accuracy compared to the uncompressed baseline. This results in an FL training that is more than 60% faster.

Elouan Colybes, Shirin Salehi, Anke Schmeink

Chair of Information Theory and Data Analytics (INDA), RWTH Aachen University, Aachen, Germany

Federated Learning Communication-Efficient Distributed Training

Ask PDF

Sep 29, 2026

Communication-Efficient Agnostic Federated Learning via Faster Convergence and Compression

Agnostic federated learning (AFL) seeks a model that performs reliably across

m

heterogeneous workers, but communication remains a bottleneck. We improve communication efficiency by reducing the number of synchronization rounds via faster convergence and the communication cost per round via compression. We first propose AFL-BR, which updates the dual weights over workers using online mirror ascent with KL divergence and blockwise restarts. It achieves an

O((\log m)^{1/4}T^{-1/8})

stationarity rate after

T

update rounds, reducing the

m

-dependence of the synchronization rounds required for convergence from polynomial to logarithmic order. Building on AFL-BR, we develop AFL-Com by applying bidirectional compression with error feedback (EF). Instead of compressing local gradients, workers apply EF to their dual-weighted gradients, enabling direct control of the aggregated compression error under time-varying weights. We then establish an

O((δ^{-1}+(\log m)^{1/4})T^{-1/8})

stationarity rate for AFL-Com under general

δ

-approximate compressors and improve the

δ

-dependence from

δ^{-1}

δ^{-1/2}

for additive-and-idempotent compressors with shared randomness (SR). With suitable compression levels, AFL-Com retains the same convergence rate as AFL-BR at a lower per-round communication cost, yielding reductions in total communication complexity by factors of

(\log m)^{1/4}

with Top-

k

and

(\log m)^{1/2}

with Rand-

k

and SR. Experiments validate the improved synchronization and communication efficiency of our methods.

Haomin Bai, Junyan Sun, Sifan Yang +2

State Key Laboratory of Novel Software Technology, Nanjing University, Nanjing, China · School of Artificial Intelligence, Nanjing University, Nanjing, China · Department of Computer Science, City University of Hong Kong, Hong Kong, China

Error Feedback Online Federated Learning Federated Learning Communication-Efficient Distributed Training Gradient Compression

Ask PDF

Sep 11, 2026

Hidden in Rounds: Predicting the Time Cost of 802.11 Contention in Federated Learning

Federated learning over IEEE~802.11 shares the wireless channel among clients that send model updates. We use ns-3 to measure the frame-delivery ratio and saturation throughput for different client densities and offered loads. A separate FedAvg trainer uses the frame-delivery ratio as a first-order proxy for the update-admission probability and uses an equation to estimate communication time. The method does not simulate the delivery of a complete model update or measure end-to-end training time. Across 720 evaluated runs with two datasets, two data partitions, six client densities, six offered loads, and five seeds, all runs reached their predefined target accuracy within the round budget. Rounds-to-target changed little with offered load, while communication time-to-target increased by about two orders of magnitude across the client-density range. A Bianchi-anchored estimator produced a mean absolute percentage error from

2.3\%

10.2\%

on held-out configurations. This error is measured against communication time constructed from the same round-duration equation, not against independently measured completion time. We also compare uniform participation with persistent heterogeneous participation. The study does not detect a statistically distinguishable excluded-class accuracy gap over five seeds, but the confidence intervals are wide. The results apply only to the evaluated configurations and do not provide a general convergence or fairness guarantee.

Satwat Bashir, Tasos Dagiuklas

Department of Computer Science London South Bank University London, UK

Wireless Local Area Networks Wireless Communications Federated Learning

Ask PDF

Abstract

Federated Learning

Explore similar work

A Full Compression Pipeline for Green Federated Learning in Communication-Constrained Environments

Communication-Efficient Agnostic Federated Learning via Faster Convergence and Compression

Hidden in Rounds: Predicting the Time Cost of 802.11 Contention in Federated Learning