cs.LGNov 9, 2026

Train High, Ship Low: Efficient LoRA Synchronization for Distributed On-Policy Post-Training

Authors: Shenghui Li, Li Ju, Thiemo Voigt

Organizations: Uppsala University, Uppsala, Sweden · Research Institutes of Sweden, Stockholm, Sweden

Abstract

Transmitting LoRA adapters to rollout workers can bottleneck distributed on-policy post-training over bandwidth-constrained networks. Lowering the training rank may limit the capacity a workload needs, while adaptive optimizer updates are difficult to compress. Rollout workers can instead reconstruct updates from gradients by maintaining replicated optimizer states. We introduce HiLo, a compressor-agnostic framework that replaces parameter transmission with compressed gradient dissemination and local optimizer replay. By applying the same decoded gradients, the learner and workers maintain consistent policy states at the same version under reproducible execution. We further develop Spar, a budget-aware gradient compressor that combines spectral truncation, adaptive rank allocation and error feedback to decouple training rank from communication cost. Experiments across on-policy distillation and reward-guided post-training show that HiLo with Spar maintains learning quality close to full synchronization while transmitting as little as 1.6% of the adapter bytes, and accelerates end-to-end updates by up to 2.2×2.2\times under bandwidth constraints.

Figures & tables

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. HiLoRe: What to Store, Compress, or Recompute for Efficient GRPO Training

    Sep 27, 2026Xinrui Chen, Mengyang Li, Ou Wu +1Group Relative Policy OptimizationActivation Compression

  2. AutoLoCo: Communication Efficient Distributed LLM Training via Adaptive Synchronization

    Sep 29, 2026Pengyu He, Yan Zhang, Ruien Li +1Communication-Efficient Distributed TrainingDistributed Training