Population Scaling or Data Dilution? Dynamics of Local Topology Evolution in Decentralized Learning

Decentralized learning is highly sensitive to communication topology under non-IID data. Adaptive peer-selection methods can exploit local model information, but broader peer discovery may require increasingly large control state, whereas direct spectral optimization typically relies on graph-wide information. We study the intermediate setting of bounded local topology search and propose Local-First Heuristic Evolution (LFHE), a representation-driven rewiring framework whose candidate discovery and scoring use only ego-neighborhood and friend-of-a-friend (FoF) information. The structural score admits an exact interpretation through graph Dirichlet energy: its sum across clients equals twice the representation Dirichlet energy, which under standard linear consensus dynamics governs the instantaneous dissipation of representation disagreement. LFHE combines this state-dependent structural signal with early exploration and degree control, while algebraic connectivity remains an offline graph diagnostic. Under bounded sparse degree, its FoF candidate state remains local rather than expanding toward population-wide peer tracking. Across four image, speech, and text benchmarks, LFHE achieves competitive decentralized learning performance. Matched-protocol controls identify the structural term as the principal empirical topology-selection signal, while comparison with broader peer discovery exposes a trade-off between predictive performance and discovery-state locality. Together, these results motivate state-aware bounded local topology search between pairwise peer selection and globally informed topology optimization.

Decentralized learning is a promising paradigm for collaborative training in mobile and pervasive systems, as it avoids a central coordinator and does not require sharing raw data. Yet, most analyses rely on idealized communication assumptions that break down in wireless settings, where connectivity is intermittent, topology changes due to mobility, and bandwidth is limited. We study decentralized averaging under client asynchrony, time-varying contact graphs, and technology-dependent throughput constraints. We implement a fully decentralized protocol that overlaps synchronization with local training and supports partial tensor-level transfers when contacts end early. Using Random Waypoint mobility and multiple wireless technologies (Bluetooth LE, LTE, and Wi-Fi), we quantify how network dynamics and link capacity impact convergence. We identify three operating regimes: (i) inter-contact time largely dictates convergence via mixing, (ii) partial updates are often well tolerated when contacts are frequent, and (iii) very dense contact patterns can trigger contention, reducing effective throughput. These findings provide a practical lens to reason about decentralized learning deployments over realistic wireless systems, highlighting when improving connectivity, increasing bandwidth, or mitigating contention is most impactful.

Decentralized training is often regarded as inferior to centralized training because the consensus errors between workers are thought to undermine convergence and generalization. This work challenges this view by introducing decentralized SGD with Adaptive Consensus (DSGD-AC), which uses a time-dependent scaling mechanism to maintain consensus errors throughout the training. We show that adaptive consensus changes the stationary variance of disagreement modes by balancing two effects: it preserves consensus-error magnitude through weaker graph damping while still allowing curvature-dependent damping to shape the disagreement directions. This balance can produce a stronger Hessian-weighted loss-envelope penalty around the deployed model, even when normalized Hessian alignment is weaker than in standard DSGD. Empirical results on image classification show that DSGD-AC reaches flatter solutions and higher test accuracy than standard DSGD and even centralized SGD. Together, these results support consensus errors as a useful implicit regularizer and open a new perspective on the design of decentralized learning algorithms.