cs.CVNov 10, 2026

Exploring Synchrony for Communication in Modular Neural Networks

Authors: Nikolas Willems, Qiyang Sun, Björn W. Schuller

Organizations: Imperial College London · Technical University of Munich

Abstract

One motivation for modular neural networks is to divide computation across components that learn distinct, reusable functions, which could then be recombined to support compositional generalisation. In practice, modular structure alone does not guarantee that such roles develop, and unrestricted communication between modules can undermine any specialisation that does emerge. This raises the question of how modular communication should be organised. The problem can be viewed as an analogue of the perceptual binding problem, where synchrony has been proposed as a mechanism for coordinating distributed representations. We introduce SyncNet, a modular architecture that separates module content from a dynamical phase state: learnt Kuramoto dynamics evolve the phases, while phase alignment determines communication through a shared channel. We evaluate SyncNet on Sort-of-CLEVR for relational reasoning and on SQOOP for systematic generalisation, where lower pair variety demands more recombination at test time. SyncNet gives the best held-out accuracy of the models compared at the three lowest pair varieties, by 5.1 percentage points at the hardest, and is competitive on Sort-of-CLEVR though below the strongest attention baselines. Test-time overrides show that performance depends on phases that evolve: fixed phases cost as much as no addressing at all, and permuting which module holds which region is nearly as damaging. A content-addressed variant trained from scratch nonetheless matches it, so the benefit appears to come from the modular decomposition rather than synchrony itself.

Explore similar work

CardsList
  1. Neural networks are more modular than single neurons suggest

    Oct 9, 2026Shreya Saha, Benjamin Bergen, Meenakshi KhoslaModular Neural NetworksNeural Network Interpretability

  2. Sparse Competition during Training For the Emergence of Specialized Modules

    Aug 31, 2026Baptiste Rossigneux, Karim HarounSparse Neural NetworksNeural Network Training Dynamics

  3. Recurrent Modular Structure in Spiking Neural Network Reinforcement Learning

    Nov 10, 2026Vikram Ramavarapu, Himanshu Udupi, Ashish GautamSpiking Neural NetworksModular Neural Networks