Modern deep learning models achieve impressive generalization through over-parameterization, but this paradigm often struggles with overfitting and memorization in few-shot regimes. Neural Cellular Automata (NCAs) offer a highly parameter-efficient alternative, yet research has focused primarily on their output, leaving the role of their internal hidden channels largely unexplored. In this paper, we investigate the internal dynamics of NCA hidden channels and introduce a novel transfer-learning mechanism that injects a pretrained teacher's hidden states into a student model to guide early optimization. Evaluated on few-shot and scale-variant MNIST benchmarks, NCAs outperform comparable recurrent and feed-forward architectures, demonstrating superior generalization with a minimal parameter budget (~9,800 parameters). Mechanistic analysis reveals that the hidden channels decouple feature extraction from uniform classification consensus by absorbing morphological complexity and converging to mutually orthogonal states. Furthermore, we demonstrate that these hidden channels capture general, scale-invariant topological primitives rather than class-specific templates. This allows a student model to achieve strong few-shot performance on unseen classes using features transferred from a teacher trained only on a subset of digits (0-5). Our results highlight the potential of utilizing hidden-state dynamics as a robust, decentralized computational substrate for parameter-efficient transfer learning
Figures & tables
Figure 1: Diagram showing how the hidden channels are extracted from a teacher NCA and used in a Student NCA. The teacher NCA sees a digit, produces a classification, and some hidden channels are used in computing. These hidden channels are extracted and detached and passed to the student NCA alongside the digit before the forward pass in training.
Figure 2: All 4 models, tested on k∈{1,5,10,25,50,100} example per class, all have ≈9800 parameters. Since there are 10 classes, 101 examples equals one example per class. Scales range from 100 percent MNIST to 25 percent MNIST, with 25 percent intervals.
Figure 3: All 4 models, tested on k∈{1,5,10,25,50,100} example per class, all have ≈9800 parameters. Since there are 10 classes, 101 examples equals one example per class. Scales range from 100 percent MNIST to 25 percent MNIST, with 25 percent intervals. NCA receives a non-solution hidden state across a portion of training.
Figure 4: Training loss (blue) and validation loss (orange) of one run of the NCA in a 1-shot environment trained with the hidden injections.
Figure 5: All 4 models, tested on k∈{1,5,10,25,50,100} example per class, all have ≈9800 parameters. Since there are 10 classes, 101 examples equals one example per class. Scales range from 100 percent MNIST to 25 percent MNIST, with 25 percent intervals. NCA receives the non-solution hidden channels across all of training and validation.
Figure 6: All 4 models, tested on k∈{1,5,10,25,50,100} example per class, all have ≈9800 parameters. Since there are 10 classes, 101 examples equals one example per class. Scales range from 100 percent MNIST to 25 percent MNIST, with 25 percent intervals. NCA receives the non-solution hidden channels across all of training and validation.
Figure 7: Top: Hidden channel representations built by the NCA on a full-size MNIST 4. Bottom: Hidden channel representations built by the same NCA on a 50% scale version of the same 4.
Figure 8: Temporal evolution of the spatial variance of the hidden channels of (a) the teacher NCA and (b) the student NCA.
Figure 9: Cross-Channel Cosine Similarity between all channel pairs over time for (a) the teacher NCA and (b) the Student NCA.
Figure 10: 2D UMAP trajectory of the hidden channels for the Teacher (blue), Student with injection (green), and Student with no injection (red).
Few-shot meta-learning traditionally formulates task adaptation either as analytical gradient descent through unrolled computational graphs or as metric-based distance comparisons over flattened 1D fea- ture vectors, which either incur costly test-time backpropagation or discard native 2D spatial geometry. In this work, we propose METALEARNNCA, a decentralized framework that achieves few-shot adapta- tion through the dynamical interaction of coupled Neural Cellular Automata (NCAs) without computing analytical gradients during inference. MetaLearnNCA decomposes task adaptation into an Active- NCA, which executes task inference conditioned on a continuous 2D spatial memory grid termed the spatial program, and a learned Meta-NCA, which acts as a decentralized cellular optimizer by diffusing spatial error residuals across local neighborhoods to dynamically update this program. METALEARN- NCA is competitive against canonical meta-learners in-distribution (96.12% on Omniglot) with Out-Of- Distribution transfer gains on MNIST, KMNIST, and Fashion-MNIST transfer across 10 independent testing seeds across 1-, 5-, and 10-shot regimes (e.g., surpassing Prototypical Networks by +10.54% on 10-shot MNIST and a +3.87% gain on 10-shot Fashion-MNIST over FOMAML). Our results establish that robust, gradient-free learning-to-learn can emerge from decentralized cellular dynamics on non-von Neumann substrates.
Etienne Guichard, Stefano Nichele
Department of Computer Science and Communication hiof.no
Self-organization is an emergent property of life, driven by the collective behavior of individual components acting on local information. Biological neurons, through local interactions transmitted through synapses, are able to learn efficiently and can adapt their connections over an organism's lifespan. Motivated by these desirable properties of adaptability and local interaction, neural cellular automata (NCA) models have been successful at learning morphogenesis solely through local update rules, demonstrating stability over many updates and robustness to perturbations. In this work, we introduce Meta Neural Cellular Automata (MetaNCA), a framework that learns local rules which self-organize the weights of artificial neural networks. A learned rule network iteratively updates the weights of a task network using only local interactions on the computation graph. We propose a novel Weight Transformer architecture for the local rule network, which uses linear attention to aggregate signals from neighboring weights and hidden states. Once trained, the rule network generates task networks of diverse architectures without backpropagation. We show that MetaNCA generates weights for feedforward MLPs, CNNs, and ResNets on MNIST and CIFAR-100, scaling to networks of 2 million parameters. We further show that MetaNCA generalizes to architectures not seen during meta-training, and that architectural diversity in the training phase strengthens this generalization.
Meet Barot, Daniel Berenberg, Sina Khajehabdollahi
Mythos Scientific, New York, USA · Independent Scholar
As Neural Cellular Automata (NCAs) are increasingly applied outside of the toy models in Artificial Life, there is a pressing need to understand how they behave and to build appropriate routes to interpret what they have learnt. By their very nature, the benefits of training NCAs are balanced with a lack of interpretability: we can engineer emergent behaviour, but have limited ability to understand what has been learnt. In this paper, we apply a variety of techniques to pry open the NCA black box and glean some understanding of what it has learnt to do. We apply techniques from manifold learning (principal components analysis and both dense and sparse autoencoders) along with techniques from topological data analysis (persistent homology) to capture the NCA's underlying behavioural manifold, with varying success. Results show that when analysis is performed at a macroscopic level (i.e. taking the entire NCA state as a single data point), the underlying manifold is often quite simple and can be captured and analysed quite well. When analysis is performed at a microscopic level (i.e. taking the state of individual cells as a single data point), the manifold is highly complex and more complicated techniques are required in order to make sense of it.
James Stovold, Mia-Katrin Kvalsund, Harald Michael Ludwig +2
University of York, UK · University of Oslo, Norway · Machine Learning Research Unit, TU Wien, Austria +2