Modern deep learning models achieve impressive generalization through over-parameterization, but this paradigm often struggles with overfitting and memorization in few-shot regimes. Neural Cellular Automata (NCAs) offer a highly parameter-efficient alternative, yet research has focused primarily on their output, leaving the role of their internal hidden channels largely unexplored. In this paper, we investigate the internal dynamics of NCA hidden channels and introduce a novel transfer-learning mechanism that injects a pretrained teacher's hidden states into a student model to guide early optimization. Evaluated on few-shot and scale-variant MNIST benchmarks, NCAs outperform comparable recurrent and feed-forward architectures, demonstrating superior generalization with a minimal parameter budget (~9,800 parameters). Mechanistic analysis reveals that the hidden channels decouple feature extraction from uniform classification consensus by absorbing morphological complexity and converging to mutually orthogonal states. Furthermore, we demonstrate that these hidden channels capture general, scale-invariant topological primitives rather than class-specific templates. This allows a student model to achieve strong few-shot performance on unseen classes using features transferred from a teacher trained only on a subset of digits (0-5). Our results highlight the potential of utilizing hidden-state dynamics as a robust, decentralized computational substrate for parameter-efficient transfer learning
Figures & tables
Figure 1: Diagram showing how the hidden channels are extracted from a teacher NCA and used in a Student NCA. The teacher NCA sees a digit, produces a classification, and some hidden channels are used in computing. These hidden channels are extracted and detached and passed to the student NCA alongside the digit before the forward pass in training.
Figure 2: All 4 models, tested on k∈{1,5,10,25,50,100} example per class, all have ≈9800 parameters. Since there are 10 classes, 101 examples equals one example per class. Scales range from 100 percent MNIST to 25 percent MNIST, with 25 percent intervals.
Figure 3: All 4 models, tested on k∈{1,5,10,25,50,100} example per class, all have ≈9800 parameters. Since there are 10 classes, 101 examples equals one example per class. Scales range from 100 percent MNIST to 25 percent MNIST, with 25 percent intervals. NCA receives a non-solution hidden state across a portion of training.
Figure 4: Training loss (blue) and validation loss (orange) of one run of the NCA in a 1-shot environment trained with the hidden injections.
Figure 5: All 4 models, tested on k∈{1,5,10,25,50,100} example per class, all have ≈9800 parameters. Since there are 10 classes, 101 examples equals one example per class. Scales range from 100 percent MNIST to 25 percent MNIST, with 25 percent intervals. NCA receives the non-solution hidden channels across all of training and validation.
Figure 6: All 4 models, tested on k∈{1,5,10,25,50,100} example per class, all have ≈9800 parameters. Since there are 10 classes, 101 examples equals one example per class. Scales range from 100 percent MNIST to 25 percent MNIST, with 25 percent intervals. NCA receives the non-solution hidden channels across all of training and validation.
Figure 7: Top: Hidden channel representations built by the NCA on a full-size MNIST 4. Bottom: Hidden channel representations built by the same NCA on a 50% scale version of the same 4.
Figure 8: Temporal evolution of the spatial variance of the hidden channels of (a) the teacher NCA and (b) the student NCA.
Figure 9: Cross-Channel Cosine Similarity between all channel pairs over time for (a) the teacher NCA and (b) the Student NCA.
Figure 10: 2D UMAP trajectory of the hidden channels for the Teacher (blue), Student with injection (green), and Student with no injection (red).