Visualising the Attractor Landscape of Neural Cellular Automata
Authors: James Stovold, Mia-Katrin Kvalsund, Harald Michael Ludwig, Varun Sharma, Alexander Mordvintsev
Organizations: University of York, UK · University of Oslo, Norway · Machine Learning Research Unit, TU Wien, Austria · CeMM Vienna, Austria · Paradigms of Intelligence Team, Google, Zurich
As Neural Cellular Automata (NCAs) are increasingly applied outside of the toy models in Artificial Life, there is a pressing need to understand how they behave and to build appropriate routes to interpret what they have learnt. By their very nature, the benefits of training NCAs are balanced with a lack of interpretability: we can engineer emergent behaviour, but have limited ability to understand what has been learnt. In this paper, we apply a variety of techniques to pry open the NCA black box and glean some understanding of what it has learnt to do. We apply techniques from manifold learning (principal components analysis and both dense and sparse autoencoders) along with techniques from topological data analysis (persistent homology) to capture the NCA's underlying behavioural manifold, with varying success. Results show that when analysis is performed at a macroscopic level (i.e. taking the entire NCA state as a single data point), the underlying manifold is often quite simple and can be captured and analysed quite well. When analysis is performed at a microscopic level (i.e. taking the state of individual cells as a single data point), the manifold is highly complex and more complicated techniques are required in order to make sense of it.
Figures & tables
Figure 1: Diagram depicting one pass of the NCA update step used in this paper. The diagram also shows the structure of the neural network. Image adapted from ( Mordvintsev et al., 2020 ) , licenced under CC BY 4.0.
Figure 2: Example classic target images used for training NCA models. Left, a green gecko used in ( Mordvintsev et al., 2020 ) ; middle, a blue gecko used in ( Stovold, 2023 ) ; right, ‘interlaced 0172’ texture from ( Cimpoi et al., 2014 ) used in ( Niklasson et al., 2021 ) .
Figure 3: The creation of the persistence diagram (right) consist of growing circles around your points with increasing radius r , and connecting components if the growing circles overlap with a point. Here, the toy data is in blue, growing cirles are in shaded grey, and edges between connected components are in orange.
H0
H1
H2
Ring
1
1
0
Two disjoint rings
2
2
0
Sphere
1
0
1
Torus
1
2
1
Table 1: Betti numbers for common topologies. In general, H0 tells us how many distinct components there are in the data, H1 tells us how many rings exist, and H2 tells us how many voids there are. Adapted from ( Kemme and Agyingi, 2025 ) .
Figure 4: Persistence diagram for noisy data. The green arrow points to a strongly persistent ring. The rest of the recorded components, like the H2 group that briefly appears at r=7.1 , can be considered noise because their persistence is short-lived.
Figure 5: The blue/green gecko model from ( Stovold, 2023 ) being trained. (Top) loss as it trains, showing transition at around 3500–4000 epochs; (middle) latent space of the autoencoder model after training for the 1000, 2000, 4000, and 10,000 epochs; (bottom) persistent homology of the underlying manifold at the corresponding training points, confirming a cycle forms ahead of 4000 epochs.
Figure 6: Perturbation on the blue/green gecko model. The left graphs show the first two principal components of the dataset, the right graphs show the persistent homology analysis. (a) A perturbation was applied on the 600th timestep, as the system was in the blue attractor. It returns to the green attractor and stays there until the episode ends. (b) The same was done, but the signal to change colour was still sent every 150 timestep. (c) State of the NCA at different timesteps.
Figure 7: Field lines superposed on the underlying latent space (captured using PCA). To visualise the vector field, we sampled this latent space, reconstructed the high-dimensional representation, ran it through the NCA for a small number of steps and then reduced it back down to the latent space. This allows us to capture which direction the NCA model would move in the latent space, and hence to visualise the behaviour of the system across the latent space.
Figure 8: A sample of 1 million data points (excluding dead cells) encoded using an autoencoder to visualise the underlying latent space. The clumps of cells indicate local attractors for individual cells. The NCA model used was the 10,000 epoch blue/green gecko model.
Figure 9: The PH diagrams (left) and the PCA projection of a) the full substrate and b) the subsampled substrate, based on 20 timesteps.
Figure 10: Per-frame manifold of the blue/green gecko NCA in SAE feature space. Each point is the mean SAE activation vector across all living cells in one frame, projected using PCA and coloured by the mean RGB value of the frame. The two attractors sit at opposite ends of the first principal plane and the trajectory traces a clean cycle between them, recovering the macroscopic cycle of fig. 5 from purely microscopic per-cell data.
Throughout the literature on Neural Cellular Automata (NCAs), it is often taken for granted that the systems learn attractors. This is shown through evolving the system for many timesteps and noting visual similarity to the goal state. There remain many questions after such an analysis. Namely, what kind of attractors do we have? Is their behavior ordered or chaotic? Can we estimate stability over very long time horizons? What really happens in the attractor when perturbations are applied? In this paper, we present a case study to help answer these questions, with methods drawn from the literature on dynamical systems theory. We use the growing gecko NCA of Mordvintsev et al. (2020) with deterministic cell updates as a case study. To the best of the authors' knowledge, we present the first visualizations of NCA attractor dynamics. We also analyze them using the Lyapunov and Fourier spectra, to reveal that the NCA displays oscillatory, periodic and quasi-periodic behavior, and that these behaviors arise early during training. This challenges the belief that NCAs learn fixed point attractors. Finally, we show that large perturbations to the attractor states can throw the NCAs into a secondary mode separate from the original attractor. We hope that this initial foray into NCA attractor dynamics expands the toolkit for NCA researchers to analyze the robustness and stability of their systems.
Mia-Katrin Kvalsund, James Stovold
Department of Physics, University of Oslo, Norway · Department of Computer Science, University of York, UK
Stephen Wolfram proclaimed in his 2003 seminal work "A New Kind Of Science" that simple recursive programs in the form of Cellular Automata (CA) are a promising approach to replace currently used mathematical formalizations, e.g. differential equations, to improve the modeling of complex systems. Over two decades later, while Cellular Automata have still been waiting for a substantial breakthrough in scientific applications, recent research showed new and promising approaches which combine Wolfram's ideas with learnable Artificial Neural Networks: So-called Neural Cellular Automata (NCA) are able to learn the complex update rules of CA from data samples, allowing them to model complex, self-organizing generative systems. The aim of this paper is to review the existing work on NCA and provide a unified modular framework and notation, as well as a reference implementation in the open-source library NCAtorch. Supplementary materials, videos, and code are available at the project website: https://www.neural-cellular-automata.org/
Martin Spitznagel, Janis Keuper
Institute for Machine Learning and Analytics (IMLA), Offenburg University, Germany · University of Mannheim, Germany
Modern deep learning models achieve impressive generalization through over-parameterization, but this paradigm often struggles with overfitting and memorization in few-shot regimes. Neural Cellular Automata (NCAs) offer a highly parameter-efficient alternative, yet research has focused primarily on their output, leaving the role of their internal hidden channels largely unexplored. In this paper, we investigate the internal dynamics of NCA hidden channels and introduce a novel transfer-learning mechanism that injects a pretrained teacher's hidden states into a student model to guide early optimization. Evaluated on few-shot and scale-variant MNIST benchmarks, NCAs outperform comparable recurrent and feed-forward architectures, demonstrating superior generalization with a minimal parameter budget (~9,800 parameters). Mechanistic analysis reveals that the hidden channels decouple feature extraction from uniform classification consensus by absorbing morphological complexity and converging to mutually orthogonal states. Furthermore, we demonstrate that these hidden channels capture general, scale-invariant topological primitives rather than class-specific templates. This allows a student model to achieve strong few-shot performance on unseen classes using features transferred from a teacher trained only on a subset of digits (0-5). Our results highlight the potential of utilizing hidden-state dynamics as a robust, decentralized computational substrate for parameter-efficient transfer learning
Etienne Guichard, Stefano Nichele
Østfold University of Applied Sciences, BRA Veien 4, 1757 Halden, Norway