ConwayNet: Interpretability, Activation Coherence, and Noise Resilience via a Game-of-Life-Inspired Lateral Inhibition in Artificial Neural Networks
Abstract
Deep neural networks achieve universal function approximation by stacking interconnected nonlinearities, but as a consequence they are sensitive to noise and resist mechanistic interpretation. In contrast, the local update rules of cellular automata, such as Conway's Game of Life, give rise to not merely interpretable but programmable dynamics. We explore a bridge between these worlds: a lateral inhibition in neural networks based on continuous Game-of-Life rules that derives inhibitory and excitatory responses from the layer input and blends them into the activation map in a state-conditioned way. The mechanism is differentiable and architecture-agnostic, and its regulation is imposed rather than learned: the rule's parameters are fixed and are not optimised, while the rule's values and derivatives shape training throughout the network. Wired into fully connected, convolutional, wide-residual and graph convolutional networks, it gives rise to several properties. Activation maps become coherent across space, depth and seeds, with coarse structure persisting through depth while fine detail is renewed at every layer. The co-activation graph of the hidden units departs from an Erdős--Rényi random graph: it is laid out on the image and its giant component grows in steps, as in a spatial graph. Networks started from nearly identical initialisations, under label noise, disagree on thirty times fewer test images. Saliency maps rest on the object rather than the background, and resilience to pixel noise not seen in training is substantially improved.
Figures & tables
Appendix figures & tables29 assets
Supplementary material from the paper’s appendix.
Appendix
| log-ratio | favours object (%) | rank | rel. | FD error, contrast dir. | FD error, random dir. (%) | |||||
|---|---|---|---|---|---|---|---|---|---|---|
| dataset | surr. | exact | surr. | exact | corr. | diff. | surr. (%) | exact | surr. | exact |
| MNIST | 100 | 100 | 0.97 | 0.75 | 657 | 25 | 36 | |||
| Fashion-MNIST | 83 | 91 | 0.97 | 0.70 | 623 | 57 | 17 | |||
| KMNIST | 95 | 97 | 0.97 | 0.57 | 570 | 35 | 17 | |||
| EMNIST | 100 | 100 | 0.97 | 0.61 | 473 | 41 | 11 | |||
| MNIST (colour) | 100 | 100 | 0.97 | 0.65 | 361 | 39 | 28 | |||
| arm | frame | rules | val | LFPR | lens@1 | lens@10 | SSI |
|---|---|---|---|---|---|---|---|
| with lateral inhibition (B5/S3) | grid | yes | 0.962 | 0.941 | 57.5 | 96.5 | 18.2 |
| plain | none | – | 0.984 | 0.090 | 9.6 | 98.4 | 1.1 |
| input blur ( ) | grid | no | 0.987 | 0.132 | 10.9 | 97.8 | 3.5 |
| output blur ( ) | grid | no | 0.984 | 0.722 | 10.4 | 97.6 | 1.0 |
| random-neighbour | random | yes | 0.301 | 0.101 | 10.1 | 22.8 | 1.1 |
| arm | val | LFPR | lens@1 | SSI |
|---|---|---|---|---|
| with lateral inhibition | 0.962 | 0.941 | 57.5 | 18.2 |
| plain | 0.984 | 0.090 | 9.6 | 1.1 |
| input blur | 0.985 | 0.097 | 8.6 | 1.7 |
| input blur | 0.987 | 0.132 | 10.9 | 3.5 |
| input blur | 0.986 | 0.274 | 31.5 | 14.6 |
| input blur | 0.985 | 0.350 | 49.0 | 15.9 |
| regulation | agreement, trained | agreement, untrained | weight average (%) |
|---|---|---|---|
| with lateral inhibition (B5/S3) | 0.83 | 0.11 | 88.8 |
| input blur | 0.01 | 0.01 | 27.7 |
| input blur | 0.22 | 0.04 | 28.8 |
| input blur | 0.97 | 0.74 | 89.1 |
| input blur | 1.00 | 0.96 | 94.2 |
| plain | 0.00 | 0.00 | 10.1 |
| regulation | val | iso | (whole-graph) | grid-local |
|---|---|---|---|---|
| with lateral inhibition (B5/S3) | 0.962 | 0.24 | 0.82 | 1.00 |
| plain | 0.984 | 0.69 | 0.75 | 0.01 |
| plain, active units only | – | 0.24 | 0.50 | 0.01 |
| local-average (grid, no rules) | 0.984 | 0.25 | 0.88 | 0.35 |
| random-neighbour (rules, no grid) | 0.301 | 0.82 | 0.33 | 0.01 |
| dataset | classes | eff. rank (with/plain) | train acc (with/plain) | iso (with/plain) |
|---|---|---|---|---|
| MNIST | 10 | 10.9 / 8.7 | 0.999 / 1.000 | 0.22 / 0.72 |
| KMNIST | 10 | 12.1 / 8.7 | 0.999 / 1.000 | 0.25 / 0.71 |
| FashionMNIST | 10 | 9.9 / 6.8 | 0.996 / 0.971 | 0.20 / 0.38 |
| EMNIST | 47 | 19.3 / 16.1 | 0.913 / 0.930 | 0.24 / 0.21 |
| with lateral inhibition, depth 10 | plain, depth 10 | with lateral inhibition, depth 100 | |||||||
| band | wavelength (px) | ||||||||
| 9.11 ∗ | 0.54 | 8.85 ∗ | |||||||
| 7–14 | 8.22 ∗ | 0.51 | 8.17 ∗ | ||||||
| 4.7–7 | 4.68 ∗ | 0.52 | 5.81 ∗ | ||||||
| 3.1–4.7 | 2.61 | 0.51 | 3.83 ∗ | ||||||
| 2.2–3.1 | 1.00 | 0.50 | 1.88 | ||||||
| regulation | clean acc (%) | noise acc (%) | saliency log-ratio | region sep. |
|---|---|---|---|---|
| CA (with lateral inhibition) | 98.6 | 91.7 | 14.5 | |
| plain | 98.9 | 76.4 | 3.2 | |
| input blur, | 98.8 | 87.2 | 3.1 | |
| input blur, | 97.9 | 90.9 | 2.4 | |
| output blur, | 99.3 | 81.4 | 1.7 | |
| output blur, | 99.2 | 83.4 | 2.1 |
| CNN, with | CNN, plain | MLP, with | MLP, plain | |
| clean accuracy (%) | 98.8 | 99.0 | 96.5 | 98.4 |
| object pixels noised (%) | 60.5 | 74.4 | 85.3 | 94.4 |
| as many background pixels noised (%) | 96.3 | 67.0 | 79.8 | 96.9 |
| saliency log-ratio , full gradient | ||||
| gate routes detached (trunk only) | – | vanishes | – | |
| untrained network |
| ECE (%) | margin | entropy | top alt. | spread | ranking stability | agreement with | |||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| network | raw / cal. | raw / cal. | raw / cal. | raw / cal. | raw / cal. | seeds | images | top | visual | errors | |
| MLP, with | 1.38 | 1.69 / 0.59 | 8.5 / 6.1 | .004 / .038 | / | 3.3 / 2.4 | 0.90 | 0.79 | 0.42 | 0.65 | 0.74 |
| MLP, plain | 1.31 | 1.16 / 0.42 | 8.9 / 6.8 | .008 / .055 | / | 0.5 / 0.4 | 0.73 | 0.82 | 0.53 | 0.54 | 0.58 |
| CNN, with | 1.15 | 0.26 / 0.44 | 9.5 / 8.2 | .001 / .004 | / | 4.7 / 4.1 | 0.97 | 0.78 | 0.40 | 0.58 | 0.68 |
| CNN, plain | 2.61 | 0.76 / 0.23 | 24.6 / 9.4 | / .001 | / | 9.1 / 3.5 | 0.96 | 0.74 | 0.41 | 0.61 | 0.64 |
| network | gate routes | clean | noise | sal. log-ratio | cls. LFPR | SSI | lens@1 |
|---|---|---|---|---|---|---|---|
| MLP-10, MNIST | kept | 96.0 | – | 0.69 | 18.2 | 62.2 | |
| cut | 94.1 | – | 0.23 | 9.8 | 54.9 | ||
| CNN, MNIST | kept | 98.5 | 92.2 | – | – | – | |
| cut | 98.6 | 91.8 | – | – | – | ||
| WRN, CIFAR-10 | kept | 90.7 | 39.1 | – | – | – | – |
| cut | 80.7 | 36.4 | – | – | – | – |
| (points) | ||||||||
|---|---|---|---|---|---|---|---|---|
| network | dataset | mean | per seed pair | seeds | images | both | mean | both |
| MLP-2 | MNIST | |||||||
| MLP-2 | Fashion | |||||||
| MLP-2 | KMNIST | |||||||
| MLP-2 | EMNIST | |||||||
| CNN | MNIST | |||||||
| network | benchmark | corruption | with (%) | plain (%) | |
|---|---|---|---|---|---|
| WRN | CIFAR-10 | clean | 91.8 | 91.0 | |
| CIFAR-10-C | Gaussian | 74.8 | 68.8 | ||
| shot | 79.5 | 74.5 | |||
| impulse | 79.2 | 76.3 | |||
| speckle | 80.3 | 75.5 |
| network | active units | giant at | largest jump | edge overlap | floor overlap | |||
| MLP | with lateral inhibition | ( – ) | ( – ) | ( – ) | ( – ) | ( – ) | ||
| plain | – | ( – ) | ( – ) | ( – ) | ( – ) | ( – ) | ( – ) | |
| blur | — | |||||||
| blur | — | — | ||||||
| CNN | with lateral inhibition | – | ( – ) | ( – ) | ( – ) | ( – ) | ( – ) | ( – ) |
| plain | – | ( – ) | ( – ) | ( – ) | ( – ) | ( – ) | ( – ) |
Neural Network Interpretability
Neural network interpretability studies how models represent information and make predictions, and how to explain, test, or control their internal computations.
See all 364 papers