Emergent Models: Intelligence from Tiny Substrates

Giacomo Bocchese; Nicola Giacobbo; Etienne Guichard; James Wiles; Akshaj Devireddy

cs.NEAug 14, 2026

Emergent Models: Intelligence from Tiny Substrates

Authors: Giacomo Bocchese, Nicola Giacobbo, Etienne Guichard, James Wiles, Akshaj Devireddy

Organizations: Wolfram Institute · Emergent Computing · Østfold University College

Abstract

Emergent Models (EMs) are a machine learning paradigm based on simple yet open-ended substrates, such as cellular automata, in which modeling is treated not as the learning of a closed-form input-output map but as the emergence, within simple dynamical systems, of computational behaviors that solve external tasks. Such substrates typically iterate a fixed local rule over a latent space for an adaptive number of steps, with an interface linking the latent state to external input/output signals. Training proceeds by evolutionary search. We hypothesize that some instances of this framework are biased toward global generalization: capturing the rule generating the data over its full domain, and therefore extrapolating beyond the training range. Theoretically, we prove that some EMs are latent-universal: with the update rule and interface held fixed, they can realize any partial computable function by varying only the initial condition of the latent state. Empirically, we study a zoo of minimal EM instantiations across discrete and continuous substrates, showing that local-recursive computation at a tiny scale (tens to hundreds of parameters) can extrapolate exactly on simple arithmetic functions, can support control behaviour and online adaptation, while still exposing several limitations. This work is foundational: it does not propose a competitive architecture, but a framework meant to widen the design space of machine learning beyond differentiable feed-forward maps.

Figures & tables

Figure 1: Emergent Model with a two-dimensional continuous-valued lattice as latent state, $s\in\mathbb{R}^{20\times 20}$ . Coloured borders mark the ports: input (green), output (blue), halting (red), and reward (orange). Interface: the input $x\in\mathbb{R}^{2}$ is injected at the two input ports through the encoder $E(x)$ ; at halting, the decoder $D(s_{T})$ reads the output $y\in\mathbb{R}^{2}$ from the two output ports. Update: the transition rule is iterated, $s_{t+1}=f(s_{t})$ , for an adaptive number of steps, halting when $H(s_{t})=1$ . The role of the reward port is explained in Section 2.5 .

Figure 2: Diagram representing the latent state of EM43, a discrete Emergent Model, evolving in time. It is based on a four-valued, radius-1 cellular automaton over a one-dimensional unbounded tape, with state $s\in\{0,1,2,3\}^{\mathbb{N}}$ . This uses a different interface from fig. 1 : input is not written to a fixed port, but placed at a position that depends on its value. The same holds for the output. Encoding: an integer input $x\in\mathbb{N}$ is written as a red cell at position $x_{0}+x$ , i.e. at offset $x$ from a fixed origin $x_{0}$ . In this case input $x=10$ . Decoding: at halting, the output $y\in\mathbb{N}$ is read as the position of the rightmost red cell relative to the same origin $x_{0}$ . In this case output $y=20$ . Update: the transition rule is a lookup table over radius-1 neighbourhoods, applied synchronously on all the tape. Halting: triggered by a relative count of cell values (Explained in section 3.2 ).

Figure 3: Graphs: 1-d unbounded tape (on the left) with radius-1 interaction ; 2-d finite lattice (on the right) with Moore neighborhood. Dots represent nodes, while lines connecting the nodes are the edges.

Figure 4: Construction of the initial state $s_{0}=p\oplus E(x)$ on the same two-dimensional continuous-valued lattice of fig. 1 ; here $\oplus$ is a sum. Left: the program $p$ , non-perturbed initial state. Centre: the encoded input $E(x)$ , nonzero only at the input ports and quiescent elsewhere. Right: their sum, giving the perturbed initial state $s_{0}$ from which the dynamics are iterated.

Figure 5: Didactic TM example: program string $\rho$ , separator $\#$ , and input string $x$ on the tape, forming the initial state $\rho\,\#\,x\,b^{\infty}$ .

task	runs	gens	$L_{p}$	params	bits	train acc	test acc
$x+1$	8	20	10	70	140	100%	100%
$2x$	8	90	10	70	140	100%	100%
$x\bmod 4$	8	150	10	70	140	100%	100%
$\mathrm{round}(x/3)$	8	400	10	70	140	100%	100%
$x\cdot(1+x\bmod 2)$	2	500	10	70	140	100%	100%
$a\cdot b$	2	800	50	110	220	$<$ 20%	$<$ 10%

Table 1: Performance of EM43 across different tasks, showing number of runs, generations, parameter size, and accuracy.

Figure 6: Space-time plot of EM43 computing $\mathrm{round}(x/3)$ for $x=47$ , using the learned initial condition and update rule. The green line represents the origin of the encoding ( $x_{0}$ ). The input is encoded as a red cell at position $(x_{0}+x)$ in the initial condition, and the output is marked by the position of the rightmost red cell at halting time with respect to the origin.

Figure 7: Spacetime plot of a model representing $x\bmod 4$ , visualized for different inputs (15, 30 and 70).

Figure 8: EM43 spacetime plot for $x\cdot(1+x\bmod 2)$ . On the left for an odd input (33), on the right for an even input (34).

Figure 9: Automaton grid, here encoding the input $x=5$ and with a two-glider program. Each encoded integer corresponds to a $2\times 2$ block of automaton cells at a specific position; to represent this faithfully, the grid shown here is scaled by a factor of two relative to the underlying automaton lattice. Tick marks on the horizontal axis indicate the integer values represented by block positions, with respect to the origin.

task	train dataset $x$	test dataset $x$
$x-2$	$[2,12]$	$[12,200]$
$x-1$	$[1,11]$	$[11,200]$
$x+1$	$[0,10]$	$[10,200]$
$x+2$	$[0,10]$	$[10,200]$
$x+5$	$[0,10]$	$[10,200]$
$x+1$ , $x$ even $x-1$ , $x$ odd	$[0,10]$	$[10,200]$

Table 2: Training and evaluation input ranges for GoL arithmetic tasks. Lower bounds are chosen so that the target output is non-negative, since the positional decoder is not designed to represent negative integers.

task	$n_{\mathrm{evals}}$	$n_{\mathrm{gliders}}$	$L_{p}$	train acc	test acc
$x-2$	40k	2	18	100%	100%
$x-1$	40k	2	18	100%	100%
$x+1$	40k	2	18	100%	100%
$x+2$	40k	2	18	100%	100%
$x+5$	40k	2	18	100%	100%
$x+1$ , $x$ even $x-1$ , $x$ odd	1M	3	26	60%	50%

Table 3: Performance of the GoL-based Emergent Model across tested tasks, trained by random search over glider positions.

n_{\mathrm{evals}}

is the search budget (number of candidate programs sampled and evaluated). Accuracies are exact-match rates.

Figure 10: Observing a few steps of the trace computing $y=x+2$

Figure 11: Board layout of the GoL-EM controller for CartPole. The learned GoL program is placed in the upper-left region, separated from the input-output interface by a buffer zone. Four horizontal lanes encode the CartPole variables $q,\dot{q},\theta,\dot{\theta}$ . In each lane, one of six spatial bins is activated according to the discretized value of the corresponding variable. Blue cells indicate input-block locations, while red and green cells indicate left and right output-collector regions used for force decoding.

Figure 12: Single-bin motif used in the GoL-EM CartPole interface. If this bin is selected, the blue $I$ zone is activated as a $\mathbf{1}_{2\times 2}$ alive block. After GoL evolution, live cells in the red $L$ and green $R$ regions contribute to the aggregate left and right force votes.

Metric	Value
GoL computation budget $T$	300
Population size	100
Training generations	10
Final population mean return	171.36
Best training return	417.20
Evaluation return	$316.90\pm 121.52$

Table 4: Results from the representative run.

Figure 13: Learning curves for the representative run, showing mean and best CartPole return across generations. The maximum attainable return is 500.

Figure 14: Snapshots of GoL evolution during 1 CartPole macrostep.

Figure 15: Layout of CEM1D. The CartPole state variables are written into designated input ports, and the force is decoded from the central output port.

metric	value
generations	70
population size	120
lattice length $L$	19
internal steps $T$	30
input cells	$(3,6,12,15)$
output cell	$9$

Table 5: Results for a representative CEM1D CartPole training run.

Figure 16: Best controller return and its standard deviation across training generations.

Figure 17: Internal "reasoning" trace of the CEM1D controller during two environment steps. Rows correspond to time evolution in microsteps, and columns correspond to lattice cells in the one-dimensional medium. The horizontal line at $t=31$ marks the boundary between consecutive macrosteps, where the input is re-injected.

Figure 18: Information transport with right and left propagating patterns.

Figure 19: Meta-Life environment. Agents (blue dots) move and collect resources (green dots).

Figure 20: Meta-Life agent with its three lidar rays.

model	hard params size( $E,D,H,f$ )	soft params size( $S$ )	structure
CEM2D	35	128	local $8\times 8$ spatial fields
mRNN	117	12	dense vector recurrence

Table 6: Meta-Life controller sizes

task	model	population $A$	generations
Food-only	CEM2D	100	700
Food-only	mRNN	100	700
Meta-Life-Adapt	CEM2D	300	510
Meta-Life-Adapt	mRNN	300	260

Table 7: Training setup for representative Meta-Life runs

Figure 21: Meta-Life-Food training return per agent per rollout in generations.

model	train return	eval return
CEM2D	155	133
mRNN	150	146

Table 8: Meta-Life-Food results. Return is measured per agent per rollout.

model	meta score	reaction meta	after-reaction meta
CEM2D	0.714	0.780	0.699
mRNN	0.822	0.913	0.796

Table 9: Meta-Life-Adapt evaluation results. Random baseline is

0

, perfect behaviour is

1

Figure 22: Learned initial condition of the CEM2D controller on Meta-Life-Adapt settings.

Figure 23: Interpretability visualization of the learned CEM2D dynamics on Meta-Life-Adapt. The two panels show the lattice activation state when the agent estimates a positive (food) phase and a negative (poison) phase. The arrow indicates the regime-correlated cell.

model	substrate	hard params	soft params	param domain
EM43	1D CA	60	10 - 50*	$\{0,1,2,3\}$
GoL-EM (arith.)	2D GoL	0	100 - 300*	$\{0,1\}$
GoL-EM (CP)	2D GoL	0	100 - 300*	$\{0,1\}$
CEM1D	1D CEM	3	19	$\mathbb{R}$
CEM2D	2D CEM	35	128	mixed ∗∗
mRNN	dense RNN	117	12	$\mathbb{R}$

Table 10: Structural summary of the experimental models. ∗ For EM43 and GoL, soft parameters are indicated as program region size. ∗∗ For CEM2D: real-valued soft and update-rule parameters and integer-valued port locations.

model	capabilities
EM43	extrapolation on simple arithmetic tasks
GoL-EM (arithmetic)	limited capabilities on arithmetic tasks
GoL-EM (CartPole)	partially capable of control, not robust
CEM1D	stronger control capabilities, still not robust
CEM2D	capable of control and adaptation
mRNN	capable of control and adaptation, slightly stronger

Table 11: Behavioral summary of the experimental models.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 24: TM generalized automata scheme, with string on tape "101100", head at position 0 and head state q0.

Figure 25: Rendell’s UTM in GoL, organized as an Emergent Model with distinct regions. The program is a finite-support initial GoL pattern occupying a disjoint region from the encoded input. The encoder $E(x)$ overwrites on an input region organized in fixed modular patterns corresponding to the UTM tape cells and their symbols.

Algorithmic Generalization

See all 18 papers

A Limit Theory of Foundation Models: A Mathematical Approach to Understanding Emergent Intelligence and Scaling Laws
Apr 27, 2026Jun Shu, Junxiong Jia, Deyu Meng +1Neural Scaling Laws AI Scaling Laws
School of Mathematics and Statistics, Xi’an Jiaotong University.
Emergence via Phase Transitions: Mechanism Landscapes and Universal Convergence Across Complex Systems
May 25, 2026Truong Xuan KhanhDynamical Systems Phase Transitions in Neural Networks
H&K Research Studio, Clevix LLC, Hanoi, Vietnam
What Shapes Emergent Misalignment? Insights from Training Dynamics, Model Priors, and Data
Jun 18, 2026Yuchen Zhang, Anietta Weckauff, Diego Garcia-Olano +1LLM Evaluation LLM Alignment
Max Planck Institute for Intelligent Systems, ELLIS Institute Tübingen, Tübingen AI Center, Tübingen, Germany · Meta.

Apr 27, 2026J/K move · Enter open · S save

A Limit Theory of Foundation Models: A Mathematical Approach to Understanding Emergent Intelligence and Scaling Laws

Jun Shu, Junxiong Jia, Deyu Meng, Zongben Xu

School of Mathematics and Statistics, Xi’an Jiaotong University.

Neural Scaling Laws AI Scaling Laws

Open paper AskPDF arXiv

Emergent intelligence have played a major role in the modern AI development. While existing studies primarily rely on empirical observations to characterize this phenomenon, a rigorous theoretical framework remains underexplored. This study attempts to develop a mathematical approach to formalize emergent intelligence from the perspective of limit theory. Specifically, we introduce a performance function E(N, P, K), dependent on data size N, model size P and training steps K, to quantify intelligence behavior. We posit that intelligence emerges as a transition from finite to effectively infinite knowledge, and thus recast emergent intelligence as existence of the limit

\lim_{N,P,K \to \infty} \mathcal{E}(N,P,K)

, with emergent abilities corresponding to the limiting behavior. This limit theory helps reveal that emergent intelligence originates from the existence of a parameter-limit architecture (referred to as the limit architecture), and that emergent intelligence rationally corresponds to the learning behavior of this limit system. By introducing tools from nonlinear Lipschitz operator theory, we prove that the necessary and sufficient conditions for existence of the limit architecture. Furthermore, we derive the scaling law of foundation models by leveraging tools of Lipschitz operator and covering number. Theoretical results show that: 1) emergent intelligence is governed by three key factors-training steps, data size and the model architecture, where the properties of basic blocks play a crucial role in constructing foundation models; 2) the critical condition Lip(T)=1 for emergent intelligence provides theoretical support for existing findings. 3) emergent intelligence is determined by an infinite-dimensional system, yet can be effectively realized in practice through a finite-dimensional architecture. Our empirical results corroborate these theoretical findings.

Abstract

Algorithmic Generalization

Explore similar work

A Limit Theory of Foundation Models: A Mathematical Approach to Understanding Emergent Intelligence and Scaling Laws

Emergence via Phase Transitions: Mechanism Landscapes and Universal Convergence Across Complex Systems

What Shapes Emergent Misalignment? Insights from Training Dynamics, Model Priors, and Data