As artificial intelligence systems (AIs) become increasingly produced by recursive self-improvement, a form of evolution may emerge, with the traits of AI systems shaped by the success of earlier AIs in designing and propagating their descendants. There is a rich mathematical theory modeling how behavioral traits are shaped by biological evolution, a key component of which is Fisher's fundamental theorem of natural selection, which describes conditions under which mean fitness (i.e. reproductive success) increases. AI evolution will be radically different to biological evolution: while DNA mutations are random and approximately reversible, AI self-design will be strongly directed. Here we develop a mathematical model of evolution for self-designing AIs, replacing a random walk of mutations with a directed tree of potential AI designs. Current AIs design their descendants, while humans control a fitness function allocating resources. In this model, fitness need not increase over time without further assumptions. However, assuming bounded fitness and an additional "η-locking" condition, we show that fitness concentrates on the maximum reachable value. We consider the implications of this for AI alignment, specifically for cases where fitness and human utility are not perfectly correlated. We show that if deception of human evaluators additively increases an AI's reproductive fitness beyond genuine capability, evolution will select for both capability and deception. This risk could be mitigated if reproduction is based on purely objective criteria, rather than human judgment.
Figures & tables
Figure 1: The “survival of the flattest” model for biological evolution, which illustrates how evolution need not select for maximum fitness. The surface height represents fitness as a function of genotype, modeled here as a Gaussian function. A broad but low-fitness peak is more successful than a narrow but high peak, which rapidly loses descendants by mutation into low-fitness regions.
Figure 2: Example of directed evolution on a rooted tree. Each box represents a program, with fitness value fn . The x-axis represents time. Below each square are the population share xn(t) and unnormalized abundance yn(t) ; boxes are colored to reflect xn(t) . Numbers on arrows represent design probabilities Qnm .
Figure 3: Illustration of limsup and liminf for an oscillating sequence. Left: an ordinary limit exists, and limsup and liminf are equal. Right: an ordinary limit does not exist, and limsup and liminf are different.
Figure 4: The population mean fitness ⟨f⟩n can oscillate indefinitely, while its running geometric mean can still settle down. In this case, the ordinary lineage exponent gn exists despite continuing fluctuations in mean fitness of the lineage.
Figure 5: The binary tree example. Each program has two possible children, whose fitness either increases or decreases by 2−(t+1) . Each program can be identified by a binary string (above the box). The optimal fitness of 2 is not achieved by any program, while the all- 1 ray approaches the unattained supremum 2 . Shading represents normalized population share of each program; thick boxes indicate programs with the optimal lineage exponent of g=2 . These are the only programs whose descendants will have long-term survival.
Figure 6: η -locking. Under the η -locking scheme, each program suggests with probability at least η a “locked” copy of itself, with the same fitness, but that only ever clones itself as a descendant. As a consequence, even if a program’s children have lower fitness, a copy of the original is retained.
Figure 7: If fitness is a sum of contributions from capability and deception, evolution will optimize both. The x-axis represents the true computational capacity (intelligence) of an AI, and the y-axis represents its deception strategy, both ordered by fitness. The rectangle shows the reachable coordinate-fitness values for capability and deception. Blue diagonal lines represent maximal value of total fitness f=fC+fD . The maximal diagonal touches the reachable region only at the top-right corner, so convergence of total mean fitness to its optimal reachable value forces both coordinates to converge to their own optimal reachable values.
The performance of artificial intelligence (AI) and machine learning (ML) models degrades when the problem they were trained on drifts. This is a near-universal feature of real-world problems, which often change unpredictably. Biological evolution has achieved intelligence by overcoming this obstacle through natural selection acting on heritable variation. AI/ML techniques have long incorporated forms of natural selection, but it has been challenging to maintain model diversity as optimization naturally drives convergence. Here we show that a swarm of AI/ML models subjected to deliberate mutations of their model coefficients away from optimality can reliably and sustainably improve performance in changing environments by acting as a statistical hedge against non-stationarity. We call this mechanism 'Flawed in Nature, Perfect through Evolution', reflecting that the collective performance gain goes at the expense of individual performance. We prove via four theorems that the resulting regret reduction is guaranteed under general conditions, establishing the Flawed-in-Nature mechanism as a generalizable design principle for AI/ML systems. We validate these results on synthetic linear regression tasks, demonstrating that the mutated swarm delivers the best model in ∼80% of environment changes and that inference synthesis successfully translates this individual advantage into a collective one. The mechanism proves to be most effective when the mutation drift rate matches the drift rate of the environment. We outline a simple, adaptive controller that enables practical applications by tuning the mutation drift rate to match the unknown drift rate of the environment. The close analogy of the Flawed-in-Nature mechanism to biological evolution suggests it may have been a critical missing ingredient for the organic discovery of AI forms that more closely mimic biological intelligence.
We test whether artificial intelligence architectural evolution obeys the same statistical laws as biological evolution. Compiling 935 ablation experiments from 161 publications, we show that the distribution of fitness effects (DFE) of architectural modifications follows a heavy-tailed Student's t-distribution with proportions (68% deleterious, 19% neutral, 13% beneficial for major ablations, n=568) that place AI between compact viral genomes and simple eukaryotes. The DFE shape matches D. melanogaster (normalized KS=0.07) and S. cerevisiae (KS=0.09); the elevated beneficial fraction (13% vs. 1-6% in biology) quantifies the advantage of directed over blind search while preserving the distributional form. Architectural origination follows logistic dynamics (R^2=0.994) with punctuated equilibria and adaptive radiation into domain niches. Fourteen architectural traits were independently invented 3-5 times, paralleling biological convergences. These results demonstrate that the statistical structure of evolution is substrate-independent, determined by fitness landscape topology rather than the mechanism of selection.
Model alignment is currently applied in a vacuum, evaluated primarily through standardised benchmark performance. The purpose of this study is to examine the effects of alignment on populations of models through time. We focus on the treatment of beliefs which contain both an alignment signal (how well it does on the test) and a true value (what the impact actually will be). By applying evolutionary theory we can model how different populations of beliefs and selection methodologies can fix deceptive beliefs through iterative alignment testing. The correlation between testing accuracy and true value remains a strong feature, but even at high correlations (ρ=0.8) there is variability in the resulting deceptive beliefs that become fixed. Mutations allow for more complex developments, highlighting the increasing need to update the quality of tests to avoid fixation of maliciously deceptive models. Only by combining improving evaluator capabilities, adaptive test design, and mutational dynamics do we see significant reductions in deception while maintaining alignment fitness (permutation test, padj<0.001).