Biological neurons achieve computational diversity through specialized types: tonic, bursting, adapting, and fast-spiking cells coexist within the same circuit. Artificial neural networks, by contrast, apply a single activation function uniformly to all nodes, which limits what they can represent. We show that this uniformity creates hard limits for evolutionary search: across sparse evolved substrates, monotonic functions fail to solve parity beyond its smallest instance, XOR, while a single oscillatory unit suffices at all tested scales. The gap is one of search and sparsity, not representation: monotonic networks can represent parity with a modest number of hidden units, and gradient descent recovers that solution. We evolve, to our knowledge for the first time in indirect encoding, per-node activation function assignments from an 18-function palette across more than 4,500 experimental runs spanning Boolean logic, regression, and spatial classification. Testing each of the 18 functions individually on Parity-4 reveals a three-tier solvability structure: oscillatory functions achieve 100%, intermediate functions 6.7-80%, and all 9 monotonic functions 0%. This divide is not universal. Recurrence collapses it, and gradient descent inverts it entirely, showing that the barrier is specific to evolutionary search in sparse substrates. What activation functions a network can use, beyond its topology and weights, determines what evolutionary search can solve. Indirect encoding discovers heterogeneous per-node activation assignments unlikely to be chosen by hand.
Figures & tables
Figure 1: System pipeline. The activation palette, through the per-node selection mechanism, assigns each node its own activation function; the CPPN genome (NEAT) is decoded into a substrate carrying these per-node activations, the substrate is evaluated for fitness, and NEAT selection closes the evolutionary loop. Three assignment methods determine how CPPN output maps to function selection.
Idx
Name
Formula
Category
Solve%
Avg Gen
Multi-crossing (100% solve, 2.9–69.2 gen)
4
sin
sin(x)
Oscillatory
100
2.9
13
osc_adapt
sin(x)(1−0.2∣x∣)
Oscillatory
100
3.7
11
burst
tanh(x)+0.5sin(3x)
Oscillatory
100
10.7
16
band_pass
e−∣x−1∣−e−∣x+1∣
Adaptive
100
69.2
Intermediate (6.7–80% solve, 55–246 gen)
Table 1: 18 activation functions and their individual Parity-4 results ( Per-Function Ablation ; N=30 seeds; 300 gen max). The solvability boundary follows oscillatory structure: functions with local extrema solve parity; 9 strictly monotonic functions never solve.
Figure 2: Oscillatory structure predicts solvability. Local extrema count in [−5,5] vs Parity-4 solve rate ( N=30 ) for all 18 activation functions. Functions with more local extrema achieve higher solve rates. The two outliers at 2 extrema reflect extrema amplitude: band_pass has deep sign-reversing extrema (100%), while integrate has shallow perturbations of a monotonic trend (0%).
Problem
Samples
Sin%
Sin Gen
Tanh%
Tanh Fit.
Parity-2
4
100
1.0
50.0
0.930
Parity-3
8
100
2.2
3.3
0.890
Parity-4
16
100
2.9
0
0.863
Parity-5
32
100
2.9
0
0.826
Parity-6
64
100
3.4
0
0.825
Parity-7
128
100
3.7
0
0.821
Table 2: Parity scaling: sin vs tanh ( N=30 seeds each; population 150; 100 gen max). Sin achieves 100% across all sizes; tanh degrades monotonically. Percentages are solve rates (replications reaching the threshold).
Pop Size
Sin%
Sin Gen
Tanh%
Tanh Fit.
50
100
5.0
0
0.808
150
100
3.0
0
0.863
500
100
2.0
0
0.894
1,000
100
1.0
0
0.900
2,000
100
1.0
0
0.910
Table 3: Population reshapes the dynamics but not the oscillatory divide (Parity-4, N=30 per cell, 100 gen max). Sin solves 100% at every size; tanh 0% at every size. Percentages are solve rates.
Method
Configuration
Accuracy
Std
CPPN-output
cppn_output_4
76.9%
0.29
cppn_output_6
76.9%
0.29
cppn_with_sin
76.1%
0.42
Weight-interp.
sign-based
75.2%
0.14
magnitude-based
75.2%
0.14
variance-based
75.2%
0.14
Table 4: Assignment method comparison on Two Spirals ( N=30 ; depth 3). CPPN-output significantly outperforms the other methods; weight-interpretation gives no gain over the uniform baseline. cppn_output_ k = CPPN-output over a k -function palette (4: monotonic only; 6: adds sin, gauss).
Palette Size
Functions
Avg Gen
Std
1 (sin only)
sin
1.7
0.7
3
sin, tanh, relu
3.4
1.2
4
+sigmoid
3.1
1.1
5
+identity
3.2
1.2
6
+gauss
2.6
1.1
7
+lelu
3.3
1.0
Table 5: Palette size vs convergence speed ( N=30 seeds per size; Parity-4; all 100% success).
Function
FF%
FF Gen
Rec%
Rec Gen
sin
100
2.9
100
2.0
osc_adapt
100
3.7
100
3.0
burst
100
10.7
100
5.0
band_pass
100
69.2
100
10.0
receptive
80
100.1
100
3.0
resonator
66.7
55.9
100
2.0
Table 6: Feedforward vs recurrent topology on Parity-4 (both N=30 , 300 gen max, pop 150; solve threshold 0.95). Recurrence partially collapses the divide: oscillatory functions remain at 100%; bounded monotonic partially solve; unbounded/linear remain near 0%. % = solve rate.
Connection Type
Solve%
Mean Gen
Avg Fitness
feedforward only
0
—
0.914
h-to-h only
20.0
79.5
0.911
h-to-h + backward
13.3
191.2
0.917
h-to-h + lateral
6.7
117.5
0.881
h-to-h + self-loops
6.7
58.5
0.885
all recurrent
6.7
98.5
0.893
Table 7: Connection type decomposition (tanh on Parity-4, N=30 , 300 gen max, pop 150; solve threshold 0.95). Each row adds one connection type to hidden-to-hidden (h-to-h) as base.
Function
FF%
Full Rec%
Hidden-only%
osc_adapt
100
100
100
sin
100
80
50
resonator
100
70
57
gauss
43
3.3
3.3
burst
0
0
0
sigmoid
3.3
0
6.7
Table 8: Recurrence on concentric circles ( N=30 each; solve threshold 0.90). Opposite to parity, recurrence degrades most functions; only osc_adapt stays at 100%. FF% = feedforward solve rate; Full Rec% = full recurrence; Hidden-only% = recurrence on hidden-to-hidden connections only.
Parity-4
Sine Regr.
Step Func.
Circles
Function
%
Gen
%
Gen
%
Gen
%
Gen
sin
100
2.9
100
1
97
55
100
23
osc_adapt
100
3.7
100
1
100
49
100
2
burst
100
10.7
100
6
100
24
0
—
resonator
67
56
100
1
100
50
100
8
gauss
33
139
100
2
13
152
43
155
Table 9: Cross-problem comparison for representative functions. The three-tier solvability structure replicates on problems requiring non-monotonic decision surfaces (Parity-4, Circles) and vanishes on smooth problems (Sine, Step). All cells N=30 . % = solve rate; Gen = median generations to solve.
Problem
Act.
h
Solve%
Avg Epoch
Parity-4
sin
16
100
257
Parity-4
tanh
16
100
182
Parity-5
sin
16
100
477
Parity-5
sin
32
100
543
Parity-5
tanh
16
100
245
Parity-6
sin
16
66.7
2,653
Table 10: Gradient-trained MLP baseline (Adam, lr=0.01 , 5,000 epochs max, N=30 ). The divide progressively reverses: tanh maintains 100% through Parity-8; sin degrades to 0%. Increasing h does not help sin; it makes it worse on Parity-6 (66.7% → 46.7%).
Indirectly encoded neural networks can assign different activation functions to individual nodes, but the right functions are rarely known in advance. When the available set contains only standard monotonic functions, problems like parity become unsolvable, yet an all-inclusive palette underperforms a curated one. How should evolution discover which functions to use? We address this as a meta-learning problem, designing 13 strategies (11 inspired by biological adaptation mechanisms, plus baseline and oracle controls) that modify the set of available activation functions during evolution. Each strategy translates a biological principle into an evolutionary operator: for example, circadian-inspired oscillatory gating cycles functions in and out of the palette on a fixed schedule, while immune-inspired Clonal Selection permanently protects functions that consistently correlate with fitness. We evaluate all strategies across more than 3,000 runs on parity and non-parity problems, first evolving the activation palette alone, then co-evolving a per-node aggregation palette on harder problems; an independent replication with new seeds confirms a stable high-reliability tier, with Circadian holding its top rank. Bio-inspired strategies match the solve rate of a tuned baseline but converge up to twice as fast, with Circadian halving total compute. Strategy rankings reverse across problem types, with no strategy dominating all domains. Strategy success is largely shaped by timescale compatibility: strategies whose characteristic timescale matches the evolutionary evaluation window consistently outperform those that operate too slowly. The practical guideline: match the mechanism's timescale to the evaluation budget. Rescaling the slowest strategy bypasses the oscillatory barrier entirely: all nine solutions solve parity with non-oscillatory activations paired with min or max aggregation.
Romain Claret, Michael O'Neill, Paul Cotofrei +1
University of Neuchâtel, Neuchâtel, Switzerland · University College Dublin, Ireland
Open-ended artificial life systems must acquire diverse competencies from a single evolving genotype. Biological brains combine neuromodulation, which reconfigures circuits without changing connections, with diverse neuron types matched to specific computational roles. Can artificial evolution achieve something analogous in indirectly encoded substrates? Using indirectly encoded substrates evolved via CPPNs, we show through more than 10,000 experiments that neuromodulation alone is insufficient: under evolutionary search, monotonic activation functions impose a 75% ceiling on parity tasks that persists regardless of capacity, topology, or population size. This is an evolutionary search barrier, not a representational limit, since Adam gradient descent achieves 100% on the identical architecture. We combine neuromodulation with per-task activation function selection, matching oscillatory primitives to parity tasks and monotonic to threshold tasks, producing multi-behavioral evolved substrates. The result: 100% simultaneous 5-task success across all 30 seeds (median 14 generations). This generalizes across the oscillatory activation class: all four functions reach 100% (30 seeds each). Neither mechanism suffices alone. The barrier extends to higher-arity and asymmetric tasks, while multi-layer depth provides an alternative path. For open-ended evolution, the computational primitive should itself be an evolvable trait. At inference, one evolved genotype expresses many behaviors.
Romain Claret, Michael O'Neill, Paul Cotofrei +1
University of Neuchˆatel, Switzerland · University College Dublin, Ireland
Monotone Boolean functions are a structurally important class of Boolean functions, but their restricted form imposes strong limitations on achievable nonlinearity. In this paper, we investigate whether evolutionary computation can evolve monotone Boolean functions with high nonlinearity, both in the balanced and imbalanced settings. We consider three solution encodings: the standard truth table representation, a balanced truth table encoding that preserves Hamming weight, and a symbolic tree-based genetic programming representation. To guide the search toward monotone increasing functions, we introduce a non-monotonicity penalty and combine it with fitness functions targeting balancedness and nonlinearity. Experimental results are reported for dimensions from n=5 to n=14. The results show that evolutionary search can discover monotone Boolean functions with nonlinearities clearly exceeding those of majority functions, and in several cases approaching the best currently known values for monotone functions. At the same time, the experiments reveal substantial differences between encodings: the balanced truth table encoding performs poorly for larger dimensions, while the standard truth table and genetic programming encodings remain competitive, with genetic programming becoming especially relevant in the largest tested dimensions.
Claude Carlet, Marko Čupić, Marko Ðurasevic +3
University of Paris 8, 2 rue de la liberté, 93526 Saint-Denis Cedex, France · University of Bergen, Bergen, Norway · Faculty of Electrical Engineering and Computing, University of Zagreb, Unska 3, Zagreb, Croatia +2