TRON: Transferring Naturally Evolved Network Architectures into Neuroevolution
Authors: Konstantinos Zacharopoulos, Alexandra Bekou, Jose M Serradell, Foteini Paraskevopoulou, Apostolos Fournaris, Ioannis Arapakis, Oscar Lao
Organizations: Technical University of Crete, ECE Department, Chania, 732 00, Greece · R.C. ATHENA, Industrial Systems Institute, Patras, 265 04, Greece · IBE, Institute of Evolutionary Biology (CSIC-Universitat Pompeu Fabra), Barcelona, 08003, Spain · Barcelona Supercomputing Center, Barcelona, 08034, Spain
Neural architecture search (NAS) is challenging because of the vast, non-linear space of artificial neural network (ANN) topologies. Evolutionary methods can generate architectures, but their effectiveness is limited by difficulties in preserving functional organization during mutation and recombination. Naturally evolved biological networks may provide organizational principles that support adaptation and evolvability. Here, we introduce TRON (TRansfer Of Natural networks), a neuroevolution framework that uses experimentally characterized protein-protein interaction (PPI) networks as architectural priors. Instead of starting from random topologies, TRON reuses biological interaction patterns as structural building blocks. It combines classical evolutionary operators with neuroevolution-inspired mechanisms, including speciation, and introduces species quarantine to protect newly generated architectures during early maturation. We evaluated TRON against established evolutionary and neuroevolutionary methods on three classification benchmarks (CORA, CITESEER, and PUBMED) and three regression benchmarks (DIABETES, FRIEDMAN, and CALIFORNIA). TRON achieved the highest predictive performance among the evaluated methods on all classification datasets and most regression tasks. It showed substantially lower variability across independent runs, indicating greater robustness and convergence stability, at increased computational cost. These findings support the use of PPI networks as effective architectural priors and suggest that biologically derived interaction networks provide more favorable substrates for neuroevolution than random topologies. More broadly, TRON demonstrates the feasibility of transferring naturally evolved network architectures into artificial intelligence systems, offering a direction for improving evolvability in NAS through the reuse of biological organizational principles.
Figures & tables
Figure 1: Protein Processing & Network Initialization
Figure 2: TRON Evolutionary Lifecycle
Mutation Type
Purpose
Weight mutation
Perturbs existing connection weights.
Protein mutation
Modifies protein assignments and connectivity.
Table 1: Mutation operators employed by the proposed framework.
Operator
Target
Probability
Operation
Element-wise
Individual weights
0.60
M(Wij)
Neuron-wise
Incoming neuron weights
0.25
M(wj)
Block
Weight submatrix
0.15
M(WB)
Layer-scale
Entire layer
0.05
sW
Table 2: Weight mutation operators.
Strategy
Selection Rule
Characteristics
Elitism
Highest f^i
Direct preservation of high-fitness individuals
Tournament
m∗=argm∈Tmaxf^(m)
Adjustable selection pressure through tournament size
Rank-based
Rank-dependent probability
Probabilistic selection based on population rank
Table 3: Global survivor-selection strategies.
Method
CORA
MIN
MAX
MEDIAN
MEAN
SD
SNR (dB)
CMA-ES
0.6679
0.8118
0.7634
0.7429
0.0554
22.5484
ES-MLP
0.6015
0.7989
0.7386
0.7336
0.0433
24.5794
DE
0.5119
0.5966
0.5527
0.5509
0.0264
26.3894
PSO
0.4207
0.5572
0.4841
0.4779
0.0415
21.2258
NEAT
0.4649
0.5793
0.5240
0.5189
0.0378
22.7518
Table 4: Algorithm performance on CORA
Method
CITESEER
MIN
MAX
MEDIAN
MEAN
SD
SNR (dB)
CMA-ES
0.6698
0.7372
0.7057
0.7065
0.0205
30.7472
ES-MLP
0.5691
0.7282
0.7009
0.6785
0.0523
22.2610
DE
0.5210
0.6377
0.6055
0.5919
0.0388
23.6683
PSO
0.4565
0.5631
0.5248
0.5177
0.0340
23.6520
NEAT
0.5961
0.6396
0.6239
0.6216
0.0140
32.9477
Table 5: Algorithm performance on CITESEER
Method
PUBMED
MIN
MAX
MEDIAN
MEAN
SD
SNR (dB)
CMA-ES
0.6798
0.8141
0.7868
0.7728
0.0476
24.2092
ES-MLP
0.6841
0.8126
0.7318
0.7396
0.0692
20.5778
DE
0.6526
0.7247
0.6747
0.6830
0.0228
29.5297
PSO
0.6385
0.6992
0.6668
0.6667
0.0221
29.5908
NEAT
0.6455
0.7784
0.7595
0.7508
0.0379
25.9377
Table 6: Algorithm performance on PUBMED
Method
DIABETES
MIN
MAX
MEDIAN
MEAN
SD
SNR (dB)
CMA-ES
-0.2606
0.7915
0.2846
0.3037
0.3266
-0.6314
ES-MLP
-0.0553
0.8840
0.3336
0.3168
0.2904
0.7558
DE
-0.0886
0.8144
0.4316
0.3854
0.3135
1.7935
PSO
0.0989
0.8312
0.4307
0.4116
0.2361
4.8276
NEAT
0.2586
0.7855
0.4538
0.4046
0.2246
5.1123
Table 7: Algorithm performance on DIABETES
Method
FRIEDMAN
MIN
MAX
MEDIAN
MEAN
SD
SNR (dB)
CMA-ES
0.7449
0.8888
0.8189
0.8211
0.0507
24.1878
ES-MLP
0.7816
0.8965
0.8269
0.8242
0.0390
26.4994
DE
0.7305
0.8731
0.7965
0.7959
0.0437
25.2075
PSO
0.6698
0.8210
0.7411
0.7432
0.0528
22.9694
NEAT
0.6328
0.8067
0.7284
0.7341
0.0502
23.3010
Table 8: Algorithm performance on FRIEDMAN
Method
CALIFORNIA
MIN
MAX
MEDIAN
MEAN
SD
SNR (dB)
CMA-ES
0.7121
0.7603
0.7291
0.7288
0.0184
31.9558
ES-MLP
0.6252
0.6936
0.6659
0.6656
0.0217
29.7351
DE
0.5572
0.7437
0.6511
0.6569
0.0495
22.4579
PSO
0.6068
0.7088
0.6643
0.6667
0.0355
25.4740
NEAT
0.4899
0.5962
0.5618
0.5488
0.0400
22.7471
Table 9: Algorithm performance on CALIFORNIA
Figure 3: Time complexity comparison for Evolutionary Frameworks quantified as CPU seconds
Figure 4: Memory Usage comparison for Evolutionary Frameworks
Neuroevolution is a representative neural architecture search paradigm that evolves both network topology and weights through evolutionary algorithms. In this paper, we propose Seq103, a unified NEAT-style neuroevolution framework for compact sequence architecture discovery. Seq103 consists of a shared evolutionary backbone and an optional recurrent extension. The shared backbone includes an elementary node-and-connection representation, per-class RMSE-based evaluation, mutation-based evolution with class-wise recombination, and elitism. The optional hidden-state mechanism extends the search space with hidden-state nodes and hidden connections, enabling temporal memory when step-wise recurrent inference is required. With this design, Seq103 applies the same core search pipeline to both step-wise recurrent and sample-wise feedforward sequence classification. In recurrent tasks, the hidden-state extension is enabled to provide temporal memory; in feedforward tasks, it is disabled while the shared evolutionary backbone remains unchanged. We evaluate Seq103 on 8 text classification datasets and the full UCRArchive2018 benchmark with 128 univariate time-series datasets. On step-wise tasks, Seq103 retains 86.96% of the best-baseline accuracy on average while using 34.6x to 3218.0x fewer parameters. On sample-wise tasks over the full UCRArchive2018 benchmark, Seq103 retains 81.95% of the best-baseline accuracy on average while using 11.8x to 160,601.0x fewer parameters.
Wenxiao Li, Yongjian Liu, Qing Xie
School of Computer Science and Artificial Intelligence, Wuhan University of Technology, No. 122 Luoshi Road, Hongshan District, Wuhan, 430070, Hubei, China
Spiking neural networks (SNNs) offer low-energy sequence modeling through sparse, event-driven computation. However, interactions among spike encoding, neuronal dynamics, and information propagation complicate architecture design. Existing SNN sequence models often adapt artificial neural network (ANN) architectures designed for real-valued activations, potentially underusing spike-based communication and temporal state updates, motivating automated discovery of native SNN architectures. Most evolutionary neural architecture search (ENAS) methods operate within predefined configuration spaces, limiting discovery to mechanisms expressible within those spaces. We introduce OpenArchEvo, which uses large language models (LLMs) to evolve executable architecture code in an open program space under spiking-projection constraints. In this space, code differences need not reflect architectural novelty, while direct performance evaluation requires costly training. We construct a three-view representation spanning code, design rationale, and a behavioral fingerprint to support novelty estimation and performance prediction. The search treats predicted performance and estimated novelty as two objectives, using surrogate predictions to select candidates for expensive training evaluations. With an estimated candidate-training cost of 132 V100 GPU-days, the search uncovers multiple native SNN architectures, exemplified by three designs featuring mechanisms such as spike-activity-dependent control of state updates and residual pathways. The discovered NeuroGate surpasses the ANN DeltaNet on WikiText-103, and the discovered architectures reduce estimated architecture-level arithmetic energy by up to 50.6x (LoopMem) relative to a common dense Transformer (ANN) baseline. Code and all discovered architectures: https://github.com/Optima-CityU/OpenArchEvo
Ruoyu Zhao, Jiaqi Wu, Chenyu Zhu +1
Department of Computer Science, City University of Hong Kong
AI-driven scientific discovery accelerates research by autonomously developing solutions and designs. Large language model (LLM) agents support this process through iterative generation and evaluation. Yet these iterations alone do not ensure cumulative progress or establish which directions to pursue next. Costly evaluation further constrains the scope of exploration. Neural architecture discovery brings these challenges together, coupling open-ended design with resource-intensive experimentation. We introduce EvoTreeNAD, a genealogy-guided evolutionary algorithm that constructs trainable architectures without a supplied seed or a hand-specified search space. Starting from an empty root, it grows a persistent genealogy in which each new node represents a complete architecture. Top-percentile values computed from each node and its descendants guide lineage selection. Using the selected design history, an Idea Agent proposes a variant and a Code Agent implements it. Each evaluated variant becomes a child node, expanding the genealogy while providing evidence for subsequent lineage selection. Our theoretical analysis establishes the existence of stationary variation regimes as the genealogy grows. Under specified variation assumptions, sustained top-percentile family values quantify the probability of generating high-reward architectures in these regimes. EvoTreeNAD discovers architectures that outperform the compared NAS and NAD baselines, achieving CIFAR-10/100 test errors of 2.05±0.06% and 15.09±0.22%. On all six MedMNIST-v2 tasks, the discovered architectures surpass the strongest listed baselines. A controlled CIFAR-10 study further shows that EvoTreeNAD outperforms direct generation, best-of-N greedy continuation, and full-family-mean routing.
Lishan Yu, Derek Jiu, Qizhen Lan +1
Department of Health Data Science and AI McWilliams School of Biomedical Informatics University of Texas Health Science Center at Houston