Organizations: State Key Laboratory of Novel Software Technology, Nanjing University, China · School of Artificial Intelligence, Nanjing University, China · Huawei Noah’s Ark Lab, China
Large language models are increasingly participating in complex real-world tasks in the form of algorithm-design agents, designing and refining algorithms. Many successful algorithm-design agents adopt pure in-context evolutionary frameworks, but they may quickly plateau in domains that require specialized knowledge. Parametric adaptation offers a way to internalize specialized knowledge, but conventional training requires abundant domain-specific corpora while high-quality algorithms are scarce in complex algorithm-design scenarios. In this paper, we propose sample-efficient parametric self-evolution where agents can explore and learn from self-generated algorithms. First, we characterize in-context evolutionary stagnation and analytically propose the Improvement Chain proposition, showing how learning successive self-generated algorithms can locally increase the likelihood of neighboring algorithms. Motivated by this local-transfer perspective, we further propose Population-Curated Policy Optimization (PCPO) to utilize a global population and a hybrid policy update scheme for retaining and reusing high-quality, diverse self-generated algorithms, shifting the policy towards stronger algorithms. In the task of learning rate schedule design for global placement in electronic design automation, trained only on 4 chip cases, PCPO outperforms the state-of-the-art in-context evolutionary methods (e.g., OpenEvolve and ShinkaEvolve) on average across 16 chip cases. With an 8B-size base model, PCPO achieves competitive performance compared to frontier closed-source models such as GPT-5.5. PCPO also reduces inference-time token cost by internalizing grounded domain knowledge and prompt distillation. Moreover, PCPO achieves significant speedups on four GPU kernel designs, with an average of 8.27× speedup against the PyTorch Eager baseline.
Figures & tables
Figure 1: Average online improvement on 4 chips against total training and inference cost. PCPO achieves the largest improvement within acceptable computational cost, and the trained model can generalize to other chip cases with minimal inference-time token cost as shown in Section 6.1 . Frozen-model methods underperform the base model due to the stagnation described in Section 4.1 and incur substantial computational cost due to repeated generation refinement and long thinking.
Figure 2: Diagnostic experiments based on Qwen3-8B.
Figure 3: An illustration of the ideal Improvement Chain where each successor in the chain gradually approaches A∗ . The optimal algorithm is approached by progressively increasing the reachability of successive chain nodes, generating and learning on chain nodes one after another.
Figure 4: The workflow of PCPO. The left part illustrates the Global Population, filtering promising candidates while avoiding incompatible reward scales and difficulties. The right part illustrates the Hybrid Policy Update scheme, supporting sustained improvement, exploration, and rapid correction.
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Setting
Symbol / name
Value
Reward and legality
Illegal / crash / missing API
—
−109
Not achieving target overflow
—
−1000
Valid placement
r
DREAMPlace get_fitness (higher better)
Illegal threshold (crash band)
rinv
−108
PCPO
Appendix
Table 4: Hyperparameters of PCPO with Qwen3-8B on Global Placement. Every T on-policy steps, we additionally run an off-policy update on the global-population top- k programs ranked by composite score S=αr~+βd , where r~ is min–max normalized reward and d is semantic diversity. Near-duplicates with diversity ≤δmax are merged unless the fitness gain is at least γmin .
Figure 5: A representative OpenEvolve thinking trace on bigblue1 versus a strong direct generation based on Qwen3-8B. Panel (a) is lightly abridged from the model’s <think> block; highlighted claims H1–H3 are factually false or internally inconsistent. Panel (b) is the program emitted from that trace: extra multiplicative decay on learning_rate_prev while overflow exceeds 0.07 compounds into ηt=η0(0.995⋅0.95)t , the optimizer stalls, and overflow never reaches the 0.07 target. Panel (c) reconstructs the expert schedule η0⋅0.995t and applies non-compounding state-dependent modulators; HPWL improving ( log_hpwl < log_hpwl_prev ) shrinks the step, the opposite of (b). Ellipses in (a) mark omitted sentences; wording is otherwise verbatim.
ID
Type
Typical claim in <think> vs. the working policy
H1
Causal inversion
Overflow remains ≈0.65⇒ “LR is not small enough”; extra ×0.99/0.95/0.9 on learning_rate_prev . Direct: non-compounding soft scale on η0⋅0.995t .
H2
Semantic inversion
“If log_hpwl is decreasing (meaning hpwl is getting worse)”. log is monotone: a drop is an improvement .
H3
Phantom input
Invents log_gradient_norm_prev , which is not in the API.
Treats −88.90 / −89.26 as “better than baseline” −88.56 (higher is better).
Appendix
Table 5: Recurring errors in OpenEvolve chain-of-thought on bigblue1 . “Direct” denotes the best sample produced by the base model directly from the same task (Fig. 5 c).
Figure 6: A second OpenEvolve trace on the legal island. H5 inverts fitness polarity ( −89.26 is worse than baseline −88.56 ). H4 compares log∥∇∥ to logHPWL , two unrelated quantities, then treats the predicate as “gradient is decreasing.” The model later notes that the previous gradient is unavailable, yet still emits the comparison.
Figure 7: Joint log-probability assigned by Qwen3-8B to five global placement learning-rate policies under a shared prompt, before and after one training update step on the anchor program. The orange bar is the log-probability of the anchor produced by PCPO; blue bars are the log-probabilities of short-edit neighbors with respect to the anchor; bars with lighter colors show the log-probability after one training update step. Token-level edit distance from the anchor is annotated on each bar.
Figure 8: Detailed modifications and improvement of the global placement learning rate schedule algorithm evolution made by PCPO on adaptec1.
Method
adaptec1
bigblue1
superblue1
superblue7
Avg rank
OpenEvolve
70.99
87.53
389.18
554.48
4.25
ShinkaEvolve
71.10
88.29
389.18
551.10
4.63
JitRL
70.99
88.36
387.61
556.13
4.38
ThetaEvolve
70.68
87.83
388.79
562.56
4.25
GRPO
70.49
87.51
388.09
551.60
2.50
PCPO
70.04
87.23
385.64
547.11
1.00
Appendix
Table 6: Average inference-time 64-rollout best HPWL ( ×106 ) of 4 independent runs on four global placement cases. Lower is better.
Method
adaptec1
bigblue1
superblue1
superblue7
Average
PCPO
70.04
87.23
385.64
547.11
–
Qwen3-8B + Skill
70.94
87.58
388.76
553.16
–
Relative gap (%)
+1.29
+0.40
+0.81
+1.11
+0.90
Appendix
Table 7: Average inference-time 64-rollout best HPWL ( ×106 ) of 4 independent runs produced by PCPO versus Qwen3-8B equipped with a generalizable skill distilled by GPT-6-Astra from PCPO’s training logs. Lower is better.
Figure 9: Ablation study on the training performance of the Regularized Off-policy Update and the Refreshing On-policy Update in the Hybrid Policy Update module of PCPO.
Figure 10: Best-so-far normalized reward along the training trajectories of PCPO and GRPO on 4 training cases.
Figure 11: Inference-time token cost statistics of PCPO and two evolutionary methods.
Figure 12: Diagnostic experiments based on Gemma4-E4B.
Large Language Model (LLM)-based automated algorithm design typically evolves algorithms as complete, indivisible programs. While this whole-program perspective simplifies the search space, it fundamentally couples the useful local logic to its host program. Consequently, valuable code snippets vanish when the overall program is discarded, making it highly difficult to assess the contribution of individual algorithmic components.To address this, we propose Primitive-Aware Code Evolution (PACE), which decouples local logic from complete programs by representing it as persistent units called Executable Algorithmic Primitives (EAPs). To enable code-level transfer, PACE maintains a dynamic set of EAPs. Algorithm evolution is driven by primitive-aware operators that structurally guarantee the retention and cross-program transfer of these components. To evaluate them effectively, PACE leverages Thompson sampling based on parent-relative performance improvements, guiding primitive selection from the set without requiring extra evaluation datasets. Experiments on four tasks demonstrate that PACE effectively discovers competitive algorithms while structurally preserving valuable algorithmic components.
Zhuoliang Xie, Ruihao Zheng, Xiang Xu +2
1Southern University of Science and Technology · 2Shenzhen University
Large language models have advanced automated algorithm discovery by synthesizing executable code, but existing frameworks trap them in rigid search pipelines with pre-defined control flows. This limitation restricts adaptive reasoning, blocks cross-paradigm transfer, and overlooks richer execution feedback. To bridge this gap, we introduce an end-to-end framework, AlgoEvo, a unified agentic architecture that transforms automated algorithm discovery into an interactive, knowledge-accumulating process. An autonomous agent dynamically inspects, diagnoses, and edits code based on runtime feedback. A design skill hub decouples paradigm-specific knowledge from the core discovery engine, allowing a unified workflow to seamlessly handle single-heuristic, multi-objective, and multi-component design. Meanwhile, a hierarchical experience bank organizes search trajectories into a task-level tree to guide exploration and consolidates cross-task patterns into reusable skills. Across six representative benchmark tasks, AlgoEvo reaches state-of-the-art performance with as little as 7% of the evaluation budget and reduced token consumption, demonstrating strong intra-task accumulation, cross-task transfer, and the ability to reproduce or exceed the strongest existing methods through flexible skill activation.
Junhao Qiu, Qinglong Hu, Ji Cheng +3
Department of Computer Science, City University of Hong Kong · Huawei Noah’s Ark Lab · Institute of Advanced Intelligence and Computing, A*STAR
Meta-Black-Box Optimization (MetaBBO) is one of the highlights in the recent AI for Optimization trend. This paradigm's bi-level workflow leverages the learnable algorithm design policy at meta level to ensure the performance and generalization improvement on the low-level optimization task. While MetaBBO helps advance the performance lower bound of the resulted optimization system, it is currently handcrafted and customized case by case to adapt different optimization problems, which inevitably introduces inherent subjectivity and hence restricts the performance upper bound and usability in practice. In this paper, we address this issue by regarding MetaBBO's design loop as coding task, where we could introduce openendedness into MetaBBO with recursive self-improvement capability of advanced coding agents. Specifically, we propose a dual-agent framework: i) a task agent continuously refines the codebase of a target MetaBBO approach through code evolution; ii) a hyper agent progressively modifies the task agent and itself to provide open-ended design behavior; iii) the evolved MetaBBO codebase is evaluated and all in-execution information is fed back to the agents for recursive self-referential improvement. As a result, given a naive MetaBBO template, our framework automates a design evolution and finds novel variants superior to up-to-date human-made MetaBBO baselines. Surprisingly, the experimental results also demonstrate that our framework supports fast adaption across different optimization domains. Solid interpretation analysis further reveals interesting design principles emerge in such open-ended process. This work serves as the first exploration on automating design of complex learning-assisted optimization algorithms.
Zipei Yu, Yue-Jiao Gong, Zeyuan Ma +2
South China University of Technology · South China Normal University · Singapore Management Univeristy