Code World Models for Parameter Control in Evolutionary Algorithms

Camilo Chacón Sartori; Guillem Rodríguez Corominas

cs.LGFeb 25, 2026

Code World Models for Parameter Control in Evolutionary Algorithms

Authors: Camilo Chacón Sartori, Guillem Rodríguez Corominas

Organizations: ICN2 – Catalan Institute of Nanoscience and Nanotechnology, Barcelona, Spain · Artificial Intelligence Research Institute (IIIA-CSIC), Barcelona, Spain · Universitat Politècnica de Catalunya (UPC), Barcelona, Spain

Abstract

Can an LLM learn how an optimizer behaves -- and use that knowledge to control it? We extend Code World Models (CWMs), LLM-synthesized Python programs that predict environment dynamics, from deterministic games to stochastic combinatorial optimization. Given suboptimal trajectories of $(1{+}1)$ - $\text{RLS}_k$ , the LLM synthesizes a simulator of the optimizer's dynamics; greedy planning over this simulator then selects the mutation strength $k$ at each step. On \lo{} and \onemax{}, CWM-greedy performs within 6% of the theoretically optimal policy -- without ever seeing optimal-policy trajectories. On \jump{ $_k$ }, where a deceptive valley causes all adaptive baselines to fail (0% success rate), CWM-greedy achieves 100% success rate -- without any collection policy using oracle knowledge of the gap parameter. On the NK-Landscape, where no closed-form model exists, CWM-greedy outperforms all baselines across fifteen independently generated instances ( $36.94$ vs.\ $36.32$ ; $p<0.001$ ) when the prompt includes empirical transition statistics. The CWM also outperforms DQN in sample efficiency (200 offline trajectories vs.\ 500 online episodes), success rate (100% vs.\ 58%), and generalization ( $k{=}3$ : 78% vs.\ 0%). Robustness experiments confirm stable synthesis across 5 independent runs.

Figures & tables

Panel of Figure 1 — Figure 1: Overview of the CWM approach. Offline : trajectory data and a problem description are assembled into a prompt; an LLM synthesizes a Python world model in a single API call ( ${\sim}$ € 0.04). Online : the greedy planner queries the CWM at each step to select $k^{*}$ . Bottom : key results across four benchmarks.

Aspect	Original CWM	This work
Environments	18 Gymnasium games	4 EA benchmarks
Dynamics	Deterministic	Stochastic combinatorial
LLM input	Env. docs + trajectories	Math / empirical stats + traj.
Synthesis	GIF-MCTS (iterative)	Single-shot + refinement
Planning	MCTS (5,000 rollouts)	Greedy 1-step lookahead
Baselines	Model-free RL	Adaptive rules & DQN

Table 1: Original CWM framework [ 3 ] vs. this work.

	LeadingOnes	OneMax	Jump k	NK
Landscape	Unimodal	Unimodal	Deceptive valley	Rugged / epistatic
Optimal $k^{*}$	Known (closed)	Known (closed)	Unknown	Unknown
Key challenge	Smooth $k$ ramp	Sharp policy cliff	Valley crossing	No math model
CWM input	Math description	Math description	Math + empirical table	Empirical table
Budget	$0.8n^{2}{=}2{,}000$	$5n\ln n{=}978$	$10{,}000$	$10{,}000$
What CWM learns	$k\downarrow$ as $i\uparrow$	Cliff at $n/2$	$k\uparrow$ at edge	Transition lookup

Table 2: Benchmark progression: each problem adds a new challenge for CWM-based parameter control.

Benchmark	Policy	Mean	Std	vs opt.
LeadingOnes	optimal	983	259	1.00 $\times$
	CWM-greedy	1,045	248	1.06 $\times$
	fifth_rule	1,256	299	1.28 $\times$
	CWM-MCTS	1,273	316	1.29 $\times$
	RLS_1	1,273	316	1.29 $\times$
OneMax	optimal	186	67	1.00 $\times$

Table 3: Unimodal benchmark results (100 runs each). LeadingOnes :

n{=}50

, budget

{=}2{,}000

. OneMax :

n{=}50

, budget

{=}978

Figure 2: LeadingOnes : CWM score heatmap over (fitness $i$ , parameter $k$ ). Black stars ( $\star$ ) mark the optimal $k^{*}(i)=\lfloor n/(i+1)\rfloor$ , snapped to the nearest column when $k^{*}$ is not in the displayed set. Stars appear only at $k$ values that are optimal for at least one fitness level; columns without stars (e.g. $k{=}20,25,30,40$ ) are never greedy-optimal for any $i$ .

Policy	Mean	Std	Median	SR%
greedy-opt	1,346	1,311	940	100%
CWM-greedy	1,342	1,328	1,003	100%
stagnation	1,506	1,427	1,194	99%
RLS_jump_k	5,226	4,176	2,820	58%
$\text{EA}_{\alpha}(2,0.5)$	10,000	0	10,000	0%
$\text{EA}_{\alpha}(1.3,0.75)$	10,000	0	10,000	0%

Table 4: Jump k results (

n=50

k=2

, budget

{}=10{,}000

, 100 runs).

Figure 3: Jump k ( $k{=}2$ ): CWM score heatmap over (fitness, parameter $k$ ). Black stars ( $\star$ ) mark the greedy-optimal $k^{*}(i)$ , snapped to the nearest displayed column. At the valley edge (fitness $=n$ ), the CWM correctly predicts that only $k{=}2$ leads to improvement—opposite to all adaptive baselines, which decrease $k$ during stagnation.

Policy	Mean	SR%	Data
optimal	1,346	100%	—
CWM-greedy	1,342	100%	200 offline traj.
stagnation	1,506	99%	— (heuristic)
DQN (500 ep.)	5,210	58%	500 online ep.
DQN (200 ep.)	10,000	0%	200 online ep.

Table 5: CWM vs. DQN on Jump k (

n=50

k=2

, 100 eval. runs).

Panel of Figure 4 — Figure 4: Left: CWM vs. DQN on Jump k : 100% success rate vs. 58%. Right: DQN learning curves ( $k{=}2$ ). Training beyond 500 episodes degrades performance—success rate plateaus at ${\sim}50\%$ while steps decrease only marginally, revealing overfitting to the $\epsilon$ -greedy exploration policy.

Policy	Mean best	Std	Max	Mean rank
CWM-greedy	36.94	1.03	39.12	1.0
static_5	36.32	1.04	38.96	2.5
random_k	36.31	1.00	39.06	2.5
fifth_rule	35.36	0.99	38.91	4.5
stagnation	35.32	0.93	38.91	5.1
self_adjusting	35.22	0.95	39.06	6.1

Table 6: NK-Landscape results (

n=50

K=2

, budget

{}=10{,}000

, 100 runs

\times

15 instances). Std: across per-instance means; Max: best single run. CWM-greedy ranks first on every instance (paired

t

-test

p<0.001

Figure 6: NK CWM score heatmap ( $K{=}2$ ). The scores are instance-independent (determined by the CWM alone); the model is evaluated on fifteen landscape instances in Table 6 . No optimal overlay exists—the optimal policy is unknown. The CWM prefers small $k$ at high fitness.

Problem	$n$	Optimal	CWM	RLS 1	CWM/Opt
LeadingOnes	50	983	1,052	1,220	1.07 $\times$
	100	3,705	4,167	4,923	1.12 $\times$
	200	15,761	16,156	20,210	1.02 $\times$
OneMax	50	185	188	192	1.02 $\times$
	100	441	452	431	1.02 $\times$
	200	1,031	1,020	1,037	0.99 $\times$

Table 7: Generalization across problem sizes. CWM trained on

n{=}50

, 50 eval. runs.

	CWM		DQN		$\text{EA}_{\alpha}$
$k$	Mean	SR	Mean	SR	Mean	SR
2	1,259	100%	5,023	58%	10,000	0%
3	21,377	78%	50,000	0%	50,000	0%
4	50,000	0%	50,000	0%	50,000	0%

Table 8: Generalization across Jump k values. CWM and DQN (best: 500 training episodes) trained on

k{=}2

, 50 eval. runs.

Figure 7: Jump k generalization: CWM 78% on $k{=}3$ vs. DQN 0%.

Benchmark	$\tau$	$\rho$	Exact	Note
LeadingOnes	0.784	0.916	60%	—
OneMax	0.614	0.939	82%	—
Jump k	0.898	0.948	—	valley-edge $\tau{=}1.0$
NK	0.553	—	—	with empirical table

Table 9: CWM quality metrics.

\tau

: Kendall’s rank correlation between CWM-predicted and true action rankings.

\rho

: Spearman correlation. Exact: fraction of states where CWM selects the same

k^{*}

as the oracle.

Code World Models

See all 15 papers

A Hybrid Nested Harness for Decoupling Structure and Parameters in LLM-Driven Optimization
Aug 8, 2026Víctor GallegoBlack-Box Optimization Large Language Model-Guided Optimization
Komorebi AI Technologies
SeaEvo: Advancing Algorithm Discovery with Strategy Space Evolution
Apr 27, 2026Sichun Luo, Yi Huang, Haochen Luo +7Automated Algorithm Discovery Evolutionary Optimization
1The University of Hong Kong · 2JIUTIAN Research, China Mobile · 3City University of Hong Kong
Relay, Don't Route: Adaptive Population Handoff for Cost-Efficient LLM-Driven Evolution
Aug 6, 2026Sichun Luo, Yi Huang, Guanzhi Deng +6Multi-Fidelity Optimization Evolutionary Optimization
1The University of Hong Kong · 2JIUTIAN Research, China Mobile · 3City University of Hong Kong +1

Aug 8, 2026J/K move · Enter open · S save

A Hybrid Nested Harness for Decoupling Structure and Parameters in LLM-Driven Optimization

Víctor Gallego

Komorebi AI Technologies

Black-Box Optimization Large Language Model-Guided Optimization LLMs for Evolutionary Optimization

Open paper AskPDF arXiv

In evolutionary algorithms powered by language models, the LLM acts as a single operator that simultaneously updates structural components (like control flow) and continuous parameters. While LLMs can be good at the first, they are not efficient at the second, wasting tokens taking discrete jumps inside a trial and error loop. We resolve this by formalizing a hybrid nested search, in which an outer loop has the LLM propose a structural sketch, with numeric gaps, and an inner numerical optimizer tunes the sketch. Both the outer and inner solvers are pluggable: any text-based optimizer can be combined with a zero-order optimizer (CMA-ES), gradient-based routines, or MCMC samplers. We validate our framework across three scientific domains: (i) meta-optimizers on closed-form test functions, (ii) code-based policies for systems research and social dilemmas; and (iii) approximate Bayesian inference tasks. Across all three, the hybrid optimizer is superior to both vanilla LLM-driven search and pure numerical optimization baselines. Code at: https://github.com/vicgalle/hybrid-nested-search

Abstract

Code World Models

Explore similar work

A Hybrid Nested Harness for Decoupling Structure and Parameters in LLM-Driven Optimization

SeaEvo: Advancing Algorithm Discovery with Strategy Space Evolution

Relay, Don't Route: Adaptive Population Handoff for Cost-Efficient LLM-Driven Evolution