cs.LGDec 23, 2024

EcoSearch: A Constant-Delay Best-First Search Algorithm for Program Synthesis

Authors: Théo Matricon, Nathanaël Fijalkow, Guillaume Lagarde

Organizations: CNRS, LaBRI and Université de Bordeaux, France · Université de Bordeaux, France

Abstract

Many approaches to program synthesis perform a combinatorial search within a large space of programs to find one that satisfies a given specification. To tame the search space blowup, previous works introduced probabilistic and neural approaches to guide this combinatorial search by inducing heuristic cost functions. Best-first search algorithms ensure to search in the exact order induced by the cost function, significantly reducing the portion of the program space to be explored. A key limitation of existing best-first search algorithms is that they slow down over time: the delay between outputting successive programs grows with the number of programs already generated. We ask whether best-first search algorithms with constant delay exist. We design Eco Search, a new best-first bottom-up search algorithm. It combines the cost tuple representation of Bee Search with the per non-terminal data structures of Heap Search, thanks to a new frugal expansion strategy, and it uses bucket queues instead of priority queues, which is possible because the costs stored in each queue always lie in a window of constant width. We prove that Eco Search is the first constant-delay best-first search algorithm for pre-generation cost functions. Experimentally, we evaluate Eco Search on two classic program-synthesis benchmarks (DeepCoder and FlashFill) and observe that Eco Search solves 12% to 70% more tasks than its predecessors, and reaches the number of tasks they solve 1.6 to 3.2 times faster. This shows that constant delay can be achieved for best-first search in program synthesis, and that it makes a difference in practice.

Figures & tables

Explore similar work

Jun 11, 2026

The program is still there, and searching for it obeys a conservation law

The length of the shortest program that generates a sequence, its Kolmogorov complexity, is uncomputable, so we ask what finding a short program costs when one exists. Many search algorithms, including Levin search, evolutionary methods, simulated annealing and the cross-entropy method, learn about a candidate program only through its score, how much of the sequence it reproduces. For such algorithms we define the coupling width of a search problem, the largest number of program fields that must change together before the score moves. We prove an unconditional worst-case lower bound on the number of candidates scored, exponential in that width. Under stated conditions, a conservation law follows, in which each bit of structural knowledge injected into a search removes at most one bit of search. The structure plus the logarithm of the remaining search is at least a fixed fraction of the sequence's Kolmogorov complexity. That fraction tends to one as the instruction set grows, so the lower bound approaches Levin's 1973 upper bound, apart from the program's running time. Search by score must pay this cost to be complete. We prove that our engine's structural inference avoids it only by giving up coverage, as does, if one-way functions exist, every method that runs in polynomial time. Our deterministic engine, built on this theory, recovers a generating program, certified by compressing its data and predicting an unseen continuation, for 2,383 of 3,914 candidate sequences in four populations of integer sequences and cellular automata, including the chaotic Rule 30.
May 19, 2026

optimize_anything: A Universal API for Optimizing any Text Parameter

Can a single LLM-based optimization system match specialized tools across fundamentally different domains? We show that when optimization problems are formulated as improving a text artifact evaluated by a scoring function, a single AI-based optimization system-supporting single-task search, multi-task search with cross-problem transfer, and generalization to unseen inputs-achieves state-of-the-art results across six diverse tasks. Our system discovers agent architectures that nearly triple Gemini Flash's ARC-AGI accuracy (32.5% to 89.5%), finds scheduling algorithms that cut cloud costs by 40%, generates CUDA kernels where 87% match or beat PyTorch, and outperforms AlphaEvolve's reported circle packing solution (n=26). Ablations across three domains reveal that actionable side information yields faster convergence and substantially higher final scores than score-only feedback, and that multi-task search outperforms independent optimization given equivalent per-problem budget through cross-task transfer, with benefits scaling with the number of related tasks. Together, we show for the first time that text optimization with LLM-based search is a general-purpose problem-solving paradigm, unifying tasks traditionally requiring domain-specific algorithms under a single framework. We open-source optimize_anything with support for multiple backends as part of the GEPA project at https://github.com/gepa-ai/gepa .
Oct 18, 2026

PRECOCIAL: Pretraining with Code Curriculums via Iterative Search for Accelerated Learning

Fully synthetic pretraining data generated by symbolic programs can bootstrap downstream training on naturalistic tasks, such as language modeling, but the generating programs have so far been designed by hand. We introduce PRECOCIAL, a method for automatically optimizing programs that generate synthetic pretraining data. Starting from an initial generator program, an LLM iteratively rewrites the program, and each rewrite is scored by pretraining a randomly-initialized transformer on its data and fine-tuning it on the downstream task. Because every score requires training, PRECOCIAL keeps a single network for the entire search, continuing its pretraining for each accepted program instead of training a new network per candidate, making it significantly more efficient. On four language and code datasets, at a fixed fine-tuning budget, optimized programs reach lower perplexity than hand-designed curricula, published procedural generators, random program mutation and the program the LLM writes in the first iteration of the search, and reach the same perplexity as the hand-designed curricula with 9-33% fewer fine-tuning steps. PRECOCIAL also ends with lower perplexity than FunSearch and AlphaEvolve, which train a new network for every candidate, at the same number of evaluations and a lower cost. We also observe that an optimized program generator from one domain transfers to datasets in other domains. Additionally, optimized programs allow for better downstream performance on four different reasoning and problem-solving tasks.