Relation Reasoning with LLMs in Expensive Optimization

Large language models (LLMs) are increasingly integrated with evolutionary computation to support optimization tasks. This survey primarily focuses on evolutionary optimization, i.e., optimization based on evolutionary computation. For brevity, we use the term optimization throughout to denote this scope. However, existing surveys typically examine isolated roles of LLMs and do not provide a unified view that connects optimization modeling with optimization solving. To address this gap, we systematically review recent developments through a workflow-oriented framework. First, we organize the literature into two primary stages: LLMs for optimization modeling and LLMs for optimization solving (in this survey, the terms optimization modeling and optimization solving are used as concise forms of optimization problem modeling and optimization problem solving, respectively). Second, we divide the solving stage into three paradigms according to the role of the LLM: stand-alone optimizers, low-level components embedded in optimization algorithms, and high-level managers for algorithm selection and generation. Third, we analyze representative methods, identify their technical limitations, and clarify their relationships with traditional optimization approaches. We further substantiate this taxonomy through benchmark systematization, baseline comparisons, and practitioner-oriented guidance, and we review interdisciplinary applications across the natural sciences, engineering, and machine learning. Based on the resulting analysis, we identify research directions toward dynamic, self-evolving, and agentic optimization ecosystems. An up-to-date collection of related literature is maintained at https://github.com/ishmael233/LLM4OPT.

Large language model (LLM)-driven evolution has shown promise for program search and algorithm discovery, but relying on strong models throughout long evolutionary runs is costly. A natural alternative is to combine cheap and strong models under a fixed inference budget. However, existing approaches typically allocate models at the level of individual queries or mutation steps, overlooking that evolutionary search is \textit{stateful}: each generated candidate changes the population from which subsequent mutations are produced. We empirically analyze LLM-driven evolutionary trajectories and find that search progress is strongly front-loaded, early trajectory performance is informative but noisy, and cheap models recover much of the early progress achieved by strong models at lower cost. Motivated by these findings, we propose \textbf{\model}, a training-free framework that shifts budget allocation from individual calls to evolving populations through adaptive \textit{population handoff}. A cheap model explores multiple trajectories in short blocks allocated by a bandit scheduler. Relay Gain, defined as the marginal improvement of a compact, quality-diverse candidate bank constructed for handoff, serves as the scheduler reward and determines when to hand off. The curated candidates initialize a shared strong model population for refinement. Across four benchmarks and three budgets, \model achieves the highest mean score in 11 of 12 settings, outperforming competitive baselines. Our results suggest that in stateful search, budget allocation should be organized around the population, not the individual call.

Evolutionary optimization of LLM prompts and agentic programs (e.g., GEPA) is dominated by fitness evaluation: scoring each candidate runs an answering LLM over a validation set, so the evaluator's price tier dictates total search cost. We restructure that search by decoupling the three roles an LLM plays, running the high-volume answering role on the cheapest tier, reserving a strong model for the rare reflection/variation operator, then exploiting upward cross-tier transfer to deploy the cheaply evolved prompt on a stronger target. We contribute a cost-controlled characterization of when cheap-tier search substitutes for target-tier search, and where it fails. Across four tasks (HotpotQA, IFBench, LiveBench-Math, HoVer) and eleven models in four model families, the resulting prompt matches or exceeds same-tier optimization while placing over 96% of search tokens on the cheapest tier, at 5.6-14x lower search cost, rising to 25-54x where reasoning tiers emit long chains of thought on every fitness call.