Automated Heuristic Design

Ask about this topic

Overview

Automated heuristic design studies how to create efficient rules that guide search and decision-making in problems such as vehicle routing, job scheduling, classical planning, and hardware design. Common approaches use genetic programming, evolutionary search, learned policies, or large language models to generate and refine executable heuristics using solver or simulator feedback. Current work increasingly combines these methods with memory, diverse heuristic portfolios, instance specialization, and transfer across tasks, while evaluating solution quality, runtime, and performance on held-out cases. A central challenge is balancing flexible search with reliable generalization, affordable evaluation, and guarantees such as execution safety or admissibility.

Latest papers

CardsList

84 papers15 papers in the last four weeks, up 200% on the four weeks before. 0.1% of all new papers.

Apr 4, 2026

TransGP: Task-Conditioned Transformer-Guided Genetic Programming for Multitask Dynamic Flexible Job Shop Scheduling

Hyper-heuristics have become a popular approach for solving dynamic flexible job shop scheduling (DFJSS) problems. They use gradient-free optimization techniques like Genetic Programming (GP) to evolve non-differentiable heuristics. However, conventional GP methods tend to converge slowly because they rely solely on evolutionary search to find good heuristics. Existing multitask GP methods can solve multiple tasks simultaneously and speed up the search by transferring knowledge across similar tasks. But they mostly exchange heuristic building blocks without truly generating heuristics conditioned on task information. In this paper, we aim to accelerate convergence and enable task-specific heuristic generation by incorporating a task-conditioned Transformer model. The Transformer works in two ways. First, it learns the distribution of elite heuristics, biasing the search toward promising regions of the heuristic space. Second, through conditional generation, it produces heuristics tailored to specific tasks, allowing the model to handle multiple scheduling tasks at once and improving overall optimization efficiency. Based on these ideas, we propose TransGP, a Task-Conditioned Transformer-Guided GP framework. This evolutionary paradigm integrates generative modeling with GP, enabling efficient multitask heuristic learning and knowledge transfer. We evaluate TransGP on a range of DFJSS scenarios. Experimental results show that TransGP consistently outperforms multitask GP baselines, widely used handcrafted heuristics, and the pure Transformer model, achieving faster convergence, superior solution quality, and enhanced robustness.

Meng Xu, Jiao Liu, Hua Yu +1

Job Shop SchedulingAutomated Heuristic DesignGenetic ProgrammingCombinatorial OptimizationMulti-Task Optimization
AskPDF
Apr 3, 2026

Breaking Validity-Induced Boundaries to Expand Algorithm Search Space: A Two-Stage AST-Based Operator for LLM-Driven Automated Heuristic Evolution

Large Language Model (LLM) based automated heuristic design (AHD) has shown great potential in discovering efficient heuristics. Most existing LLM-AHD frameworks use semantic evolutionary operators that rely entirely on the LLM's pre-trained knowledge. These one-stage methods strictly require the generated code to be valid during the operation and often rely on a ``thought-code'' representation. We argue that this end-to-end generation fundamentally limits the exploration ability within the algorithm search space. In this paper, we propose a two-stage, structure-based evolutionary operator for LLM-AHD. In the first stage, our approach directly performs crossover and mutation on the Abstract Syntax Trees (ASTs) of the heuristic code, intentionally generating diverse but often invalid structural variants. In the second stage, the LLM is employed to repair these invalid heuristics into executable, high-quality code. Depending on the underlying framework, either the raw invalid variants or the repaired heuristics are integrated into the population to preserve potential structural patterns. We demonstrate that the proposed operator can significantly enhance the search ability of state-of-the-art LLM-AHD algorithms, such as EoH-S. Experimental results on the Traveling Salesman Problem (TSP) and the Online Bin Packing Problem (OBP) show that our method effectively improves both optimization performance and convergence speed.

Sun Shengming, Shi Jialong

School of Mathematics and Statistics, Xi’an Jiaotong University, Xi’an, China

Automated Heuristic DesignLarge Language Model-Guided OptimizationGenetic ProgrammingCombinatorial Optimization
AskPDF
Mar 30, 2026

Transferable knowledge graphs with executable learned operators for algorithm design

Procedural knowledge in algorithm design is embedded in source code and rebuilt for each new domain. We introduce Generative Executable Algorithm Knowledge Graphs (GEAKG), a representation in which this knowledge is stored as a generative, executable, transferable graph: typed nodes hold validated operators, edges encode admissible compositions, and learned edge weights record effective sequences. The same engine instantiates the structure across domains by changing only a role ontology (RoleSchema) and a binding. We study GEAKG as a representation mechanism rather than a state-of-the-art optimizer, asking what transfers and when. Layer ablations localize transfer by granularity: within a neural-architecture-search family the learned snapshot transfers across 70 dataset pairs - its weights stay correlated across datasets and one frozen snapshot remains competitive with Regularized Evolution at zero deployment-token cost; across combinatorial domains only the ontology-constrained executable structure transfers, not the learned weights. That structure pays off where target-side search is expensive - a Traveling Salesman snapshot beats an equally untuned from-scratch search on large scheduling instances even at one-fifth its budget - but does not improve on an effective local search where one is cheap, as in assignment and linear ordering. Executable procedural knowledge can thus be acquired offline, compacted, inspected, and reused without runtime language-model calls.

Camilo Chacón Sartori, José H. García, Andrei Voicu Tomut +1

Catalan Institute of Nanoscience and Nanotechnology (ICN2), CSIC and BIST, Campus UAB, Bellaterra, 08193, Barcelona, Spain · Artificial Intelligence Research Institute (IIIA-CSIC), Bellaterra, Spain

Knowledge GraphsAutomated Algorithm DiscoveryAutomated Heuristic DesignKG ConstructionCombinatorial Optimization
AskPDF
Mar 29, 2026

DSevolve: Enabling Real-Time Adaptive Scheduling on Dynamic Flexible Job Shop with LLM-Evolved Heuristic Portfolios

In dynamic flexible job shops, order arrivals, machine breakdowns, and processing-time deviations continually reshape the scheduling state and the priority trade-offs behind dispatching decisions. Dispatching rules are well suited to this setting because they are fast, interpretable, and easy to deploy, and recent LLM-assisted automatic heuristic design further expands their expressiveness by evolving composite priority functions. The key challenge is to make these evolved rule behaviors state-adaptive without losing the rapid response needed for online rescheduling. This paper proposes a dynamic self-evolutionary framework DSevolve, which separates offline rule-library construction from online state-conditioned rule selection. Offline, an LLM-guided quality-diversity search combines multi-persona seeding, a MAP-Elites behavioral archive, and behavior-guided variation to evolve a library of complementary rules rather than a single elite, and event-level simulation then trains a neural selector to rank the rules by state. Online, after each rescheduling event, a neural selector maps a 22-dimensional state fingerprint to rule scores and dispatches the top-ranked rule within about a second, meeting the response-time requirement after each disruption. Experiments on dynamic instances derived from standard flexible job shop benchmarks show that DSevolve achieves lower mean makespan than individual LLM-evolved rules, classical dispatching rules, and learning-based baselines under a one-active-rule deployment protocol. Trained only on small instances, the selector transfers zero-shot to substantially larger dynamic shops. These results show that state-conditioned selection preserves the speed and interpretability of dispatching rules while improving adaptability.

XinLei Zhou, Jin Huang, Jie Yang +2

School of Mechanical Science and Engineering, Huazhong University of Science and Technology, Wuhan, 430074, PR China

Flexible Job Shop SchedulingReal-Time SchedulingCombinatorial OptimizationAutomated Heuristic DesignMetaheuristic Optimization
AskPDF
Mar 25, 2026

The DeepXube Software Package for Solving Pathfinding Problems with Learned Heuristic Functions and Search

DeepXube is a free and open-source Python package and command-line tool that seeks to automate the solution of pathfinding problems by using machine learning to learn heuristic functions that guide heuristic search algorithms tailored to deep neural networks (DNNs). DeepXube is comprised of the latest advances in deep reinforcement learning, heuristic search, and formal logic for solving pathfinding problems. This includes limited-horizon Bellman-based learning, hindsight experience replay, batched heuristic search, and specifying goals with answer-set programming. A robust multiple-inheritance structure simplifies the definition of pathfinding domains and the generation of training data. Training heuristic functions is made efficient through the automatic parallelization of the generation of training data across central processing units (CPUs) and reinforcement learning updates across graphics processing units (GPUs). Pathfinding algorithms that take advantage of the parallelism of GPUs and DNN architectures, such as batch weighted A* and Q* search and beam search are easily employed to solve pathfinding problems through command-line arguments. Finally, several convenient features for visualization, code profiling, and progress monitoring during training and solving are available. The GitHub repository is publicly available at https://github.com/forestagostinelli/deepxube.

Forest Agostinelli

Department of Computer Science and Engineering University of South Carolina

Automated Heuristic DesignGraph-Based PlanningRL for Combinatorial Optimization
AskPDF
Mar 21, 2026

Large Neighborhood Search meets Iterative Neural Constraint Heuristics

Neural networks are being increasingly used as heuristics for constraint satisfaction. These neural methods are often recurrent, learning to iteratively refine candidate assignments. In this work, we make explicit the connection between such iterative neural heuristics and Large Neighborhood Search (LNS), and adapt an existing neural constraint satisfaction method-ConsFormer-into an LNS procedure. We decompose the resulting neural LNS into two standard components: the destroy and repair operators. On the destroy side, we instantiate several classical heuristics and introduce novel prediction-guided operators that exploit the model's internal scores to select neighborhoods. On the repair side, we utilize ConsFormer as a neural repair operator and compare the original sampling-based decoder to a greedy decoder that selects the most likely assignments. Through an empirical study on Sudoku, Graph Coloring, and MaxCut, we find that adapting the neural heuristic to an LNS procedure yields substantial gains over its vanilla settings and improves its competitiveness with classical and neural baselines. We further observe consistent design patterns across tasks: stochastic destroy operators outperform greedy ones, while greedy repair is more effective than sampling-based repair for finding a single high-quality feasible assignment. These findings highlight LNS as a useful lens and design framework for structuring and improving iterative neural approaches.

Yudong W. Xu, Wenhao Li, Scott Sanner +1

Department of Mechanical & Industrial Engineering, University of Toronto · Vector Institute

Constraint SatisfactionLarge Neighborhood SearchAutomated Heuristic Design
AskPDF
Mar 20, 2026

From Language Models to Control Policies: Search, Selection, and Verification

Industrial process control demands policies that are interpretable and auditable, requirements that black-box neural policies struggle to meet. LLM-driven heuristic synthesis offers an alternative: a language model iteratively proposes and refines human-readable Python controllers using rich behavioral feedback from a simulator. We study this form of programmatic policy search on hot steel rolling, a physics-simulated, multi-pass control problem with competing objectives and hard equipment constraints, and address three questions that arise once the search is run at scale: how to split a fixed budget of LLM iterations between restarting and refining, how to deploy the resulting pool of heuristics, and how to verify it. Treating each search run as a randomized algorithm, we find across 52 independent runs that runs of moderate length match the hindsight-optimal budget allocation, and that Luby-style universal restarts reach this regime without problem-specific tuning. Because each heuristic is cheap to simulate, a small portfolio selected per job outperforms the best single heuristic, also under cross-validation and on out-of-distribution operating conditions on which the best heuristic stalls. Finally, an automated audit pipeline combining static analysis, Z3 proofs, and property-based testing proves key safety properties for all 174 synthesized heuristics; the stall, which neither the search reward nor these properties reveal, shows that the specifications must also cover progress.

Nima H. Siboni, Seyedreza Kiamousavi, Vahid Rostami +1

Juna.ai, Kastanienallee 32, 10435 Berlin · Institute of Metal Forming, RWTH Aachen, Intzestr. 10, 52072 Aachen

Automated Heuristic DesignLLM-Based Program SynthesisManufacturingFeedback Control
AskPDF
Mar 16, 2026

Evolutionary Transfer Learning for Dragonchess

Dragonchess, a three-dimensional chess variant introduced by Gary Gygax, presents unique strategic and computational challenges that make it an ideal environment for studying the transfer of artificial intelligence (AI) heuristics across domains. In this work, we introduce Dragonchess as a novel testbed for AI research and provide an open-source, Python-based game engine for community use. Our research investigates evolutionary transfer learning by adapting heuristic evaluation functions directly from Stockfish, a leading chess engine, and subsequently optimizing them using Covariance Matrix Adaptation Evolution Strategy (CMA-ES). Initial trials showed that direct heuristic transfers were inadequate due to Dragonchess's distinct multi-layer structure and movement rules. However, evolutionary optimization significantly improved AI agent performance, resulting in superior gameplay demonstrated through empirical evaluation in a 50-round Swiss-style tournament. This research establishes the effectiveness of evolutionary methods in adapting heuristic knowledge to structurally complex, previously unexplored game domains.

Jim O'Connor, Annika Hoag, Sarah Goyette +1

Computer Science Department, Connecticut College, New London, United States

Game-Playing AgentsTransfer LearningCross-Domain Transfer LearningAutomated Heuristic Design
AskPDF
Mar 16, 2026

Planning as Goal Recognition: Deriving Heuristics from Intention Models -- Extended Version

Classical planning aims to find a sequence of actions, a plan, that maps a starting state into one of the goal states. If a trajectory appears to be leading to the goal, should we prioritise exploring it? Seminal work in goal recognition (GR) has defined GR in terms of a classical planning problem, adopting classical solvers and heuristics to recognise plans. We come full circle, and study the adoption and properties of GR-derived heuristics for seeking solutions to classical planning problems. We propose a new divergence-based framework for assessing goal intention, which informs a new class of efficiently-computable heuristics. As a proof of concept, we derive two such heuristics, and show that they can already yield improvements for top-scoring classical planners. Our work provides foundational knowledge for understanding and deriving probabilistic intention-based heuristics for planning.

Giacomo Rosa, Jean Honorio, Nir Lipovetzky +1

The University of Melbourne · RMIT University

Symbolic PlanningAutomated Heuristic Design
AskPDF
Mar 9, 2026

Efficient Policy Learning with Hybrid Evaluation-Based Genetic Programming for Uncertain Agile Earth Observation Satellite Scheduling

The Uncertain Agile Earth Observation Satellite Scheduling Problem (UAEOSSP) is a novel combinatorial optimization problem and a practical engineering challenge that aligns with the current demands of space technology development. It incorporates uncertainties in profit, resource consumption, and visibility, which may render pre-planned schedules suboptimal or even infeasible. Genetic Programming Hyper-Heuristic (GPHH) shows promise for evolving interpretable scheduling policies; however, their simulation-based evaluation incurs high computational costs. Moreover, the design of the constructive method, denoted as Online Scheduling Algorithm (OSA), directly affects fitness assessment, resulting in evaluation-dependent local optima within the policy space. To address these issues, this paper proposes a Hybrid Evaluation-based Genetic Programming (HE-GP) for effectively solving UAEOSSP. A Hybrid Evaluation (HE) mechanism is integrated into the policy-driven OSA, combining exact and approximate filtering modes: exact mode ensures evaluation accuracy through elaborately designed constraint verification modules, while approximate mode reduces computational overhead via simplified logic. HE-GP dynamically switches between evaluation models based on real-time evolutionary state information. Experiments on 16 simulated instance sets demonstrate that HE-GP significantly outperforms handcrafted heuristics and single-evaluation based GPHH, achieving substantial reductions in computational cost while maintaining excellent scheduling performance across diverse scenarios. Specifically, the average training time of HE-GP was reduced by 17.77% compared to GP employing exclusively exact evaluation, while the optimal policy generated by HE-GP achieved the highest average ranks across all scenarios.

Junhua Xue, Yuning Chen, Mingyan Shao +3

Genetic ProgrammingAutomated Heuristic DesignDecision-Making under UncertaintyRemote Sensing
AskPDF
Mar 9, 2026

Advancing Automated Algorithm Design via Evolutionary Stagewise Design with LLMs

With the rapid advancement of human science and technology, problems in industrial scenarios are becoming increasingly challenging, bringing significant challenges to traditional algorithm design. Automated algorithm design with LLMs emerges as a promising solution, but the currently adopted black-box modeling deprives LLMs of any awareness of the intrinsic mechanism of the target problem, leading to hallucinated designs. In this paper, we introduce Evolutionary Stagewise Algorithm Design (EvoStage), a novel evolutionary paradigm that bridges the gap between the rigorous demands of industrial-scale algorithm design and the LLM-based algorithm design methods. Drawing inspiration from CoT, EvoStage decomposes the algorithm design process into sequential, manageable stages and integrates real-time intermediate feedback to iteratively refine algorithm design directions. To further reduce the algorithm design space and avoid falling into local optima, we introduce a multi-agent system and a "global-local perspective" mechanism. We apply EvoStage to the design of two types of common optimizers: designing parameter configuration schedules of the Adam optimizer for chip placement, and designing acquisition functions of Bayesian optimization for black-box optimization. Experimental results across open-source benchmarks demonstrate that EvoStage outperforms human-expert designs and existing LLM-based methods within only a couple of evolution steps, even achieving the historically state-of-the-art half-perimeter wire-length results on every tested chip case. Furthermore, when deployed on a commercial-grade 3D chip placement tool, EvoStage significantly surpasses the original performance metrics, achieving record-breaking efficiency. We hope EvoStage can significantly advance automated algorithm design in the real world, helping elevate human productivity.

Chen Lu, Ke Xue, Chengrui Gao +5

National Key Laboratory for Novel Software Technology, Nanjing University, China · School of Artificial Intelligence, Nanjing University, China · Huawei Noah’s Ark Lab, China

Automated Algorithm DiscoveryAutomated Heuristic DesignEvolutionary OptimizationElectronic Design AutomationBayesian Optimization
AskPDF
Mar 8, 2026

CDEoH: Category-Driven Automatic Algorithm Design With Large Language Models

With the rapid advancement of large language models (LLMs), LLM-based heuristic search methods have demonstrated strong capabilities in automated algorithm generation. However, their evolutionary processes often suffer from instability and premature convergence. Existing approaches mainly address this issue through prompt engineering or by jointly evolving thought and code, while largely overlooking the critical role of algorithmic category diversity in maintaining evolutionary stability. To this end, we propose Category Driven Automatic Algorithm Design with Large Language Models (CDEoH), which explicitly models algorithm categories and jointly balances performance and category diversity in population management, enabling parallel exploration across multiple algorithmic paradigms. Extensive experiments on representative combinatorial optimization problems across multiple scales demonstrate that CDEoH effectively mitigates convergence toward a single evolutionary direction, significantly enhancing evolutionary stability and achieving consistently superior average performance across tasks and scales.

Yu-Nian Wang, Shen-Huan Lyu, Ning Chen +3

Key Laboratory of Water Big Data Technology of Ministry of Water Resources, College of Computer Science and Software Engineering, Hohai University, Nanjing, China · Department of Computer Science, City University of Hong Kong, Hong Kong, China · State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China

Automated Heuristic DesignAutomated Algorithm DiscoveryLarge Language Model-Guided OptimizationEvolutionary OptimizationCombinatorial Optimization
AskPDF
Mar 7, 2026

DyACE: Dynamic Algorithm Co-evolution for Online Automated Heuristic Design with Large Language Model

The prevailing paradigm in Automated Heuristic Design (AHD) typically relies on the assumption that a single, fixed algorithm can effectively navigate the shifting dynamics of a combinatorial search. This static approach often proves inadequate for Perturbative Heuristics, where the optimal algorithm for escaping local optima depends heavily on the specific search phase. To address this limitation, we reformulate heuristic design as a Non-stationary Bi-level Control problem and introduce DyACE (Dynamic Algorithm Co-evolution). Distinct from standard open-loop solvers, DyACE use a Receding Horizon Control architecture to continuously co-evolve the heuristic logic alongside the solution population. A core element of this framework is the Look-Ahead Rollout Search, which queries the landscape geometry to extract Search Trajectory Features. This sensory feedback allows the Large Language Model (LLM) to function as a grounded meta-controller, prescribing phase-specific interventions tailored to the real-time search status. We validate DyACE on three representative combinatorial optimization benchmarks. The results demonstrate that our method significantly outperforms state-of-the-art static baselines, exhibiting superior scalability in high-dimensional search spaces. Furthermore, ablation studies confirm that dynamic adaptation fails without grounded perception, often performing worse than static algorithms. This indicates that DyACE's effectiveness stems from the causal alignment between the synthesized logic and the verified gradients of the optimization landscape.

Guidong Lu, Yiping Liu, Xiangxiang Zeng

Automated Heuristic DesignCombinatorial OptimizationLarge Language Model-Guided Optimization
AskPDF
Mar 7, 2026

Large Language Model-Driven Full-Component Evolution of Adaptive Large Neighborhood Search

Adaptive Large Neighborhood Search (ALNS) is a prominent metaheuristic and a widely adopted approach for production and logistics optimization. However, it has long relied on hand-crafted components built on expert experience, which makes development slow and costly to adapt to new problems. This paper proposes a closed-loop, large-language-model-driven evolutionary framework that decouples ALNS and automatically rebuilds all of its components. We break ALNS into seven key modules: destroy, repair, operator selection, weight update, initial solution construction, acceptance rule, and destroy-rate control, and evolve each module through a dedicated task. By incorporating the Multi-dimensional Archive of Phenotypic Elites mechanism, the framework maintains a multi-dimensional elite archive to simultaneously drive the evolution of solution quality and strategic diversity. In addition, we design multiple mechanisms, including parallel and sequential multi-module evolution as well as single-expert-driven and multi-expert-driven evolution, to systematically evaluate the impact of different evolutionary paradigms on algorithm generation performance. Evaluations on Traveling Salesman Problem and Capacitated Vehicle Routing Problem benchmarks demonstrate that evolved algorithms consistently outperform optimized classic ALNS baselines under both fixed-iteration and fixed-time limits. The framework also shows a degree of generalizability and cross-problem transferability. Code analysis also uncovers several counterintuitive yet meaningful design patterns that emerged naturally during evolution, offering practical and theoretical insights for future ALNS design. Finally, comparisons across multiple language models highlight clear differences in their ability to support evolutionary algorithm design, helping guide model selection for real-world engineering use.

Shaohua Yu, Tianyu Chen, Linyan Liu +1

School of Intelligent Science and Technology, Nanjing University of Science and Technology, Nanjing 210094, China · School of Mechanical Engineering, Nanjing University of Science and Technology, Nanjing 210094, China · Université Paris-Saclay, CentraleSupélec, Laboratoire Génie Industriel, 91190, Gif-sur-Yvette, France +1

Automated Heuristic DesignLarge Neighborhood SearchLarge Language Model-Guided OptimizationEvolutionary Optimization
AskPDF
Mar 6, 2026

Twitch: Learning Abstractions for Equational Theorem Proving

Several successful strategies in automated reasoning rely on human-supplied guidance about which term or clause shapes are interesting. In this paper we aim to discover interesting term shapes automatically. Specifically, we discover abstractions : term patterns that occur over and over again in relevant proofs. We present our tool Twitch which discovers abstractions with the help of Stitch, a tool originally developed for discovering reusable library functions in program synthesis tasks. Twitch can produce abstractions in two ways: (1) from a partial, failed proof of a conjecture; (2) from successful proofs of other theorems in the same domain. We have also extended Twee, an equational theorem prover, to use these abstractions. We evaluate Twitch on a set of unit equality (UEQ) problems from TPTP, and show that it can prove 12 rating-1 problems as well as yielding significant speed-ups on many other problems.

Guy Axelrod, Moa Johansson, Nicholas Smallbone

Chalmers University, of Technology, Gothenburg, Sweden · University of Gothenburg, Gothenburg, Sweden

Automated Theorem ProvingAutomated Heuristic Design
AskPDF
Mar 6, 2026

LUMINA: LLM-Guided GPU Architecture Exploration via Bottleneck Analysis

GPU design space exploration (DSE) for modern AI workloads, such as Large-Language Model (LLM) inference, is challenging because of GPUs' vast, multi-modal design spaces, high simulation costs, and complex design optimization objectives (e.g. performance, power and area trade-offs). Existing automated DSE methods are often prohibitively expensive, either requiring an excessive number of exploration samples or depending on intricate, manually crafted analyses of interdependent critical paths guided by human heuristics. We present LUMINA, an LLM-driven GPU architecture exploration framework that leverage AI to enhance the DSE efficiency and efficacy for GPUs. LUMINA extracts architectural knowledge from simulator code and performs sensitivity studies to automatically compose DSE rules,which are auto-corrected during exploration. A core component of LUMINA is a DSE Benchmark that comprehensively evaluates and enhances LLMs' capabilities across three fundamental skills required for architecture optimization, which provides a principled and reproducible basis for model selection and ensuring consistent architectural reasoning. In the design space with 4.7 million possible samples, LUMINA identifies 6 designs of better performance and area than an A100 GPU efficiently, using only 20 steps via LLM-assisted bottleneck analysis. In comparison, LUMINA achieves 17.5x higher than design space exploration efficiency, and 32.9% better designs (i.e. Pareto Hypervolume) than Machine-Learning baselines, showcasing its ability to deliver high-quality design guidance with minimal search cost.

Tao Zhang, Rui Ma, Shuotao Xu +2

Microsoft Research China

Automated Heuristic DesignLarge Language Model-Guided Optimization
AskPDF
Mar 5, 2026

ReVEL: Multi-Turn Reflective LLM-Guided Heuristic Evolution via Structured Performance Feedback

Designing effective heuristics for NP-hard combinatorial optimization problems remains challenging and often requires substantial domain expertise. Recent LLM-guided evolutionary methods have shown promise for automated heuristic generation, but most existing approaches refine heuristics independently or through limited pairwise feedback. We propose ReVEL: Multi-Turn Reflective LLM-Guided Heuristic Evolution via Structured Performance Feedback, a framework for group-wise multi-turn heuristic refinement. ReVEL organizes heuristics into behavior-aware reflective groups, including similarity-driven groups for localized refinement and diversity-driven groups for exploratory search. Within each group, the LLM performs iterative multi-turn refinement using accumulated performance feedback, enabling related heuristics to be jointly analyzed and progressively improved across evolutionary iterations. Experiments on standard combinatorial optimization benchmarks show that ReVEL generally improves optimization performance over existing LLM-guided evolutionary baselines across multiple settings and LLM backbones. Additional analyses suggest that behavior-aware grouping contributes to more consistent refinement trajectories during iterative heuristic evolution.

Cuong Van Duc, Minh Nguyen Dinh Tuan, Tam Vu Duc +4

VinMotion · Hanoi University of Science and Technology · VinUniversity +2

Automated Heuristic DesignLarge Language Model-Guided OptimizationEvolutionary OptimizationCombinatorial OptimizationLLM Self-Refinement
AskPDF
Mar 3, 2026

From Heuristic Selection to Automated Algorithm Design: LLMs Benefit from Strong Priors

Large Language Models (LLMs) have already been widely adopted for automated algorithm design, demonstrating strong abilities in generating and evolving algorithms across various fields. Existing work has largely focused on examining their effectiveness in solving specific problems, with search strategies primarily guided by adaptive prompt designs. In this paper, through investigating the token-wise attribution of the prompts to LLM-generated algorithmic codes, we show that providing high-quality algorithmic code examples can substantially improve the performance of the LLM-driven optimization. Building upon this insight, we propose leveraging prior benchmark algorithms to guide LLM-driven optimization and demonstrate superior performance on two black-box optimization benchmarks: the pseudo-Boolean optimization suite (pbo) and the black-box optimization suite (bbob). Our findings highlight the value of integrating benchmarking studies to enhance both efficiency and robustness of the LLM-driven black-box optimization methods.

Qi Huang, Furong Ye, Ananta Shahane +2

Leiden Institute of Advanced Computer Science (LIACS), Leiden University, 2333 CC Leiden, The Netherlands

Large Language Model-Guided OptimizationAutomated Algorithm DiscoveryAutomated Heuristic DesignBlack-Box Optimization
AskPDF
Feb 26, 2026

Enhancing CVRP Solver through LLM-driven Automatic Heuristic Design

The Capacitated Vehicle Routing Problem (CVRP), a fundamental combinatorial optimization challenge, focuses on optimizing fleet operations under vehicle capacity constraints. While extensively studied in operational research, the NP-hard nature of CVRP continues to pose significant computational challenges, particularly for large-scale instances. This study presents AILS-AHD (Adaptive Iterated Local Search with Automatic Heuristic Design), a novel approach that leverages Large Language Models (LLMs) to revolutionize CVRP solving. Our methodology integrates an evolutionary search framework with LLMs to dynamically generate and optimize ruin heuristics within the AILS method. Additionally, we introduce an LLM-based acceleration mechanism to enhance computational efficiency. Comprehensive experimental evaluations against state-of-the-art solvers, including AILS-II and HGS, demonstrate the superior performance of AILS-AHD across both moderate and large-scale instances. Notably, our approach establishes new best-known solutions for 8 out of 10 instances in the CVRPLib large-scale benchmark, underscoring the potential of LLM-driven heuristic design in advancing the field of vehicle routing optimization.

Zhuoliang Xie, Fei Liu, Zhenkun Wang +1

Southern University of Science and Technology · City University of Hong Kong

Vehicle RoutingAutomated Heuristic DesignMetaheuristic OptimizationLarge Language Model-Guided Optimization
AskPDF
Feb 25, 2026

Survey on Neural Routing Solvers

Neural routing solvers (NRSs) that leverage deep learning to tackle vehicle routing problems have demonstrated notable potential for practical applications. By learning implicit heuristic rules from data, NRSs replace the handcrafted counterparts in classic heuristic frameworks, thereby reducing reliance on costly manual design and trial-and-error adjustments. This survey makes two main contributions: (1) The heuristic nature of NRSs is highlighted, and existing NRSs are reviewed from the perspective of heuristics. A hierarchical taxonomy based on heuristic principles is further introduced. (2) A generalization-focused evaluation pipeline is proposed to address limitations of the conventional pipeline. Comparative benchmarking of representative NRSs across both pipelines uncovers a series of previously unreported gaps in current research.

Yunpeng Ba, Xi Lin, Changliang Zhou +7

Guangdong Provincial Key Laboratory of Fully Actuated System Control Theory and Technology, School of Automation and Intelligent Manufacturing, Southern University of Science and Technology, Shenzhen, China. · the School of Mathematics and Statistics, Xi’an Jiaotong University, Xi’an, China. · the Institute of Big Data Science and Industry, Shanxi University, Taiyuan, China. +1

Neural Combinatorial OptimizationVehicle RoutingAutomated Heuristic Design
AskPDF
Feb 17, 2026

Heuristic Search as Language-Guided Program Optimization

Large Language Models (LLMs) have advanced Automated Heuristic Design (AHD) in combinatorial optimization (CO) in the past few years. However, existing discovery pipelines often require extensive manual trial-and-error or reliance on domain expertise to adapt to new or complex problems. This stems from tightly coupled internal mechanisms that limit systematic improvement of the LLM-driven design process. To address this challenge, we propose a structured framework for LLM-driven AHD that explicitly decomposes the heuristic discovery process into modular stages: a forward pass for evaluation, a backward pass for analytical feedback, and an update step for program refinement. This separation provides a clear abstraction for iterative refinement and enables principled improvements of individual components. We validate our framework across four diverse real-world CO domains, where it consistently outperforms baselines, achieving up to 0.170.17 improvement in QYI on unseen test sets. Finally, we show that several popular AHD methods are restricted instantiations of our framework. By integrating them in our structured pipeline, we can upgrade the components modularly and significantly improve their performance.

Mingxin Yu, Ruixiao Yang, Chuchu Fan

Massachusetts Institute of Technology, Cambridge, MA, USA

Automated Heuristic DesignLarge Language Model-Guided OptimizationCombinatorial OptimizationLLM-Based Program Synthesis
AskPDF
Dec 31, 2025

Vulcan: Instance-specialized, Verifiable Systems Heuristics Through LLM-driven Search

Systems resource management tasks rely primarily on hand-designed heuristics. However, growing hardware heterogeneity and workload diversity require heuristics specialized to particular deployment instances, making manual design expensive and difficult to scale. In this paper, we explore how to synthesize systems heuristics using LLMs. The main challenge is ensuring that generated heuristics execute safely, integrate correctly with the surrounding system, and still achieve strong performance. We propose Vulcan, a framework that identifies LLM-friendly interfaces that isolate core decision logic from the rest of the implementation. With Vulcan, LLM-generated code is restricted to simple stateless decision functions, while trusted runtime abstractions provide rich derived statistics for meaningful policy exploration without system-integration bugs. To ensure execution safety, LLMs synthesize heuristics in a restricted language, Anvil, that guarantees important properties by construction. We evaluate Vulcan across three well-studied domains and demonstrate up to 4.9×\times higher savings for spot-VM scheduling, up to 2×\times lower miss ratios for cache eviction, and up to 14% higher application performance for tiered-memory systems, while ensuring execution safety throughout.

Rohit Dwivedula, Divyanshu Saxena, Sujay Yadalam +3

The University of Texas at Austin Austin, TX, USA

Automated Heuristic DesignLLM-Based Program SynthesisCache Replacement
AskPDF
Oct 8, 2025

VRPAgent: LLM-Driven Discovery of Heuristic Operators for Vehicle Routing Problems

Designing high-performing heuristics for vehicle routing problems (VRPs) is a complex task that requires both intuition and deep domain knowledge. Large language model (LLM)-based code generation has recently shown promise across many domains, but it still falls short of producing heuristics that rival those crafted by human experts. In this paper, we propose VRPAgent, a framework for VRPs that integrates LLM-generated components into a metaheuristic and refines them through a novel genetic search. By using the LLM to generate problem-specific operators, embedded within a shared metaheuristic framework for VRPs, VRPAgent keeps tasks manageable, guarantees correctness, and still enables the discovery of novel and powerful strategies. Across multiple VRPs, including the capacitated VRP, the VRP with time windows, and the prize-collecting VRP, our method discovers heuristic operators that outperform handcrafted methods and recent learning-based approaches while requiring only a single CPU core. To our knowledge, VRPAgent is one of the first LLM-based methods to advance the state-of-the-art in VRPs, highlighting a promising future for automated heuristics discovery.

André Hottung, Federico Berto, Chuanbo Hua +9

Bielefeld University · Radical Numerics · KAIST +6

Vehicle RoutingAutomated Heuristic DesignLanguage Model-Guided Search HeuristicsMetaheuristic Optimization
AskPDF
Jan 30, 2025

Successor-Generator Planning with LLM-generated Heuristics

Heuristics are a central component of deterministic planning, particularly in domain-independent settings where general applicability is prioritized over task-specific tuning. This work revisits that paradigm in light of recent advances in large language models (LLMs), which enable the automatic synthesis of heuristics directly from problem definitions -- bypassing the need for handcrafted domain knowledge. We present a method that employs LLMs to generate problem-specific heuristic functions from planning tasks specified through successor generators, goal tests, and initial states written in a general-purpose programming language. These heuristics are compiled and integrated into standard heuristic search algorithms, such as greedy best-first search. Our approach achieves competitive, and in many cases state-of-the-art, performance across a broad range of established planning benchmarks. Moreover, it enables the solution of problems that are difficult to express in traditional formalisms, including those with complex numeric constraints or custom transition dynamics. We provide an extensive empirical evaluation that characterizes the strengths and limitations of the approach across diverse planning settings, demonstrating its effectiveness.

Alexander Tuisov, Yonatan Vernik, Alexander Shleyfman

Independent Researcher · Computer Science Department, Bar-Ilan University

Automated Heuristic DesignLLM Planning
AskPDF