Large Language Model-Guided Optimization

Ask about this topic

Overview

This field studies how language models can use learned patterns and natural-language understanding to help search for better algorithms, programs, designs, prompts, and agent workflows. Common approaches combine model-generated proposals with evolutionary search, Bayesian optimization, reinforcement learning, or classical solvers, then use execution, simulator, or benchmark feedback to select and repair candidates. Applications range from combinatorial and scientific optimization to GPU kernels, compiler code, engineering design, and tuning language-model systems. A central open tension is that language models can suggest useful structures but may be unreliable, costly, or prone to overfitting, so robust evaluation, constraint checks, and comparisons with numerical methods remain essential.

Latest papers

CardsList

613 papers101 papers in the last four weeks, up 60% on the four weeks before. 0.6% of all new papers.

  1. A Systematic Survey on Large Language Models for Evolutionary Optimization: From Modeling to Solving

    Sep 10, 2025Yisong Zhang, Ran Cheng, Guoxing Yi +1Evolutionary OptimizationLarge Language Model-Guided Optimization

  2. Access Paths for Efficient Ordering with Large Language Models

    Aug 30, 2025Fuheng Zhao, Jiayue Chen, Yiming Pan +7Large Language Model-Guided Optimization

    Snowflake · University of Chicago · 3UC Los Angeles +1

  3. LLM-Guided Evolution of Symbolic Acceleration Inequality Families for MILP

    Aug 16, 2025Milad Yazdani, Mahdi Mostajabdaveh, Samin Aref +1Mixed-Integer Linear ProgrammingInteger Linear Programming

    Department of Electrical and Computer Engineering, University of British Columbia, Vancouver, BC, Canada · Huawei Technologies Canada, Burnaby, BC, Canada · Department of Mechanical and Industrial Engineering, University of Toronto, Toronto, ON, Canada

  4. Optimizing Prompt Sequences using Monte Carlo Tree Search for LLM-Based Optimization

    Aug 8, 2025Fei Xu Yu, Gina Adam, Nathaniel D. Bastian +1Natural Language to Optimization ModelingLarge Language Model-Guided Optimization

    The George Washington University · United States Military Academy

  5. CUDA-L1: Improving CUDA Optimization via Contrastive Reinforcement Learning

    Jul 18, 2025Xiaoya Li, Albert Wang, Guoyin Wang +2Reinforcement LearningGPU Kernel Optimization

    DeepReinforce Team

  6. Stabilizing Black-Box Prompt Optimization with Textual Regularization and Signal Aggregation

    Jul 14, 2025MohammadReza Davari, Utkarsh Garg, Weixin Cai +1Large Language Model-Guided OptimizationAutomatic Prompt Optimization

    † Concordia University · ‡ Mila – Quebec AI Institute · ⋄Microsoft

  7. Scaling Textual Gradients via Sampling-Based Momentum

    May 31, 2025Zixin Ding, Junyuan Hong, Zhan Shi +6Large Language Model-Guided OptimizationPrompt Optimization

    University of Chicago · University of Texas at Austin · Santa Clara University +3

  8. Search-Based Software Engineering and AI Foundation Models: Current Landscape and Future Roadmap

    May 26, 2025Hassan Sartaj, Shaukat Ali, Paolo Arcaini +1Software EngineeringLarge Language Model-Guided Optimization

    Simula Research Laboratory Oslo, Norway · National Institute of Informatics Tokyo, Japan · Kristiania University of Applied Sciences and Oslo Metropolitan University Oslo, Norway

  9. Generalising from Self-Produced Data: Model Training Beyond Human Constraints

    Apr 7, 2025Alfath Daryl Alhajir, Jennifer Dodgson, Joseph Lim +4Large Language Model-Guided OptimizationSelf-Evolving Agents

    Eigenform

  10. Evolutionary Guided Decoding: Iterative Value Refinement for LLMs

    Mar 4, 2025Zhenhua Liu, Lijun Li, Ruizhe Chen +5LLM AlignmentLanguage Model Decoding

    Shanghai Artificial Intelligence Laboratory · Soochow University · Zhejiang University +1

  11. LLM Program Optimization via Retrieval Augmented Search

    Jan 31, 2025Sagnik Anupam, Alexander Shypula, Osbert BastaniLarge Language Model-Guided OptimizationIn-Context Learning

    University of Pennsylvania

  12. OptiMUS-0.3: Using Large Language Models to Model and Solve Optimization Problems at Scale

    Jul 29, 2024Ali AhmadiTeshnizi, Wenzhi Gao, Herman Brunborg +3Mixed-Integer Linear ProgrammingLarge Language Model-Guided Optimization

    a School of Management Science and Engineering, Stanford University, Stanford, California 94305 · b Institute for Computational and Mathematical Engineering, Stanford University, Stanford, California 94305

  13. Small Molecule Optimization with Large Language Models

    Jul 26, 2024Philipp Guevorguian, Menua Bedrosian, Tigran Fahradyan +3Molecular OptimizationEvolutionary Optimization

    YerevaNN research lab, Yerevan, Armenia.