Large Language Model-Guided Optimization

Overview

This field studies how language models can use learned patterns and natural-language understanding to help search for better algorithms, programs, designs, prompts, and agent workflows. Common approaches combine model-generated proposals with evolutionary search, Bayesian optimization, reinforcement learning, or classical solvers, then use execution, simulator, or benchmark feedback to select and repair candidates. Applications range from combinatorial and scientific optimization to GPU kernels, compiler code, engineering design, and tuning language-model systems. A central open tension is that language models can suggest useful structures but may be unreliable, costly, or prone to overfitting, so robust evaluation, constraint checks, and comparisons with numerical methods remain essential.

Latest papers

CardsList

660 papers58 papers in the last four weeks, down 45% on the four weeks before. 0.4% of all new papers.

  1. NeuroWeaver: An Autonomous Evolutionary Agent for Exploring the Programmatic Space of EEG Analysis Pipelines

    Feb 13, 2026Guoan Wang, Shihao Yang, Feng LiuEvolutionary OptimizationEEG Decoding

  2. Self-EvolveRec: Self-Evolving Recommender Systems with LLM-based Directional Feedback

    Feb 13, 2026Sein Kim, Sangwu Park, Hongseok Kang +6Large Language Model-Guided OptimizationLLM-Based Program Synthesis

  3. ImprovEvolve: Basin-Hopping Meets LLM-Guided Evolutionary Search

    Feb 10, 2026Alexey Kravatskiy, Valentin Khrulkov, Ivan OseledetsLarge Language Model-Guided Scientific DiscoveryEvolutionary Optimization

  4. MOSAIC: More LLMs, Less Cost through Collaborative Reasoning for Compiler Optimizations

    Feb 2, 2026Annabelle Sujun Tang, Christopher Priebe, Lianhui Qin +1Large Language Model-Guided OptimizationCompiler Optimization

  5. Tree-Structured Synergy of Large Language Models and Bayesian Optimization for Efficient CASH

    Jan 18, 2026Beicheng Xu, Weitong Qian, Lingching Tung +2Automated Machine LearningHyperparameter Optimization

  6. PACEvolve: Enabling Progress-Aware Consistent Evolution

    Jan 15, 2026Minghao Yan, Bo Peng, Benjamin Coleman +13Evolutionary OptimizationLarge Language Model-Guided Optimization

  7. Test-Time Detoxification without Training or Learning Anything

    Jan 14, 2026Baturay Saglam, Dionysis KalogeriasLarge Language Model-Guided OptimizationLLM Safety

  8. KernelEvolve: Scaling Agentic Kernel Coding for Heterogeneous AI Accelerators at Meta

    Dec 29, 2025Gang Liao, Hongsen Qin, Ying Wang +36GPU Kernel OptimizationGPU Acceleration

  9. Shuttling Compiler for Trapped-Ion Quantum Computers Based on Fine-Tuned Large Language Models

    Dec 19, 2025Fabian Kreppel, Reza Salkhordeh, Ferdinand Schmidt-Kaler +1Quantum Circuit CompilationLLM Fine-Tuning

  10. Informing Acquisition Functions via Foundation Models for Molecular Discovery

    Dec 15, 2025Qi Chen, Fabio Ramos, Alán Aspuru-Guzik +1Bayesian OptimizationLarge Language Model-Guided Optimization

  11. LLM-Auction: Generative Auction towards LLM-Native Advertising

    Dec 11, 2025Chujie Zhao, Qun Hu, Shiping Song +4Online AdvertisingMechanism Design

  12. Evolving Excellence: Automated Optimization of LLM-based Agents

    Dec 9, 2025Paul Brookes, Vardan Voskanyan, Rafail Giavrimis +18Large Language Model-Guided OptimizationLLM Agent Workflow Optimization

  13. Towards Training-free Automatic Proxy Discovery via Large Language Models for Mixed Precision Quantization

    Dec 8, 2025Haidong Kang, Jun Du, Guo YuMixed-Precision QuantizationLarge Language Model-Guided Optimization

  14. AutoPBO: LLM-powered Optimization for Local Search PBO Solvers

    Sep 4, 2025Yi Chu, Jinyuan Li, Yiwen Sun +2Combinatorial OptimizationAutomated Heuristic Design

  15. Access Paths for Efficient Ordering with Large Language Models

    Aug 30, 2025Fuheng Zhao, Jiayue Chen, Yiming Pan +7Large Language Model-Guided Optimization

  16. LLM-Guided Evolution of Symbolic Acceleration Inequality Families for MILP

    Aug 16, 2025Milad Yazdani, Mahdi Mostajabdaveh, Samin Aref +1Mixed-Integer Linear ProgrammingInteger Linear Programming

  17. Optimizing Prompt Sequences using Monte Carlo Tree Search for LLM-Based Optimization

    Aug 8, 2025Fei Xu Yu, Gina Adam, Nathaniel D. Bastian +1Natural Language to Optimization ModelingLarge Language Model-Guided Optimization

  18. Stabilizing Black-Box Prompt Optimization with Textual Regularization and Signal Aggregation

    Jul 14, 2025MohammadReza Davari, Utkarsh Garg, Weixin Cai +1Large Language Model-Guided OptimizationAutomatic Prompt Optimization

  19. Scaling Textual Gradients via Sampling-Based Momentum

    May 31, 2025Zixin Ding, Junyuan Hong, Zhan Shi +6Large Language Model-Guided OptimizationPrompt Optimization

  20. Search-Based Software Engineering and AI Foundation Models: Current Landscape and Future Roadmap

    May 26, 2025Hassan Sartaj, Shaukat Ali, Paolo Arcaini +1Software EngineeringLarge Language Model-Guided Optimization

  21. LLINBO: Trustworthy LLM-in-the-Loop Bayesian Optimization

    May 20, 2025Chih-Yu Chang, Milad Azvar, Chinedum Okwudire +1Bayesian OptimizationBlack-Box Optimization

  22. Evolutionary Guided Decoding: Iterative Value Refinement for LLMs

    Mar 4, 2025Zhenhua Liu, Lijun Li, Ruizhe Chen +5LLM AlignmentLanguage Model Decoding

  23. LLM Program Optimization via Retrieval Augmented Search

    Jan 31, 2025Sagnik Anupam, Alexander Shypula, Osbert BastaniLarge Language Model-Guided OptimizationIn-Context Learning

  24. OptiMUS-0.3: Using Large Language Models to Model and Solve Optimization Problems at Scale

    Jul 29, 2024Ali AhmadiTeshnizi, Wenzhi Gao, Herman Brunborg +3Mixed-Integer Linear ProgrammingLarge Language Model-Guided Optimization

  25. Small Molecule Optimization with Large Language Models

    Jul 26, 2024Philipp Guevorguian, Menua Bedrosian, Tigran Fahradyan +3Molecular OptimizationEvolutionary Optimization