Large Language Model-Guided Optimization
Overview
This field studies how language models can use learned patterns and natural-language understanding to help search for better algorithms, programs, designs, prompts, and agent workflows. Common approaches combine model-generated proposals with evolutionary search, Bayesian optimization, reinforcement learning, or classical solvers, then use execution, simulator, or benchmark feedback to select and repair candidates. Applications range from combinatorial and scientific optimization to GPU kernels, compiler code, engineering design, and tuning language-model systems. A central open tension is that language models can suggest useful structures but may be unreliable, costly, or prone to overfitting, so robust evaluation, constraint checks, and comparisons with numerical methods remain essential.
Latest papers
660 papers58 papers in the last four weeks, down 45% on the four weeks before. 0.4% of all new papers.
Self-EvolveRec: Self-Evolving Recommender Systems with LLM-based Directional Feedback
ImprovEvolve: Basin-Hopping Meets LLM-Guided Evolutionary Search
MOSAIC: More LLMs, Less Cost through Collaborative Reasoning for Compiler Optimizations
Tree-Structured Synergy of Large Language Models and Bayesian Optimization for Efficient CASH
PACEvolve: Enabling Progress-Aware Consistent Evolution
Test-Time Detoxification without Training or Learning Anything
KernelEvolve: Scaling Agentic Kernel Coding for Heterogeneous AI Accelerators at Meta
Shuttling Compiler for Trapped-Ion Quantum Computers Based on Fine-Tuned Large Language Models
Informing Acquisition Functions via Foundation Models for Molecular Discovery
LLM-Auction: Generative Auction towards LLM-Native Advertising
Evolving Excellence: Automated Optimization of LLM-based Agents
Towards Training-free Automatic Proxy Discovery via Large Language Models for Mixed Precision Quantization
CUDA-L2: Surpassing cuBLAS Performance for Matrix Multiplication through Reinforcement Learning
OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
A Systematic Survey on Large Language Models for Evolutionary Optimization: From Modeling to Solving
AutoPBO: LLM-powered Optimization for Local Search PBO Solvers
Access Paths for Efficient Ordering with Large Language Models
LLM ORDER BY semantic operator as a logical abstraction and conduct a systematic study of its physical implementations. First, we propose several improvements to existing semantic sorting algorithms and introduce a semantic-aware external merge sort algorithm. Our extensive evaluation reveals that no single implementation offers universal optimality on all datasets. From our evaluations, we observe a general scaling relationship between sorting cost and the ordering quality for comparison-based algorithms. Building on these insights, we design a budget-aware optimizer that utilizes heuristic rules, LLM-as-Judge evaluation, and consensus aggregation to dynamically select the near-optimal access path for LLM ORDER BY. In our extensive evaluations, our optimizer consistently achieves ranking accuracy on par with or superior to the best static methods across all benchmarks. We believe that this work provides foundational insights into the principled optimization of semantic operators essential for building robust, large-scale LLM-powered analytic systems.