Automatic Prompt Optimization

Ask about this topic

Overview

Automatic prompt optimization studies how to improve natural-language instructions, examples, and sometimes prompt structure without changing model weights. Common approaches use black-box evaluation with evolutionary search, textual-gradient or reflective edits, Bayesian and bandit selection, and targeted example selection; newer systems also optimize multi-step workflows, decoding settings, and task-specific prompt components. Work increasingly treats evaluation budget, inference cost, interpretability, personalization, and stability as objectives alongside accuracy, using held-out tests and multi-objective selection to limit regressions. A central tension is that gains can depend on the task and model: noisy or narrow feedback may cause overfitting, prompt hacking, or regressions, so reliable transfer matters as much as peak benchmark scores.

Latest papers

CardsList

103 papers21 papers in the last four weeks, up 91% on the four weeks before. 0.1% of all new papers.

  1. Batching the Reflective Optimization Loop: Parallel Proposals Make GEPA Faster and Better

    Oct 27, 2026Jialin Zhang, Lakshya A Agrawal, Shangyin Tan +6Prompt OptimizationLarge Language Model-Guided Optimization

    University of California, Berkeley

  2. When Does GEPA Merge Help? A Mechanism-Level Diagnostic

    Oct 25, 2026Chong Zhao, Ruyi Yang, Changming Liu +1Automatic Prompt OptimizationPrompt Optimization

    Harvard University, Cambridge, MA, USA.

  3. Can LLM Annotators Match Human Consistency? Benchmarking Human Agreement and Optimizing Multi-Agent with Contrastive Knowledge

    Oct 23, 2026Xiaopeng Ke, Zhixin Wang, Xuebo Liu +5LLM-Assisted AnnotationMulti-Agent LLM Systems

    ANT Group · Harbin Institute of Technology (Shenzhen)

  4. Explore More, Validate Less: Efficient Reflective Prompt Optimization

    Oct 19, 2026Xiaofan Zhou, Lu ChengPrompt OptimizationAutomatic Prompt Optimization

    Pennsylvania State University, USA

  5. Shaping the Document, Not the Model: Optimized Suffixes for Context Following

    Oct 19, 2026Gia-Bao Dinh Ho, Sekh Mainul Islam, Masahiro Kaneko +2Knowledge Conflicts in Language ModelsAutomatic Prompt Optimization

    Mohamed bin Zayed University of Artificial Intelligence · University of Copenhagen

  6. A Cross-Domain Evaluation of Prompting Techniques for Large Language Models

    Oct 17, 2026Kevin Krazius, Tim Schlippe, Paul LibbrechtLLM PromptingLLM Evaluation

    IU International University of Applied Sciences, Germany

  7. Designing Textual Interfaces To Enhance Llm Reasoning On Audio-Visual Question Answering

    Oct 14, 2026Yi Wang, Tianzi Wang, Tong Lei +2Multimodal ReasoningAutomatic Prompt Optimization

    School of Informatics, University of Edinburgh, UK · Department of Systems Engineering and Engineering Management, Chinese University of Hong Kong, Hong Kong SAR · Key Laboratory of Modern Acoustics, Institute of Acoustics, Nanjing University, China

  8. CORE: Contrast-and-Reflect for Self-Supervised Prompt Optimization in Anomaly Synthesis

    Oct 14, 2026Haoxiang Rao, Fang Zhao, Zhao Wang +3Synthetic Anomaly GenerationSynthetic Data Augmentation

    School of Intelligence Science and Technology, Nanjing University · China Mobile Zijin Innovation Institute

  9. HERO: Harnessing Evidence for Test-time Refinement and Prompt Optimization in Program Generation

    Oct 13, 2026Mingyu Lu, Timothy Ossowski, Tristan Naumann +2Automatic Prompt OptimizationLLM Code Generation

    University of Washington · Microsoft Research · Recursion

  10. Beyond Aggregate Accuracy: Item-Level Transitions in Prompt Optimization

    Oct 13, 2026Chinmoy Mitra, Tarek MahmudAutomatic Prompt OptimizationLanguage Model Generation Evaluation

    Department of Computer Science & Engineering, Rajshahi University of Engineering & Technology, Rajshahi, Bangladesh · Texas A&M University–Kingsville, Kingsville, Texas, USA

  11. RLPE: Teaching LLMs to Prompt with Reinforcement Learning

    Oct 13, 2026Rahul Singhal, Karime Maamari, Pradyumna TambwekarLLM PromptingAutomatic Prompt Optimization

  12. Prompt or Policy? Optimizing Physician Slate Presentations with LLM User Simulators

    Oct 13, 2026Andrei Niculae, Adrian Cosma, Emilian RadoiAutomatic Prompt OptimizationLarge Language Model-Based Recommendation

    National University of Science and Technology POLITEHNICA Bucharest, Romania · IDSIA, Dalle Molle Institute for Artificial Intelligence, Lugano, Switzerland

  13. Reflective Prompt Optimization of a Deep-Research Agent: A Topic-Disjoint, Multi-Seed Protocol for Measuring Transferable Gains and Prompt-Level Overfitting

    Oct 8, 2026Elias Lumer, Sahil Sen, Vamse Kumar Subbiah +1Automatic Prompt OptimizationLarge Language Model-Guided Optimization

  14. Goldsmith: Gold-Loss-Guided Definition Optimization with an Agentic Annotation Harness

    Oct 7, 2026Yihan Li, Hanyi Zhang, Xiaoxi Jiang +1LLM-Assisted AnnotationLarge Language Model-Guided Optimization

    Sun Yat-sen University, Guangzhou, China · South China University of Technology, Guangzhou, China

  15. Joint Workflow and Prompt Optimization for User Behavior Simulation

    Oct 6, 2026Nipun B Nair, Tongtong Wu, Hongzhi Yin +2Multi-Agent LLM SystemsAutomatic Prompt Optimization

    Monash University · The University of Queensland · Xiamen University

  16. DP-ES: Differentially Private Evolution Strategies for Prompt Optimization

    Oct 5, 2026Ziniu Liu, Aiping Li, Yue Han +5Large Language Model-Guided OptimizationEvolution Strategies

    National University of Defense Technology · CRRC Zhuzhou Electric Locomotive Research Institute Co., Ltd., China Academy of Railway Sciences

  17. How Should a Prompt Optimizer Spend a Tight Budget? BudgetAPO with Noise-Adaptive Evaluation

    Oct 5, 2026Haoyue Liu, Zhichao Wang, Huanyu Yan +1Automatic Prompt OptimizationPrompt Optimization

    School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen · Shenzhen Future Network of Intelligence Institute (FNii-Shenzhen) · Guangdong Provincial Key Laboratory of Future Networks of Intelligence, CUHK(SZ)

  18. AdaGEPA: Adaptive Feedback Allocation for Reflective Prompt Optimization

    Sep 30, 2026Junyang Chen, Zecheng Wang, Jingbang ChenAutomatic Prompt OptimizationPrompt Optimization

    The Chinese University of Hong Kong, Shenzhen · Shenzhen Loop Area Institute

  19. Benchmarking Prompt Optimization of Large Language Models With Chess

    Sep 30, 2026Timothée Lesort, Alejandra López de Aberasturi Gómez, Tristan Karch +4LLM EvaluationAutomatic Prompt Optimization

    imec AI.labs

  20. MERGE: Multi-LLM Ensemble for Retrieval via Generative Enrichment

    Sep 29, 2026Tzu-I Ho, Yung-Yu Shih, Shang-Yu Su +2LLM PromptingAutomatic Prompt Optimization

    University of Waterloo · National Taiwan University · Rakuten Group, Inc. +1

  21. From Weak Task Specifications to Scientific Extraction Agents: Optimizing Task Construction

    Sep 28, 2026Zixiao Dong, Wei Yang, Zihao Liu +4LLM Agent Self-ImprovementAutomatic Prompt Optimization

    School of Computer Science and Technology, University of Science and Technology of China · State Key Laboratory of Cognitive Intelligence · University of Science and Technology of China +1

  22. APOLO: Automatic Prompt Optimization for Ontology Learning

    Sep 28, 2026Huu Tan Mai, Roman Kochnev, Cuong Xuan Chu +3Automatic Prompt OptimizationOntology Learning

    Bosch Center for Artificial Intelligence, Robert-Bosch-Campus 1, 71272 Renningen, Germany · University of Mannheim, Schloss, 68131 Mannheim, Germany · University of Würzburg, Sanderring 2, 97070 Würzburg, Germany +1

  23. Streamlined Reflective Evolution for Task-Adaptive Self-Refinement Pipelines

    Sep 26, 2026Xiaofan Zhou, Lu ChengLLM Self-RefinementLLM Agent Workflow Optimization

    Pennsylvania State University

  24. What Should the Reflector See? An Empirical Study of Evidence in Reflective Prompt Optimization

    Sep 26, 2026Xiaofan Zhou, Lu ChengPrompt OptimizationAutomatic Prompt Optimization

    Pennsylvania State University

  25. On the Capability and Limitation of Hard Prompt

    Sep 26, 2026Lijia Yu, Shuaitong Liu, Gaojie Jin +2LLM PromptingPrompt Learning

    Institute of AI for Industries, Chinese Academy of Sciences · Southwest University · Department of Artificial Intelligence&Institute of Artificial Intelligence and Brain Sciences, University of Macau +3

  26. ClusterFewshot: Improving Few-shot Optimization for LLMs workflow

    Sep 22, 2026Omri Bar Haim, Shahar Katz, Lior WolfIn-Context Example SelectionLarge Language Model-Guided Optimization

    Blavatnik School of Computer Science, Tel Aviv University

  27. STEVE: Stabilizing Textual Gradient-Based Prompt Optimization via Error-Driven Refinement and Regularized Verification

    Sep 20, 2026Yifan Xu, Yixuan Li, Xinzhuo Li +4Large Language Model-Guided OptimizationAutomatic Prompt Optimization

    University of Illinois Urbana-Champaign · Google DeepMind

  28. One Prompt Does Not Fit All: Self-Meta-Evolve for Personalized Information Extraction

    Sep 18, 2026Hongliang Li, Lu Wang, Yong Xu +6Automatic Prompt OptimizationLLM Information Extraction

    Microsoft · Beijing Jiaotong University

  29. Failure-Guided Co-Evolution of Prompts and Training Data

    Sep 14, 2026Tianyu Yuan, Zhuzhong QianLarge Language Model-Guided OptimizationAutomatic Prompt Optimization

    Nanjing University

  30. InsightChain: Optimized Chain-of-Insight Analytics for LLM-driven Data Visualization

    Sep 6, 2026Hanya Sun, Chen Zhang, Sheng Liang +2Data VisualizationLLM Prompting

    Huawei Technologies Co., Ltd.