Automatic Prompt Optimization

Overview

Automatic prompt optimization studies how to improve natural-language instructions, examples, and sometimes prompt structure without changing model weights. Common approaches use black-box evaluation with evolutionary search, textual-gradient or reflective edits, Bayesian and bandit selection, and targeted example selection; newer systems also optimize multi-step workflows, decoding settings, and task-specific prompt components. Work increasingly treats evaluation budget, inference cost, interpretability, personalization, and stability as objectives alongside accuracy, using held-out tests and multi-objective selection to limit regressions. A central tension is that gains can depend on the task and model: noisy or narrow feedback may cause overfitting, prompt hacking, or regressions, so reliable transfer matters as much as peak benchmark scores.

Latest papers

CardsList

111 papers11 papers in the last four weeks, down 48% on the four weeks before. 0.1% of all new papers.

  1. Simplicity Goes Far: Auditing Prompt Optimizers with Demonstration-Conditioned Prompt Search

    Nov 17, 2026Bohe ChenAutomatic Prompt OptimizationLarge Language Model-Guided Optimization

  2. EmoHarness: Boosting Multimodal Emotion Understanding with Meta-Harness Evolving

    Nov 9, 2026Fenghai Li, Bingxuan Li, Jiawen Zhang +4Multimodal Emotion RecognitionMultimodal Large Language Models

  3. Chatter Attack: Resource Consumption Attack for Large Language Models

    Nov 8, 2026Yibo Miao, Yichuan Cao, Xiao-Shan Gao +1Adversarial Attacks on LLMsLLM Security

  4. PromptOps: Iterative Prompt Optimization for LLM-as-a-Judge Systems

    Nov 4, 2026Sven Kleinkop, Tamás Janusko, Ricardo Bochnia +4LLM-as-a-JudgePrompt Optimization

  5. IntentGEO: Intent-aware Generative Engine Optimization for Recommendation

    Nov 3, 2026Runlin Lei, Shiqi Zhang, Xiaokui Xiao +1Generative Engine OptimizationAutomatic Prompt Optimization

  6. Compile as Hypothesis: A Headroom Typology for Prompt Optimization on Real Reasoning Models

    Oct 31, 2026Zhengshuai Yang, Changyun LiAutomatic Prompt OptimizationLarge Reasoning Models

  7. Self-Refining Prompt Induction for Streaming Text-to-SQL

    Oct 31, 2026Gaetano Rossiello, Alfio Gliozzo, Nandana Mihindukulasooriya +3Text-to-SQLAutomatic Prompt Optimization

  8. When Does GEPA Merge Help? A Mechanism-Level Diagnostic

    Oct 25, 2026Chong Zhao, Ruyi Yang, Changming Liu +1Automatic Prompt OptimizationPrompt Optimization

  9. Shaping the Document, Not the Model: Optimized Suffixes for Context Following

    Oct 19, 2026Gia-Bao Dinh Ho, Sekh Mainul Islam, Masahiro Kaneko +2Knowledge Conflicts in Language ModelsAutomatic Prompt Optimization

  10. A Cross-Domain Evaluation of Prompting Techniques for Large Language Models

    Oct 17, 2026Kevin Krazius, Tim Schlippe, Paul LibbrechtLLM PromptingLLM Evaluation

  11. Designing Textual Interfaces To Enhance Llm Reasoning On Audio-Visual Question Answering

    Oct 14, 2026Yi Wang, Tianzi Wang, Tong Lei +2Multimodal ReasoningAutomatic Prompt Optimization

  12. CORE: Contrast-and-Reflect for Self-Supervised Prompt Optimization in Anomaly Synthesis

    Oct 14, 2026Haoxiang Rao, Fang Zhao, Zhao Wang +3Synthetic Anomaly GenerationSynthetic Data Augmentation

  13. HERO: Harnessing Evidence for Test-time Refinement and Prompt Optimization in Program Generation

    Oct 13, 2026Mingyu Lu, Timothy Ossowski, Tristan Naumann +2Automatic Prompt OptimizationLLM Code Generation

  14. Beyond Aggregate Accuracy: Item-Level Transitions in Prompt Optimization

    Oct 13, 2026Chinmoy Mitra, Tarek MahmudAutomatic Prompt OptimizationLanguage Model Generation Evaluation

  15. Prompt or Policy? Optimizing Physician Slate Presentations with LLM User Simulators

    Oct 13, 2026Andrei Niculae, Adrian Cosma, Emilian RadoiAutomatic Prompt OptimizationLarge Language Model-Based Recommendation

  16. Goldsmith: Gold-Loss-Guided Definition Optimization with an Agentic Annotation Harness

    Oct 7, 2026Yihan Li, Hanyi Zhang, Xiaoxi Jiang +1Automated Data AnnotationLLM-Assisted Annotation

  17. Joint Workflow and Prompt Optimization for User Behavior Simulation

    Oct 6, 2026Nipun B Nair, Tongtong Wu, Hongzhi Yin +2Multi-Agent LLM SystemsAutomatic Prompt Optimization

  18. DP-ES: Differentially Private Evolution Strategies for Prompt Optimization

    Oct 5, 2026Ziniu Liu, Aiping Li, Yue Han +5Large Language Model-Guided OptimizationEvolution Strategies

  19. How Should a Prompt Optimizer Spend a Tight Budget? BudgetAPO with Noise-Adaptive Evaluation

    Oct 5, 2026Haoyue Liu, Zhichao Wang, Huanyu Yan +1Automatic Prompt OptimizationPrompt Optimization

  20. AdaGEPA: Adaptive Feedback Allocation for Reflective Prompt Optimization

    Sep 30, 2026Junyang Chen, Zecheng Wang, Jingbang ChenAutomatic Prompt OptimizationPrompt Optimization

  21. Benchmarking Prompt Optimization of Large Language Models With Chess

    Sep 30, 2026Timothée Lesort, Alejandra López de Aberasturi Gómez, Tristan Karch +4LLM EvaluationAutomatic Prompt Optimization

  22. MERGE: Multi-LLM Ensemble for Retrieval via Generative Enrichment

    Sep 29, 2026Tzu-I Ho, Yung-Yu Shih, Shang-Yu Su +2LLM PromptingAutomatic Prompt Optimization

  23. From Weak Task Specifications to Scientific Extraction Agents: Optimizing Task Construction

    Sep 28, 2026Zixiao Dong, Wei Yang, Zihao Liu +4LLM Agent Self-ImprovementAutomatic Prompt Optimization

  24. APOLO: Automatic Prompt Optimization for Ontology Learning

    Sep 28, 2026Huu Tan Mai, Roman Kochnev, Cuong Xuan Chu +3Automatic Prompt OptimizationOntology Learning