Prompt Stability Matters: Evaluating and Optimizing Auto-Generated Prompt in General-Purpose Systems
Authors: Ke Chen, Xucheng Yu, Yufei Zhou, Haohan Wang
Organizations: School of Information Sciences, University of Illinois Urbana-Champaign, Urbana, USA · Department of Economics, Duke University, Durham, USA
Automatic prompt generation plays a crucial role in enabling general-purpose multi-agent systems to perform diverse tasks autonomously. Existing methods typically evaluate prompts based on their immediate task performance, overlooking the intrinsic qualities that determine their reliability. This outcome-centric view not only limits interpretability but also fails to account for the inherent stochasticity of large language models (LLMs). In this work, we bring attention to prompt stability-the consistency of model responses across repeated executions-as a key factor for building robust and effective prompt generation systems. To quantify this, we propose semantic stability as a criterion for assessing the response consistency of prompts. Based on the proposed metric, we developed the first stability-aware general-purpose prompt generation system that leverages stability feedback to iteratively enhance both prompt quality and system-level performance. Furthermore, we establish a logical chain between prompt stability and task success by analyzing the structural dependencies within our system, proving stability as a necessary condition for effective system-level execution. Empirical results across general and domain-specific tasks demonstrate that our stability-aware framework improves both accuracy and output consistency. By shifting the focus from one-off results to persistent reliability, our work offers a new perspective on prompt design and contributes practical tools for building more trustworthy general-purpose systems. The UI and source code for our system are publicly available at: https://xucheng63.github.io/MyDataPilot/.
Figures & tables
Figure 1: Promptor system pipeline of our stability-aware prompt generation framework.
Figure 2: Illustration of semantic stability as a metric.
Feature
Importance (mean ± std)
Stability Score (ours)
0.544 ± 0.062
Mutual Information
0.391 ± 0.027
Prompt Entropy
0.303 ± 0.012
Clarity
0.149 ± 0.021
Specificity
0.135 ± 0.019
Coherence
0.115 ± 0.011
Table 1: Feature importance of different metrics on XGB Regressor.
System
Math Reasoning
Data Analysis
Code Writing
Machine Learning
Avg
Promptor
0.63
0.88
0.94
0.86
0.84
DI
0.63
0.95
0.84
0.80
0.82
AutoGen
0.50
0.71
0.88
0.82
0.73
DA-Agent
0.43
0.65
0.73
0.58
0.62
EvoMac
0.60
0.43
0.95
0.44
0.63
Table 2: Comparison of different systems across four general task domains.
Table 5
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 3: Performance on the MATH benchmark. The horizontal axis shows cumulative accuracy defined as CumAcc=∑k=14Acck , where Acck∈[0,1] denotes the accuracy on each of the four subtasks (counting and probability, number theory, pre-algebra, and pre-calculus). Each colored segment corresponds to one subtask.
System prompt optimization improves agent behavior without modifying the underlying model, yielding human-readable, model-agnostic instructions. Existing methods build a prompt agent that refines task agents' system prompts, yet leave the prompt agent's own system prompt hand-engineered and fixed. We propose Self-Evolving Prompt Optimization (SePO), which treats the prompt agent's own system prompt as an optimization target alongside task agents' system prompts. SePO adopts a self-referential design. A single prompt agent improves both task agents' system prompts and its own under an open-ended evolutionary search that maintains an archive of candidate prompts as stepping stones. Training proceeds in two stages: pre-training evolves the prompt agent on a multi-task pool, and fine-tuning then applies it to a target task. Across five benchmarks spanning math (AIME'25), abstract reasoning (ARC-AGI-1), graduate-level science (GPQA), code generation (MBPP), and logic puzzles (Sudoku), SePO consistently outperforms Manual-CoT, TextGrad, and MetaSPO, improving the average accuracy by 4.49 points compared to Manual-CoT. The prompt optimization skill from pre-training also generalizes to tasks beyond the pre-training mixture, rather than memorizing per-task prompts.
Wangcheng Tao, Han Wu, Weng-Fai Wong
National University of Singapore · City University of Hong Kong
Automatic prompt optimization is a promising approach for adapting large language models (LLMs) to downstream tasks, yet existing methods typically search for a specific prompt specialized to a fixed task. This paradigm limits generalization across heterogeneous queries and prevents models from accumulating reusable prompting knowledge over time. In this paper, we propose MemAPO, a memory-driven framework that reconceptualizes prompt optimization as generalizable and self-evolving experience accumulation. MemAPO maintains a dual-memory mechanism that distills successful reasoning trajectories into reusable strategy templates while organizing incorrect generations into structured error patterns that capture recurrent failure modes. Given a new prompt, the framework retrieves both relevant strategies and failure patterns to compose prompts that promote effective reasoning while discouraging known mistakes. Through iterative self-reflection and memory editing, MemAPO continuously updates its memory, enabling prompt optimization to improve over time rather than restarting from scratch for each task. Experiments on diverse benchmarks show that MemAPO consistently outperforms representative prompt optimization baselines while substantially reducing optimization cost.
Guanbao Liang, Yuanchen Bei, Sheng Zhou +5
Zhejiang University · University of Illinois Urbana-Champaign · Alibaba Group
Large language model (LLM)-based Multi-agent systems (MAS) have shown promise in tackling complex collaborative tasks, where agents are typically orchestrated via role-specific prompts. While the quality of these prompts is pivotal, jointly optimizing them across interacting agents remains a non-trivial challenge, primarily due to the misalignment between local agent objectives and holistic system goals. To address this, we introduce MASPO, a novel framework designed to automatically and iteratively refine prompts across the entire system. A core innovation of MASPO is its joint evaluation mechanism, which assesses prompts not merely by their local validity, but by their capacity to facilitate downstream success for successor agents. This effectively bridges the gap between local interactions and global outcomes without relying on ground-truth labels. Furthermore, MASPO employs a data-driven evolutionary beam search to efficiently navigate the high-dimensional prompt space. Extensive empirical evaluations across 6 diverse tasks demonstrate that MASPO consistently outperforms state-of-the-art prompt optimization methods, achieving an average accuracy improvement of 2.9. We release our code at https://github.com/wangzx1219/MASPO.
Zhexuan Wang, Xuebo Liu, Li Wang +4
Institute of Computing and Intelligence, Harbin Institute of Technology, Shenzhen, China.