$p1$: Better Prompt Optimization with Fewer Prompts

Zhaolin Gao; Yu; Wang; Bo Liu; Thorsten Joachims; Kianté Brantley; Wen Sun

cs.LGApr 9, 2026

$p1$ : Better Prompt Optimization with Fewer Prompts

Authors: Zhaolin Gao, Yu, Wang, Bo Liu, Thorsten Joachims, Kianté Brantley, Wen Sun

Organizations: Sid · Cornell University · Microsoft · Harvard University ·

Abstract

Prompt optimization improves language models without updating their weights by searching for a better system prompt, but its effectiveness varies widely across tasks. We study what makes a task amenable to prompt optimization. We show that the reward variance across different system prompts can be decomposed into two components: variance among responses, which captures generation stochasticity, and variance among system prompts, which captures differences in system prompt quality. Prompt optimization succeeds when variance among system prompts is sufficiently large, but fails when variance among responses dominates the variance of the system prompts. Surprisingly, we further show that scaling to more user prompts can hurt optimization by reducing variance among system prompts, especially on heterogeneous datasets where different user prompts favor different system prompts. Motivated by this insight, we propose $p1$ , a simple user prompt filtering method that selects a small subset of user prompts with high variance across candidate system prompts. This subset of user prompts allows one to distinguish a good system prompt from a bad one, making system optimization easier. Experiments on reasoning benchmarks show that $p1$ substantially improves prompt optimization over training on the full dataset and outperforms strong baselines such as GEPA. Notably, training on only two prompts from AIME 24 yields a system prompt that generalizes well to other reasoning benchmarks.

Figures & tables

Figure 1: Comparison of

p1

against the base model and baseline methods. For all methods, the system prompt is optimized based on AIME 24 and Qwen3-4B-Instruct-2507, and directly applied to these benchmarks and to Qwen3-30B-A3B-Instruct-2507. The results are averaged over

64

generations per user prompt.

Figure 2: Training reward and evaluation accuracy on IFBench and AIME with

M\in\{1,2\}

Figure 3: Variances across responses and system prompts for IFBench training set and AIME 24. On IFBench, the variance in reward across system prompts is substantially larger than the variance across sampled responses. In contrast, on AIME, the variance across system prompts is smaller than the variance across responses. The visualizations provide an intuitive illustration of this distinction. On IFBench, different system prompts are more clearly separated, making it easier to identify which system prompt is better, whereas on AIME, the small variance makes such distinctions much harder to detect.

Figure 4: Variance among system prompts vs. training reward improvement when training on one AIME prompt.

Figure 5: Variance among responses and among system prompts for AIME and IFBench. The first row shows results with varying

K

while fixing

M=128

. The second row shows results with varying both

K

and

M

while keeping

KM=128

fixed. The last column reports the ratio of the variance among system prompts to the variance among responses. Note that each line is plotted with its own y-axis scale.

	Method	$S$	$M$	Training	AIME 25	AIME 26	HMMT 25	HMMT 26
	Method	$S$	$M$	Reward Imp.	AIME 25	AIME 26	HMMT 25	HMMT 26
Qwen3-4B-Instruct-2507	base	/	/	/	47.03	54.38	40.68	27.89
	GEPA	15/15	/	/	46.87	54.22	40.57	27.08
	GEPA	[1,23]/28	/	/	46.72	53.49	40.10	27.46
	GEPA	28/[1,23]	/	/	46.35	53.91	40.21	27.32
	RL	all 30	1	0	47.24	54.58	40.26	27.04

Table 1: Performance on AIME and HMMT under different methods and selected subsets

S

. The subset

S

is a set of indices (within []) or the number of prompts used for training. For GEPA,

S

shows the (training prompts / validation prompts) split. The best performing method is highlighted in bold and the second best is underlined .

Method	$S$	$M$	Training	IFBench
Method	$S$	$M$	Reward Imp.	IFBench
Qwen3-4B-Instruct-2507
base	/	/	/	35.03
GEPA	32/32	/	/	39.12
RL	64	2	0.15	39.46
$p1$	16	8	0.18	37.41

Table 2: Performance on IFBench under different methods, subset

S

and

M

Panel of Figure 6 — Figure 6: Examples of learned system prompts from $p1$ and GEPA on AIME 24 with Qwen3-4B-Instruct-2507. $p1$ produces a more general reasoning-oriented prompt, while GEPA produces a more task-specific prompt that appears to memorize training-set patterns.

Appendix figures & tables22 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Training Dataset Card	Evaluation Dataset Card	Generation Length ( $\|y\|$ )
IFBench	allenai/IF_multi_constraints_upto5	allenai/IFBench_test	$2{,}048$
AIME	math-ai/aime24	math-ai/aime25	$16{,}384$

Appendix

Table 3: Dataset details and maximum generation length

	Prompt
Meta prompt $s$ for $\pi^{\prime}$ on IFBench	You are designing a system prompt for a large language model (LLM) whose primary goal is precise instruction following. The LLM should be optimized to correctly handle tasks with strict formatting, stylistic, and content constraints. Directly generate a high-quality system prompt that instructs an LLM to reliably solve instruction following problems. Generate the system prompt only.
Meta prompt $s$ for $\pi^{\prime}$ on AIME	Directly generate an effective system prompt for a large language model (LLM) designed specifically to solve competition-level math problems. Instruct the model to think and reason like a human.
Prompt for $\pi$	{System Prompt} {Prompt}

Appendix

Table 4: Input Prompts

Dataset	Training Dataset Card	Generation Length ( $\|y\|$ )	Generation Length ( $\|y\|$ )
Dataset	Training Dataset Card	Qwen3-4B-Instruct-2507	Qwen3-1.7B
IFBench	allenai/IF_multi_constraints_upto5	$2{,}048$	$4{,}096$
AIME	math-ai/aime24	$16{,}384$	$32{,}768$

Appendix

Table 5: Dataset details and maximum generation length

	Prompt
Meta prompt $s$ for $\pi^{\prime}$ on IFBench	You are designing a system prompt for a large language model (LLM) whose primary goal is precise instruction following. The LLM should be optimized to correctly handle tasks with strict formatting, stylistic, and content constraints. Directly generate a high-quality system prompt that instructs an LLM to reliably solve instruction following problems. Generate the system prompt only.
Meta prompt $s$ for $\pi^{\prime}$ on AIME	Directly generate an effective system prompt for a large language model (LLM) designed specifically to solve competition-level math problems. Instruct the model to think and reason like a human.
Prompt for $\pi$	{System Prompt} {Prompt}

Appendix

Table 6: Input Prompts

Setting	Parameters
GEPA	maximum training time: 3 days	reflection minibatch size: 4
RL & $p1$ Qwen3-4B-Instruct-2507	$N$ : 16 learning rate: 1e-6 maximum training time: 3 days $\pi$ temperature : 0.6 $\pi$ top_p : 0.95 $\pi$ top_k : -1	$\pi^{\prime}$ training temperature : 1 $\pi^{\prime}$ training top_p : 1 $\pi^{\prime}$ training top_k : -1 $\pi^{\prime}$ validation temperature : 0.6 $\pi^{\prime}$ validation top_p : 0.95 $\pi^{\prime}$ validation top_k : -1
RL & $p1$ Qwen3-1.7B	$N$ : 16 learning rate: 1e-6 maximum training time: 3 days $\pi$ temperature : 0.6 $\pi$ top_p : 0.95 $\pi$ top_k : 20	$\pi^{\prime}$ training temperature : 1 $\pi^{\prime}$ training top_p : 1 $\pi^{\prime}$ training top_k : -1 $\pi^{\prime}$ validation temperature : 0.6 $\pi^{\prime}$ validation top_p : 0.95 $\pi^{\prime}$ validation top_k : -1

Appendix

Table 20

Automatic Prompt Optimization

Automatic prompt optimization uses model or task feedback to revise instructions for stronger, more reliable language-model performance.

See all 103 papers

Why Prompt Optimization Works, and Why It Sometimes Doesn't: A Causal-Inspired Edit-Level Analysis
May 26, 2026Shuzhi Gong, Hechuan WenAutomatic Prompt Optimization LLM Reasoning
The University of Melbourne, Melbourne, Victoria, Australia · The University of Queensland, Brisbane, Queensland, Australia
Adaptive Prompt Structure Factorization: A Framework for Self-Discovering and Optimizing Compositional Prompt Programs
Apr 8, 2026Haoyue Liu, Zhichao Wang, Yongxin Guo +2LLM Input Optimization LLM Prompting
The Chinese University of Hong Kong, Shenzhen · Taobao and Tmall Group
RLMOpt: Adaptive Prompt Optimization via Recursive Language Models
Aug 11, 2026Subhash Bangalore Satheesha, Nirvik Pande, Deepthi Duddempudi +1Large Language Model-Guided Optimization LLM Agents
Autonomize AI · Carnegie Mellon University

May 26, 2026J/K move · Enter open · S save

Why Prompt Optimization Works, and Why It Sometimes Doesn't: A Causal-Inspired Edit-Level Analysis

Shuzhi Gong, Hechuan Wen

The University of Melbourne, Melbourne, Victoria, Australia · The University of Queensland, Brisbane, Queensland, Australia

Automatic Prompt Optimization LLM Reasoning

Open paper AskPDF arXiv

Automated prompt optimization methods (e.g., DSpy, TextGrad) can substantially improve the performance of large language model (LLM), however, their generalization ability across different tasks remains underperformed. In practice, the superiority of the optimized prompt on one benchmark often fails to transfer to another, and this limitation persists even when switching across different LLM backbones. To investigate the underexplored sources of heterogeneity in prompt performance, we conduct a causal inference-inspired observational analysis of optimized prompts across a diverse set of optimization frameworks, LLM backbones, and NLP benchmarks. To achieve the goal, we build upon the propensity-adjusted associational analysis together with multiple complementary representations of prompt edits, where the consistent task-conditioned edits patterns are identified. We find that complexity-increasing and meta-instructional edits are negatively associated with mathematical and multi-hop reasoning performance, whereas step-by-step and meta-cognitive edits improve logical and sequential reasoning tasks. These effects are robust across cognitive-load annotations, surface-level text features, and edit-motif analyses, and can generalize across optimization frameworks. Overall, these results indicate that prompt optimization failures arise from systematic interactions between edit families and task characteristics rather than random optimization artifacts, providing feature-level characterization of optimizer behavior and motivating future task-conditioned optimizer design.

Abstract

Automatic Prompt Optimization

Explore similar work

Why Prompt Optimization Works, and Why It Sometimes Doesn't: A Causal-Inspired Edit-Level Analysis

Adaptive Prompt Structure Factorization: A Framework for Self-Discovering and Optimizing Compositional Prompt Programs

RLMOpt: Adaptive Prompt Optimization via Recursive Language Models