cs.LGApr 9, 2026

p1p1: Better Prompt Optimization with Fewer Prompts

Authors: Zhaolin Gao, Yu, Wang, Bo Liu, Thorsten Joachims, Kianté Brantley, Wen Sun

Organizations: Sid · Cornell University · Microsoft · Harvard University · 

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Prompt optimization improves language models without updating their weights by searching for a better system prompt, but its effectiveness varies widely across tasks. We study what makes a task amenable to prompt optimization. We show that the reward variance across different system prompts can be decomposed into two components: variance among responses, which captures generation stochasticity, and variance among system prompts, which captures differences in system prompt quality. Prompt optimization succeeds when variance among system prompts is sufficiently large, but fails when variance among responses dominates the variance of the system prompts. Surprisingly, we further show that scaling to more user prompts can hurt optimization by reducing variance among system prompts, especially on heterogeneous datasets where different user prompts favor different system prompts. Motivated by this insight, we propose p1p1, a simple user prompt filtering method that selects a small subset of user prompts with high variance across candidate system prompts. This subset of user prompts allows one to distinguish a good system prompt from a bad one, making system optimization easier. Experiments on reasoning benchmarks show that p1p1 substantially improves prompt optimization over training on the full dataset and outperforms strong baselines such as GEPA. Notably, training on only two prompts from AIME 24 yields a system prompt that generalizes well to other reasoning benchmarks.

Figures & tables

Figure 1: Comparison of p1p1 against the base model and baseline methods. For all methods, the system prompt is optimized based on AIME 24 and Qwen3-4B-Instruct-2507, and directly applied to these benchmarks and to Qwen3-30B-A3B-Instruct-2507. The results are averaged over 6464 generations per user prompt.
Figure 2: Training reward and evaluation accuracy on IFBench and AIME with M∈{1,2}M\in\{1,2\} .
Figure 3: Variances across responses and system prompts for IFBench training set and AIME 24. On IFBench, the variance in reward across system prompts is substantially larger than the variance across sampled responses. In contrast, on AIME, the variance across system prompts is smaller than the variance across responses. The visualizations provide an intuitive illustration of this distinction. On IFBench, different system prompts are more clearly separated, making it easier to identify which system prompt is better, whereas on AIME, the small variance makes such distinctions much harder to detect.
Figure 4: Variance among system prompts vs. training reward improvement when training on one AIME prompt.
Figure 5: Variance among responses and among system prompts for AIME and IFBench. The first row shows results with varying KK while fixing M=128M=128 . The second row shows results with varying both KK and MM while keeping KM=128KM=128 fixed. The last column reports the ratio of the variance among system prompts to the variance among responses. Note that each line is plotted with its own y-axis scale.
 MethodSSMMTrainingAIME 25AIME 26HMMT 25HMMT 26
 Reward Imp.
Qwen3-4B-Instruct-2507base///47.0354.3840.6827.89
GEPA15/15//46.8754.2240.5727.08
GEPA[1,23]/28//46.7253.4940.1027.46
GEPA28/[1,23]//46.3553.9140.2127.32
RLall 301047.2454.5840.2627.04
Table 1: Performance on AIME and HMMT under different methods and selected subsets SS . The subset SS is a set of indices (within []) or the number of prompts used for training. For GEPA, SS shows the (training prompts / validation prompts) split. The best performing method is highlighted in bold and the second best is underlined .
MethodSSMMTrainingIFBench
Reward Imp.
Qwen3-4B-Instruct-2507
base///35.03
GEPA32/32//39.12
RL6420.1539.46
p1p11680.1837.41
Table 2: Performance on IFBench under different methods, subset SS and MM .
Figure 6: Examples of learned system prompts from p1p1 and GEPA on AIME 24 with Qwen3-4B-Instruct-2507. p1p1 produces a more general reasoning-oriented prompt, while GEPA produces a more task-specific prompt that appears to memorize training-set patterns.
Appendix figures & tables22 assets

Supplementary material from the paper’s appendix.

Appendix

DatasetTraining Dataset CardEvaluation Dataset CardGeneration Length ( ∣y∣|y| )
IFBenchallenai/IF_multi_constraints_upto5allenai/IFBench_test2,0482{,}048
AIMEmath-ai/aime24math-ai/aime2516,38416{,}384
Appendix
Table 3: Dataset details and maximum generation length
 Prompt
Meta prompt ss for π′\pi^{\prime} on IFBenchYou are designing a system prompt for a large language model (LLM) whose primary goal is precise instruction following. The LLM should be optimized to correctly handle tasks with strict formatting, stylistic, and content constraints. Directly generate a high-quality system prompt that instructs an LLM to reliably solve instruction following problems. Generate the system prompt only.
Meta prompt ss for π′\pi^{\prime} on AIMEDirectly generate an effective system prompt for a large language model (LLM) designed specifically to solve competition-level math problems. Instruct the model to think and reason like a human.
Prompt for π\pi{System Prompt} {Prompt}
Appendix
Table 4: Input Prompts
Figure 11
Figure 12
Figure 13
Figure 14
Figure 15
Figure 16
Figure 17
DatasetTraining Dataset CardGeneration Length ( ∣y∣|y| )Generation Length ( ∣y∣|y| )
Qwen3-4B-Instruct-2507Qwen3-1.7B
IFBenchallenai/IF_multi_constraints_upto52,0482{,}0484,0964{,}096
AIMEmath-ai/aime2416,38416{,}38432,76832{,}768
Appendix
Table 5: Dataset details and maximum generation length
 Prompt
Meta prompt ss for π′\pi^{\prime} on IFBenchYou are designing a system prompt for a large language model (LLM) whose primary goal is precise instruction following. The LLM should be optimized to correctly handle tasks with strict formatting, stylistic, and content constraints. Directly generate a high-quality system prompt that instructs an LLM to reliably solve instruction following problems. Generate the system prompt only.
Meta prompt ss for π′\pi^{\prime} on AIMEDirectly generate an effective system prompt for a large language model (LLM) designed specifically to solve competition-level math problems. Instruct the model to think and reason like a human.
Prompt for π\pi{System Prompt} {Prompt}
Appendix
Table 6: Input Prompts
SettingParameters 
GEPAmaximum training time: 3 daysreflection minibatch size: 4
RL & p1p1 Qwen3-4B-Instruct-2507NN : 16 learning rate: 1e-6 maximum training time: 3 days π\pi temperature : 0.6 π\pi top_p : 0.95 π\pi top_k : -1π′\pi^{\prime} training temperature : 1 π′\pi^{\prime} training top_p : 1 π′\pi^{\prime} training top_k : -1 π′\pi^{\prime} validation temperature : 0.6 π′\pi^{\prime} validation top_p : 0.95 π′\pi^{\prime} validation top_k : -1
RL & p1p1 Qwen3-1.7BNN : 16 learning rate: 1e-6 maximum training time: 3 days π\pi temperature : 0.6 π\pi top_p : 0.95 π\pi top_k : 20π′\pi^{\prime} training temperature : 1 π′\pi^{\prime} training top_p : 1 π′\pi^{\prime} training top_k : -1 π′\pi^{\prime} validation temperature : 0.6 π′\pi^{\prime} validation top_p : 0.95 π′\pi^{\prime} validation top_k : -1
Appendix
Table 20
Figure 21
Figure 22
Figure 23
Figure 24
Figure 25
Figure 26
Figure 27
Figure 28
Figure 29
Figure 30
More in this topic

Automatic Prompt Optimization

Automatic prompt optimization uses model or task feedback to revise instructions for stronger, more reliable language-model performance.

See all 103 papers

Explore similar work

CardsList
May 26, 2026

Why Prompt Optimization Works, and Why It Sometimes Doesn't: A Causal-Inspired Edit-Level Analysis

Automated prompt optimization methods (e.g., DSpy, TextGrad) can substantially improve the performance of large language model (LLM), however, their generalization ability across different tasks remains underperformed. In practice, the superiority of the optimized prompt on one benchmark often fails to transfer to another, and this limitation persists even when switching across different LLM backbones. To investigate the underexplored sources of heterogeneity in prompt performance, we conduct a causal inference-inspired observational analysis of optimized prompts across a diverse set of optimization frameworks, LLM backbones, and NLP benchmarks. To achieve the goal, we build upon the propensity-adjusted associational analysis together with multiple complementary representations of prompt edits, where the consistent task-conditioned edits patterns are identified. We find that complexity-increasing and meta-instructional edits are negatively associated with mathematical and multi-hop reasoning performance, whereas step-by-step and meta-cognitive edits improve logical and sequential reasoning tasks. These effects are robust across cognitive-load annotations, surface-level text features, and edit-motif analyses, and can generalize across optimization frameworks. Overall, these results indicate that prompt optimization failures arise from systematic interactions between edit families and task characteristics rather than random optimization artifacts, providing feature-level characterization of optimizer behavior and motivating future task-conditioned optimizer design.

Shuzhi Gong, Hechuan Wen

The University of Melbourne, Melbourne, Victoria, Australia · The University of Queensland, Brisbane, Queensland, Australia

Automatic Prompt OptimizationLLM Reasoning
AskPDF
Apr 8, 2026

Adaptive Prompt Structure Factorization: A Framework for Self-Discovering and Optimizing Compositional Prompt Programs

Automated prompt optimization is crucial for eliciting reliable reasoning from large language models (LLMs), yet most API-only prompt optimizers iteratively edit monolithic prompts, coupling components and obscuring credit assignment, limiting controllability, and wasting tokens. We propose Adaptive Prompt Structure Factorization (aPSF), an API-only framework (prompt-in/text-out; no access to model internals) that uses an Architect model to discover task-specific prompt structures as semantic factors. aPSF then performs interventional, single-factor updates: interventional factor-level scoring estimates each factor's marginal contribution via validation-performance changes, and error-guided factor selection routes updates to the current dominant failure source for more sample-efficient optimization. Across multiple advanced reasoning benchmarks, aPSF outperforms strong baselines including principle-aware optimizers, improving accuracy by up to +2.16 percentage points on average, and reduces optimization cost by 45--87% tokens on MultiArith while reaching peak validation in 1 step.

Haoyue Liu, Zhichao Wang, Yongxin Guo +2

The Chinese University of Hong Kong, Shenzhen · Taobao and Tmall Group

LLM Input OptimizationLLM PromptingAutomatic Prompt OptimizationLarge Language Model-Guided Optimization
AskPDF
Aug 11, 2026

RLMOpt: Adaptive Prompt Optimization via Recursive Language Models

Prompt optimizers automate the search for prompts that improve language-model performance, but existing methods rely on a predefined optimization procedure: the algorithm determines which candidates to explore and how the search progresses, while the language model generates or refines prompt proposals. We introduce RLMOpt, a prompt optimizer that makes the search policy itself language-model-driven through a recursive language model (RLM). The RLM agent operates over a tool-based environment, inspecting task information, analyzing failures, generating candidates, allocating evaluation budget, and deciding when to stop. A deterministic harness complements the agent by enforcing objective scoring, Pareto-based selection, and regression constraints. We evaluate RLMOpt across four benchmarks spanning structured clinical information extraction (Chia), multi-hop question answering (HotpotQA), verifiable instruction following (IFBench-2025), and multi-turn tool-calling agents (BFCL). In a matched comparison at a single seed, RLMOpt obtains the best held-out score on all four benchmarks and leads the four-task mean (0.610 against 0.589 for GEPA). Repeating each benchmark across seeds yields 11 matched benchmark-seed comparisons, in which RLMOpt outperforms GEPA in 9 cases. Across all 11 runs, it never produced a prompt that underperformed its seed, whereas GEPA fell below its starting point twice. It is also more efficient, achieving these results with fewer search rollouts while producing prompts that are 27-79% the size of those produced by GEPA. Our results further show that optimization gains are determined primarily by the headroom available in the seed prompt, rather than by the search budget. Efficient optimization therefore depends on reaching the available headroom reliably and with minimal search

Subhash Bangalore Satheesha, Nirvik Pande, Deepthi Duddempudi +1

Autonomize AI · Carnegie Mellon University

Large Language Model-Guided OptimizationLLM AgentsPrompt OptimizationRecursive Language ModelsLLM Input Optimization
AskPDF