Prompt Interference in LLM Post-Training: Why Pass@k Optimization Can Degrade Pass@1

Anas Barakat; Souradip Chakraborty; Khushbu Pahwa; Amrit Singh Bedi

cs.LGFeb 24, 2026

Prompt Interference in LLM Post-Training: Why Pass@k Optimization Can Degrade Pass@1

Authors: Anas Barakat, Souradip Chakraborty, Khushbu Pahwa, Amrit Singh Bedi

Organizations: Singapore University of Technology and Design · University of Maryland, College Park · University of Central Florida

Abstract

Pass@ $k$ is a widely used performance metric for verifiable large language model tasks, including mathematical reasoning, code generation, and short-answer reasoning. It defines success if any of $k$ independently sampled solutions passes a verifier. This multi-sample inference metric has motivated inference-aware fine-tuning methods that directly optimize pass@ $k$ . However, prior work reports a trade-off: pass@ $k$ improves while pass@1 degrades under such methods. This trade-off is practically important because pass@1 often remains a hard operational constraint due to latency and cost budgets, imperfect verifier coverage, and the need for a reliable single-shot fallback. We study the origin of this trade-off and provide a theoretical characterization of when pass@ $k$ policy optimization can reduce pass@1 through gradient conflict induced by prompt interference. This interference arises because shared model parameters couple prompts, so improving performance on some prompts can adversely affect others. We show that pass@ $k$ policy gradients can conflict with pass@1 gradients because pass@ $k$ optimization implicitly reweights prompts toward low-success prompts; when these prompts are negatively interfering, their upweighting can rotate the pass@ $k$ update direction away from the pass@1 direction. We illustrate our findings with large language model experiments on verifiable mathematical reasoning and coding tasks.

Figures & tables

Figure 1: (a) Empirical trade-off. Under pass@ $k$ policy optimization, pass@ $k$ increases while pass@ $1$ may decrease. We explain this empirically observed trade-off in (b) and (c) , which schematically illustrate the pass@ $1$ and pass@ $k$ ( $k>1$ ) gradients for three prompts and their expectations in policy-parameter space. (b) Pass@1 gradients with negatively interfering prompt. This panel shows a setting in which prompt $3$ is negatively interfering with prompts $1$ and $2$ , i.e., the per-prompt pass@1 gradient for prompt $3$ has negative inner product with the per-prompt pass@ $1$ gradients for prompts $1$ and $2$ (see Section 3 ). Here, $\nabla$ pass@ $1$ denotes the population pass@ $1$ gradient, given by the average (expectation over prompts; here under a uniform distribution) of the per-prompt pass@ $1$ gradients. (c) Pass@ $k$ vs pass@1 gradient conflict. Per-prompt pass@k gradients are scaled versions of the corresponding per-prompt pass@1 gradients (Eq. ( 2 )). This reweighting amplifies the magnitude of the pass@ $k$ gradient for the negatively interfering prompt $3$ , causing the resulting population pass@ $k$ gradient to conflict with the population pass@ $1$ gradient (their inner product becomes negative, corresponding to an obtuse angle, shown here as approximately $120^{\circ}$ ). Hence, a policy update in the pass@ $k$ gradient direction can increase pass@ $k$ while decreasing pass@1.

Figure 2: The prompt distribution has prompts with negative and positive agreement scores

a_{\theta}(x)

( ) on the toy example (Appendix ). Prompts with negative scores are responsible for gradient conflict and hence pass@1 decrease. After pass@

k

optimization, negative scores get even more negative.

Panel of Figure 4 — Figure 4: Pass@ $k$ -induced reweighting produces population-gradient conflict in filtered MATH subsets. Both models demonstrate how pass@ $k$ reweighting causes gradient conflict with pass@1, despite different threshold settings. Agreement scores are computed relative to the pass@1 gradient of filtered prompts. Left column (A): Agreement scores show separation between hard negatives (red) and easy positives (green), with unweighted means shown as blue dashed lines. Middle column (B): Pass@ $k$ weights grouped by pass@1 probability, showing extreme weight disparity ( $\sim$ 10 28 :1) between hard and easy prompts. Right column (C): Weighted contributions (estimates of pass@ $k$ vs pass@1 inner products) demonstrate how pass@ $k$ reweighting induces negative pass@ $k$ /pass@1 gradient alignment. Purple arrows highlight the critical downward shift from the unweighted mean (blue dotted, same as plot A) to the weighted mean (red dashed). (a) DeepSeek-Llama: The downward shift from $+2.80\times 10^{-3}$ to $-1.12\times 10^{-3}$ ( $\Delta=-3.92\times 10^{-3}$ ) flips the gradient from positive to negative, resulting in inner product $-0.613$ . (b) DeepSeek-Qwen: The dramatic downward shift from $+2.97\times 10^{-2}$ to $-2.74\times 10^{-1}$ ( $\Delta=-3.04\times 10^{-1}$ ) demonstrates even stronger gradient conflict. Despite only 29 hard prompts vs 627 easy (21.6:1 ratio), the extreme weight disparity causes hard negatives to completely dominate, resulting in inner product $-181$ .

Figure 5: Cosine kernel heatmap: $\cos(\nabla p_{\theta}(x),\nabla p_{\theta}(x^{\prime}))$ for subsamples of prompts: 120 easy and 80 hard among a total of 6000 samples. Blue regions show negative prompt interference.

Figure 6: Contour plots of pass@1 and pass@

k

objectives in the policy parameter space. Gradients of pass@

k

and pass@1 with respect to policy parameters are conflicting in the gray area.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 7 — Figure 7: Gradient conflict across all threshold configurations (DeepSeek-R1-Distill-Llama-8B). Each panel shows agreement scores, weights, and weighted contributions for a different $(\delta_{1},\delta_{2})$ configuration. All exhibit negative inner products, confirming robust misalignment across thresholds. The consistent downward shift from unweighted to weighted means shows pass@ $k$ optimization systematically opposes pass@1 improvement.

Panel of Figure 8 — Figure 8: Pass@ $k$ amplification pattern across models and thresholds. Each point represents one prompt, with the corresponding agreement score (x-axis), pass@ $k$ weight (y-axis), and pass@1 probability shown by color (red = low, green = high). Consistent pattern: Prompts with negative alignment (left), low pass@1 (red), and high weights (4-12) cluster upper-left, while prompts with positive alignment (right), high pass@1 (green), and zero weights ( $\sim$ 10 -28 ) appear lower-right. Top row (a-c): DeepSeek-R1-Distill-Llama-8B across three $\delta_{2}$ values (0.05, 0.10, 0.15). Bottom row (d-f): DeepSeek-R1-Distill-Qwen-7B with same thresholds.

Panel of Figure 9 — Figure 9: Pass@ $k$ amplification and negative prompt interference in HumanEval dataset using a Qwen 1.5B model for different values of $k$ . Each point represents one prompt, with the corresponding agreement score (x-axis), pass@ $k$ weight (y-axis), and pass@1 probability shown by color (red = low, green = high). Consistent pattern: Prompts with negative alignment (left), low pass@1 (red), and high weights (4-12) cluster upper-left, while prompts with positive alignment (right), high pass@1 (green), and small weights appear lower-right.

RL for Language Model Reasoning

Reinforcement learning for language model reasoning uses reward-driven training to improve multi-step problem solving, tool use, and reliable answers.

See all 1,094 papers

Beyond Compression: Diagnosing How Post-Training Changes Mathematical Reasoning
Sep 29, 2026Hongyang Li, Yiming Zhu, Xiao Li +3Language Model Distillation RL Post-Training
University of Luxembourg · Seafill Open-Source Community · Université Paris-Saclay
Does CoT-Pass@k Really Check the CoT? A Multilingual Mathematical Audit
Sep 26, 2026Tarık Tuna Taşaltı, Burcu Hüdaverdi, David SemedoLLM-as-a-Judge LLM Mathematical Reasoning
Dokuz Eylül University, ˙Izmir, Türkiye · NOVA School of Science and Technology, Universidade NOVA de Lisboa, Caparica, Portugal
Model Capability Dominates: Inference-Time Optimization Lessons from AIMO 3
Mar 29, 2026Natapong NitarachLLM Reasoning LLM Prompting

Sep 29, 2026J/K move · Enter open · S save

Beyond Compression: Diagnosing How Post-Training Changes Mathematical Reasoning

Hongyang Li, Yiming Zhu, Xiao Li, Caesar Wu+2

University of Luxembourg · Seafill Open-Source Community · Université Paris-Saclay

Language Model Distillation RL Post-Training LLM Post-Training LLM Mathematical Reasoning

Open paper AskPDF arXiv

Post-training is central to mathematical reasoning in modern large language models (LLMs), but endpoint pass@1 alone underidentifies what has changed. Gains may reflect newly reachable solutions, cheaper sampling of latent solutions, surface robustness, or memorisation. We compare three post-training paths under a common diagnostic readout: our sufficiently trained off-policy distillation trajectories, released Qwen3 off-policy-plus-on-policy distillation endpoints, and a released DeepSeek-Math endpoint trained with Group Relative Policy Optimisation (GRPO). Our probe uses cross-surface pass@K over verbatim prompts, paraphrases, numerical isomorphisms, and translations, plus consistency, distribution-shape, and verified supervised-fine-tuning (SFT) membership analyses. We find two regimes. On easier AMC problems, large-K ceilings are near saturation, so post-training mainly compresses sample cost. On harder AIME problems, post-training expands the large-K ceiling over the base model: sufficient off-policy distillation already raises this ceiling, Qwen3 released endpoints raise it further, and DeepSeek-Math GRPO does not dominate sufficient off-policy distillation at large K. English-dominant distillation improves non-English reasoning but preserves language-tier gaps. A controlled-overfit audit finds limited sensitivity in current SFT-membership probes. Compression is one regime of post-training, not a universal explanation.

Abstract

RL for Language Model Reasoning

Explore similar work

Beyond Compression: Diagnosing How Post-Training Changes Mathematical Reasoning

Does CoT-Pass@k Really Check the CoT? A Multilingual Mathematical Audit

Model Capability Dominates: Inference-Time Optimization Lessons from AIMO 3