Coding agents operating under ambiguous instructions or user prompts must decide whether to ask clarifying questions or attempt a solution directly. While clarification from the user may improve the correctness of the agent's solution, each back-and-forth interaction incurs user and system costs, forming an explicit accuracy vs. efficiency tradeoff. Existing works study clarification behavior but do not train policies under enforceable clarification budgets; penalty-based approaches typically require separate coefficient tuning swept across all clarification budget levels. We formulate clarification as a Constrained Markov Decision Process (CMDP) and post-train Qwen2.5-Coder-7B-Instruct with PPO-Lagrangian to optimize coding accuracy, subject to an expected question-budget constraint. Evaluated on HumanEvalComm with a GPT-4o-mini oracle simulator, the resulting policies reveal that untuned clarification behavior is Pareto-inefficient: budget-constrained policies can simultaneously achieve higher accuracy and lower clarification rates than the baseline model. Across budget levels, we observe a log-shaped Pareto frontier with diminishing returns to additional clarification. Gains arise not from simply asking more questions overall, but from improved question targeting and better code generation under ambiguity. Without explicit supervision, trained policies learn to allocate clarification budget non-uniformly, asking more frequently on tougher (multi-degradation) tasks. These results suggest that unconstrained interactive LLM systems may systematically use clarification inefficiently.
Figures & tables
Figure 1: Pareto frontier in ( qˉ , pass@1 ) space for all trained policies and the untrained baseline (gray diamond), evaluated on 417 held-out problems under greedy decoding. The log fit illustrates diminishing returns; from d1=1.0 onward Lagrangian enforcement weakens or ceases.
Figure 2: Mean pass@1 by degradation type (417 problems, greedy decoding). d1∈{0.5,0.75,1.0} results averaged across two independent runs. Error bars are ±1 standard error of the mean (SEM).
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Component
Hyperparameter
Value
Policy (LoRA)
Rank r
16
Scale α
32
Trainable parameters
≈ 40M
Learning rate
5×10−6
LR warmup
20 steps (linear)
Value heads
Hidden dim
1024
Appendix
Table 1: Hyperparameters used for all training runs.
Figure 3: Training curves for d1=0.5 .
Figure 4: Training curves for d1=0.75 .
Figure 5: Training curves for d1=1.0 .
Figure 6: Mean number of questions per episode by degradation complexity group, evaluated on the 417-problem held-out set under greedy decoding. Settings match Figure 2 .
Figure 7: pass@1 by degradation type and number of questions asked, for single-degradation types (1a, 1c, 1p). Columns correspond to episodes where 0, exactly 1, or 2+ clarifying questions were asked. Per-run pooling: for policies with two independent runs, each run contributes its episodes independently, classified by that run’s question count, then pooled before computing means. Error bars are ±1 SEM.
Figure 8: pass@1 by degradation type and number of questions asked, for multi-degradation types (2ac, 2ap, 2cp, 3acp). Same layout and pooling as Figure 7 . Cells with very small n (notably 3acp and the 2+ Q column) should be interpreted cautiously.
Large language model post-training adapts pretrained models into more useful, reliable, and specialized assistants through supervised and preference-based learning.
Coding agents can generate code that appears correct but implements behavior the user never intended. This mismatch can arise when an agent silently resolves underspecified requirements through its own assumptions. As subsequent development builds on these assumptions, correcting the resulting behavior can become increasingly costly. Early clarification can help prevent such mismatches, but unnecessary questions can interrupt developers and slow down development. Existing methods struggle to identify key clarification questions while avoiding unnecessary ones. Therefore, we propose CONTRA, a training-free method that combines broad question discovery with semantic and execution-based question qualification. CONTRA first generates candidate questions and filters out those unrelated to required behavior or already resolved by the requirement. For each remaining question, it generates programs conditioned on two plausible answers and checks for stable behavioral differences on shared inputs. It then uses the interaction history to select among qualified questions or stop asking. Experiments on ClarifyCodeBench show that CONTRA achieves the highest F1 with all four coding agents, exceeding the best baseline macro-average F1 by 13.88 percentage points. With the same LLM and evaluation protocol, CONTRA also achieves higher clarification recall and F1 than the coding harnesses Claude Code and OpenHands. To support practical use, we also implement CONTRA as a Claude Code plugin that integrates selective clarification into everyday development.
Instruction-tuned LLMs faced with underspecified queries often commit to a single interpretation rather than ask for clarification, producing confidently wrong answers. In our experiments, prompting alone is insufficient: models either ask for clarification on every query or ask vague questions that fail to recover the missing information. Addressing this failure requires learning two coupled skills: when to ask rather than answer and how to ask a question that recovers the disambiguating information. Existing recipes either address only one of these skills or require a separately trained critic. We propose CIGAsk, an RL recipe that teaches both skills through two complementary reward signals within a multi-turn GRPO loop. Counterfactual Information Gain (CIG) compares the gold-answer log-likelihood under a frozen reference model with and without the user response, providing per-turn credit that guides how to ask. The Asymmetric Ambiguity Bonus assigns a signed reward at the terminal token based on the gold ambiguity label, guiding when to ask. Across three clarification benchmarks spanning table, passage, and open-domain QA, CIGAsk-7B outperforms the strongest external baseline despite using a smaller backbone. It also transfers across datasets without per-dataset tuning while preserving single-turn QA performance on out-of-distribution benchmarks.
Yunxiang Li, Xixin Wu, Helen Meng
The Chinese University of Hong Kong, Hong Kong SAR, China
LLM agents can execute increasingly complex tasks, but effective assistance also requires aligning their actions with goals and constraints that users may leave unstated. We study this problem as interactive intent alignment through clarification. Yet clarification is useful only when the agent can turn the answer into better decisions. As training changes how the agent handles incomplete information and uses additional answers, the same clarification can become more or less useful. We introduce CLEAT (CLarification through valuE-guided Agent co-Training), a framework that co-trains a clarification-value controller and the task policy from shared interaction outcomes. We formulate residual query value to measure the task utility gained by asking rather than proceeding under the current policy, and estimate it through paired rollouts from the same interaction history. These paired returns train the controller to predict clarification value and the task policy to select useful questions and act on the answers. The controller guides subsequent rollout collection, while outcomes from the updated agent provide new value labels that allow clarification guidance to adapt as the agent learns. With Qwen3-4B, CLEAT achieves the highest mean score in all ten evaluation settings across four benchmarks. Gains over the strongest baseline reach 13.2 and 6.5 percentage points on UserGym Travel and IN3 underspecification judgment accuracy, respectively. Ablations show that co-training improves the task policy even when the controller is removed at inference, while adapting clarification guidance to the evolving policy yields further gains.
Jiatan Huang, Zheyuan Zhang, Ziming Li +4
University of Connecticut · University of Notre Dame · University of Massachusetts, Amherst