Coding agents can generate code that appears correct but implements behavior the user never intended. This mismatch can arise when an agent silently resolves underspecified requirements through its own assumptions. As subsequent development builds on these assumptions, correcting the resulting behavior can become increasingly costly. Early clarification can help prevent such mismatches, but unnecessary questions can interrupt developers and slow down development. Existing methods struggle to identify key clarification questions while avoiding unnecessary ones. Therefore, we propose CONTRA, a training-free method that combines broad question discovery with semantic and execution-based question qualification. CONTRA first generates candidate questions and filters out those unrelated to required behavior or already resolved by the requirement. For each remaining question, it generates programs conditioned on two plausible answers and checks for stable behavioral differences on shared inputs. It then uses the interaction history to select among qualified questions or stop asking. Experiments on ClarifyCodeBench show that CONTRA achieves the highest F1 with all four coding agents, exceeding the best baseline macro-average F1 by 13.88 percentage points. With the same LLM and evaluation protocol, CONTRA also achieves higher clarification recall and F1 than the coding harnesses Claude Code and OpenHands. To support practical use, we also implement CONTRA as a Claude Code plugin that integrates selective clarification into everyday development.
Figures & tables
Figure 1: Overview of Contra for selective clarification.
GPT-5-mini
Method
F1 ↑
TKQR ↑
P ↑
R ↑
Direct Prompting
16.86
15.09
20.62
14.25
Ask-or-Assume
17.06
17.00
17.06
17.06
ClarifyGPT
19.92
30.94
13.58
37.37
Contra
35.36
34.32
36.20
34.56
Table 1: Clarification results (%) on ClarifyCodeBench. P and R denote precision and recall. Bold values indicate the best result for each metric and coding agent.
Method
F1 ↑
TKQR ↑
P ↑
R ↑
Direct Prompting
25.30
20.64
41.61
19.28
Ask-or-Assume
27.32
24.93
33.02
23.60
ClarifyGPT
20.37
25.99
16.59
28.29
Contra
41.20
38.32
45.42
38.01
Table 2: Macro-average results (%) across the four agents. Bold values indicate the best result.
Figure 2: Recall and precision across discovery rounds. Each point evaluates all distinct candidates found up to that round. Initial denotes the first generation step.
Figure 3: F1 and TKQR@5 across Contra stages. Before Adaptive Question Selection, F1 evaluates all retained candidates and TKQR@5 evaluates the first five questions in discovery order.
Variant
F1 ↑
TKQR ↑
P ↑
R ↑
Contra
35.36
34.32
36.20
34.56
w/o REL
32.54
32.91
31.27
33.91
w/o RES
29.13
32.98
26.46
32.40
w/o BV
32.58
37.10
28.04
38.88
w/o AQS
26.69
30.70
20.96
36.72
Table 3: Component ablations (%) with GPT-5-mini. Bold marks the best result for each metric.
Method
Gemini 3.1 Pro
GPT-5.5
Direct Prompting
56.10
62.50
Ask-or-Assume
55.49
63.72
ClarifyGPT
53.35
64.02
Contra
57.32
64.63
Table 4: Downstream pass@1 (%) on ClarifyCodeBench. Bold marks the best result.
Method
F1 ↑
TKQR ↑
P ↑
R ↑
FPR ↓
Claude Code
9.63
6.59
20.86
6.26
5.00
OpenHands
10.61
8.99
14.34
8.42
32.50
Contra
19.86
18.79
19.05
20.73
5.00
Table 5: Clarification results (%) with Qwen3.8-27B. Bold marks the best result for each metric.
Figure 4: Efficiency and quality of Contra -Flash. Open/filled markers denote Contra /Flash. Dashed lines show the best baseline for each agent and metric in Table 1 .
Information reported
Claude Code
+ Contra
Unspecified information
4.3
59.7
Alternative interpretation
7.1
47.8
All three elements
2.9
46.3
Table 6: Completeness of reported decisions (%).
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: A completion report can leave implementation choices implicit. Those choices may become consequential later. The plugin makes them available for review while the requirement and change are still in context.
Figure 6: English rendering of the supplied plugin demonstration, condensed for print. It preserves the five decisions, three categories, and test-artifact path. The verification indicator applies to the fifth entry.