LLM agents can execute increasingly complex tasks, but effective assistance also requires aligning their actions with goals and constraints that users may leave unstated. We study this problem as interactive intent alignment through clarification. Yet clarification is useful only when the agent can turn the answer into better decisions. As training changes how the agent handles incomplete information and uses additional answers, the same clarification can become more or less useful. We introduce CLEAT (CLarification through valuE-guided Agent co-Training), a framework that co-trains a clarification-value controller and the task policy from shared interaction outcomes. We formulate residual query value to measure the task utility gained by asking rather than proceeding under the current policy, and estimate it through paired rollouts from the same interaction history. These paired returns train the controller to predict clarification value and the task policy to select useful questions and act on the answers. The controller guides subsequent rollout collection, while outcomes from the updated agent provide new value labels that allow clarification guidance to adapt as the agent learns. With Qwen3-4B, CLEAT achieves the highest mean score in all ten evaluation settings across four benchmarks. Gains over the strongest baseline reach 13.2 and 6.5 percentage points on UserGym Travel and IN3 underspecification judgment accuracy, respectively. Ablations show that co-training improves the task policy even when the controller is removed at inference, while adapting clarification guidance to the evolving policy yields further gains.
Figures & tables
Figure 1: Interactive intent alignment. CLEAT targets questions that improve task outcomes, not merely reduce uncertainty.
Figure 2: Overview of CLEAT . Value-guided paired rollouts train the controller and task policy from shared outcomes. At inference, the controller gates clarification by predicted incremental utility.
Method
UserGym
τ2 -Bench
IN3
AskBench
Travel
Turtle
Inten.
Tele.
Air.
Ret.
Telec.
Judg.Acc
AskMind
AskOv.
Prompting-based
Vanilla
26.74 ± 1.49
17.97 ± 1.73
185.50 ± 2.00
49.12 ± 1.87
23.78 ± 2.89
20.89 ± 2.61
20.00 ± 1.25
65.74 ± 1.54
38.70 ± 1.27
57.24 ± 1.18
ReAct
28.66 ± 1.87
19.93 ± 1.63
192.17 ± 3.81
53.66 ± 2.44
26.67 ± 2.89
24.54 ± 1.29
20.83 ± 2.77
70.99 ± 1.42
44.34 ± 2.33
56.55 ± 1.18
Reflexion
29.34 ± 1.76
21.70 ± 3.96
195.67 ± 3.95
52.03 ± 4.28
26.67 ± 1.55
24.26 ± 1.89
21.33 ± 2.20
71.61 ± 4.18
45.01 ± 1.59
58.75 ± 1.52
Proactive CoT
27.01 ± 0.32
23.96 ± 4.06
176.17 ± 3.41
52.97 ± 2.84
25.44 ± 3.03
27.50 ± 2.50
20.83 ± 1.44
69.14 ± 2.33
42.58 ± 0.57
57.83 ± 1.23
Table 1: Task performance with Qwen3-4B is reported as mean ± std over 3 seeds (percentages except cumulative Intention credits), bold and underlined values indicate the best and second-best means, respectively.
Figure 3: Effects of rollout guidance (top, gains over unguided collection in percentage points) and component ablations (bottom, mean ± std over 3 seeds) on task performance.
Figure 4: Instruction recovery on IN3 and complete clarification coverage on AskBench measure how fully agents resolve user requests (mean ± std over 3 seeds).
Figure 5: Answer accuracy and clarification coverage on AskBench, with gains over the stronger baseline in points.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Require: task policy π0 , updates T , budgets (Kp,Kg,Kc)
1:
Bootstrap f0 on paired rollouts collected with μ=pˉπ0
2:
for t=0,1,…,T−1 do
3:
Sample a batch Bt of clarification-eligible states (h,G)
4:
for each (h,G)∈Bt do
5:
μt(z∣h)∝pˉπt(z∣h)exp{βft(h,z)}
6:
Draw guided target zg∼μt(⋅∣h)
Appendix
Algorithm 1 Online co-training in CLEAT
Component
Parameter
Setting
Task policy
LoRA rank / scaling α
16 / 32
Adapted modules
Attention and MLP projections
Optimizer / learning rate
AdamW / 2×10−5
Policy-ratio clipping
0.2
Reference-KL coefficient
0.01
Maximum context length
8,192 tokens
Appendix
Table 2: Shared training configuration of CLEAT .
Variant
UserGym
τ2 -Bench
AskBench
Travel
Turtle
Airline
Retail
Telecom
AskOv.
CLEAT (full)
63.72 ± 0.48
29.79 ± 1.15
35.00 ± 2.00
32.50 ± 1.29
24.94 ± 1.77
61.27 ± 2.19
Deployment-time ablation
w/o controller
59.49 ± 0.59 [-1pt] ( − 4.23)
28.89 ± 1.16 [-1pt] ( − 0.90)
28.86 ± 3.86 [-1pt] ( − 6.14)
28.50 ± 2.66 [-1pt] ( − 4.00)
24.50 ± 2.00 [-1pt] ( − 0.44)
59.08 ± 1.38 [-1pt] ( − 2.19)
Training-time ablations
w/o co-training
61.77 ± 0.62 [-1pt] ( − 1.95)
27.19 ± 1.41 [-1pt] ( − 2.60)
28.30 ± 3.19 [-1pt] ( − 6.70)
30.00 ± 2.61 [-1pt] ( − 2.50)
23.67 ± 2.29 [-1pt] ( − 1.27)
59.25 ± 1.95 [-1pt] ( − 2.02)
Appendix
Table 3: Ablation of CLEAT (mean ± std over 3 seeds, Qwen3-4B ). Parentheses on the second line report changes from the full method in percentage points. Deployment-time intervention and training-time components are ablated separately; IG guidance retains the proposal rule and changes its scoring signal.
Method
Travel
IN3
AskOv.
Pass-1
Recover
Judg.Acc
Acc
Cov
Vanilla
26.74
54.26
65.74
57.24
40.98
Menu-Vanilla
28.35
55.13
66.18
57.78
41.75
GRPO
42.26
53.68
66.05
58.67
43.50
Menu-GRPO
44.83
58.47
70.03
59.17
44.58
w/o guidance
56.17
64.49
78.95
57.28
47.70
Appendix
Table 4: Menu-augmented variants (Qwen3-4B; mean scores over 3 seeds, %).
Model
UserGym
τ2 -Bench
IN3
AskBench
Travel
Turtle
Inten.
Tele.
Air.
Ret.
Telec.
Judg.Acc
AskMind
AskOv.
Qwen3-4B
26.74 ± 1.49
17.97 ± 1.73
185.50 ± 2.00
49.12 ± 1.87
23.78 ± 2.89
20.89 ± 2.61
20.00 ± 1.25
65.74 ± 1.54
38.70 ± 1.27
57.24 ± 1.18
Qwen3-8B
42.90 ± 2.41
35.62 ± 2.25
189.42 ± 2.14
52.85 ± 2.84
28.33 ± 2.89
31.47 ± 3.20
23.78 ± 2.89
66.97 ± 2.83
45.23 ± 0.50
61.08 ± 1.26
Qwen3-14B
55.69 ± 1.56
50.27 ± 2.20
191.25 ± 2.79
53.66 ± 1.79
30.33 ± 3.41
35.67 ± 1.55
24.33 ± 2.00
70.06 ± 2.83
43.58 ± 1.79
63.58 ± 2.13
CLEAT (Qwen3-4B)
63.72 ± 0.48
29.79 ± 1.15
209.17 ± 3.52
57.67 ± 2.45
35.00 ± 2.00
32.50 ± 1.29
24.94 ± 1.77
83.95 ± 1.42
48.37 ± 2.66
61.27 ± 2.19
Appendix
Table 5: Comparison with larger backbones. Qwen3-4B, 8B, and 14B without additional task-specific training are compared with CLEAT on Qwen3-4B (mean ± std over 3 seeds). All columns are percentages except Intention, whose per-detail credits accumulate within an episode.
Method
AskMind
AskOverconfidence
Acc
Cov
Acc
Cov
Qwen3-1.7B
Backbone
31.93 ± 0.93
24.54 ± 1.20
47.79 ± 1.04
26.95 ± 0.88
GRPO
27.66 ± 0.36
22.97 ± 0.83
48.50 ± 0.90
28.92 ± 1.53
CLEAT
36.89 ± 1.34
27.56 ± 1.34
49.00 ± 1.00
36.17 ± 1.28
Qwen3-4B
Appendix
Table 6: Accuracy and coverage across model sizes (mean ± std over 3 seeds).
Method
IN3
AskBench
Recover
AskMind Cov
AskOv. Cov
Prompting-based
Vanilla
54.26 ± 2.21
24.23 ± 0.75
40.98 ± 2.02
ReAct
59.62 ± 3.51
23.97 ± 2.36
39.64 ± 1.34
Reflexion
57.94 ± 3.36
26.57 ± 0.62
39.74 ± 2.03
Proactive CoT
55.23 ± 0.42
24.24 ± 2.40
41.12 ± 0.68
Appendix
Table 7: Clarification-specific metrics on Qwen3-4B (mean ± std over 3 seeds).
Method
AskMind
AskOv.
IN3
Acc
Cov
Acc
Cov
Recover
Judg.Acc
ACT
45.30 ± 0.96
26.51 ± 0.56
59.00 ± 1.38
42.33 ± 1.59
51.17 ± 0.59
66.05 ± 1.42
CollabLLM
44.84 ± 1.53
25.86 ± 2.48
58.08 ± 1.38
39.83 ± 1.42
56.44 ± 3.24
67.90 ± 3.50
CLEAT
48.37 ± 2.66
34.87 ± 2.20
61.27 ± 2.19
51.49 ± 0.58
67.58 ± 3.80
83.95 ± 1.42
Appendix
Table 8: Additional comparisons with Qwen3-4B (mean ± std over 3 seeds; all metrics in %).
Figure 7: Vague ask rate versus overall judgment accuracy on IN3. The star denotes CLEAT .
Figure 8: Training diagnostics for CLEAT on AskOverconfidence (top) and AskMind (bottom), with raw per-step values in light and a moving average in dark. Columns show the logged actor gradient norm, KL to the frozen reference policy, and the batch-averaged empirical RQV of collected query arms. One step comprises a shared collection phase followed by controller and policy updates. Empirical RQV is the paired return difference Uˉ\textscquery(z)−Uˉ\textscpass from Equation 3 .
Large Language Model (LLM) agents often operate under underspecified user instructions, where latent uncertainty over user intent leads to erroneous tool actions. To address this challenge, we propose a goal-oriented clarification framework that aligns clarification behavior with ambiguity resolution. Central to our approach is the Information Gain Reward, a metric that quantifies the utility of clarification questions by measuring the Bayesian belief update towards the ground-truth goal induced by the clarification exchange. We train the clarifier (LLM) using this reward to optimize for high information gain, ensuring that clarifications effectively reduce uncertainty and improve task completion within the agent-tool-user environment. We validate our framework within a clarification-enhanced τ-Bench environment, conducting cross-agent evaluations across five heterogeneous backbones. Empirical results demonstrate that our method consistently improves the success rate by 3.7% over the no-clarification baseline, while adding only 0.3 total interaction steps on average.
Mengyi Deng, Zhiwei Li, Xin Li +4
Information Hub, HKUST(GZ) · Jilin University · et al.
Instruction-tuned LLMs faced with underspecified queries often commit to a single interpretation rather than ask for clarification, producing confidently wrong answers. In our experiments, prompting alone is insufficient: models either ask for clarification on every query or ask vague questions that fail to recover the missing information. Addressing this failure requires learning two coupled skills: when to ask rather than answer and how to ask a question that recovers the disambiguating information. Existing recipes either address only one of these skills or require a separately trained critic. We propose CIGAsk, an RL recipe that teaches both skills through two complementary reward signals within a multi-turn GRPO loop. Counterfactual Information Gain (CIG) compares the gold-answer log-likelihood under a frozen reference model with and without the user response, providing per-turn credit that guides how to ask. The Asymmetric Ambiguity Bonus assigns a signed reward at the terminal token based on the gold ambiguity label, guiding when to ask. Across three clarification benchmarks spanning table, passage, and open-domain QA, CIGAsk-7B outperforms the strongest external baseline despite using a smaller backbone. It also transfers across datasets without per-dataset tuning while preserving single-turn QA performance on out-of-distribution benchmarks.
Yunxiang Li, Xixin Wu, Helen Meng
The Chinese University of Hong Kong, Hong Kong SAR, China
Large Language Models (LLMs) are increasingly deployed in interactive systems where understanding user intent precisely is paramount. A key capability for such systems is effective question clarification, especially when user queries are ambiguous or underspecified. This paper introduces a novel tri-agent framework for the robust evaluation of an LLM's ability to engage in clarifying dialogue. Our framework comprises three distinct LLM-based agents: (1) a Question Clarifying Agent (QCA), the system under evaluation, tasked with identifying ambiguities and posing clarifying questions; (2) a Respondent Agent (RA), designed to simulate human user responses, potentially including irrelevant or challenging replies; and (3) an Evaluator Agent (EA), an LLM-as-a-judge, which assesses the quality of the dialogue based on a comprehensive set of metrics. We detail a methodology for synthetic data generation in the supply chain domain as an example. We propose metrics evaluating ambiguity handling, question quality, dialogue efficiency, language appropriateness, and final intent alignment. We also briefly discuss the validation of the EA against human judgments. This work provides a structured approach to benchmark, validate, and improve the clarification capabilities of conversational LLM applications.
Yikai Zhao, Saurabh Pandey, Pradeep Kumar Misra
Amazon Dallas, Texas, USA · Amazon Seattle, Washington, USA