CLEAT: Learning Value-Guided Clarification for Interactive Agents

Jiatan Huang; Zheyuan Zhang; Ziming Li; Tianyi Ma; Mingchen Li; Yanfang Ye; Chuxu Zhang

cs.AIOct 8, 2026

CLEAT: Learning Value-Guided Clarification for Interactive Agents

Authors: Jiatan Huang, Zheyuan Zhang, Ziming Li, Tianyi Ma, Mingchen Li, Yanfang Ye, Chuxu Zhang

Organizations: University of Connecticut · University of Notre Dame · University of Massachusetts, Amherst

Abstract

LLM agents can execute increasingly complex tasks, but effective assistance also requires aligning their actions with goals and constraints that users may leave unstated. We study this problem as interactive intent alignment through clarification. Yet clarification is useful only when the agent can turn the answer into better decisions. As training changes how the agent handles incomplete information and uses additional answers, the same clarification can become more or less useful. We introduce CLEAT (CLarification through valuE-guided Agent co-Training), a framework that co-trains a clarification-value controller and the task policy from shared interaction outcomes. We formulate residual query value to measure the task utility gained by asking rather than proceeding under the current policy, and estimate it through paired rollouts from the same interaction history. These paired returns train the controller to predict clarification value and the task policy to select useful questions and act on the answers. The controller guides subsequent rollout collection, while outcomes from the updated agent provide new value labels that allow clarification guidance to adapt as the agent learns. With Qwen3-4B, CLEAT achieves the highest mean score in all ten evaluation settings across four benchmarks. Gains over the strongest baseline reach 13.2 and 6.5 percentage points on UserGym Travel and IN3 underspecification judgment accuracy, respectively. Ablations show that co-training improves the task policy even when the controller is removed at inference, while adapting clarification guidance to the evolving policy yields further gains.

Figures & tables

Figure 1: Interactive intent alignment. CLEAT targets questions that improve task outcomes, not merely reduce uncertainty.

Figure 2: Overview of CLEAT . Value-guided paired rollouts train the controller and task policy from shared outcomes. At inference, the controller gates clarification by predicted incremental utility.

Prompting-based
Method	UserGym				$\bm{\tau^{2}}$ -Bench			IN3	AskBench
Method	Travel	Turtle	Inten.	Tele.	Air.	Ret.	Telec.	Judg.Acc	AskMind	AskOv.
Vanilla	26.74 $\pm$ 1.49	17.97 $\pm$ 1.73	185.50 $\pm$ 2.00	49.12 $\pm$ 1.87	23.78 $\pm$ 2.89	20.89 $\pm$ 2.61	20.00 $\pm$ 1.25	65.74 $\pm$ 1.54	38.70 $\pm$ 1.27	57.24 $\pm$ 1.18
ReAct	28.66 $\pm$ 1.87	19.93 $\pm$ 1.63	192.17 $\pm$ 3.81	53.66 $\pm$ 2.44	26.67 $\pm$ 2.89	24.54 $\pm$ 1.29	20.83 $\pm$ 2.77	70.99 $\pm$ 1.42	44.34 $\pm$ 2.33	56.55 $\pm$ 1.18
Reflexion	29.34 $\pm$ 1.76	21.70 $\pm$ 3.96	195.67 $\pm$ 3.95	52.03 $\pm$ 4.28	26.67 $\pm$ 1.55	24.26 $\pm$ 1.89	21.33 $\pm$ 2.20	71.61 $\pm$ 4.18	45.01 $\pm$ 1.59	58.75 $\pm$ 1.52
Proactive CoT	27.01 $\pm$ 0.32	23.96 $\pm$ 4.06	176.17 $\pm$ 3.41	52.97 $\pm$ 2.84	25.44 $\pm$ 3.03	27.50 $\pm$ 2.50	20.83 $\pm$ 1.44	69.14 $\pm$ 2.33	42.58 $\pm$ 0.57	57.83 $\pm$ 1.23

Table 1: Task performance with Qwen3-4B is reported as mean

\pm

std over 3 seeds (percentages except cumulative Intention credits), bold and underlined values indicate the best and second-best means, respectively.

Figure 3: Effects of rollout guidance (top, gains over unguided collection in percentage points) and component ablations (bottom, mean

\pm

std over 3 seeds) on task performance.

Figure 4: Instruction recovery on IN3 and complete clarification coverage on AskBench measure how fully agents resolve user requests (mean

\pm

std over 3 seeds).

Figure 5: Answer accuracy and clarification coverage on AskBench, with gains over the stronger baseline in points.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Require: task policy $\pi_{0}$ , updates $T$ , budgets $(K_{p},K_{g},K_{c})$
1:	Bootstrap $f_{0}$ on paired rollouts collected with $\mu=\bar{p}_{\pi_{0}}$
2:	for $t=0,1,\ldots,T-1$ do
3:	Sample a batch $\mathcal{B}_{t}$ of clarification-eligible states $(h,G)$
4:	for each $(h,G)\in\mathcal{B}_{t}$ do
5:	$\mu_{t}(z\mid h)\propto\bar{p}_{\pi_{t}}(z\mid h)\exp\{\beta f_{t}(h,z)\}$
6:	Draw guided target $z_{g}\sim\mu_{t}(\cdot\mid h)$

Appendix

Algorithm 1 Online co-training in CLEAT

Component	Parameter	Setting
Task policy	LoRA rank / scaling $\alpha$	$16$ / $32$
	Adapted modules	Attention and MLP projections
	Optimizer / learning rate	AdamW / $2\times 10^{-5}$
	Policy-ratio clipping	$0.2$
	Reference-KL coefficient	$0.01$
	Maximum context length	$8{,}192$ tokens

Appendix

Table 2: Shared training configuration of CLEAT .

Deployment-time ablation
Variant	UserGym		$\bm{\tau^{2}}$ -Bench			AskBench
Variant	Travel	Turtle	Airline	Retail	Telecom	AskOv.
CLEAT (full)	63.72 $\pm$ 0.48	29.79 $\pm$ 1.15	35.00 $\pm$ 2.00	32.50 $\pm$ 1.29	24.94 $\pm$ 1.77	61.27 $\pm$ 2.19
w/o controller	59.49 $\pm$ 0.59 [-1pt] ( $-$ 4.23)	28.89 $\pm$ 1.16 [-1pt] ( $-$ 0.90)	28.86 $\pm$ 3.86 [-1pt] ( $-$ 6.14)	28.50 $\pm$ 2.66 [-1pt] ( $-$ 4.00)	24.50 $\pm$ 2.00 [-1pt] ( $-$ 0.44)	59.08 $\pm$ 1.38 [-1pt] ( $-$ 2.19)
Training-time ablations
w/o co-training	61.77 $\pm$ 0.62 [-1pt] ( $-$ 1.95)	27.19 $\pm$ 1.41 [-1pt] ( $-$ 2.60)	28.30 $\pm$ 3.19 [-1pt] ( $-$ 6.70)	30.00 $\pm$ 2.61 [-1pt] ( $-$ 2.50)	23.67 $\pm$ 2.29 [-1pt] ( $-$ 1.27)	59.25 $\pm$ 1.95 [-1pt] ( $-$ 2.02)

Appendix

Table 3: Ablation of CLEAT (mean

\pm

std over 3 seeds, Qwen3-4B ). Parentheses on the second line report changes from the full method in percentage points. Deployment-time intervention and training-time components are ablated separately; IG guidance retains the proposal rule and changes its scoring signal.

Method	Travel	IN3		AskOv.
	Pass-1	Recover	Judg.Acc	Acc	Cov
Vanilla	26.74	54.26	65.74	57.24	40.98
Menu-Vanilla	28.35	55.13	66.18	57.78	41.75
GRPO	42.26	53.68	66.05	58.67	43.50
Menu-GRPO	44.83	58.47	70.03	59.17	44.58
w/o guidance	56.17	64.49	78.95	57.28	47.70

Appendix

Table 4: Menu-augmented variants (Qwen3-4B; mean scores over 3 seeds, %).

Model	UserGym				$\bm{\tau^{2}}$ -Bench			IN3	AskBench
Model	Travel	Turtle	Inten.	Tele.	Air.	Ret.	Telec.	Judg.Acc	AskMind	AskOv.
Qwen3-4B	26.74 $\pm$ 1.49	17.97 $\pm$ 1.73	185.50 $\pm$ 2.00	49.12 $\pm$ 1.87	23.78 $\pm$ 2.89	20.89 $\pm$ 2.61	20.00 $\pm$ 1.25	65.74 $\pm$ 1.54	38.70 $\pm$ 1.27	57.24 $\pm$ 1.18
Qwen3-8B	42.90 $\pm$ 2.41	35.62 $\pm$ 2.25	189.42 $\pm$ 2.14	52.85 $\pm$ 2.84	28.33 $\pm$ 2.89	31.47 $\pm$ 3.20	23.78 $\pm$ 2.89	66.97 $\pm$ 2.83	45.23 $\pm$ 0.50	61.08 $\pm$ 1.26
Qwen3-14B	55.69 $\pm$ 1.56	50.27 $\pm$ 2.20	191.25 $\pm$ 2.79	53.66 $\pm$ 1.79	30.33 $\pm$ 3.41	35.67 $\pm$ 1.55	24.33 $\pm$ 2.00	70.06 $\pm$ 2.83	43.58 $\pm$ 1.79	63.58 $\pm$ 2.13
CLEAT (Qwen3-4B)	63.72 $\pm$ 0.48	29.79 $\pm$ 1.15	209.17 $\pm$ 3.52	57.67 $\pm$ 2.45	35.00 $\pm$ 2.00	32.50 $\pm$ 1.29	24.94 $\pm$ 1.77	83.95 $\pm$ 1.42	48.37 $\pm$ 2.66	61.27 $\pm$ 2.19

Appendix

Table 5: Comparison with larger backbones. Qwen3-4B, 8B, and 14B without additional task-specific training are compared with CLEAT on Qwen3-4B (mean

\pm

std over 3 seeds). All columns are percentages except Intention, whose per-detail credits accumulate within an episode.

Qwen3-1.7B
Method	AskMind		AskOverconfidence
Method	Acc	Cov	Acc	Cov
Backbone	31.93 $\pm$ 0.93	24.54 $\pm$ 1.20	47.79 $\pm$ 1.04	26.95 $\pm$ 0.88
GRPO	27.66 $\pm$ 0.36	22.97 $\pm$ 0.83	48.50 $\pm$ 0.90	28.92 $\pm$ 1.53
CLEAT	36.89 $\pm$ 1.34	27.56 $\pm$ 1.34	49.00 $\pm$ 1.00	36.17 $\pm$ 1.28
Qwen3-4B

Appendix

Table 6: Accuracy and coverage across model sizes (mean

\pm

std over 3 seeds).

Prompting-based
Method	IN3	AskBench
Method	Recover	AskMind Cov	AskOv. Cov
Vanilla	54.26 $\pm$ 2.21	24.23 $\pm$ 0.75	40.98 $\pm$ 2.02
ReAct	59.62 $\pm$ 3.51	23.97 $\pm$ 2.36	39.64 $\pm$ 1.34
Reflexion	57.94 $\pm$ 3.36	26.57 $\pm$ 0.62	39.74 $\pm$ 2.03
Proactive CoT	55.23 $\pm$ 0.42	24.24 $\pm$ 2.40	41.12 $\pm$ 0.68

Appendix

Table 7: Clarification-specific metrics on Qwen3-4B (mean

\pm

std over 3 seeds).

Method	AskMind		AskOv.		IN3
Method	Acc	Cov	Acc	Cov	Recover	Judg.Acc
ACT	45.30 $\pm$ 0.96	26.51 $\pm$ 0.56	59.00 $\pm$ 1.38	42.33 $\pm$ 1.59	51.17 $\pm$ 0.59	66.05 $\pm$ 1.42
CollabLLM	44.84 $\pm$ 1.53	25.86 $\pm$ 2.48	58.08 $\pm$ 1.38	39.83 $\pm$ 1.42	56.44 $\pm$ 3.24	67.90 $\pm$ 3.50
CLEAT	48.37 $\pm$ 2.66	34.87 $\pm$ 2.20	61.27 $\pm$ 2.19	51.49 $\pm$ 0.58	67.58 $\pm$ 3.80	83.95 $\pm$ 1.42

Appendix

Table 8: Additional comparisons with Qwen3-4B (mean

\pm

std over 3 seeds; all metrics in %).

Figure 7: Vague ask rate versus overall judgment accuracy on IN3. The star denotes CLEAT .

Figure 8: Training diagnostics for CLEAT on AskOverconfidence (top) and AskMind (bottom), with raw per-step values in light and a moving average in dark. Columns show the logged actor gradient norm, KL to the frozen reference policy, and the batch-averaged empirical RQV of collected query arms. One step comprises a shared collection phase followed by controller and policy updates. Empirical RQV is the paired return difference

\bar{U}_{\textsc{query}(z)}-\bar{U}_{\textsc{pass}}

from Equation 3 .

LLM Agents

Large language model agents use language models to plan, use tools, and act across multi-step tasks and environments.

See all 658 papers

Uncertainty-Aware Clarification in LLM Agents with Information Gain
Jun 2, 2026Mengyi Deng, Zhiwei Li, Xin Li +4Clarification Question Generation Tool-Using Agents
Information Hub, HKUST(GZ) · Jilin University · et al.
When and How Should an Agent Clarify? CIGAsk: Teaching LLMs to Clarify via Counterfactual Information Gain
Sep 21, 2026Yunxiang Li, Xixin Wu, Helen MengClarification Question Generation LLM Agent Training
The Chinese University of Hong Kong, Hong Kong SAR, China
A Tri-Agent Framework for Evaluating and Aligning Question Clarification Capabilities of Large Language Models
Sep 2, 2026Yikai Zhao, Saurabh Pandey, Pradeep Kumar MisraMulti-Agent LLM Systems Clarification Question Generation
Amazon Dallas, Texas, USA · Amazon Seattle, Washington, USA

Jun 2, 2026J/K move · Enter open · S save

Uncertainty-Aware Clarification in LLM Agents with Information Gain

Mengyi Deng, Zhiwei Li, Xin Li, Tingyu Zhu+3

Information Hub, HKUST(GZ) · Jilin University · et al.

Clarification Question Generation Tool-Using Agents LLM Uncertainty Estimation LLM Agent Training

Open paper AskPDF arXiv

Large Language Model (LLM) agents often operate under underspecified user instructions, where latent uncertainty over user intent leads to erroneous tool actions. To address this challenge, we propose a goal-oriented clarification framework that aligns clarification behavior with ambiguity resolution. Central to our approach is the Information Gain Reward, a metric that quantifies the utility of clarification questions by measuring the Bayesian belief update towards the ground-truth goal induced by the clarification exchange. We train the clarifier (LLM) using this reward to optimize for high information gain, ensuring that clarifications effectively reduce uncertainty and improve task completion within the agent-tool-user environment. We validate our framework within a clarification-enhanced

τ

-Bench environment, conducting cross-agent evaluations across five heterogeneous backbones. Empirical results demonstrate that our method consistently improves the success rate by 3.7% over the no-clarification baseline, while adding only 0.3 total interaction steps on average.

Abstract

LLM Agents

Explore similar work

Uncertainty-Aware Clarification in LLM Agents with Information Gain

When and How Should an Agent Clarify? CIGAsk: Teaching LLMs to Clarify via Counterfactual Information Gain

A Tri-Agent Framework for Evaluating and Aligning Question Clarification Capabilities of Large Language Models