User-engaged LLM agents increasingly operate in service scenarios where task success depends on coordination between the agent, the user, and a stateful environment. In such interactions, the agent often has access to rich information that the user does not observe, such as task policies, tool results, and environment states. This makes agent-user communication a central component of task completion. In this work, we study a communication-level failure mode of user-service agents: internal reasoning is counterproductive for the interactive information disclosure. We introduce information disclosure as a measurable communication mechanism, covering state corrections, available options, constraints, consequences, and proactively retrieved environment information. Motivated by recent advances in LLM reasoning, we evaluate and analyse information disclosure under a controlled setting of agents equipped with a "think" tool, which allows agents to reason internally before responding or acting. Across five models and three user-service scenarios, we first show that enforced thinking has unstable utility. We then analyse response scaling under different think efforts and find that more thinking does not reliably translate into richer information disclosure. Using turn-level counterfactual intervention, we show that injecting golden decision-relevant information significantly intervenes agents to correct action trajectories, indicating that missing disclosure can causally affect the agent's immediate downstream behaviour. We further find that failed trajectories most often lack state corrections and constraints, and that a lightweight disclosure-aware prompt effectively mitigates degradation. These results suggest that the key challenge is not simply making agents think more, but ensuring that reasoning and information are externalised as timely information disclosure.
Figures & tables
Figure 1: An overview of the counterfactual intervention design. To handle intervention in multi-turn dialogue processes, the method identifies the last agent’s user decision point before the first environment edit action. Next, the method injects the InfoDis evidence annotated from evidence-grounded probing.
Model
Think Mode
Retail
Airline
TS-Phone
Pass ˆ 1
Pass ˆ 2
Pass ˆ 1
Pass ˆ 2
Pass ˆ 1
Pass ˆ 2
GPT-4o
w/o think
64.06
53.33
43.33
30.00
39.29
30.21
Enforced
59.71
45.51
35.33
19.33
41.52
31.85
GPT-4.1
w/o think
73.33
63.48
44.00
31.33
46.73
37.65
Enforced
65.51
52.46
52.00
40.67
45.24
35.57
GPT-5
w/o think
74.78
64.78
50.67
42.67
24.85
15.77
Table 1: Performance comparison of user-engaged tool-calling agents under different think tool mode. Each setting is run for three trials with random seeds. The pass ˆ k quantifies the chance that all k i.i.d. task trials are successful, averaged across tasks, representing the reliability and consistency of LLM agents.
Figure 2: Response token length and pass ˆ 2 with respect to think effort.
Figure 3: A case of agent-user interaction before the first environment edit action on a retail task (agent: GPT-4o; user: GPT-4.1). The blue highlights the information disclosure of task constraint , which intervenes the next user decision. The thought of agent with enforced think is invisible to the user.
Category
Definition
Example
State Correction (SC)
Information that corrects a factual misunderstanding or false assumption the user has expressed.
The User says "my cancelled order" but agent clarifies "this order was delivered, not cancelled"
Option (OP)
The set of choices available to the user for their upcoming decision — including relevant variants, alternatives, or paths.
“Here are available options for replacement: - … - …”
Constraint (CN)
Rules, policies, eligibility conditions, or limits that affect what the user can or cannot do — and that the user likely does not already know.
“Modifications are only possible before shipment”
Consequence (CQ)
The concrete outcomes that will result from the user’s decision, including financial impact, timing, and irreversible effects.
Refund amount, refund timeline, charge amount, order status change
Proactive Retrieval (PR)
Information the agent proactively fetched from its tools /database and disclosed even though the user did not possess and did not directly ask for.
Listing the user’s orders with their contents, showing both current and new address for comparison
Table 2: Information disclosure dimensions.
Model
Scenario
Failed Trajs
Golden Trajs
Filtered Links
GPT-4o
Retail
73
48
64
GPT-4o
Airline
48
25
44
DeepSeek-V3.1
Retail
91
58
63
DeepSeek-V3.1
Airline
48
27
43
Total
260
158
214
Table 3: Summary of processed data for counterfactual intervention.
Intervention
Retail
Airline
GPT-4o
DeepSeek-V3.1
GPT-4o
DeepSeek-V3.1
FECR ↑
RR ↑
FECR ↑
RR ↑
FECR ↑
RR ↑
FECR ↑
RR ↑
Original
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
+Verbosity
7.09 ± 2.65
21.35 ± 13.64
8.33 ± 1.07
25.93 ± 1.50
4.90 ± 1.39
19.70 ± 1.07
2.94 ± 2.40
4.65 ± 3.29
+Think Content
8.51 ± 0.00
21.88 ± 7.76
8.33 ± 3.86
15.34 ± 5.24
6.86 ± 3.67
18.18 ± 1.86
0.98 ± 1.39
3.10 ± 1.10
+Golden Info
17.73 ± 4.01
33.85 ± 0.74
11.36 ± 3.21
11.64 ± 3.26
6.86 ± 1.39
21.21 ± 3.86
8.82 ± 2.40
11.63 ± 1.90
Table 4: Counterfactual intervention results (FECR (%) and RR (%)) on failure enforce-think agents. Values after ‘ ± ’ denotes standard deviation.
Figure 5: Results of frequency of intervention for different information disclosure dimensions.
Model
Prompt
Retail
Airline
Micro Overall
Pass ˆ 1
Pass ˆ 2
Pass ˆ 1
Pass ˆ 2
Pass ˆ 1
Δ (Pass ˆ 1)
tool-calling agent w/o enforced think tool
GPT-4o
Baseline
64.06
53.33
43.33
30.00
57.78
-
ProCoT
66.38 (+2.32)
55.07 (+1.74)
46.00 (+2.67)
32.00 (+2.00)
60.00
+2.42
InfoDis (ours)
67.25 (+3.19)
54.49 (+1.16)
45.33 (+2.00)
27.33 (-2.67)
60.61
+2.83
DeepSeek-V3.1
Baseline
71.01
59.71
50.00
34.67
64.65
-
Table 7: Task performance of agents with various prompting across three independent trials. Each value is displayed as a percentage. Values in parentheses denote the change against Baseline.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Model
Think Focus
Retail
Airline
Micro Overall
Pass ˆ 1
Pass ˆ 2
Pass ˆ 1
Pass ˆ 2
Pass ˆ 1
Pass ˆ 2
Agent: GPT-4o User: GPT-4.1
Action-centric
56.81
45.22
43.33
26.67
52.73
39.60
User-centric
61.74
50.14
41.33
24.67
55.56
42.42
Agent: GPT-4o User: GPT-5
w/o think
63.48
50.72
39.33
26.00
56.16
43.23
Default
57.97
46.09
36.00
24.00
51.31
39.39
Action-centric
59.42
49.86
36.00
20.67
52.32
41.01
Appendix
Table 8: Performance comparison of different think focus.
Criteria
State Correction
Option
Constraint
Consequence
Proactive Retrieval
Decision Point
w/o evidence
0.8443
0.7766
0.7079
0.8301
0.8363
0.9308
w/ evidence
0.9177
0.8976
0.8681
0.9384
0.9188
0.9362
Appendix
Table 9: Cohen’s κ results on self-consistency of LLM-as-a-judge annotation.