Thinking Is Not Telling: Information Disclosure in User-Service LLM Agents
Organizations: University of Wisconsin–Madison · William & Mary
Abstract
User-engaged LLM agents increasingly operate in service scenarios where task success depends on coordination between the agent, the user, and a stateful environment. In such interactions, the agent often has access to rich information that the user does not observe, such as task policies, tool results, and environment states. This makes agent-user communication a central component of task completion. In this work, we study a communication-level failure mode of user-service agents: internal reasoning is counterproductive for the interactive information disclosure. We introduce information disclosure as a measurable communication mechanism, covering state corrections, available options, constraints, consequences, and proactively retrieved environment information. Motivated by recent advances in LLM reasoning, we evaluate and analyse information disclosure under a controlled setting of agents equipped with a "think" tool, which allows agents to reason internally before responding or acting. Across five models and three user-service scenarios, we first show that enforced thinking has unstable utility. We then analyse response scaling under different think efforts and find that more thinking does not reliably translate into richer information disclosure. Using turn-level counterfactual intervention, we show that injecting golden decision-relevant information significantly intervenes agents to correct action trajectories, indicating that missing disclosure can causally affect the agent's immediate downstream behaviour. We further find that failed trajectories most often lack state corrections and constraints, and that a lightweight disclosure-aware prompt effectively mitigates degradation. These results suggest that the key challenge is not simply making agents think more, but ensuring that reasoning and information are externalised as timely information disclosure.
Figures & tables
| Model | Think Mode | Retail | Airline | TS-Phone | |||
|---|---|---|---|---|---|---|---|
| Pass ˆ 1 | Pass ˆ 2 | Pass ˆ 1 | Pass ˆ 2 | Pass ˆ 1 | Pass ˆ 2 | ||
| GPT-4o | w/o think | 64.06 | 53.33 | 43.33 | 30.00 | 39.29 | 30.21 |
| Enforced | 59.71 | 45.51 | 35.33 | 19.33 | 41.52 | 31.85 | |
| GPT-4.1 | w/o think | 73.33 | 63.48 | 44.00 | 31.33 | 46.73 | 37.65 |
| Enforced | 65.51 | 52.46 | 52.00 | 40.67 | 45.24 | 35.57 | |
| GPT-5 | w/o think | 74.78 | 64.78 | 50.67 | 42.67 | 24.85 | 15.77 |
| Category | Definition | Example |
|---|---|---|
| State Correction (SC) | Information that corrects a factual misunderstanding or false assumption the user has expressed. | The User says "my cancelled order" but agent clarifies "this order was delivered, not cancelled" |
| Option (OP) | The set of choices available to the user for their upcoming decision — including relevant variants, alternatives, or paths. | “Here are available options for replacement: - … - …” |
| Constraint (CN) | Rules, policies, eligibility conditions, or limits that affect what the user can or cannot do — and that the user likely does not already know. | “Modifications are only possible before shipment” |
| Consequence (CQ) | The concrete outcomes that will result from the user’s decision, including financial impact, timing, and irreversible effects. | Refund amount, refund timeline, charge amount, order status change |
| Proactive Retrieval (PR) | Information the agent proactively fetched from its tools /database and disclosed even though the user did not possess and did not directly ask for. | Listing the user’s orders with their contents, showing both current and new address for comparison |
| Model | Scenario | Failed Trajs | Golden Trajs | Filtered Links |
|---|---|---|---|---|
| GPT-4o | Retail | 73 | 48 | 64 |
| GPT-4o | Airline | 48 | 25 | 44 |
| DeepSeek-V3.1 | Retail | 91 | 58 | 63 |
| DeepSeek-V3.1 | Airline | 48 | 27 | 43 |
| Total | 260 | 158 | 214 | |
| Intervention | Retail | Airline | ||||||
|---|---|---|---|---|---|---|---|---|
| GPT-4o | DeepSeek-V3.1 | GPT-4o | DeepSeek-V3.1 | |||||
| FECR | RR | FECR | RR | FECR | RR | FECR | RR | |
| Original | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 |
| +Verbosity | 7.09 2.65 | 21.35 13.64 | 8.33 1.07 | 25.93 1.50 | 4.90 1.39 | 19.70 1.07 | 2.94 2.40 | 4.65 3.29 |
| +Think Content | 8.51 0.00 | 21.88 7.76 | 8.33 3.86 | 15.34 5.24 | 6.86 3.67 | 18.18 1.86 | 0.98 1.39 | 3.10 1.10 |
| +Golden Info | 17.73 4.01 | 33.85 0.74 | 11.36 3.21 | 11.64 3.26 | 6.86 1.39 | 21.21 3.86 | 8.82 2.40 | 11.63 1.90 |
| Model | Prompt | Retail | Airline | Micro Overall | |||
| Pass ˆ 1 | Pass ˆ 2 | Pass ˆ 1 | Pass ˆ 2 | Pass ˆ 1 | (Pass ˆ 1) | ||
| tool-calling agent w/o enforced think tool | |||||||
| GPT-4o | Baseline | 64.06 | 53.33 | 43.33 | 30.00 | 57.78 | - |
| ProCoT | 66.38 (+2.32) | 55.07 (+1.74) | 46.00 (+2.67) | 32.00 (+2.00) | 60.00 | +2.42 | |
| InfoDis (ours) | 67.25 (+3.19) | 54.49 (+1.16) | 45.33 (+2.00) | 27.33 (-2.67) | 60.61 | +2.83 | |
| DeepSeek-V3.1 | Baseline | 71.01 | 59.71 | 50.00 | 34.67 | 64.65 | - |
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
| Model | Think Focus | Retail | Airline | Micro Overall | |||
|---|---|---|---|---|---|---|---|
| Pass ˆ 1 | Pass ˆ 2 | Pass ˆ 1 | Pass ˆ 2 | Pass ˆ 1 | Pass ˆ 2 | ||
| Agent: GPT-4o User: GPT-4.1 | Action-centric | 56.81 | 45.22 | 43.33 | 26.67 | 52.73 | 39.60 |
| User-centric | 61.74 | 50.14 | 41.33 | 24.67 | 55.56 | 42.42 | |
| Agent: GPT-4o User: GPT-5 | w/o think | 63.48 | 50.72 | 39.33 | 26.00 | 56.16 | 43.23 |
| Default | 57.97 | 46.09 | 36.00 | 24.00 | 51.31 | 39.39 | |
| Action-centric | 59.42 | 49.86 | 36.00 | 20.67 | 52.32 | 41.01 | |
| Criteria | State Correction | Option | Constraint | Consequence | Proactive Retrieval | Decision Point |
|---|---|---|---|---|---|---|
| w/o evidence | 0.8443 | 0.7766 | 0.7079 | 0.8301 | 0.8363 | 0.9308 |
| w/ evidence | 0.9177 | 0.8976 | 0.8681 | 0.9384 | 0.9188 | 0.9362 |
Human-AI Interaction
Human-AI interaction studies how people and AI systems communicate, collaborate, make decisions, and shape one another over time.
See all 354 papers