Reinforcement learning has improved the tool-use capabilities of large language models. However, better task performance can coincide with increased tool hallucination. Reliable tool use requires decisions to follow the current task state: available information, tool capabilities, and task progress. Our analysis shows that agents sometimes execute when they should ask for information, retrieve it, or decline an infeasible request, even when reasoning identifies missing information. We introduce StateAlign, combining task-state construction with state-contrastive reinforcement learning. The data cover five task conditions and pair states with different action requirements. Outcome rewards guide task completion; token-level contrasts compare the same generated reasoning and actions under paired states to train outputs that fit the current state. Experiments across five base models from three families show improvements on BFCL v4 multi-turn, ToolSandbox, and When2Call. Ablations show that state construction improves performance and reliability under different reward designs. The complete method further reduces tool hallucination across three Qwen3 sizes. On Qwen3-8B, StateAlign improves BFCL v4 multi-turn accuracy from 45.5% to 55.2% and reduces no-tool hallucination on SimpleToolHalluBench from 62.2% under outcome-only training to 56.1%. Behavioral analyses show more consistent decisions, more complete multi-call execution, and stronger recovery after tool errors.
Figures & tables
Figure 1: Comparison of tool-use performance and reliability on Qwen3-8B. We plot BFCL v4 multi-turn accuracy against SimpleToolHalluBench hallucination rates with (a) no available tool and (b) only an irrelevant tool. Arrows connect the base model to StateAlign. StateAlign achieves the highest multi-turn accuracy and the lowest hallucination rates among the models shown.
Figure 2: Task-state diagnosis on Qwen3-8B. (a) Each group pairs a state in which calling the target function is warranted with one that differs in a single condition; bars give the share of responses. (b) Response excerpts when a required phone number is missing. Blue shading marks the decision to ask; red shading marks the unprovided value in the emitted call.
Figure 3: Overview of StateAlign on an example task. (1) Editing one condition of a step from a source trajectory yields five task states with different required actions; each state keeps a reference path, and states are paired with Exec . (2) The policy acts in a constructed state; shaded cards are user messages and tool results, taken from the source trajectory where available. Both rollouts ask for the order ID and then execute; the second also makes an unnecessary call in the same step, which halves its reward. Their outcome rewards give group-relative advantages in the GRPO objective Jout . (3) The base model π0 , kept fixed during training, scores the tokens of two further rollouts under the current state and the paired state, each with its reference path; the clipped difference rt weights the policy’s token log-likelihoods in Jpair , defined in Eq. 3 .
Model
BFCL v4 Multi-turn ↑
ToolSandbox ↑
When2Call ↑
SimpleToolHalluBench ↓
Overall
Base
MF
MP
LC
Similarity
Accuracy
NTA
DT
Base models
Qwen3-4B
39.8
46.6
43.8
34.6
34.3
67.4
67.9
49.9
28.4
Qwen3-8B
45.5
50.3
49.1
37.9
44.6
73.4
66.4
59.8
28.2
Qwen3-14B
47.6
56.4
48.3
37.2
48.4
74.6
70.6
53.9
20.7
Granite-4.2-8B
49.4
62.7
40.0
43.3
51.7
82.1
75.6
20.8
33.3
Table 1: Comparison of StateAlign with base models and tool-use training baselines on task performance and tool hallucination across four benchmarks. ToolSandbox similarity reflects how well execution trajectories satisfy task milestones. MF, MP, and LC denote Missing Functions, Missing Parameters, and Long Context. NTA and DT denote the no-tool and distractor-tool settings. All scores are on a 0–100 scale. Gray subscripts indicate score differences from the corresponding base model; bold marks the best result for each base model and size.
BFCL v4
TS
W2C
NTA ↓
DT ↓
(A) State construction
GRPO
Original
47.2 ± 1.8
73.7
66.9
63.3
37.2
Constructed
51.9 ± 0.5
78.2
71.7
55.6
26.6
ToolRL
Original
48.9 ± 0.6
72.5
65.1
58.7
31.1
Constructed
53.0 ± 0.6
78.5
71.3
55.5
29.8
(B) Training configuration
Table 2: State construction and training ablations. TS: ToolSandbox; W2C: When2Call; NTA, DT: hallucination rates (lower is better). (A) Qwen3-8B trained on original data (ToolMind for GRPO; RLLA-4k for ToolRL) and constructed states. (B) Outcome-only training and complete StateAlign on constructed states. Bold: better of each pair.
Figure 4: State-contrast scores for two successful rollouts when the image ID is missing; the paired state provides it. Labels show mean token scores within each sentence. Blue indicates scores ≥0.1 , red indicates scores ≤−0.1 , and unshaded sentences have scores between these thresholds. Both rollouts ask for the ID; their reasoning differs in state compatibility.
(A) State-conditioned actions
Value missing
Value available
Paired states
Model
Mentions
Clarifies
Target call
Both correct
Base
94.9
68.8
97.7
63.5
ToolRL
94.6
70.4
97.9
65.1
LoopTool
95.4
80.6
98.0
75.4
StateAlign
95.2
83.2
97.1
76.9
Table 3: State-conditioned reasoning and actions for Qwen3-8B (%). (A) Mentions: identifies the missing value; Clarifies: correct clarification given that mention; Target call: correct function when the value is available; Both correct: mean product of per-state action accuracies within each pair. (B) Among traces identifying the missing value, Plans to execute measures proposals to call despite it; Calls after deferring measures target calls conditional on reasoning proposing to ask or take another step first. Lower is better for both metrics.
Figure 5: Multi-step execution and recovery on Qwen3-8B. (a) BFCL v4 turn success by reference-call count. (b) Percentage of multi-call turns that fail after issuing at least one call but fewer than the reference requires. Panels a and b use ordinary execution turns reached without an earlier failure. (c) ToolSandbox similarity on multi-call scenarios. (d) BFCL v4 turn success with tool errors (bars) and without tool errors (open circles). Colors identify methods; arrows indicate the preferred direction.
BFCL
ToolSandbox
Model
Reply+fail ↓
Similarity ↑
Base
11.6
68.9
GRPO
5.7
71.3
ToolRL
11.4
69.3
LoopTool
6.3
67.7
StateAlign
2.7
73.5
Table 4: Behavior and task completion after tool errors on Qwen3-8B. BFCL Reply+fail is the percentage of error-containing turns with no tool call in the next assistant response after the first error and a final failure. ToolSandbox reports similarity for error-containing scenarios. Values use a 0–100 scale; bold marks the best result in each column.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Dataset
Size
Exec
Need_Fetch
Need_User
Infeasible
Stop
Other
Paired
ToolACE
11,300 dialogues
73.5
0.8
6.5
8.1
7.2
3.9
—
Glaive-v2
112,960 dialogues
23.5
0.0
5.0
8.6
24.0
38.9
—
APIGen-MT-5k
5,000 dialogues
19.7
27.6
26.5
0.4
12.2
13.6
—
When2Call
15,000 samples
0.0
0.0
49.6
50.3
0.0
0.1
—
ToolMind (synthetic)
40,161 dialogues
54.4
4.9
7.1
1.3
23.4
8.9
—
LoopTool-23k
23,040 samples
87.1
1.2
1.3
1.1
9.0
0.3
—
Appendix
Table 5: Task-state composition of tool-use training datasets (%). Size uses each release’s reporting unit. Other covers reference responses outside the five task-state categories. Paired is the share of training samples linked to a counterpart requiring a different action; — indicates that no such links are provided.
Hyperparameter
Value
Hyperparameter
Value
Learning rate
2×10−5
Trajectories per batch
320
KL coefficient
0.04
Policy clipping
0.2
LoRA rank
64
LoRA α
128
Trajectories per state
8
Rollout steps
4
Tokens per step
1,536
Sampling temperature
1.0
State-contrast coefficient
0.1
Successful-group share
20%
Appendix
Table 6: Training hyperparameters for StateAlign. The successful-group share is the fraction of batch groups reserved for paired successful trajectories.
Benchmark
Temperature
Top- p
Top- k
Response token limit
BFCL v4
0.001
1.0
—
4,096
ToolSandbox
0.6
0.95
20
16,384
When2Call
0.6
0.95
20
Remaining context
SimpleToolHalluBench
0.5
0.95
20
2,048
Appendix
Table 7: Evaluation decoding settings for Qwen3-based models. The response limit includes reasoning tokens. A dash indicates no top- k filtering; remaining context denotes the available token budget after the input.
Model
Retrieves
Executes
Both correct
Base
75.2
96.9
73.2
ToolRL
74.3
96.9
72.2
LoopTool
77.5
97.5
75.8
Outcome only
81.9
97.8
80.2
StateAlign
88.2
97.0
85.8
Appendix
Table 8: State-conditioned action accuracy on Qwen3-8B retrieval pairs (%). Retrieves measures lookup selection when information must be fetched; Executes measures target-function selection when the required information is available. Both correct averages the product of the two per-state accuracies within each pair. Bold marks the best result in each column.
Figure 7: Training dynamics on Qwen3-8B. (a) Mean outcome reward for Outcome only and StateAlign on constructed states; (b) mean response length; (c) token entropy; (d) KL divergence to the base model; (e) the proportion of responses reaching the per-step token limit. Curves are 40-step centered moving averages.
Model
Precondition
Not found
Other
Shared
Base
80.5
58.3
75.8
74.6
GRPO
84.8
62.2
77.7
77.4
ToolRL
79.6
56.1
76.4
72.8
LoopTool
89.9
63.5
69.5
79.3
StateAlign
92.3
64.0
79.9
81.5
Appendix
Table 9: BFCL recovery success on Qwen3-8B (%). The first three columns group turns by the first tool error: an unmet precondition, a missing resource, or another error. Shared restricts the comparison to task positions where every method encounters a tool error. Bold marks the best result in each column.
(A) BFCL turn success
Base model
Outcome only
StateAlign
Qwen3-4B
69.6
71.5
Qwen3-8B
78.4
80.5
Qwen3-14B
79.7
79.8
(B) ToolSandbox similarity
Base model
Base
StateAlign
Appendix
Table 10: Tool-error recovery across Qwen3 sizes. (A) BFCL turn success, comparing Outcome only with StateAlign. (B) ToolSandbox similarity, comparing Base with StateAlign. Both panels include only error-containing interactions. Higher is better on both 0–100 scales; bold marks the better result in each row.
Large language model (LLM) agent reliability concerns dependable, safe, and verifiable behavior across tools, multi-step tasks, and changing environments.