From State to Action: OODA-Tool for Reliable Multi-Turn Tool Use

Rongfeng Guo; Yinxuan Huang; Yusen Wu; Maoqing Zhong; Yunlu Chen; Meng Tang; Teng Long; Vincent Tao Hu

cs.AIAug 25, 2026

From State to Action: OODA-Tool for Reliable Multi-Turn Tool Use

Authors: Rongfeng Guo, Yinxuan Huang, Yusen Wu, Maoqing Zhong, Yunlu Chen, Meng Tang, Teng Long, Vincent Tao Hu

Organizations: Huazhong University of Science and Technology · Knowin AI, Shenzhen, China · Fujian University of Technology · Jiangxi University of Science and Technology ·

Abstract

Reliable multi-turn tool use requires an agent to preserve an evolving task state and ensure that each action remains consistent with it. However, direct function-calling and ReAct-style policies learn state tracking and action generation within the same autoregressive trajectory. This coupling creates state-action competition: the pressure to produce the next call can overwrite or ignore information accumulated earlier in the interaction. Inspired by Boyd's Observe-Orient-Decide-Act cycle, we introduce OODA-Tool, a typed closed-loop policy designed to mitigate this competition by separating state preservation from action realization. Rather than generating an action directly from the interaction history, OODA-Tool routes each decision through controller-checked intermediate states, ensuring that the final output remains grounded in the current task state. Specifically, Observe reconstructs the task state, Orient determines whether execution is warranted, Decide forms an admissible action structure, and Act realizes the external output. We evaluate OODA-Tool against direct function-calling and ReAct policies using Qwen3 models ranging from 0.6B to 14B across multi-turn, multi-tool, and incomplete-information settings. OODA-Tool consistently improves task success across model sizes, with larger gains on smaller models and on tasks whose actions depend strongly on information accumulated across turns and prior tool results. Controlled variants, stage-level ablations, and transfer evaluations further demonstrate the robustness of these improvements.

Figures & tables

Figure 1: Comparison between a single-pass tool agent and our staged OODA-Tool agent.

Figure 2: Overview of OODA-Tool. The left panel shows the closed-loop tool-use process under central-controller validation. The upper-right panel illustrates Specialized OODA with a separate LoRA adapter for each stage over a shared frozen Qwen3 backbone. The lower-right panel summarizes the typed intermediate states and the unified constraint policy used for stage supervision and final-action validation.

Method	Structure [-0.3ex] / adapters	Task Success $\uparrow$					Tool Exact $\uparrow$					Ask–Act Accuracy $\uparrow$
Method	Structure [-0.3ex] / adapters	0.6B	1.7B	4B	8B	14B	0.6B	1.7B	4B	8B	14B	0.6B	1.7B	4B	8B	14B
Base Instruct	–	0.10	0.43	3.04	7.26	11.63	0.20	0.67	6.18	13.84	20.45	69.50	73.76	78.42	82.15	85.27
Direct-LoRA	1 adapter	78.24	78.67	80.31	83.58	90.42	88.36	82.41	84.29	87.31	96.21	94.06	95.03	96.08	97.05	98.04
ReAct-LoRA	1 adapter	65.43	73.18	79.62	81.35	84.77	71.56	77.21	82.39	85.44	90.83	91.32	94.71	95.26	96.59	97.84
Direct-SC@4	4 samples	78.90	78.90	81.12	83.92	90.72	88.70	82.68	84.57	87.45	96.34	94.10	95.07	96.15	97.12	98.09
$\alpha$ -UMi	3 roles	82.90	83.94	85.80	88.10	93.90	97.90	98.60	98.72	98.94	99.15	98.35	98.70	98.80	98.92	99.04

Table 1: ToolDial results across Qwen3 backbone sizes. Specialized OODA achieves the highest Task Success at every scale, with larger gains on smaller backbones. Bold indicates the best result, and underlining indicates the strongest non-OODA baseline.

Figure 3: Task Success gains of Specialized OODA over Direct-LoRA across model scales and ToolDial complexity subsets. Gains are largest on state-intensive tasks and smallest on parallel calls; values are percentage-point differences.

Figure 4: Component ablations and execution-grounding errors. Panel (a) reports Task Success decreases under controlled ablations, and panel (b) reports premature-call and stale-or-ungrounded-argument rates; larger ablation drops are worse, while lower error rates are better. Orient and the Decide–Act separation contribute most, whereas stage-specific adapters have a smaller effect.

Figure 5: Normalized transfer advantage of Specialized OODA over

\alpha

-UMi across Qwen3 backbone sizes on BFCL, MTU-Eval, and FAIL-TaLMs. Higher values indicate larger gains. The advantage is strongest on BFCL and smaller backbones and narrows with scale.

Figure 6: Identifier grounding. Baselines copy a local ID, whereas OODA traces the selection to the Wikidata ID.

Tool Use

Tool use studies how AI agents select, invoke, and combine external tools to solve tasks beyond their internal knowledge.

See all 91 papers

Critical-State RL: Diagnosing Trainable States for Multi-Turn Tool Use
Sep 21, 2026Zixiang Chen, Wenting Zhao, Zhepeng Cen +9Reinforcement Learning Credit Assignment in RL
Salesforce AI Research
State-OPD: State-Grounded On-Policy Distillation for Multi-Turn Agents
Oct 9, 2026Zhentao Zhang, Yanzhao Zheng, Yanlin Liu +9On-Policy Distillation
Alibaba Group · Tsinghua University · Shanghai Jiao Tong University +1
When History Lies: Evaluating and Improving Tool Use under Misleading Multi-Turn Histories
Aug 6, 2026Xiaoqing Wu, Xingyu Fan, Feifei Li +1Tool-Use Evaluation LLM Tool Use
WeChat, Tencent Inc., Beijing, China

Sep 21, 2026J/K move · Enter open · S save

Critical-State RL: Diagnosing Trainable States for Multi-Turn Tool Use

Zixiang Chen, Wenting Zhao, Zhepeng Cen, Akshara Prabhakar+8

Salesforce AI Research

Reinforcement Learning Credit Assignment in RL RL for Tool Use

Open paper AskPDF arXiv

Multi-turn tool-use failures can hinge on a single model call, yet reward variation alone does not reveal which call would benefit from training. When rewards depend on later interactions, their variation can reflect downstream randomness rather than differences between the current actions. We introduce Critical-State RL to identify trainable states in multi-turn interactions. Given task-defined candidate calls and local rewards, the method assesses whether each reward captures the action's effect on task success and whether improvement over a reference policy is possible. It then uses nested sampling to separate action-dependent reward variation from continuation noise and optimizes the policy at the selected states using contextual-bandit training. Experiments on the Berkeley Function Calling Leaderboard (BFCL) v4 compare training at diagnostic-selected states with training at alternative states. For missing-function tasks, the diagnostic selects the response after the tool becomes available; for missing-argument tasks, it selects the response before the missing argument is supplied. Training the selected responses improves performance, including about 14 percentage points on the missing-function task, while training the alternatives leaves performance flat or worse. We further apply the recipe across models and tasks, including logged repeat-call avoidance and memory management.

Abstract

Tool Use

Explore similar work

Critical-State RL: Diagnosing Trainable States for Multi-Turn Tool Use

State-OPD: State-Grounded On-Policy Distillation for Multi-Turn Agents

When History Lies: Evaluating and Improving Tool Use under Misleading Multi-Turn Histories