Of Errors and Echoes: Responsibility-Aware Credit Assignment for Tool-Use Agents

Xingjian Diao; Tianyu Yang; Wenjun Huang; Xiangchi Yuan; Chunhui Zhang; Xingchen Zhao; Weiyi Wu; Haotian Xu

cs.CLOct 12, 2026

Of Errors and Echoes: Responsibility-Aware Credit Assignment for Tool-Use Agents

Authors: Xingjian Diao, Tianyu Yang, Wenjun Huang, Xiangchi Yuan, Chunhui Zhang, Xingchen Zhao, Weiyi Wu, Haotian Xu,

Organizations: Dartmouth College · University of Notre Dame · University of California, Irvine · Georgia Institute of Technology ·

Abstract

Reinforcement learning improves tool-use agents, yet fine-grained credit assignment remains challenging even with step-level feedback. An observed error may be inherited from an earlier decision, while multiple responsible decisions may share the same downstream consequence or participate in later recovery. We introduce Error-Chain-Aware Optimization (ECHO), a responsibility-aware credit assignment framework that separates responsibility identification from credit allocation. A frozen judge combines retrospective trajectory analysis with decision-time assessment from the policy-visible prefix to distinguish newly introduced errors, inherited consequences, missed corrections, and successful corrections. ECHO then counts shared downstream consequences once across responsible decisions and constrains rewards for repairing self-induced errors through source-funded recovery, producing bounded action-level supervision alongside terminal supervision. Experiments on six diverse tool-use benchmarks at both 4B and 8B scales show strong and consistent improvements over the compared baselines, with ECHO achieving the highest score on five of six benchmarks at each scale. These results support decision responsibility as a useful basis for fine-grained credit assignment in tool-use agents. Our code is available at https://github.com/xid32/ECHO.

Figures & tables

Figure 1: ECHO on an error chain. Left: Responsibility depends on decision-time visibility: Step 2 inherits the error before the mismatch is visible, while Step 3 misses a correction after disclosure. Right: Credit allocation counts the wrong-item consequence once across Steps 1 and 3 and funds Step 4’s repair credit from Step 1.

Figure 2: Overview of ECHO. (a) Full-trajectory review proposes candidate events, propagation links, and restoration evidence, while decision-time assessment classifies each selected action as New , Inh , Miss , or Corr . (b) Shared consequences are counted once across responsible actions, then weighted, normalized, and adjusted by source-funded recovery to yield signed action-level corrections

\delta_{t}

. (c) Terminal and local supervision are optimized through separately clipped outcome-level and action-level branches with KL regularization.

		Overall score
Model		BFCL-v3	BFCL-v4	$\tau$ -bench	$\tau^{2}$ -Bench	ToolSandbox	ACEBench
4B	Qwen3 ( Yang et al., 2025 )	25.38	38.61	33.44	15.83	52.89	55.28
	EnvScaler ( Song et al., 2026 )	37.25	52.92	48.78	33.05	62.25	63.33
	Arctic-AWM ( Wang et al., 2026b )	38.89	49.55	26.58	22.57	68.24	60.58
	ECHO (ours)	41.24	53.71	50.77	35.81	76.36	62.46
8B	Qwen3 ( Yang et al., 2025 )	28.88	41.58	36.65	26.44	59.74	60.00

Table 1: Overall results on six tool-use benchmarks at the 4B and 8B scales, spanning multi-turn function calling, conversational task completion, and stateful tool execution. For BFCL-v3, we report the overall score on the Multi-Turn (MT) subset. For ACEBench, we report results on the Agent subset. Scores are percentages. Bold and underlined values mark the best and second-best results within each scale.

		$\tau$ -bench			$\tau^{2}$ -Bench
Model		Retail	Airline	Overall	Retail	Airline	Telecom	Overall
4B	Qwen3 ( Yang et al., 2025 )	40.87	26.00	33.44	19.96	21.00	9.43	15.83
	EnvScaler ( Song et al., 2026 )	49.57	48.00	48.78	50.00	36.00	13.16	33.05
	Arctic-AWM ( Wang et al., 2026b )	30.15	23.00	26.58	30.26	19.00	16.45	22.57
	ECHO (ours)	52.03	49.50	50.77	54.09	36.00	17.33	35.81
8B	Qwen3 ( Yang et al., 2025 )	45.79	27.50	36.65	34.43	26.50	18.42	26.44

Table 2: Domain-level results on

\tau

-bench and

\tau^{2}

-Bench at the 4B and 8B scales. All scores are reported as percentages. Bold and underlined values mark the best and second-best results within each model scale.

Model		Base	Missing Function	Missing Parameter	Long Context	Overall
4B	Qwen3 ( Yang et al., 2025 )	32.00	20.00	24.00	25.50	25.38
	EnvScaler ( Song et al., 2026 )	46.50	38.50	28.50	35.50	37.25
	Arctic-AWM ( Wang et al., 2026b )	48.55	36.50	34.22	36.30	38.89
	ECHO (ours)	50.50	46.58	32.59	35.27	41.24
8B	Qwen3 ( Yang et al., 2025 )	32.00	33.50	22.00	28.00	28.88
	EnvScaler ( Song et al., 2026 )	50.50	41.50	29.50	44.00	41.38

Table 3: BFCL-v3 Multi-Turn results across Base, Missing Function, Missing Parameter, and Long Context conditions at the 4B and 8B scales. Overall is the unweighted mean across the four conditions. All scores are reported as percentages. Bold and underlined values mark the best and second-best results within each model scale.

Figure 3: Case studies comparing agent behavior before (Qwen3) and after (ECHO) training on the same task.

Figure 4: Ablation of ECHO components on

\tau^{2}

-Bench and BFCL-v3 Multi-Turn at the 8B scale.

Figure 5: Reward ablation on

\tau^{2}

-Bench at the 8B scale. Each group compares five supervision configurations, and Overall denotes the mean across the three domains.

Figure 6: Frozen-judge comparison at the 8B scale. Left: downstream ECHO performance on

\tau^{2}

-Bench. Right: responsibility identification accuracy on 100 manually verified samples before reliability gating.

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Reward ablation on BFCL-v3 Multi-Turn at 8B. Overall is the mean across Base, Missing Function, Missing Parameter, and Long Context.

Semantic family	Observed diagnostics	Evidence required for responsibility assignment
Format, schema, and types	Malformed calls; parameter or type mismatches.	Visible schema constraints, actionable rejections, and execution of a corrected call.
Tool selection and call count	Incorrect function, call count, or relevance decision.	Available tools, selection obligations, and provenance of an inherited choice.
Arguments, entities, and values	Incorrect argument, entity, value, or parallel-call match.	Visible value constraints, the source of reused data, and verified value correction.
State and prerequisites	State mismatch; unmet database, environment, or milestone checks.	Applicable preconditions, the responsible state transition, and verified restoration.
Execution and omitted work	Missing or mismatched execution; process-score deficits.	Required operations and current progress, with a feasible continuation or correction.
Retrieval, memory, and answers	Expected answer absent after search or memory use.	Retrieved evidence, stored-state provenance, and correction of the affected fact.

Appendix

Table 4: Semantic families of checker diagnostics and the evidence each requires for responsibility assignment. Diagnostics (23 literal error_type codes and 19 derived signals) are drawn from 14,014 scored Qwen3 base-model trajectories (7,007 each at 4B and 8B) across the six benchmarks. Families can overlap within a trajectory. A further 2,319 unlocalized or unmapped occurrences fall outside these families (§ H ).

Exact diagnostic string	Semantic family	Occurrences
agentic:answer_not_found	Retrieval, memory, and answers	752
ast_decoder:decoder_failed	Format, schema, and types	3
class attributes wrong	State and prerequisites	53
context_limit_exceeded	Progress, stopping, and limits; Runtime and external boundary	4
irrelevance_error:decoder_success	Tool selection and call count	540
multiple_function_checker:wrong_count	Tool selection and call count	47

Appendix

Table 5: Literal error_type codes recorded by benchmark checkers, with their semantic families and occurrence counts. A code mapped to two families lists both, separated by a semicolon.

Exact diagnostic string	Semantic family	Occurrences
acebench:process_score<1	Execution and omitted work	53
audit:score<1_without_listed_diagnostic	Unmapped or unlocalized	2,278
record:explicit_exception_or_error	Runtime and external boundary	51
tau2:action_checks:action_match=false	Execution and omitted work	897
tau2:communicate_checks:met=false	Communication obligations	38
tau2:db_match=false	State and prerequisites	365

Appendix

Table 6: Derived diagnostic signals computed from benchmark records, with their semantic families and occurrence counts. Signals marked as unmapped or unlocalized fall outside the eleven families of Table 4 .

Panel of Figure 8 — Figure 8: Base-model New events.

Panel of Figure 9 — Figure 9: Base-model Inh events.

Panel of Figure 10 — Figure 10: Base-model Miss events.

Panel of Figure 11 — Figure 11: Base-model Corr events.

Panel of Figure 12 — Figure 12: Paired comparisons of Qwen3 (before training) and ECHO (after training) on the same tasks.

Panel of Figure 13 — Figure 13: Paired comparisons of Qwen3 and ECHO on the same tasks (continued).

Panel of Figure 14 — Figure 14: Paired comparisons of Qwen3 and ECHO on the same tasks (continued).

Panel of Figure 15 — Figure 15: Paired comparisons of Qwen3 and ECHO on the same tasks (continued).

Panel of Figure 16 — Figure 16: Paired comparisons of Qwen3 and ECHO on the same tasks (continued).

Credit Assignment

Credit assignment studies how to attribute outcomes to the actions, agents, or components that caused them.

See all 71 papers

TRACE: Turn-level Reward Assignment via Credit Estimation for Long-Horizon Agents
Jul 15, 2026Leitian Tao, Baolin Peng, Wenlin Yao +5Credit Assignment in RL Step-Level Credit Assignment in RL
University of Wisconsin–Madison · Microsoft Research
Teach the Magnitude, Not the Direction: Verifier-Bounded Credit Assignment for Multi-Turn Multi-step LLM Agents
Aug 13, 2026Zechuan Wang, Siyuan Lu, Hongxuan Zhang +3Token-Level Credit Assignment Step-Level Credit Assignment in RL
Zhejiang University · AWorld Team, Inclusion AI · Shanghai Innovation Institute +2
ToolOPD: Credit-Aware On-Policy Distillation for Tool-Use Agents
Oct 19, 2026Chu Zhao, Lei Tang, Yanming Wang +5On-Policy Distillation Step-Level Credit Assignment in RL
Northeastern University · Honor Device Co., Ltd

Jul 15, 2026J/K move · Enter open · S save

TRACE: Turn-level Reward Assignment via Credit Estimation for Long-Horizon Agents

Leitian Tao, Baolin Peng, Wenlin Yao, Tao Ge+4

University of Wisconsin–Madison · Microsoft Research

Credit Assignment in RL Step-Level Credit Assignment in RL Dense Reward RL Temporal-Difference Learning Step-Level Credit Assignment RL Post-Training+1

Open paper AskPDF arXiv

Multi-turn agents solve complex tasks through extended sequences of tool interactions before producing a final answer, making credit assignment a fundamental challenge during post-training. Outcome rewards provide reliable supervision for short-horizon reasoning, but become sparse and high-variance as trajectories grow to tens or hundreds of tool calls. They can also be misleading: a failed rollout may contain many useful actions that move the agent closer to the goal, yet outcome-only training assigns them the same negative advantage as the eventual mistake. We propose TRACE (Turn-level Reward Assignment via Credit Estimation), a dense credit-assignment method for agentic reinforcement learning. TRACE represents rollouts as state transitions at tool-call boundaries, obtains gold-answer log-probabilities from a frozen reference model, transforms them into log-ratio state values, and derives per-action rewards as Temporal-Difference changes in those values. This requires no additional critic or process-label training, and its one-step log-ratio TD component telescopes across redundant tool calls. On long-horizon complex search, TRACE substantially improves base-model tool-use ability using pure RL, without a cold-start supervised fine-tuning stage, an agentic mid-training stage, or training on live-web data. On the closed-web BrowseComp-Plus benchmark, it raises Qwen3-4B from

7.2

35.6

and Qwen3-30B-A3B from

8.4

42.6

. The learned search behavior also transfers to open-web benchmarks, and the learning curves show earlier improvement and faster convergence during RL training.

Abstract

Credit Assignment

Explore similar work

TRACE: Turn-level Reward Assignment via Credit Estimation for Long-Horizon Agents

Teach the Magnitude, Not the Direction: Verifier-Bounded Credit Assignment for Multi-Turn Multi-step LLM Agents

ToolOPD: Credit-Aware On-Policy Distillation for Tool-Use Agents