Learning What to Skip: Counterfactual Credit Assignment for Efficient Multi-Agent LLM Workflows

Jinfeng Xu; Zheyu Chen; Ziyue Peng; Zheng Lin; Shuo Yang; Jinze Li; Zheng Xing; Mengran Li

cs.AISep 25, 2026

Learning What to Skip: Counterfactual Credit Assignment for Efficient Multi-Agent LLM Workflows

Authors: Jinfeng Xu, Zheyu Chen, Ziyue Peng, Zheng Lin, Shuo Yang, Jinze Li, Zheng Xing, Mengran Li,

Organizations: The University of British Columbia, Canada · The Hong Kong Polytechnic University, Hong Kong · The Hong Kong University of Science and Technology, Hong Kong · University of Luxemburg, Luxemburg ·

Abstract

Multi-agent LLM workflows use planning, execution, verification, and summarization to improve task performance, yet the value of each component depends on the state already produced. Executing every component can waste computation or overwrite a correct intermediate answer. We formulate component omission as counterfactual credit assignment: full-workflow logs reveal the executed trajectory's reward, while controlled skip interventions reveal the consequences of omitting a future step. We introduce Learning What to Skip (LW2S), which learns action-specific safety models from these interventions and combines held-out calibration with domain-native guards to select skips. When an early skip is rejected, the controller can continue execution and reconsider a later component. Across mathematical reasoning, multiple-choice QA, and code generation with two instruction-model families, LW2S reduces recorded token cost while matching or improving aggregate full-workflow accuracy in the evaluated settings. Scale-up and second-topology experiments further examine component redundancy, while shared-error cases reveal why agreement alone is insufficient for skip selection. These findings connect efficient workflow execution to learning the conditional utility of individual components.

Figures & tables

Figure 1: Overview of our LW2S framework.

Backbone	Setting	Metric	Full	RouteLLM	GraphPlanner	AutoMix	Prompt-LLM	LW2S
Qwen2.5-7B	MATH verifier	Acc.	65.2	64.8 1	65.2 1	65.6	64.8 1	66.4
	MATH verifier	Save	0.0	14.6	11.2	18.3	20.5	25.1
	MATH summarizer	Acc.	65.2	64.8 1	65.2	65.2	64.8 1	65.2
	MATH summarizer	Save	0.0	11.8	9.5	14.2	16.5	18.4
	GSM8K verifier	Acc.	88.5	88.0 1	88.5	88.0 1	88.0 1	88.5
	GSM8K verifier	Save	0.0	12.7	8.9	15.8	19.1	20.6

Table 1: Main same-protocol evidence matrix. Acc. and Save are percentages; Acc. superscripts count task-level degradations against the full workflow. No superscript means zero observed degradation. Mistral MMLU averages three rotations.

Figure 2: Qwen2.5-7B method summary; lower right is better.

Figure 3: Diagnostics. (a) LW2S coverage, saving, and Wilson degradation-risk bounds on Qwen2.5-7B. (b) MBPP component credit under slice shift; LW2S reallocates verifier and summarizer skips while preserving reward.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Block	Backbone	Workflow	Tasks	Traces	Tokens (M)	Latency (h)	GPU	API
Main matrix	Qwen2.5-7B	Five-role	1100	6600	12.74	9.02	PRO6000	none
Model generalization	Mistral-7B	Five-role	1490	8940	20.31	14.52	PRO6000	none
Scale-up diagnostic	Qwen2.5-14B	Five-role	250	1500	3.27	4.62	PRO6000	none
Second workflow	Qwen2.5-7B	Debate	200	1200	3.14	2.90	PRO6000	none
Second workflow	Qwen2.5-7B	Debate	250	1500	3.83	2.97	PRO6000	none
Total	–	–	3290	19740	43.30	34.04	PRO6000	none

Appendix

Table 2: Resource accounting for reported open-weight traces. Five-role denotes planner, two executors, verifier, and summarizer; Debate denotes debate-refine. Latency is summed component latency and is not used for threshold selection; no commercial API calls are used.

Workflow family	Benchmarks	Backbones	Evidence role
Five-role	MATH, GSM8K, MMLU, MBPP	Qwen2.5-7B, Mistral-7B	Main safety–efficiency matrix with same-protocol named baselines and calibrated skip policies.
Scale-up fixed template	MATH, MMLU, MBPP	Qwen2.5-14B	Larger-backbone diagnostic for skip-safe redundancy and unsafe high-saving shortcuts.
Debate-refine	MBPP	Qwen2.5-7B	Topology diagnostic with different roles and information flow from the main workflow.

Appendix

Table 3: Workflow-family coverage. The main matrix uses a controlled five-role template; scale-up and debate-refine rows test persistence beyond one backbone and one topology.

Method family	Decision target	Supervision or evidence	Boundary to LW2S
Model routers	choose a model before execution	preference or task reward	Optimize model/cost trade-off; no component-level skip intervention labels or sequential fall-through.
Graph/workflow routers	choose a graph or agent allocation	end-to-end reward	Estimate workflow utility; usually does not identify whether a specific future component is safely omissible.
Selective prediction and early exit	emit, reject, or exit at a confidence gate	labels plus reject loss	Controls prediction risk–coverage; does not intervene on verifier/summarizer components in an observed workflow prefix.
Counterfactual MARL credit	assign action advantage during policy learning	counterfactual returns	Uses counterfactual credit, but it is not a calibrated deployment-time skip policy over LLM workflow components.
Workflow compaction	replace an evaluated workflow before deployment	workflow search or evaluation	Finds fewer turns or a single-agent simulation; no per-task fall-through after a blocked component skip.
LW2S	omit a specific future component after observing a prefix	full and single-skip traces	Learns action-specific skip labels, calibrates source-stable risk, and can fall through to later skip decisions.

Appendix

Table 4: Technical boundary between LW2S and adjacent efficiency paradigms. The key distinction is component-level skip supervision from controlled interventions after a workflow prefix has been observed.

Action	Observed prefix	Removed component	Emitted/evaluated output	Accounting
skip verifier	after both executors	verifier	summarizer or finalizer consumes executor answers; the domain parser scores the final answer	token cost; latency logged separately
skip summarizer	after verifier	summarizer	verified answer is emitted directly and parsed by the same task parser	token cost; latency logged separately
two-stage MBPP	after executors, then after verifier if blocked	one component per task	selected recorded intervention: skip verifier, skip summarizer, or full workflow; unit tests score the result	token cost of selected trace; latency logged separately

Appendix

Table 5: Controlled intervention protocol. Prompts, decoding settings, parsers, and workflow templates are fixed within each task family. Each selected continuation omits a single target component.

Method	Input state	Training signal	Calibration data	Deployment rule
RouteLLM	observed prefix features	binary skip-safe label	same split	threshold sweep; fallback to full workflow
GraphPlanner	prefix plus memory features	binary skip-safe label	same split	threshold sweep; fallback to full workflow
AutoMix	prefix confidence and action value	trace reward and cost	same split	value threshold; fallback to full workflow
Prompt-LLM	task, prefix, candidate action	in-context skip judgment	calibration split	probability threshold; fallback to full workflow
LW2S	prefix, agreement, metadata	action-specific intervention label	same split plus source checks	source-stable threshold with guard; fallback to full or later skip

Appendix

Table 6: Baseline inputs, supervision, and decision rules under the shared component-omission protocol. Methods use the same saved traces and data partition.

Setting	Policy	Full	Acc.	Deg.	Save	Interpretation
MATH seed55	fixed skip verifier	66.0	70.0	0	25.0	verifier removal improved reward on this evaluated slice
MATH seed55	executor-consensus verifier skip	66.0	68.0	0	13.5	guarded redundancy remained after scaling the backbone
MATH seed55	fixed skip summarizer	66.0	66.0	0	17.2	summarizer removal preserved full-workflow reward
MMLU seed2	fixed skip verifier	82.0	83.0	3	23.6	boundary case: high saving did not imply safe skipping
MMLU seed2	executor-consensus verifier skip	82.0	84.0	2	22.0	consensus did not remove the shared-error failure mode
MBPP slice1	fixed skip verifier	69.0	70.0	0	24.5	verifier redundancy persisted on the scaled coding slice

Appendix

Table 7: Qwen2.5-14B scale-up diagnostics. Rows test whether skip-safe and boundary phenomena persist under a larger backbone; Save is token-cost reduction.

Policy	Full	Acc.	Save	Interpretation
Full workflow	67.0	67.0	0.00	reference execution of the debate-refine workflow
LW2S after executors	67.0	68.0	16.65	learned verifier/summarizer skip decision after observing proposer and opponent outputs
LW2S after verifier	67.0	68.0	16.65	learned summarizer skip decision after the critic/verifier state is available
Hindsight reward-improving skip	67.0	69.0	25.79	hindsight diagnostic, not a deployed policy
Fixed skip verifier	67.0	67.0	27.42	high-saving fixed intervention that is safe on this evaluated slice
Fixed skip summarizer	67.0	68.0	16.65	fixed late-component omission in the debate-refine workflow

Appendix

Table 8: MBPP debate-refine diagnostic on Qwen2.5-7B. Acc. is pass rate, Deg. counts task-level degradations, and Save is token-cost reduction.

Setting	Backbone	Workflow	Policy	Full	Acc.	UCB 95	Save
MATH verifier	Mistral-7B	Five-role	LW2S transfer	12.8	12.8	1.5	19.1
MATH verifier	Qwen2.5-7B	Five-role	LW2S final	65.2	66.4	1.5	25.1
MATH verifier	Qwen2.5-7B	Five-role	Full workflow	65.2	65.2	1.5	0.0
MATH summarizer	Mistral-7B	Five-role	LW2S transfer	12.8	14.0	1.5	21.8
MATH summarizer	Qwen2.5-7B	Five-role	LW2S final	65.2	65.2	1.5	18.4
MATH summarizer	Qwen2.5-7B	Five-role	Full workflow	65.2	65.2	1.5	0.0

Appendix

Table 9: Unified safety-efficiency ledger for headline LW2S evidence. UCB 95 is a Wilson upper bound on observed task-level degradation for the evaluated slice; Save is token-cost reduction.

Setting	cal. $n$	$\tau$	UCB 80	Cal. saving
MATH summarizer	140	0.95	0.5	10.5
GSM8K verifier	50	0.00	1.4	26.2
MBPP verifier	150	1.00	0.5	24.5
MBPP summarizer	150	0.00	0.5	16.6

Appendix

Table 10: Representative LW2S calibration results. UCB 80 is the Wilson upper bound used by the acceptance rule (

\alpha=0.2

); saving is measured on the calibration traces.

Setting	$n$	Selected	Deg.	UCB 95	Save
MATH verifier	250	250	0	1.5	25.1
MATH summarizer	250	250	0	1.5	18.4
GSM8K verifier	200	173	0	1.9	20.6
MMLU verifier	100	100	1	5.4	8.6
MBPP slice0	100	100	0	3.7	28.5
MBPP slice1	100	100	0	3.7	31.2

Appendix

Table 11: Coverage and uncertainty for the Qwen2.5-7B LW2S results in Table 1 . Selected counts tasks assigned a skip; UCB 95 bounds degradation over all evaluated tasks.

Slice	Method	Full	Acc.	Deg.	$n_{v}$	$n_{s}$	Save
MBPP slice0	RouteLLM	69.0	68.0	1	–	–	16.9
	GraphPlanner	69.0	69.0	0	–	–	13.4
	AutoMix	69.0	69.0	0	–	–	22.1
	Fixed skip verifier	69.0	70.0	0	100	0	24.2
	Fixed skip summarizer	69.0	69.0	0	0	100	16.3
	LW2S (ours)	69.0	70.0	0	69	31	28.5

Appendix

Table 12: MBPP component-credit diagnostic.

n_{v}

and

n_{s}

count verifier- and summarizer-skip selections; – denotes unreported counts. Baseline accuracy, degradation, and saving match Table 1 .

Regime	Observable state	Mechanism	Evidence	LW2S response
Shared wrong consensus	Executors agree on the same answer, but the verifier changes it to the gold answer	Agreement is not a certificate when agents share a misconception or shortcut	MMLU verifier boundary; Table 15 ; Qwen2.5-14B MMLU rows	Accept calibrated actions; retain the verifier when acceptance fails
Superficial code agreement	Programs look similar, but visible tests expose a shared bug	Non-executable agreement can preserve the same incorrect logic across executors	MBPP slice1 AST-guard degradation in Table 14	Use executable guards and fall through to later decisions when early removal is blocked
Redundant late summarization	A verifier or checked intermediate already contains a parsable final answer or executable code	The final summarizer can add cost without improving reward, and can overwrite a correct intermediate answer	MATH summarizer, MBPP two-stage, and debate-refine rows	Prefer late-component omission when calibrated evidence supports reward preservation

Appendix

Table 13: Failure and abstention taxonomy. The same visible prefix can indicate safe redundancy in one domain and an unsafe shared error in another.

Slice	Condition	Full	Acc.	Selected
slice0	both executors pass	69.0	70.0	69
	Executor AST equality	69.0	70.0	93
	both pass + AST	69.0	70.0	65
	verifier passes tests	69.0	69.0	69
	verifier/summarizer AST (hindsight)	69.0	69.0	99
slice1	both executors pass	63.0	63.0	58

Appendix

Table 14: MBPP guard and structural diagnostics on unseen 100-task slices. Verifier/summarizer AST equality is a hindsight diagnostic; all other conditions use outputs available at their decision stage.

Example	Subject	Short prompt cue	Failure signal	Skip	Verifier	Gold
mmlu_test_13860	virology	What is a virus pandemic?	executor consensus A/A	A	C	C

Appendix

Table 15: MMLU verifier-skip degradation case. Skipping retains the executors’ shared wrong option A, while verification corrects it to the gold option C.

LLM Agent Workflow Optimization

LLM agent workflow optimization studies how to design and run multi-step agents for better task success, reliability, latency, and cost.

See all 260 papers

COVENANT: Natural-Language Workflow Compilation for Aligned Agent Execution
Jul 28, 2026Jincheng Wang, Min Zheng, Tao WeiAgentic Workflows LLM Agent Harnesses
Ant Group
CausalFlow: Causal Attribution and Counterfactual Repair for LLM Agent Failures
May 25, 2026Akash Bonagiri, Devang Borkar, Gerard Janno Anderias +2Agent Failure Analysis Causal Interventions in Language Models
Department of Computer Science University of California, Davis Davis, CA 95616
Pay for the Fault, Not the Flow: Label-Free In-Flow Multi-Agent Workflow Optimization
Oct 1, 2026Xuehang Guo, Haoyu Wang, Shengyu Chen +4Multi-Agent Coordination Multi-Agent Orchestration
William & Mary · NEC Corporation of America

Jul 28, 2026J/K move · Enter open · S save

COVENANT: Natural-Language Workflow Compilation for Aligned Agent Execution

Jincheng Wang, Min Zheng, Tao Wei

Ant Group

Agentic Workflows LLM Agent Harnesses LLM Agent Reliability

Open paper AskPDF arXiv

Large language model (LLM) agents are increasingly entrusted with natural-language workflow instructions (e.g., retail-payment policies) that specify not only what outcome to achieve, but also which steps, branches, and tool interactions are permitted. When these instructions are supplied as prompt context, however, the model retains control over both procedure selection and step execution. As interactions accumulate, an agent can skip required steps, take unsupported branches, or execute a valid step with unsupported arguments or effects--a failure mode we call workflow misalignment. In this work, we propose COVENANT, a compiler-and-interpreter architecture for workflow-aligned agent execution. Our key insight is to treat workflow instructions as source programs rather than prompts. COVENANT converts the instructions into a workflow abstract syntax tree (WAST) and lowers it to a workflow control-flow graph (WCFG). At runtime, a controller interprets the WCFG one node at a time, checks each proposal against requirements extracted from the instructions before committing controller state or advancing the graph, and returns diagnostic feedback for repair. To evaluate COVENANT, we use 120 cases from three existing benchmarks, spanning seven workflow scenarios. Compared with state-of-the-art LLM agents, COVENANT improves benchmark success from 50.00% to 83.33% and reduces the workflow-misalignment failure rate from 42.50% to 15.83% (62.75% relative). These results show that COVENANT substantially mitigates workflow misalignment, moving LLM-agent alignment beyond isolated prompt following toward reliable execution of complex and multi-step workflows.

Abstract

LLM Agent Workflow Optimization

Explore similar work

COVENANT: Natural-Language Workflow Compilation for Aligned Agent Execution

CausalFlow: Causal Attribution and Counterfactual Repair for LLM Agent Failures

Pay for the Fault, Not the Flow: Label-Free In-Flow Multi-Agent Workflow Optimization