Right-Sizing LLM-Agent Decomposition in VAT Determination: A Pilot Controlled Sweep

Pedro Santos

cs.MAAug 24, 2026

Right-Sizing LLM-Agent Decomposition in VAT Determination: A Pilot Controlled Sweep

Authors: Pedro Santos

Organizations: Independent Researcher

Abstract

Recent LLM-agent systems make conflicting design bets: decompose work across many narrow agents, or use one strong tool-using agent. This pilot studies that choice on bounded cross-border VAT determination with reverse charge, where every case has an oracle label and each intermediate decision is independently scoreable. We hold the activity surface fixed (subtasks, tools, I/O schemas, validation checks, orchestrator, base model, and merge policy) and vary only the assignment of subtasks to workers across four orchestrated configurations, from one wide worker to five narrow ones, against S0, a tuned no-orchestrator single agent, with a deterministic rule engine as oracle. The program spans 4,400 runs: a 40-case, five-repeat main sweep, matched-token arms separating prompt-budget from agent-count effects, and three failure-injection arms, all judged against pre-registered falsification criteria. The two intermediate configurations lead on accuracy (0.830, against endpoints at 0.720 and 0.770) but miss the pre-stated bar against the fine endpoint, so the intermediate-optimum hypothesis remains unsupported at pilot scale. The single agent does not Pareto-dominate the orchestrated set. The matched-token criterion fires: the budget-matched single agent lands 6.5 points below the leader, but the interval includes zero, so any advantage is consistent with a prompt-budget explanation. Under injection, availability faults are absorbed at every granularity, with wide-scope restart over-recovering its baseline by +0.160, while one schema-conforming hallucinated record degrades every configuration and inverts the ordering, hitting fragmented configurations hardest. The contribution is a bounded, preregistered pilot heuristic for right-sizing decomposition (place one partition boundary at the dependency-layer midpoint), released with oracle, dataset, harness, raw traces, and analysis pipeline.

Figures & tables

Figure 1: The fixed activity surface and the experimental conditions. Top: the five subtasks and the transitive reduction of the dependency order

\mathcal{D}

, with dependency layers

L_{1}

–

L_{4}

. Bottom: worker partitions C1–C4 and the no-orchestrator baseline S0; the dashed vertical line marks the dependency-layer midpoint used by the merge policy (§4.4) and the matched-token reference (§6.3).

Condition	Accuracy [95% CI]	Tokens [95% CI]	Med. latency, s [95% CI]	Trace cons.	Term. fail.
S0	0.755 [0.665, 0.840]	11,899 [11,217, 12,628]	11.8 [11.0, 12.7]	1.000	0/200
C1	0.720 [0.625, 0.810]	11,317 [10,599, 12,000]	12.1 [11.3, 12.6]	1.000	0/200
C2	0.830 [0.750, 0.910]	14,590 [12,971, 16,416]	15.8 [15.2, 16.3]	0.985	3/200
C3	0.830 [0.745, 0.915]	13,591 [12,507, 14,928]	16.7 [16.2, 17.4]	0.985	3/200
C4	0.770 [0.685, 0.855]	15,845 [15,457, 16,238]	18.4 [17.8, 19.7]	1.000	0/200

Table 1: Main sweep: final-answer accuracy, token cost, and median latency per condition, with 95% case-clustered bootstrap CIs (§6.5). Latency is the median of case-level medians.

Figure 2: Main-sweep final-answer accuracy by condition with 95% case-clustered bootstrap CIs (Table 1 ). The orchestrated sequence C1–C4 is non-monotonic; S0 is plotted separately because it is not a point on the granularity curve (§4.5).

n=40

cases

\times

five repeats (200 runs per condition).

Contrast	$\Delta$ accuracy [95% CI]	$d_{z}$	Perm. $p$	Holm $p$	Reject
S0 vs. C1	$+0.035$ [ $-0.035$ , $+0.110$ ]	0.14	0.436	0.843	no
C1 vs. C4	$-0.050$ [ $-0.160$ , $+0.050$ ]	$-0.15$	0.422	0.843	no
C2 vs. C1	$+0.110$ [ $+0.010$ , $+0.210$ ]	0.34	0.043	0.172	no
$\mathrm{S0}^{\prime}_{C2}$ vs. C2	$-0.065$ [ $-0.165$ , $+0.025$ ]	$-0.20$	0.265	0.794	no

Table 2: Pre-specified headline family (§6.5) with Holm–Bonferroni correction. No test in the family survives correction.

Figure 3: Cost–quality plane: mean total tokens per case vs. final-answer accuracy, with 95% CIs on both axes (Table 1 ). The dashed staircase traces the non-dominated set

\{

C1, S0, C3

\}

; open diamonds are the matched-token arms

\mathrm{S0}^{\prime}_{C2}

and

\mathrm{S0}^{\prime}_{C^{\star}}

(Table 4 ). Each point aggregates

n=40

cases

\times

five repeats (200 runs per condition).

Condition	JUR	RAT	RCH	Total
S0	30	0	19	49
C1	23	0	33	56
C2	0	15	19	34
C3	7	10	17	34
C4	6	1	39	46

Table 3: Error localization: earliest failing subtask per incorrect main-sweep run (§6.1), over the 219 incorrect runs.

Arm	Accuracy [95% CI]	Tokens (eval)	Target	Deviation	In $\pm 10\%$ band
S0	0.755 [0.665, 0.840]	11,899	—	—	—
$\mathrm{S0}^{\prime}_{C2}$	0.765 [0.655, 0.865]	13,230	14,590	$-9.3\%$	yes
$\mathrm{S0}^{\prime}_{C^{\star}}$	0.760 [0.650, 0.860]	12,864	13,591	$-5.3\%$	yes

Table 4: Matched-token arms (§6.3). Tuning used only the development split; evaluation-set usage is reported as observed.

Injection	Cond.	Subst. success	All-case acc.	Val.-trace acc.	$\Delta$ acc. [95% CI]	Token penalty
Timeout	S0	1.000	0.735	0.735	$-0.020$ [ $-0.110$ , $+0.065$ ]	$-1{,}442$
	C1	1.000	0.880	0.880	$+0.160$ [ $+0.065$ , $+0.265$ ]	$+21{,}793$
	C2	0.965	0.810	0.839	$-0.020$ [ $-0.090$ , $+0.045$ ]	$+4{,}611$
	C3	0.965	0.835	0.865	$+0.005$ [ $-0.030$ , $+0.045$ ]	$+4{,}433$
	C4	0.995	0.775	0.779	$+0.005$ [ $-0.055$ , $+0.065$ ]	$-87$
Outage	S0	1.000	0.780	0.780	$+0.025$ [ $-0.060$ , $+0.110$ ]	$+1{,}138$

Table 5: Localized fallback under injection (§6.4), per (injection, configuration) cell.

\Delta

accuracy is the paired delta against the condition’s own un-injected baseline (Table 1 ), with case-clustered 95% bootstrap CIs.

Figure 4: Paired per-case accuracy deltas under injection, by condition, with case-clustered 95% bootstrap CIs (Table 5 ). Availability perturbations (left, center) straddle zero in every cell except the C1 timeout overshoot (

+0.160

); the content perturbation (right) is negative in every condition. Each cell aggregates

n=40

cases

\times

five repeats (200 runs per (injection, condition) cell).

Multi-Agent LLM Systems

Multi-agent large language model systems coordinate multiple AI agents to solve tasks through communication, specialization, shared tools, and memory.

See all 1,040 papers

Attention Tax, Handoff Tax: A Stylised Model of When Multi-Agent LLM Systems Help
Oct 5, 2026Akshit Anchan, Nayonika SenMulti-Agent LLM Systems Multi-Agent Reasoning
University of Amsterdam Amsterdam, The Netherlands · Northeastern University Boston, MA, United States of America
Specialize Roles, Mix Deployments: Pushing the Cost-Accuracy Frontier of LLM Agent Teams
May 28, 2026Yinsicheng Jiang, Liang Cheng, Yeqi Huang +6Multi-Agent LLM Systems LLM Agent Evaluation
University of Edinburgh · 2Microsoft Research
Markets, Not Planners: Decentralized Orchestration of LLM Agents with Private Information
Aug 24, 2026Xiao Liu, Haoyang Li, Songwei Li +4Multi-Agent Orchestration LLM Agent Orchestration
Knowledge Lab, University of Chicago · University of Virginia · Tsinghua University +2

Oct 5, 2026J/K move · Enter open · S save

Attention Tax, Handoff Tax: A Stylised Model of When Multi-Agent LLM Systems Help

Akshit Anchan, Nayonika Sen

University of Amsterdam Amsterdam, The Netherlands · Northeastern University Boston, MA, United States of America

Multi-Agent LLM Systems Multi-Agent Reasoning Multi-Agent Communication LLM Agent Reliability

Open paper AskPDF arXiv

Recent work on multi-agent LLM systems reaches sharply different conclusions: some results show that a single agent with the same information and compute should dominate a delegated system, others that multi-agent gains grow with task depth. We argue that much of the disagreement comes from modelling different bottlenecks, and introduce a stylised reliability model built around two trade-offs. Decomposition reduces the burden of long contexts but incurs a handoff tax when information is compressed or transferred between agents. Redundancy gains from multiple samples, but its benefit depends on how much their failures are shared. With reasoning budget, verification, and task structure added, the model yields two crossover conditions: decomposition becomes preferable once the attention cost avoided by resetting context exceeds the handoff cost, and parallel sampling at equal budget is eventually preferable when its shared-failure floor lies below the error floor of one agent thinking longer. We connect these regimes to recent theoretical and empirical results. On a ledger-reconciliation task we measure the context-degradation curve and the handoff tax from single-agent and handoff runs alone. From these the model places the crossover at depth 10 and predicts decomposition to win at depths 20, 50, and 100. It does, on step-level and final-balance accuracy, and the decomposed system's success, which the prediction never sees, lands within 9 percentage points of the predicted rate at every depth.

Abstract

Multi-Agent LLM Systems

Explore similar work

Attention Tax, Handoff Tax: A Stylised Model of When Multi-Agent LLM Systems Help

Specialize Roles, Mix Deployments: Pushing the Cost-Accuracy Frontier of LLM Agent Teams

Markets, Not Planners: Decentralized Orchestration of LLM Agents with Private Information