Local-Splitter: A Measurement Study of Seven Tactics for Reducing Cloud LLM Token Usage on Coding-Agent Workloads

Justice Owusu Agyemang; Jerry John Kponyo; Elliot Amponsah; Godfred Manu Addo Boakye; Kwame Opuni-Boachie Obour Agyekum

cs.DCApr 14, 2026

Local-Splitter: A Measurement Study of Seven Tactics for Reducing Cloud LLM Token Usage on Coding-Agent Workloads

Authors: Justice Owusu Agyemang, Jerry John Kponyo, Elliot Amponsah, Godfred Manu Addo Boakye, Kwame Opuni-Boachie Obour Agyekum

Organizations: Sperix Labs · VIA Cybersecurity Lab, KNUST · Quantum and Assistive Technologies Lab, KNUST

Abstract

We present a systematic measurement study of seven tactics for reducing cloud LLM token usage when a small local model can act as a triage layer in front of a frontier cloud model. The tactics are: (1) local routing, (2) prompt compression, (3) semantic caching, (4) local drafting with cloud review, (5) minimal-diff edits, (6) structured intent extraction, and (7) batching with vendor prompt caching. We implement all seven in an open-source shim that speaks both MCP and the OpenAI-compatible HTTP surface, supporting any local model via Ollama and any cloud model via an OpenAI-compatible endpoint. We evaluate each tactic individually, in pairs, and in a greedy-additive subset across four coding-agent workload classes (edit-heavy, explanation-heavy, general chat, RAG-heavy). We measure tokens saved, dollar cost, latency, and routing accuracy. Our headline finding is that T1 (local routing) combined with T2 (prompt compression) achieves 45-79% cloud token savings on edit-heavy and explanation-heavy workloads, while on RAG-heavy workloads the full tactic set including T4 (draft-review) achieves 51% savings. We observe that the optimal tactic subset is workload-dependent, which we believe is the most actionable finding for practitioners deploying coding agents today.

Figures & tables

Figure 2: Cloud token savings (%) per tactic in isolation. Error bars show the half-range across two runs.

Tactic	WL1 (edit)	WL2 (explain)	WL3 (chat)	WL4 (RAG)
T1 route	29.2%	68.8%	58.9%	38.0%
T2 compress	22.4%	19.3%	$-$ 2.6%	18.9%
T3 cache	9.6%	$-$ 1.0%	$-$ 3.8%	2.4%
T4 draft	$-$ 35.0%	$-$ 40.5%	12.6%	$-$ 31.1%
T5 diff	5.1%	$-$ 3.4%	$-$ 4.4%	39.3%
T6 intent	5.0%	$-$ 5.5%	0.3%	$-$ 1.7%

Table 1: Cloud token savings (%) per tactic in isolation. Local model: Llama 3.2 3B. Cloud model: Gemma 3 4B (simulated locally). Negative values indicate the tactic increased cloud tokens. Mean of two runs (10 samples per workload each); T1–T4 varied by

\pm

2–7 pp; T5–T7 showed higher variance (

\pm

3–14 pp) due to model non-determinism and small sample size.

Figure 3: Cloud token savings (%) for tactic combinations. Error bars show the half-range across two runs.

Subset	WL1 (edit)	WL2 (explain)	WL3 (chat)	WL4 (RAG)
T1+T3	33.7%	70.4%	57.4%	36.2%
T1+T2	45.0%	79.0%	57.4%	44.3%
T1+T2+T3	42.6%	79.6%	59.6%	43.8%
all	29.4%	71.6%	59.1%	51.1%

Table 2: Cloud token savings (%) for tactic combinations. Mean of two runs; individual runs varied by

\pm

1–5 pp.

Figure 4: Token savings vs. dollar cost per 10-sample workload. Points closer to the lower-right are Pareto-optimal (more savings, less cost). Faded markers at 0% savings show the baseline.

Subset	Baseline	Treatment	Tie	Incon.	Errors
T1	15	5	0	17	3
T1+T2	15	6	1	17	1

Table 3: Quality evaluation: judge-model pairwise verdicts for T1 and T1+T2 vs. baseline (40 pairs each, all workloads combined). High inconsistency reflects the 4B judge model’s limited discrimination.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Workload	Subset	Cloud tok.	Local tok.	Saved (%)	Cost ($)	Latency (ms)
WL1	baseline	11,007	0	—	0.0043	6,933
WL1	T1	7,675	2,618	30.3	0.0031	17,218
WL1	T2	8,355	0	24.1	0.0037	17,481
WL1	T5	10,074	0	2.3	0.0042	11,358
WL1	T6	10,500	0	$-$ 1.8	0.0040	10,288
WL1	T7	10,833	0	$-$ 5.0	0.0041	10,187

Appendix

Table 4: Full evaluation metrics per workload and tactic subset (representative run; Tables 1 – 2 report means of two runs). Cloud tokens = input + output sent to the cloud model. Cost uses OpenAI gpt-4o-mini rate card as a proxy.

LLM Agent Workflow Optimization

LLM agent workflow optimization studies how to design and run multi-step agents for better task success, reliability, latency, and cost.

See all 256 papers

Inference Economics of Enterprise Coding Agents: A Case Study of Cloud vs. On-Premise LLMs
Jul 13, 2026Sheng-Wei Peng, Yi-Hsun Lin, Yi-Pei LeeLLM Serving Software Engineering Agents
Pegatron Corporation Taipei, Taiwan
Cross-Lingual Token Arbitrage: Optimizing Code Agent Context Windows via Local LLM Preprocessing
Jun 2, 2026Mehmet Utku ColakAI Coding Agents LLM Prompting
Department of Computer Engineering, Istanbul Technical University · Istanbul, Turkey
TraceLab: Characterizing Coding Agent Workloads for LLM Serving
Jun 29, 2026Kan Zhu, Mathew Jacob, Chenxi Ma +4LLM Serving AI Coding Agents
University of Washington · 2Wuhan University of Technology · *Work done while interning at the University of Washington. +1

Jul 13, 2026J/K move · Enter open · S save

Inference Economics of Enterprise Coding Agents: A Case Study of Cloud vs. On-Premise LLMs

Sheng-Wei Peng, Yi-Hsun Lin, Yi-Pei Lee

Pegatron Corporation Taipei, Taiwan

LLM Serving Software Engineering Agents Cost-Aware Inference LLM Agent Reliability

Open paper AskPDF arXiv

Autonomous coding agents force engineering organizations to choose between API-based frontier models -- strong reasoning at high token cost -- and on-premise quantized open-weights models, which promise low-marginal-cost scaling and data sovereignty at some loss of reasoning fidelity. We study this trade-off through a single-developer, non-randomized longitudinal case study over two contiguous 28-day periods on a production monorepo: an API-based Claude Opus 4.7/4.8 configuration using Claude Code versus an on-premise GLM-5.1/5.2 configuration using Opencode, quantized to NVFP4, on NVIDIA Blackwell hardware. Analyzing LLM telemetry and Git history, we find that prompt caching (99.3% hit rate) cuts realized API cost by 88.6% to an effective $0.57 per million tokens -- below even the $2.83 amortized unit cost of the shared on-premise slice (a utilization-dependent inversion; total realized spend and total cost of ownership (TCO) are the robust quantities). At comparable gross code churn, the local configuration was associated with a far higher defect-repair burden: a Fix Commit Ratio (FCR) of 74.9% versus 45.9%, with the odds of a commit being a repair 2.6 to 4.9 times higher within every difficulty tier (Mantel-Haenszel OR = 3.61). Under Taiwan-market parameters and a symmetric labor model, on-premise deployment nonetheless saves 40.1% of true TCO under shared GPU allocation, whereas dedicated reservation costs 43.8% more than the cached API. Under shared allocation, the genuine penalty is not monetary but a measurable developer-experience burden -- timestamp indicators show more work trapped in debugging spirals and a slower commit cadence -- and an offline replay shows hybrid routing gateways trade defect rate for infrastructure savings along a cost-quality frontier rather than dominate the pure-API baseline.

Abstract

LLM Agent Workflow Optimization

Explore similar work

Inference Economics of Enterprise Coding Agents: A Case Study of Cloud vs. On-Premise LLMs

Cross-Lingual Token Arbitrage: Optimizing Code Agent Context Windows via Local LLM Preprocessing

TraceLab: Characterizing Coding Agent Workloads for LLM Serving