LazyAgent: Demand-Driven Materialization and Physical Optimization of Agentic Programs

Xin Heng

cs.AISep 19, 2026

LazyAgent: Demand-Driven Materialization and Physical Optimization of Agentic Programs

Authors: Xin Heng

Organizations: Tote AI

Abstract

Current agent runtimes that plan before acting generally execute a step once it becomes ready. We present LazyAgent, a unified execution framework for agent-authored programs organized around a live, goal-derived demanded set. LazyAgent refreshes a backward closure from requested outputs as execution state changes and materializes a ready node only when the active goal requires it. This replaces repeated local judgments with one linear-time graph analysis followed by constant-time membership tests, allowing programs to remain broad while execution stays request-specific. On programs that describe more than the current request needs, LazyAgent consistently outperforms the strongest goal-stopping eager baseline by refusing unrelated work before it starts. Adding one unrelated product raises the eager bill by 22.5% and LazyAgent's by 0.0%. LazyAgent saves 42.0% of measured CPU on production scientific workflows and 51.7% of container time on a live release gate spanning four repositories. We also prove and verify exact equivalence when the request reaches the whole graph, leaving no unrelated work to avoid. Beyond permission, goal-relative output projection saves up to approximately 90% of a shared step on two third-party test suites while the identical eager control saves 0.0%; the advantage disappears when the omitted output has no other consumer or the request needs it. Ordering, reuse, and pruning can also save cost, but do not replace permission. Finally, we show that current public benchmarks are eager-shaped and contain almost no unrequested work. A pre-registered planning intervention did not broaden them. These findings motivate benchmarks built from standing programs and sequences.

Figures & tables

Panel of Figure 1 — Figure 1: The two permission regions in one program. Inside the closure the candidate patches form a disjunctive pool: either may be the one that satisfies the goal, so nothing there can be refused in advance, and lazy and stop_eager must tie. Outside the closure the other issues’ branches are ready but outside today’s request; only demand refuses them reliably, while stop_eager skips them when the visit order is kind and pays for them when it is not.

System or line of work	Goal- relativity	Demand re-evaluated on observation	Ready-queue ordering	Field projection
Declarative and incremental execution
Make, Bazel [ 14 , 8 ]	S	$\circ$	$\circ$
Build systems à la carte [ 30 ]	S	✓	✓
Adapton [ 18 ]	S	✓	$\circ$
TensorFlow graph mode [ 1 ]	S		$\circ$
Volcano, Eddies [ 17 , 7 ]	S	✓	✓	✓

Table 1: Positioning. Goal-relativity: S — execution is restricted to a backward closure rooted at requested or designated outputs; P — restricted by a predicate over the request, model features, or observations; — — no such restriction is established. Other columns: ✓ present,

\circ

partial, blank absent. Checkmarks mark component precedents; novelty resides in the unified LazyAgent row.

Arena	Mode	Cost unit	Shape	What it is
WfCommons / WfInstances [ 13 , 15 ]	replay	measured CPU s	multi-product	154 readable production scientific-workflow traces, 86 with several output products; 303 seeded product requests.
DataSciBench [ 48 ]	live	real tokens	multi-product	The benchmark supplies executable dependency-structured plans; our adaptation retains 95 goals over 28 tasks, including 28 multi-goal follow-up tasks.
SWE-bench Verified [ 20 , 40 ]	replay; live gate	recorded; container seconds	pool	500 human-validated tasks; also the substrate for a live release gate with real agent patches run in Docker.
Defects4J v1.1 [ 21 , 39 ]	replay	wall seconds	pool	395 bugs; 185 settle under our replay criterion; also composed into portfolios.
SimulCost [ 10 ]	replay; live	analytic cost	pool	2,304 multi-round problems from the pinned arXiv v3 release; also composed into portfolios.
Bazel [ 8 ]	replay	action CPU s	build graph	419 native targets on two production packages; exploratory second structural domain.

Table 2: Arenas. Cost columns are arena-native and never rescaled; “live” means the materializer ran real code or a real model during our measurement.

Study	Mode	$n$	Headline vs stop_eager	Control
WfCommons permission	replay	303 requests	42.0% [37.5, 46.7]; 211 / 0 / 92	68 single-product instances give exactly 0.0%
WfCommons shape law	replay	235 observations	50.3% (one product), 21.4% (half), 0.0% (all)	86 / 86 exact ties when the goal names everything
Pool arenas (SWE-bench, SimulCost, Defects4J)	replay	3,199 asks	0.0% , bit-identical bills	Predicted by Prop. 2 ; a saving here would be a bug
DataSciBench, agent-authored	live	95 goals, 28 tasks	12.8% clustered [7.7, 18.2]; 15 / 0 / 13	Quality matched on every task
Portfolio position sweep	replay	3,572 rows	0.0% asking first $\to$ 85.5% asking last	893 / 893 all-products ties; 0 losses
Program width, $N{=}3..6$	replay	10,680 rows	capture 100% ( lazy ) vs 0% open / $\approx$ 50% settleable	2,538 / 2,538 controls tie; deltas 100% off-path

Table 3: Evidence at a glance. One row per study, reported against stop_eager . The last column is the control that had to pass for the row to mean anything; every one of them did.

Panel of Figure 2 — Figure 2: Same program, same policies, same cost column — only the ask changes. Requesting the product that sorts first is the equivalence region; requesting a later product is the separation region. A Defects4J product is a repair target; a SimulCost product is a design study. The zero-valued first Defects4J bar is drawn as a 1-point stub solely for visibility; its label and reported value remain 0.0%.

Panel of Figure 3 — Figure 3: Separating what the experiment manufactures from what the policy achieves. Panel (a), the share of the guard-eager bill outside the demanded product, tracks $(N{-}1)/N$ . Panel (b), the fraction of that opportunity each policy avoids, is flat in $N$ : lazy captures all of it everywhere; goal stopping captures none when the demanded product cannot settle and about half when it can.

Operation	Arena	Costs	Result	Read it as
Project	Live DataSciBench, 95 goals	live	$+3.2\%$ ; narrow asks bill $-14.3\%$ ; no significant blind-judge preference ( $p=0.29$ )	Fewer tokens; no detected preference
Project	Authored isolation pair	authored	$+84.3\%$ (named consumer) vs $+0.0\%$ (no consumer)	Pays iff the dropped field has another consumer
Project	CI verify job, 2 third-party repos	measured cpu	$+89.1\%$ , $+80.1\%$ (named, off-path); $+0.0\%$ with no consumer and when both demanded	Same shape, prices from a clock; controls zero independently
Project	Authored sessions, coverage sweep	authored	$+34.3\%$ , $+35.6\%$ ; $+35.4\%\to+72.9\%$	Grows as session coverage falls
Fidelity	Authored ladder, named consumer / no consumer / needed	authored	$+96.1\%$ / $+0.0\%$ / $+0.0\%$	Projection one granularity down; sign pattern, not magnitude
Reorder	No rule may stop the run; 5 arenas, every task	ex-ante, replay, live	0.0% bill change, incl. 8 random orders	Exact identity under C1–C3

Table 4: Physical operations at a glance. Remainder is demand-specific: lever worth to lazy minus lever worth to the matched eager arm. A negative remainder means the eager control captured more of the lever’s absolute effect than lazy did, not that the lever failed. The final block is not a remainder: speculation’s rows report a cost/latency pair against the no-speculation lazy arm, where positive latency is faster and negative cost is more expensive. Speculation’s latency is measured in rounds containing real work , never in seconds; Appendix B.6 says why a clock cannot be used here.

Cell	Cost unit	Preconditions	Result
All three hold — separation predicted
Live release gate, SWE-bench	container seconds	all three	$51.7\%$ , $52.1\%$
Live projection, Jinja and Flask	measured CPU s	all three	$+89.1\%$ , $+80.1\%$
Composed portfolios, Defects4J and SimulCost	native arena costs	all three	separation region
DataSciBench projection	real tokens	all three	$+3.2\%$ pooled; $14.3\%$ on the 126 narrowable nodes ( gpt-5.6-terra ; sign is planner-bound, § 8.4 )
Fidelity ladder	authored	all three	$+96.1\%=(206-8)/206$ ; magnitude authored

Table 5: Selected controlled separation and equivalence cells. Ready-but-undemanded means the program contains off-path work; separable means that work can be skipped physically; and material means its cost survives the accounting. Projection and fidelity additionally require a demand-specific gap relative to the static control.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Limitation	Where	How we handle it, or why it stands
Largest separation result is replayed workflows, not LLM programs; product requests are seeded, not observed	§ 7.1	The agentic arena carries the live version of the same claim at smaller magnitude; we never pool the two
Near-complete coverage has a structural benchmark-design account; its historical cause is not identified	§ 5 , § 7.5	The census (98.6% demanded, 27 of 28 plans with no off-path work) is measured. Across 168 valid plans from three paired repeats, demand billing produced 0.0% off-path work against 0.8% under readiness billing (paired per-plan difference $-0.9$ points [ ${-2.4}$ , $0.0$ ], $p=0.501$ ); the prompt-level null rules out the tested billing intervention as evidence, not the benchmark-design account
Agentic corpus sits at session coverage $\approx 1.0$ , so the effect there is small	§ 7.4	Reported as a property of the corpus, with a node-level census; the conditional and the average are both given
Portfolio topology is authored	§ 7.6	Native costs, exhaustive position sweeps, retained zeros, exact-tie controls, setup-share sensitivity, normalization by opportunity
The four widths regroup one task pool rather than forming four datasets	§ 7.7	Stated; capture rates rather than raw savings are the quantity compared across widths
Each live gate run is 12 rows from 4 clusters	§ 7.8	Case study, never a population claim; smallest exact Wilcoxon $p$ at 4 clusters is 0.125 and no $p$ is quoted off it

Appendix

Table 6: Limits on what the evidence covers.

Limitation	Where	How we handle it, or why it stands
Fidelity’s $+96.1\%$ magnitude is authored, not measured	§ 8.2	Projection has measured support: $+3.2\%$ on live DataSciBench tokens and $+89.1\%$ / $+80.1\%$ on measured Jinja/Flask CPU. Fidelity remains authored ladder arithmetic, so only its sign and two zero controls transfer
Every remainder is computed in one resource	§ 8	Prune already shows the resource can decide the verdict while the factorial stays internally consistent; the other rows are not audited for it, and reordering — a latency lever scored on spend — is the obvious suspect
Speculation counts rounds rather than seconds, on uniform priors, leaning on one deep program	App. B.6	A clock in a replay would time the harness; the priors are optimistically ordered relative to an indifferent speculator, and both are stated
Live leg of the ordering identity is one authored program at one width, one vendor	§ 8.3	Reported as the identity on the executed set, with the bill allowed to move by the materializer’s own run-to-run noise
Ties are classified order-contingent from one recorded visit order	§ 7.1	A shuffled-selector run is done for the operation rows but not for every permission tie
Measured population varies from run to run	§ 8.4	Repeating one planner flips 6 of 31 tasks in or out (Jaccard 81%), so the disjoint sets across planners are substantially run churn and not a capability ranking; shared-goal comparisons use the 77-goal intersection, while own-population figures, including projection, are labeled as such

Appendix

Table 7: Limits on the instruments.

Scenario term	Meaning
Named-consumer case	An output omitted from today’s request is still required by another consumer recorded in the standing program, so a static eager control cannot discard it.
No-consumer control	No node reads the omitted output, so ordinary dead-output elimination is available to both eager and lazy arms.
All-demanded control	Today’s request needs every output, so neither arm may project any of them away.
Standing session	Several requests arrive over time against one persistent graph; prior artifacts may be reused when still valid.
Session coverage	Fraction of the standing program’s products requested at least once during the session.
Invalidation	Fraction of requests whose changed input makes an earlier artifact and its downstream artifacts stale.

Appendix

Table 8: Reader-facing vocabulary for the controlled scenarios in this appendix. These terms describe structural conditions, not software identifiers.

Scenario	Worth to stop_eager	Worth to lazy	Remainder
Gate output only; trace has another consumer (bills 256 / 256 / 56)	$0.0\%$	$+78.1\%$	$+78.1\%$
Gate and trace both requested (bills 261 / 261 / 261)	$0.0\%$	$+0.0\%$	$+0.0\%$
Named-consumer case	0.0%	$+84.3\%$	$+84.3\%$
No-consumer control	$+84.3\%$	$+84.3\%$	$+0.0\%$
Release-package standing session	0.0%	$+34.3\%$	$+34.3\%$
Model-refresh standing session	0.0%	$+35.6\%$	$+35.6\%$

Appendix

Table 9: Projection on authored programs, where we set the prices. The named-consumer/no-consumer pair is the one-edge isolation: the programs differ only in whether the expensive undemanded output has a downstream consumer. The gate-output/all-demanded pair separately varies whether today’s goal requests that output. The session rows carry the same shape into a standing program under a ticket sequence. Magnitudes are authored; the sign and control pattern are the claim, and the measured version is in Section 8.1 .

Arena	Cost unit	$n$	Stop rule	Demand	Perm. gap	Overlap
Release-package session	authored CPU	5	42.0%	0.0%	59.5%	70.7%
Model-refresh session	authored CPU	5	25.8%	0.0%	41.7%	61.8%
WfCommons (60 req., 200-task cap)	recorded CPU	60	1.8%	0.0%	5.0%	36.0%
DataSciBench	real tokens	95	4.2%	0.0%	19.8%	21.4%

Appendix

Table 10: Overlap by arena — of the cost difference between the stopping rule and demand, the share a price heuristic recovers with no notion of what was asked for. All four rows come from the one ordering grid, so the columns are commensurable.

Scenario	Stopping	Stopping + cheapest-first	Worth
Named-consumer projection case	332	932	$-180.7\%$
No-consumer projection control	332	1,102	$-231.9\%$

Appendix

Table 11: Cheapest-first handed to both arms. Top: the authored pair, where the eager arm pays about three times what arbitrary order cost it. Bottom: how often the inversion appears per arena, and the lazy column that is zero everywhere.

LLM Agent Workflow Optimization

LLM agent workflow optimization studies how to design and run multi-step agents for better task success, reliability, latency, and cost.

See all 262 papers

Aug 25, 2026

PeakBench: Benchmarking Resource-Aware Tool Invocation in LLM Agents

LLM agents increasingly solve tasks by invoking multiple tools, where parallel execution is essential for low latency but difficult to manage safely. Existing agent benchmarks primarily evaluate tool selection, argument generation, and end-to-end success under mostly serial execution, largely overlooking valid parallelization and resource-constrained scheduling. This missing scheduling dimension creates a practical failure mode: serial execution is safe but slow, while resource-agnostic parallel execution is fast but prone to avoidable resource overflows. To address this gap, we introduce PeakBench, a benchmark of executable multi-tool workflows with execution-grounded dependency annotations and measured resource profiles. A central challenge in evaluating such workflows is attribution: failures and inefficiencies may arise from incorrect dependency planning, poor resource-constrained scheduling, or both. PeakBench addresses this challenge with a two-part evaluation framework that disentangles logical planning from physical scheduling, with dedicated metrics for each dimension. Using this framework, we show that strong logical planning does not reliably translate into safe or efficient execution under resource constraints. We further show that exposing resource information can reduce avoidable overflows and improve resource utilization, making PeakBench a useful testbed for diagnosing resource-aware agent behavior. Code is available at https://github.com/Czzzk/Staggering-the-Peaks.

Zhi-Kai Chen, Xu-Xiang Zhong, Song-Yan Li +2

School of Artificial Intelligence, Nanjing University, China · National Key Laboratory for Novel Software Technology, Nanjing University, China · Nanjing University, China

Tool-Using Agents LLM Agent Evaluation AI Agent Benchmarks LLM Agent Planning

Ask PDF

Aug 1, 2026

AgentSLABench: Evaluating and Benchmarking Agentic Systems Under Resource Constraints

We present AgentSLABench, a resource-aware evaluation framework for autonomous AI agents that measures correctness alongside latency, cost, compute, memory, and network usage under declared resource budgets. Unlike standard benchmarks that report only accuracy, AgentSLABench produces a multi-dimensional profile per agent per task - the same way systems profilers (perf, pprof, cProfile) measure resource consumption of code, but extended with task correctness as a first-class dimension. AgentSLABench provides 16 task environments across 6 categories (5 core: multi-hop QA, retail substitution, code generation, web shopping, travel planning; 11 extended) with isolated Docker containers, declared CPU/memory/time/network budgets, sealed test sets with SHA256 hashes, and a standardized profiling protocol. We profile 5 general-purpose baseline agents (ReAct, PlanAndSolve, Reflexion, CoT, Random) plus 4 task-specialized agents, finding that specialized agents achieve 100% success on 3/5 core tasks (fact_qa, web_shopping, travel_planning) and 66.7-83.3% on retail and code_gen, while general baselines fail entirely on 4/5 domain tasks. Crucially, we report the Efficiency-Adjusted Success Rate (EASR) - success weighted by resource consumption relative to declared budgets - revealing that high accuracy at unbounded cost is not production-viable. We release the full infrastructure, sealed test sets, and profiling results to enable reproducible, resource-aware agent evaluation.

Meher Bhaskar Madiraju, Meher Sai Preetam Madiraju

AI Agent Evaluation Agent Evaluation AI Agent Benchmarks

Ask PDF

Sep 17, 2026

Not All AI Agents Are Equal: Characterizing Resource and Performance Dynamics

LLM-based AI agents process user requests through iterative reasoning and tool execution, often involving the invocation of remote LLM APIs with local tool containers. This execution model can make the optimization of agent serving difficult because latency, local resource demand, and container bottlenecks inter-mix across requests. However, the current agent ecosystem runs without much consideration of resource dynamics, which results in significant waste of the precious resources. This paper analyzes the resource inter-mix of AI agents for three representative tasks: retrieval-augmented question answering, web search, and software coding. To this end, we characterize the latency with respect to the resource dynamics of processing multiple requests and tasks concurrently. Our measurements show that agents have a wide range of behaviors depending on tasks, so that even the same tool can differ substantially in resource dynamics. We also find that running multiple requests concurrently exposes task-dependent bottlenecks in resource dynamics such as CPU, disk I/O, and memory. Furthermore, we uncover that faster LLM responses or more CPU cores do not always accelerate agents. Based on these observations, we demonstrate new optimization opportunities that exploit the resource dynamics of tasks: CPU-aware tool admission and task-aware CPU allocation. Our results show that the latency of CPU-sensitive agent tasks improves

\sim

5.4

\times

, and the average latency across multiple tasks is reduced

\sim

32% compared to native agents.

Wonmi Choi, Minuk Park, Zhixiong Niu +3

Department of Computer Science and Engineering, Korea University · Microsoft Research Asia

LLM Inference Efficiency LLM Agent Workflow Optimization LLM Agent Serving

Ask PDF

Abstract

LLM Agent Workflow Optimization

Explore similar work

PeakBench: Benchmarking Resource-Aware Tool Invocation in LLM Agents

AgentSLABench: Evaluating and Benchmarking Agentic Systems Under Resource Constraints

Not All AI Agents Are Equal: Characterizing Resource and Performance Dynamics