cs.DCJan 8, 2026

Nalar: Workflow-Aware Management of Agentic Applications

Authors: Saurabh Agarwal, Marco Laju, Donghyun Son, Nitin Kedia, Myungjin Lee, Jayanth Srinivasa, Aditya Akella

Organizations: UT-Austin · Cisco-Research

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

LLM-driven agentic applications automate complex, multi-step tasks, but serving them efficiently remains difficult due to heterogeneous components, dynamic model-driven control flow, long-lived state, and highly variable latencies. Nalar is a serving framework for agent workflows that separates workflow specification from execution while providing the runtime visibility and control needed for robust performance. Nalar preserves ordinary Python interfaces and control flow through lightweight auto-generated stubs that turn agent and tool invocations into futures carrying dependency and execution-context metadata. A two-level control architecture combines global policy computation with local event-driven enforcement to support adaptive routing, scheduling, and resource management across evolving workflows. A workflow-aware KV-cache layer enables the runtime to manage cache placement and lifetime. Together, these mechanisms enable scalable, efficient, policy-driven serving of heterogeneous agentic applications without burdening developers with orchestration logic. Across three agentic workloads, Nalar reduces tail latency by 34-74% and achieves up to 3.38x speedups.

Figures & tables

Figure 1 . An example agentic application: OpenSWE ( LangChain, 2025 ) workflow for software development.
Figure 2 . Challenges serving agentic application: Serving concurrent agent workflows without global orchestration. Independent component scheduling leads to head-of-line blocking and wasted recomputation, demonstrating the need to account for workflow progress as well as cluster-wide resource information.
Figure 3 . Workload heterogeneity in a workflow.
Figure 4 . Nalar Overview: Nalar takes user-specified files and generates stubs (§ 3.1 , § 3.2 ) that replace original function calls with controllable hooks to generate futures (§ 3.3 ). These stubs connect the user program to the runtime controllers. At deployment, Nalar launches and manages the runtime (§ 4 ), where component-level controllers and the global controller coordinate to enforce scheduling, routing, and resource policies while managing KV caches.
Figure 5 . Example Agent : A software developer agent definition. It calls a documentation lookup tool, a shared inference engine and another testing agent. These calls look like calls to local objects.
Figure 6 . Three-agent workflow: The planner agent decomposes a natural-language coding request into subtasks. Each subtask is sent to a Developer agent from Figure 5 , which returns a future indicating test success or failure. The program creates and consumes these futures, retrying failing subtasks. Unless the user explicitly desires(Line 29) they don’t need to interact with futures.
MetadataStructureDescription
dependencieslist(agentA:ip, …)List of dependencies required to compute the future’s output
creatoragentName:ipComponent that created the future
executoragentName:ipComponent instance currently assigned to execute the future
consumerslist(future-id, …)Components waiting to consume the future’s value
session-idstr:UUIDThe user-request associated with the future
Table 1 . Nalar ’s future metadata.
Figure 7 . Future Generation Timeline: For the agent workflow depicted in Figure 6 we depict a timeline for future generation and how their consumers are updated and their values realized in Nalar .
Figure 8 . Nalar ’s two-level control. Each component has an associated controller with it. Each node has a local node store. The global controller communicates with each agent and workflow driver, through the node store.
HintValuesDescriptions
statefulBooleanTrue indicates for a session successive calls to the agent will be routed to the same instance
batchableBooleanTrue indicates that module can accept a batch of request
reusableBooleanResource created during a session can be reused by subsequent invocations within the same session.
preemptablefunctionTrue indicates that running invocations of the component may be preempted by the runtime.
max_instancesIntegerIndicates the max number of instances to initialize
min_instancesIntegerIndicates the min number of instances the framework should keep alive
Table 2 . Nalar ’s hint interface
Figure 9 . End-to-End Evaluation The bars represent average latency, the whiskers represent P50, P95 and P99 latencies.
Figure 12
Number of FuturesOne-Level DesignTwo-level Design
Time(ms)Time(ms)
10241.20.1
20482.30.1
40962.80.2
81923.40.4
163843.90.4
Table 3 . Impact of Two-level Control.
Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

CategoryMetricDescription
Future metadataSession IDUniqueID associated with a session.
 Future IDUnique ID associated
 ComponentIdentifies the agent or tool invoked by the future.
 CreatorComponent that created the future.
 ExecutorAgent or tool instance assigned to execute the future.
 DependenciesFutures whose values must be available before the future becomes ready.
Appendix
Table 4 . Nalar telemetry collected by component-level controllers. Per-future telemetry captures workflow dependencies and timing, while instance-level metrics summarize local load and resource availability.
Figure 13 . Control Interaction in Nalar . The above figure shows the interaction and relevant updates to metadata when a future is being migrated. An important feature is that it’s entirely locally coordinated, i.e. , the global controller only issues the migrate command, the component level controllers coordinate it among themselves.

Explore similar work

CardsList

  1. Pythia: Exploiting Workflow Predictability for Efficient Agent-Native LLM Serving

    Apr 28, 2026Shan Yu, Junyi Shu, Yuanjiang Ni +14WorkloadCache

    UCLA · Alibaba Cloud Computing · Alibaba Group +6

  2. A Workflow-Aware Serving Layer for Agentic Applications

    Jul 3, 2026Jiayi Qian, Zishen Wan, Hanchen Yang +3Agentic WorkflowsAgentic Inference

    Columbia University New York, NY, USA · Intel Los Angeles, CA, USA

  3. GraphFlow: A Graph-Based Workflow Management for Efficient LLM-Agent Serving

    May 21, 2026Ao Li, Shangpeng Yang, Fahao Chen +3Agentic Workflow DesignGraph Representations

    School of Cyber Science and Engineering, Xi’an Jiaotong University, Xi’an, China · School of Artificial Intelligence, Shandong University, Jinan, China · 3Shanghai Advanced Research Institute, Chinese Academy of Sciences, Shanghai, China.