AID: A Framework for AI Infrastructure Dynamics

Abi Aryan

cs.DCOct 3, 2026

AID: A Framework for AI Infrastructure Dynamics

Authors: Abi Aryan

Organizations: Joule

Abstract

A useful model of AI inference infrastructure must specify the system state, the information available to an observer, and the decisions the model is intended to support. We introduce AID (AI Infrastructure Dynamics), a framework for describing this learning problem across coupled physical, computational, networking, and serving processes. The formulation allows structured and variable-size state, asynchronous observations, multiple physical timescales, and demand that responds to service. We distinguish representations that support prediction under an existing policy from those that preserve service outcomes under changed actions, and separate both from identifying intervention responses. Two analytical results describe a lower bound on prediction error when available observations cannot distinguish models and a sufficient condition for exact controlled state reduction. These results apply established information and state-abstraction principles to AI infrastructure. We then describe a validation protocol for cache representations, workload histories, measurement availability, and imposed actions.

Figures & tables

Panel of Figure 1 — Figure 1: The AID system boundary. Solid arrows show demand, actions, observations, service, and schematic cross-layer coupling, not an exhaustive dependency graph or a serial execution pipeline. The dashed path represents service-dependent arrivals through client state; it is absent in open-loop replay. External intent and client rules are distinguished from realized demand.

Panel of Figure 2 — Figure 2: From latent infrastructure state to a candidate learned representation. The observation arrow summarizes a history of partial measurements, not a direct observation of the current state. The mapping $\phi$ uses only available information. Prediction under changed actions requires testing what the representation preserves; the diagram does not establish sufficiency or identify intervention responses from logs.

Panel of Figure 3 — Figure 3: Observational ambiguity in a deterministic cache example. The two admissible hidden states have the same occupancy-only history but different responses to the same imposed action. Here $\delta_{1}$ and $\delta_{0}$ are point masses for a binary cache-hit indicator. The action is prescribed in both states; equal telemetry does not imply equal response. This is a mathematical construction, not an empirical cache result.

Explore similar work

INFRAMIND: Infrastructure-Aware Multi-Agent Orchestration
Jun 9, 2026Ahasan Kabir, Jiaqi Xue, Mengxin Zheng +1Multi-Agent Orchestration Agentic Inference
University of Central Florida
The Organization of Inference: Information, Resource Constraints, and AI Production
Sep 17, 2026Yukun Zhang, Kemu Xu, Yishen ChenAgentic Inference Artificial Intelligence Systems
The Chinese University of Hong Kong Hong Kong, China · University of Edinburgh Edinburgh, United Kingdom · The Chinese University of Hong Kong, Shenzhen Shenzhen, China
Grounded Inference: Principles for Deterministically Encapsulated Generative Models
Jun 18, 2026Marty O'NeillGenerative Models Principles
Odenton, MD, USA

Jun 9, 2026, cs.AIJ/K move · Enter open · S save

INFRAMIND: Infrastructure-Aware Multi-Agent Orchestration

Ahasan Kabir, Jiaqi Xue, Mengxin Zheng, Qian Lou

University of Central Florida

Multi-Agent Orchestration Agentic Inference Infrastructure Multi-Agent Large Language Model Systems

Open paper AskPDF arXiv

Existing multi-agent LLM orchestration methods, ranging from brute-force ensembles to learned routers, select models and topologies based on task and model features. However, these methods do not consider the runtime state of the serving infrastructure. On shared GPU clusters under concurrent load, this infrastructure blindness causes systematic resource underutilization: preferred models accumulate deep request queues while equally capable alternatives sit idle. In multi-agent pipelines, where each query triggers multiple sequential model calls, these delays then compound across every downstream step. Closing this gap is challenging because the relevant infrastructure signals (queue depths, KV-cache pressure, latencies) are dynamic and noisy, and they must drive three different decisions: planning, per-step routing, and scheduling. We introduce INFRAMIND, a framework that makes the entire multi-agent stack infrastructure-aware. An infra-aware planner conditions topology and role selection on real-time system load and remaining budget, biasing toward simpler graphs under congestion and richer ones at low load. An infra-aware executor then observes per-model queue depths, cache utilization, and response latencies at each agent step to decide which model to call and how deeply to reason; a budget-aware scheduler further reorders each model's queue so that urgent requests are served first. Cast as a hierarchical constrained MDP and solved end-to-end via reinforcement learning, the system learns to balance quality against latency automatically. Across five benchmarks, INFRAMIND delivers up to +7.6 pp accuracy over the prior baseline at low load with up to 7x lower latency, and sustains up to 99.9% SLO compliance under high load where every baseline drops below 50%.

Abstract

Explore similar work

INFRAMIND: Infrastructure-Aware Multi-Agent Orchestration

The Organization of Inference: Information, Resource Constraints, and AI Production

Grounded Inference: Principles for Deterministically Encapsulated Generative Models