AI Runtime Infrastructure

Christopher Cruz

cs.AIFeb 28, 2026

AI Runtime Infrastructure

Authors: Christopher Cruz

Abstract

We introduce AI Runtime Infrastructure, a distinct execution-time layer that operates above the model and below the application, actively observing, reasoning over, and intervening in agent behavior to optimize task success, latency, token efficiency, reliability, and safety while the agent is running. Unlike model-level optimizations or passive logging systems, runtime infrastructure treats execution itself as an optimization surface, enabling adaptive memory management, failure detection, recovery, and policy enforcement over long-horizon agent workflows.

Figures & tables

Figure 1: The full agentic AI systems stack. AI runtime infrastructure operates as an execution-time control layer between agent orchestration and model serving, observing execution state and intervening during runtime to optimize task success, efficiency, reliability, and safety. Observability and evaluation systems span the stack but do not influence execution-time behavior.

Runtime Enforcement for AI Agents

Runtime enforcement for AI agents checks and constrains actions as they execute, helping prevent unsafe or unauthorized effects.

See all 276 papers

InfraBench: Evaluating Infrastructure Agents Across Layers, Lifecycle, and Risk
Jul 31, 2026Yuan Gao, Zeren Yang, Junnan Li +6AI Agent Reliability AI Agent Evaluation
Wanxiang · University of Wisconsin–Madison · Iowa State University
INFRAMIND: Infrastructure-Aware Multi-Agent Orchestration
Jun 9, 2026Ahasan Kabir, Jiaqi Xue, Mengxin Zheng +1Multi-Agent System Optimization Multi-Agent Orchestration
University of Central Florida
Beyond Autonomy: A Dynamic Tiered AgentRunner Framework for Governable and Resilient Enterprise AI Execution
May 11, 2026Kai Pan, Rong HouLLM Agent Orchestration AI Agent Safety
A2A Lab

Jul 31, 2026J/K move · Enter open · S save

InfraBench: Evaluating Infrastructure Agents Across Layers, Lifecycle, and Risk

Yuan Gao, Zeren Yang, Junnan Li, Shawn+5

Wanxiang · University of Wisconsin–Madison · Iowa State University

AI Agent Reliability AI Agent Evaluation AI Agent Benchmarks AI Agent Safety Benchmarks

Open paper AskPDF arXiv

Managing modern computing infrastructure has become a steadily harder problem due to the ever-increasing complexity. Recent advances in AI agents create a timely opportunity to automate infrastructure management tasks, but it remains unclear how well such agents can handle real-world infrastructure complexity. We present InfraBench, a benchmark for infrastructure agents organized around system layers, the operational lifecycle, and risk. The current prototype couples container, VM, and bare-metal backends with task-specific verifiers that distinguish functional repair from durability and cleanup, and analyzes retained action-trace findings for unsafe shortcuts. We evaluate 18 agent-model configurations on 12 seed tasks, with three attempts per task. Mean effective scores range from 45.5% to 89.2%; the strongest configuration completes 77.8% of attempts perfectly. In the available lifecycle checks, pass rates fall from 91.5% for functional repair to 80.9% for durability and 38.9% for cleanup. These results show why immediate recovery is an incomplete measure of operational capability. We use this evidence to motivate a planned expansion toward 40-60 tasks centered on managed cloud infrastructure and redesign after incidents, including provider-native Kubernetes networking. This extension is a research direction; the reported results remain those of the 12-task prototype. InfraBench, its tasks, evaluation harness, and live leaderboard are available at infraben.ch.

Abstract

Runtime Enforcement for AI Agents

Explore similar work

InfraBench: Evaluating Infrastructure Agents Across Layers, Lifecycle, and Risk

INFRAMIND: Infrastructure-Aware Multi-Agent Orchestration

Beyond Autonomy: A Dynamic Tiered AgentRunner Framework for Governable and Resilient Enterprise AI Execution