cs.CRAug 29, 2026

AgentKernel: The Trust-Native Agentic Operating System

Authors: Zhenhua Zou, Sheng Guo, Qiuyang Zhan, Lepeng Zhao, Shuo Li, Zhuotao Liu

Organizations: DeepKernel Lab

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Modern AI agents routinely cross trust boundaries: they ingest untrusted content, combine it with privileged instructions, persist intermediate beliefs in long-term memory, and invoke privileged tools. This creates an attack surface in which malicious payloads can enter through model inputs and cause harmful tool actions. Yet current governance stacks remain application-level middleware that share a process trust boundary with the agents they monitor. We argue that agents need an operating-system substrate providing mandatory, non-bypassable services for identity, input mediation, memory governance, and execution control. We introduce AgentKernel, a trust-native agent operating system built around the premise that security must be a first-class design constraint. AgentKernel wraps the agent lifecycle in a mandatory enforcement boundary organized into four pillars: Identity, Perception, Cognition, and Execution. Each pillar adapts classical OS security principles to failures at the semantic plane, including delegation abuse, prompt injection, memory poisoning, and tool misuse. AgentKernel treats structural security as a capability multiplier. Kernel-managed identity supports trustworthy cross-organization collaboration; graduated perception replaces brittle single-point filters; information-flow-controlled memory improves retrieval fidelity while limiting poisoning; and semantic-to-kernel enforcement permits broader tool privileges behind a non-bypassable boundary. We position AgentKernel as the missing OS layer beneath orchestration frameworks, agent runtimes, governance platforms, and execution sandboxes, and use systematic comparison and security analysis to show how a single integrated architecture can enforce security across the full agent lifecycle.

Figures & tables

Figure 1: AgentKernel ’s position in the AI agent technology stack. Agent applications (Claude Code, OpenClaw, mobile/GUI agents, etc.) sit atop AgentKernel , which mediates all interactions with LLM models, tools, and the outside world. AgentKernel and the traditional OS kernel jointly form the AI-Native OS : AgentKernel provides semantic-level services (identity, perception, cognition, execution governance) while leveraging the traditional kernel for syscall-level enforcement via eBPF hooks and process-tree monitoring.
Figure 2: Three-domain model. All inputs traverse the AgentKernel kernel before reaching the protected agent core; all outputs are identity-verified, intent-aligned, and permission-converged before leaving the kernel.
DimensionSecurity ConsequenceCapability Consequence
IdentityImpersonation and unauthorized multi-hop delegation without binding principals to code and authorizationVerifiable delegation chains across organizations stall; operators fall back on pairwise trust or manual approval gates
Unverifiable third-party skills/plugins enabling supply-chain capability injectionVersion freezes and strict whitelists block safe participation in open third-party capability ecosystems
PerceptionPrompt injection via tools, web pages, and files; untrusted content co-mingled with privileged instructionsBrowser- and repository-backed toolchains are avoided or over-stripped, narrowing tasks an agent can safely cover
Encoding bypasses, jailbreaks, and spoofed UI or sensor readings that misstate device or application stateMultimodal and high-entropy pipelines require human checkpoints, capping throughput and end-to-end autonomy
CognitionCross-session memory poisoning; instruction–data confusion in long, multi-source context windowsCross-session RAG and long-horizon memory cannot scale without per-entry provenance or costly write-side audits
Cross-tenant memory contamination; weak edge-side summarisation or routing exploited as a chain weak linkShared memory pools across agents or business units are blocked; teams revert to siloed stores and manual reconciliation
Table 1: Infrastructure gaps across the agent lifecycle: missing OS services create both security vulnerabilities and capability limitations.
Figure 3: Defense-in-depth across the agent lifecycle. Each pillar independently intercepts a distinct threat class. Provenance labels flow across pillars (dashed arrows), enabling downstream pillars to make informed decisions based on upstream assessments. Compromise of one pillar does not cascade to others.
Figure 4: AgentKernel architecture. The security kernel mediates all interactions between the untrusted input world and the auditable output world. Four security pillars operate around the protected agent core, each enforcing independent invariants.
Figure 5: Identity as a kernel-managed resource. The Agent Kernel holds the private key and mediates all identity operations on behalf of the agent. Identity-derived trust signals flow into each of the other three security pillars through the kernel’s control plane, making identity the shared trust substrate for the entire security architecture.
PillarInvariantSoundness Argument
IdentityI1. AIC unforgeabilityEd25519 signatures verified against GAR root at every presentation; KprivK_{\mathsf{priv}} never leaves kernel boundary (§ 4.2 ).
 I2. Delegation monotonicity: Smax⁡child⊆Smax⁡parentS_{\max}^{\text{child}}\subseteq S_{\max}^{\text{parent}}Kernel verifies subset relation before issuing child AIC; child validity is capped to parent’s remaining lifetime.
 I3. Action non-repudiationAll execution traces kernel-signed by KprivK_{\mathsf{priv}} ; signing is kernel-mediated, never performed by agent code.
 I4. Session anti-replayFresh nonces with TTL per session; context-bound tokens; lifecycle ephemerality (§ 4.2 ).
PerceptionP1. Provenance completeness: all content entering LLM context carries origin tagsAll input paths traverse P1 (Source Tagger) as mandatory first stage; tags persist through P2–P4.
 P2. Layered independence: bypassing one filter does not help bypass othersP1–P4 operate on orthogonal features (source identity, syntactic patterns, semantic intent, multi-turn history).
Table 2: Security invariants of the AgentKernel architecture.
DimensionAgentKernelAGT [ 26 ]AIOS [ 22 ]OpenFang [ 23 ]SmythOS [ 24 ]Letta [ 25 ]nono [ 29 ]
Pillar 1: Identity (§ 4.2 )
Cryptographic workload identity∙\bullet∙\bullet—∘\circ——∘\circ
Multi-dimensional provenance∙\bullet——————
Cross-org A2A attestation∙\bullet∘\circ—————
Pillar 2: Perception (§ 4.3 )
Input provenance tagging∙\bullet——∘\circ———
Table 3: Comparison of representative systems across systematic Agent OS dimensions. Rather than treating security and capability as isolated features, AgentKernel demonstrates how OS-level security primitives (e.g., identity, taint tracking) directly enable advanced agent capabilities (e.g., cross-org A2A, provenance-aware retrieval). ∙\bullet = comprehensive support as a core design element; ∘\circ = partial or optional support; — = absent or not a design focus.
SystemDeploymentPrimary focusSecurity
AIOSHTTP server + in-proc agentsLLM schedulingNo auth; stub ACL
OpenFangDaemon / in-proc kernelSkill/Plugin sandboxMandatory for hosted; opt-in for external
SmythOSIn-proc library (npm)Unified resourcesACL + SES sandbox
LettaStandalone serverMemory managementOptional sandbox
AgentKernelForm-agnosticSecurity kernelMandatory, non-bypassable
Table 4: Agent runtime deployment models and enforcement characteristics. Deployment indicates whether the runtime runs in the same process as the agent ( in-proc ), as a separate service ( service ), or in a hybrid mode. Security enforcement indicates whether security mechanisms are mandatory or opt-in. AgentKernel is included for comparison; its deployment form is explicitly secondary to the non-bypass invariant (Principle 5).
More in this topic

AI Agent Security

Artificial intelligence agent security studies attacks and safeguards for tool-using, persistent, and collaborative systems across digital and physical settings.

See all 517 papers

Explore similar work

CardsList
Sep 20, 2026

When the Agent Becomes the Kernel: A Systematization of Security on the Path to AI-Native Operating Systems

Large language model agents are now privileged principals that take consequential actions: editing code repositories, operating inboxes, completing purchases. Their authority is kernel-grade, but it comes without what classical systems security requires: a trusted mediator interposed on every access. Operating-system vendors are now rebuilding the platform around this de-facto agent kernel, inheriting complete mediation as a design problem. We systematize the security of such systems around a single distinction: a crossing mediated over provenance admits a deterministic check, while one over content semantics does not. A trust-boundary taxonomy locates where mediation must occur and isolates the central mediation gap at two kinds of semantic judgment: distinguishing data from instruction in untrusted input, and an authorized action from an unauthorized one. We argue that this gap leaves an irreducible residual of undetected attacks wherever inputs and actions are not restricted in advance to an enumerated set. The same distinction makes attack-success statistics actionable, placing each number on a spectrum from deployment debt (a sound deterministic mediator left unused) to a structural gap (no such mediator known). We systematize defenses across runtime monitoring, architectural separation, and authorization, and show that current evaluations tend to overstate deployed security through evaluation-validity failures. Finally, we carry that analysis forward beyond the de-facto kernel, to an architecture in which the model itself becomes the arbitration core, and derive the design constraints, open challenges, and research agenda for a security-first AI-native OS.

Li Zhang, Yang Sun, Jie Shi

Huawei

CybersecurityAI Agent SecurityTool Access Control for LLM AgentsAI Agent Monitoring
AskPDF
Jun 1, 2026

Agent Operating Systems (AOS): Integrating Agentic Control Planes into, and Beyond, Traditional Operating Systems

Traditional operating systems were designed around deterministic programs, explicit control flow, and human initiated workflows. Their core abstractions processes, threads, system calls, files, and permissions assume bounded behavior and predictable interaction patterns. Agentic AI systems introduce a different execution model: long-lived, goal-directed entities that reason probabilistically, invoke tools dynamically, and adapt behavior based on feedback. While agents can be implemented as user-space applications today, their execution characteristics stress OS boundaries in scheduling, memory and state management, security, observability, and governance. This paper introduces the concept of an Agent Operating System (AOS), a systems architecture that integrates an agentic control plane into existing operating systems or, in some models, subsumes selected OS responsibilities over time. We provide a precise definition of an AOS, explicit assumptions and non-goals, and a structured decomposition of AOS responsibilities into schedulers, context and memory management, tool and capability registries, policy and trust enforcement, and observability and audit. We analyze limitations of classical OS abstractions for agent workloads, propose integration models from user-space runtimes to distributed control planes, and map AOS concepts onto Linux and Windows primitives. We present security and safety implications, including agent specific threat models, and define evaluation criteria that emphasize deterministic enforcement, auditability, and operator comprehensibility. The objective is not to replace operating systems wholesale, but to establish a rigorous systems foundation for agentic computation that remains controllable, accountable, and secure at scale.

Ankur Sharma, Deep Shah

Independent Researcher San Francisco, USA

AI Agent SecurityAI Agent MonitoringAI Agent Governance
AskPDF
May 6, 2026

AgentTrust: Runtime Safety Evaluation and Interception for AI Agent Tool Use

Modern AI agents execute real-world side effects through tool calls such as file operations, shell commands, HTTP requests, and database queries. A single unsafe action, including accidental deletion, credential exposure, or data exfiltration, can cause irreversible harm. Existing defenses are incomplete: post-hoc benchmarks measure behavior after execution, static guardrails miss obfuscation and multi-step context, and infrastructure sandboxes constrain where code runs without understanding what an action means. We present AgentTrust, a runtime safety layer that intercepts agent tool calls before execution and returns a structured verdict: allow, warn, block, or review. AgentTrust combines a shell deobfuscation normalizer, SafeFix suggestions for safer alternatives, RiskChain detection for multi-step attack chains, and a cache-aware LLM-as-Judge for ambiguous inputs. We release a 300-scenario benchmark across six risk categories and an additional 630 independently constructed real-world adversarial scenarios. On the internal benchmark, the production-only ruleset achieves 95.0% verdict accuracy and 73.7% risk-level accuracy at low-millisecond end-to-end latency. On the 630-scenario benchmark, evaluated under a patched ruleset and not claimed as zero-shot, AgentTrust achieves 96.7% verdict accuracy, including about 93% on shell-obfuscated payloads. AgentTrust is released under the AGPL-3.0 license and provides a Model Context Protocol server for MCP-compatible agents.

Chenglin Yang

Independent Researcher

AI Agent SecurityAI Agent Security BenchmarksAI Agent MonitoringSafety FilteringRuntime Enforcement for AI AgentsAI Agent Safety Benchmarks+1
AskPDF