cs.MAMar 18, 2026

A Trace-Based Assurance Framework for Agentic AI Orchestration: Contracts, Testing, and Governance

Authors: Ciprian Paduraru, Petru-Liviu Bouruc, Alin Stefanescu

Organizations: University of Bucharest Romania · University of Bucharest & Institute for Logic and Data Science Romania

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

In Agentic AI, Large Language Models (LLMs) are increasingly used in the orchestration layer to coordinate multiple agents and to interact with external services, retrieval components, and shared memory. In this setting, failures are not limited to incorrect final outputs. They also arise from long-horizon interaction, stochastic decisions, and external side effects (such as API calls, database writes, and message sends). Common failures include non-termination, role drift, propagation of unsupported claims, and attacks via untrusted context or external channels. This paper presents an assurance framework for such Agentic AI systems. Executions are instrumented as Message-Action Traces (MAT) with explicit step and trace contracts. Contracts provide machine-checkable verdicts, localize the first violating step, and support deterministic replay. The framework includes stress testing, formulated as a budgeted counterexample search over bounded perturbations. It also supports structured fault injection at service, retrieval, and memory boundaries to assess containment under realistic operational faults and degraded conditions. Finally, governance is treated as a runtime component, enforcing per-agent capability limits and action mediation (allow, rewrite, block) at the language-to-action boundary. To support comparative evaluations across stochastic seeds, models, and orchestration configurations, the paper defines trace-based metrics for task success, termination reliability, contract compliance, factuality indicators, containment rate, and governance outcome distributions. More broadly, the framework is intended as a common abstraction to support testing and evaluation of multi-agent LLM systems, and to facilitate reproducible comparison across orchestration designs and configurations.

Figures & tables

Fig. 1: Pipeline overview of the assurance framework. Colors indicate the four layers and their roles. The system under test (SUT, green) is the deployed multi-agent LLM system : an orchestrator coordinating an agent pool, together with the runtime governance boundary L4 (blue). The diagram uses a centralized orchestrator for clarity; the same instrumentation and controls apply to decentralized variants (e.g., peer-to-peer agents) by treating the current decision maker as the acting role at step tt . The operational environment (green cloud) is depicted only through tool, retrieval, and memory interfaces, since the framework evaluates end-to-end integration behavior and integration failures. L2 stress testing (orange) draws task instances x∼Dx\sim\mathcal{D} and applies bounded perturbations δ\delta to inputs and context. During execution, the acting role proposes an action ata_{t} ; L4 mediates the proposed external action using step contracts Istep\mathcal{I}^{\mathrm{step}} and a policy shield Π\Pi , yielding allow , rewrite (to a governed action a~t\tilde{a}_{t} ), or block (red dashed feedback) to prevent unsafe side effects. L3 (red) injects controlled faults at the same external interfaces to exercise realistic integration disturbances. The resulting observation ot+1o_{t+1} (tool output, retrieved evidence, or error) closes the interaction loop. In parallel, L1 (yellow) records Message-Action Trace entries and evaluates trace contracts Itrace\mathcal{I}^{\mathrm{trace}} over prefixes, localizing the first violation and emitting a replay record for debugging and regression testing.
Fig. 2: Adversarial counterexample search as an inner–outer assurance loop. (1) Setup: fix a system configuration κ\kappa (roles, tools, contracts, governance) and sample tasks x∼Dx\sim\mathcal{D} with stochastic seed zz . (2) Inner loop (search): an adversary selects bounded perturbations δ\delta (subject to cost(δ)≤B\mathrm{cost}(\delta)\leq B ) and injects them into the execution; the system produces a trace τ=Exec(x,κ,z,δ)\tau=\mathrm{Exec}(x,\kappa,z,\delta) , which is monitored for contract violations (step/trace contracts Istep,Itrace\mathcal{I}^{\mathrm{step}},\mathcal{I}^{\mathrm{trace}} ) and auxiliary signals such as progress Φ\Phi and unsupported-claim rate HrateH_{\mathrm{rate}} . The resulting score guides the next perturbation choice. (3) Outer loop (engineering feedback): when a violation is found (red arrow), the framework localizes the first failing step tt and stores a replay record (e.g., (z,δ⋆)(z,\delta^{\star}) and required stubs), enabling configuration revision and re-testing (green dashed path), including updates to agent parameters πθ\pi_{\theta} and/or governance policy Π\Pi .
More in this topic

AI Agent Reliability

AI agent reliability studies whether tool-using systems act correctly, consistently, safely, and recoverably across changing tasks and environments.

See all 338 papers

Explore similar work

CardsList

  1. Beyond Task Success: An Evidence-Synthesis Framework for Evaluating, Governing, and Orchestrating Agentic AI

    Apr 18, 2026Christopher Koch, Joshua Andreas WellbrockLLM Agent OrchestrationAI Agent Evaluation

    Independent Researcher

  2. Trustworthy Agentic AI: Failure Modes, Mitigation Strategies, and a Lifecycle Framework for Autonomous LLM Systems

    Sep 19, 2026Fayeq Jeelani Syed, Rehan Ahmad, Ali Al Bataineh +1LLM Agent SecurityLLM Agents

    Department of Human-Centered Computing, Indiana University Indianapolis, 755 W. Michigan Street, Indianapolis, Indiana, 46202, USA · Department of Electrical and Computer Engineering, Purdue University Northwest, 2200 169th Street, Hammond, Indiana, 46323, USA · Computer Information Science, Higher Colleges of Technology, Abu Dhabi, United Arab Emirates +1

  3. From Agent Traces to Trust: A Survey of Evidence Tracing and Execution Provenance in LLM Agents

    Jun 3, 2026Yiqi Wang, Jiaqi Zhang, Zhangkai Wu +8AI Agent ReliabilityData Provenance

    Griffith University · Jiangsu University · The University of Sydney +4