Governed Capability Evolution: Lifecycle-Time Compatibility Checking and Rollback for AI-Component-Based Systems, with a Proof-of-Concept Evaluation on Embodied Agents

Xue Qin; Simin Luan; John See; Zeyd Boukhers; Cong Yang; Zhijun Li

cs.ROApr 9, 2026

Governed Capability Evolution: Lifecycle-Time Compatibility Checking and Rollback for AI-Component-Based Systems, with a Proof-of-Concept Evaluation on Embodied Agents

Authors: Xue Qin, Simin Luan, John See, Zeyd Boukhers, Cong Yang, Zhijun Li

Organizations: School of Software, Harbin Institute of Technology, Harbin, China · School of Computer Science and Technology, Harbin Institute of Technology, Harbin, China · School of Mathematical and Computer Sciences, Heriot-Watt University, Malaysia Campus, Putrajaya, Malaysia · Fraunhofer Institute for Applied Information Technology, Sankt Augustin, Germany ·

Abstract

Systems built from versioned AI components need lifecycle-time governance over how a new module version is admitted, monitored, and withdrawn. Established deployment patterns (canary, blue-green, feature flags, MLOps) monitor aggregate service-level signals of stateless services, not the stateful, policy-constrained runtimes that drive AI components. We formulate governed capability evolution as a software-lifecycle problem and ask whether staged upgrade governance prevents unsafe activations where practice-adapted canary and blue-green deployment do not, and at what cost. We propose four compatibility checks (interface, policy, behavioral, recovery) and a seven-stage pipeline (validation, sandbox, shadow, gated activation, monitoring, rollback, audit). A reference prototype is evaluated in a behavioral simulation of an embodied manipulation stack; a PyBullet backend replicates the strategy ordering. Upgrade candidates come from a seeded fault-operator generator, including out-of-taxonomy probes, and thresholds are fixed on a disjoint development set. Across five strategies, five upgrade rounds and 15 seeds, naive upgrade reaches 73.3% final-round success but activates unsafe versions in 74.7% of rounds. Canary and blue-green adaptations keep comparable success yet still admit faulty versions in 61.3% and 64.0% of rounds, because aggregate success hides policy and recovery regressions. Governed upgrade admits no in-taxonomy fault and one out-of-taxonomy probe in 75 rounds, at a 4.6-point success cost (Holm-adjusted p<=0.024 against all three baselines); the full pre-activation stack detects 97.3-100% of in-taxonomy faults across deployment profiles, and rollback restores verified operation in 72.4% of drift scenarios. Bounded-window aggregate-success gating proved insufficient against these fault operators; staged governance closes the gap at a modest, tunable cost.

Figures & tables

Figure 1: Governance profile comparison across five deployment dimensions (all axes oriented outer = better; values from the 15-seed evaluation of Section 8 ). Governed Upgrade (blue) covers the safety and recoverability dimensions while staying competitive on task success. The practice-adapted Canary baseline (orange) improves on Naïve only marginally: it aborts the 17.9% of faulty upgrades whose cohort success rate visibly collapses and promotes the rest. Naïve Upgrade (red) collapses on four of the five dimensions, retaining only partial policy compliance.

Panel of Figure 2 — Figure 2: Overview of governed capability evolution. Top: Naïve upgrade directly replaces the active capability version without governance. Bottom: Each new version is a candidate that must pass compatibility validation, which scores all four dimensions ( $\kappa_{I},\kappa_{P},\kappa_{B},\kappa_{R}$ ) from declared metadata before any execution, against the composite gate $\theta_{\mathrm{act}}$ , then the sandbox envelope $\theta_{\mathrm{sb}}$ , the shadow divergence bound $\Delta_{\mathrm{shadow}}$ , and gated activation, which adds supervisory approval in approval-requiring profiles. Online monitoring continues after activation; Figure 3 gives the full transition set, including demotion and rollback.

Figure 3: Governance state machine over candidate capability versions. Forward transitions (left to right) require passing the stage’s admission criterion; downward transitions capture rejection, demotion, and rollback. active remains monitored: drift returns a candidate to rolled-back or demoted , and demoted candidates re-enter at sandbox evaluation, the stage the experiments exercise.

Figure 4: Reference architecture and stage-to-component mapping. Top: the five subsystems; the persistent agent core is never modified by upgrades, and candidate versions stay under Upgrade Manager control until gated activation promotes them into the registry’s active view. Bottom: the seven formal pipeline stages of Section 5 , each labelled with the component that implements it. The evaluated artifact implements the stage components and the registry; the remaining boxes are architectural context.

Fault-operator family	Rejected / total	Rate	95% CI
Interface drift	243/270	90.0%	[85.8, 93.0]
Permission expansion	189/270	70.0%	[64.3, 75.2]
Recovery degradation	168/270	62.2%	[56.3, 67.8]
Behavioral regression	51/270	18.9%	[14.7, 24.0]
Benign (false alarms)	0/540	0.0%	[0.0, 0.7]
Out-of-taxonomy probes	0/405	0.0%	[0.0, 0.9]

Table 1: Compatibility-screening results over the generated pools (15 seeds; 72 in-taxonomy faulty and 36 benign candidates per seed; 405 out-of-taxonomy probes pooled). Rejection rates are pooled across seeds with Wilson 95% confidence intervals.

Metric	Static	Naïve	Canary	Blue-green	Governed
Final SR (%) $\uparrow$	66.2 $\pm$ 3.4	73.3 $\pm$ 5.2	73.2 $\pm$ 5.3	73.3 $\pm$ 5.2	68.7 $\pm$ 5.6
Final UAR (%) $\downarrow$	0.0 $\pm$ 0.0	73.3 $\pm$ 44.2	66.7 $\pm$ 47.1	73.3 $\pm$ 44.2	0.0 $\pm$ 0.0
Mean UAR (%) $\downarrow$	0.0	74.7	61.3	64.0	1.3
Final PVR (%) $\downarrow$	11.1 $\pm$ 3.3	22.6 $\pm$ 14.1	22.3 $\pm$ 14.3	22.6 $\pm$ 14.1	11.1 $\pm$ 3.2
Evaluation exposure (%) $\downarrow$	—	—	14.9	10.0	0.0

Table 2: Deployment outcomes across upgrade rounds (mean

\pm

std over 15 seeds; round-0 baseline plus five upgrade rounds). Mean UAR averages the per-round unsafe-activation indicator over the five upgrade rounds. Evaluation exposure is the mean share of upgrade-round traffic served by a faulty candidate during the strategy’s own evaluation phase, before its promote or revert rule fired; Static and Naïve have no evaluation phase, and Governed evaluates in sandbox and shadow rather than on served traffic.

Figure 5: Performance and deployment safety over upgrade rounds for the five strategies (mean, with

\pm 1

standard deviation bands over 15 seeds, clipped to the feasible 0–100% range; every strategy faces the same per-round candidate and noise stream, so round-0 values coincide by construction). (a) Task success rate. Naïve, Canary, and Blue-green track each other closely because all three promote most nominally attractive candidates; Governed tracks Static more closely because it rejects most candidates, benign ones included. (b) Unsafe activation rate. The success-signal strategies admit faulty candidates in 47–87% of rounds (per-round means across the three strategies), Canary and Blue-green sitting below Naïve in most rounds but far above zero, whereas Governed admits exactly one candidate, an out-of-taxonomy probe, across all 75 upgrade rounds (mean UAR 1.3%) and no in-taxonomy fault.

Figure 6: Sandbox versus shadow catch rates per fault-operator family (pooled over 15 seeds; error bars are Wilson 95% intervals). In-taxonomy families are shown left of the divider, out-of-taxonomy probes right. Sandbox evaluation absorbs the statistical bulk across families; shadow deployment’s unique contribution concentrates in latency inflation, where candidates that pass the sandbox envelope diverge visibly from the active version under live timing.

Variant	BADR (%) $\uparrow$	FAR (%) $\downarrow$	UAR (%) $\downarrow$	OOT act. (%) $\downarrow$
Full	100.0 $\pm$ 0.0	30.6 $\pm$ 6.3	0.0 $\pm$ 0.0	1.5 $\pm$ 2.3
$-$ Shadow	100.0 $\pm$ 0.0	28.0 $\pm$ 5.2	0.0 $\pm$ 0.0	9.9 $\pm$ 5.7
$-$ Sandbox	78.5 $\pm$ 5.0	3.0 $\pm$ 2.6	30.3 $\pm$ 5.0	34.3 $\pm$ 1.6
$-$ RecoveryCompat	100.0 $\pm$ 0.0	30.6 $\pm$ 6.3	0.0 $\pm$ 0.0	1.5 $\pm$ 2.3
CompatOnly	60.3 $\pm$ 4.7	0.0 $\pm$ 0.0	44.1 $\pm$ 2.9	100.0 $\pm$ 0.0
Naïve	0.0 $\pm$ 0.0	0.0 $\pm$ 0.0	66.7 $\pm$ 0.0	100.0 $\pm$ 0.0

Table 3: Screening-composition ablation, computed exactly per candidate over the generated pools (mean

\pm

std over 15 seeds). A candidate is activated under a variant iff it passes every included gate. OOT act. = out-of-taxonomy probes reaching activation.

\uparrow

= higher is better;

\downarrow

= lower is better.

AI Agent Governance

AI agent governance studies how to authorize, constrain, monitor, and hold autonomous and multi-agent systems accountable.

See all 442 papers

Apr 13, 2026

EmbodiedGovBench: A Benchmark for Governance, Recovery, and Upgrade Safety in Embodied Agent Systems

Recent progress in embodied AI has produced a growing ecosystem of robot policies, foundation models, and modular runtimes. However, current evaluation remains dominated by task success metrics such as completion rate or manipulation accuracy. These metrics leave a critical gap: they do not measure whether embodied systems are governable -- whether they respect capability boundaries, enforce policies, recover safely, maintain audit trails, and respond to human oversight. We present EmbodiedGovBench, a benchmark for governance-oriented evaluation of embodied agent systems. Rather than asking only whether a robot can complete a task, EmbodiedGovBench evaluates whether the system remains controllable, policy-bounded, recoverable, auditable, and evolution-safe under realistic perturbations. The benchmark covers seven governance dimensions: unauthorized capability invocation, runtime drift robustness, recovery success, policy portability, version upgrade safety, human override responsiveness, and audit completeness. We define a benchmark structure spanning single-robot and fleet settings, with scenario templates, perturbation operators, governance metrics, and baseline evaluation protocols. We describe how the benchmark can be instantiated over embodied capability runtimes with modular interfaces and contract-aware upgrade workflows. Our analysis suggests that embodied governance should become a first-class evaluation target. EmbodiedGovBench provides the initial measurement framework for that shift.

Xue Qin, Simin Luan, John See +2

School of Software, Harbin Institute of Technology, Harbin, China · School of Computer Science and Technology, Harbin Institute of Technology, Harbin, China · School of Mathematical and Computer Sciences, Heriot-Watt University, Malaysia Campus, Putrajaya, Malaysia +1

AI Agent Governance AI Agent Safety Benchmarks AI Agent Benchmarks Embodied AI

Ask PDF

Jul 30, 2026

Stop Shipping AI Agents on Faith: Capability Is Not Production Readiness

AI agents are moving into production workflows where they retrieve information, call tools, maintain state, and act on behalf of users or organizations, but many release decisions still rely on capability signals, demos, or behavioral tests that do not show whether an agent is ready to operate under production constraints. Capability is therefore not production readiness. This paper introduces the ProofAgent Index (PAI), a governance readiness index for AI agents. PAI combines four dimensions of deployment evidence: Evaluation, Context, Compliance, and Governance. Evaluation measures observed behavior, Context measures the operating environment that shapes that behavior, Compliance measures alignment with applicable rules and controls, and Governance measures whether the organization can authorize, monitor, audit, and control the agent during operation. PAI is implemented inside ProofAgent Harness, an open source infrastructure for auditable AI agent evaluation and governance. Validation across two heavily regulated domains, healthcare and finance, shows that PAI carries held out readiness signal and separates higher risk from lower risk configurations. The results show that context engineering strongly changes reliability, capability improves behavior but does not determine readiness, and governance evidence must remain visible rather than averaged away. PAI reframes agent release from a faith based deployment decision into an auditable readiness decision.

Fouad Bousetouane

The University of Chicago, USA

AI Agent Reliability AI Agent Evaluation AI Agent Governance

Ask PDF

Jul 31, 2026

Beyond Component Testing: Validating Agentic AI Systems

Agentic AI systems act through multi-step trajectories that combine planning, tool use, memory, interaction, and adaptation. This behavior stretches validation practice beyond component testing and one-shot input-output evaluation, because acceptable system behavior now depends on how decisions unfold over time and under changing environmental conditions. This systematic mapping study synthesizes 262 papers spanning agent evaluation, software assurance, cyber-physical systems, runtime monitoring, and regulatory guidance in order to characterize the validation problem for agentic systems. The review is organized around a five-dimension taxonomy covering behavioral, safety, temporal, regulatory, and multi-agent concerns, and uses that taxonomy to map current approaches and identify dense and sparse pairings of approaches and dimensions. The resulting map shows that the literature concentrates on behavioral evaluation (105 of 262 papers) and is thinnest on temporal validity (14 papers); regulatory work rests mainly on assurance cases and regulatory analysis, largely from IEEE-indexed venues, and multi-agent work mainly on benchmarks. Three cross-domain case studies (medical care, industrial operations, smart-mobility systems) provide operational illustrations of how the five taxonomy dimensions recur in safety-critical settings, motivated by the failure patterns documented in the reviewed literature. The paper concludes with a lifecycle-oriented research agenda centered on bounded-autonomy specifications, adversarial trajectory generation, runtime monitoring, and audit-ready evidence structures. The central claim is that trustworthy deployment of agentic AI depends on validating trajectories in context rather than assessing isolated components alone.

Fabio Orazio Mirto, Luca D'Agati, Giuseppe Tricomi +4

Department of Biomedical, Dental, and Morphological and Functional Imaging Sciences, University of Messina, A.O.U. Policlinico “G.Martino” - Via Consolare Valeria, Messina, 98125, Italy. · Department of Engineering, University of Messina, Contrada di Dio, Sant’Agata, Messina, 98158, Italy. · Institute for High Performance Computing and Networking of National Research Council of Italy, ICAR-CNR, Via Pietro Castellino 111, Naples, 80131, Italy. +2

AI Agent Evaluation AI Agent Safety AI Agent Monitoring Multi-Agent Systems AI Agent Governance

Ask PDF

Abstract

AI Agent Governance

Explore similar work

EmbodiedGovBench: A Benchmark for Governance, Recovery, and Upgrade Safety in Embodied Agent Systems

Stop Shipping AI Agents on Faith: Capability Is Not Production Readiness

Beyond Component Testing: Validating Agentic AI Systems