Governed Capability Evolution: Lifecycle-Time Compatibility Checking and Rollback for AI-Component-Based Systems, with a Proof-of-Concept Evaluation on Embodied Agents

Xue Qin; Simin Luan; John See; Zeyd Boukhers; Cong Yang; Zhijun Li

cs.ROApr 9, 2026

Governed Capability Evolution: Lifecycle-Time Compatibility Checking and Rollback for AI-Component-Based Systems, with a Proof-of-Concept Evaluation on Embodied Agents

Authors: Xue Qin, Simin Luan, John See, Zeyd Boukhers, Cong Yang, Zhijun Li

Organizations: School of Software, Harbin Institute of Technology, Harbin, China · School of Computer Science and Technology, Harbin Institute of Technology, Harbin, China · School of Mathematical and Computer Sciences, Heriot-Watt University, Malaysia Campus, Putrajaya, Malaysia · Fraunhofer Institute for Applied Information Technology, Sankt Augustin, Germany ·

Abstract

Systems built from versioned AI components need lifecycle-time governance over how a new module version is admitted, monitored, and withdrawn. Established deployment patterns (canary, blue-green, feature flags, MLOps) monitor aggregate service-level signals of stateless services, not the stateful, policy-constrained runtimes that drive AI components. We formulate governed capability evolution as a software-lifecycle problem and ask whether staged upgrade governance prevents unsafe activations where practice-adapted canary and blue-green deployment do not, and at what cost. We propose four compatibility checks (interface, policy, behavioral, recovery) and a seven-stage pipeline (validation, sandbox, shadow, gated activation, monitoring, rollback, audit). A reference prototype is evaluated in a behavioral simulation of an embodied manipulation stack; a PyBullet backend replicates the strategy ordering. Upgrade candidates come from a seeded fault-operator generator, including out-of-taxonomy probes, and thresholds are fixed on a disjoint development set. Across five strategies, five upgrade rounds and 15 seeds, naive upgrade reaches 73.3% final-round success but activates unsafe versions in 74.7% of rounds. Canary and blue-green adaptations keep comparable success yet still admit faulty versions in 61.3% and 64.0% of rounds, because aggregate success hides policy and recovery regressions. Governed upgrade admits no in-taxonomy fault and one out-of-taxonomy probe in 75 rounds, at a 4.6-point success cost (Holm-adjusted p<=0.024 against all three baselines); the full pre-activation stack detects 97.3-100% of in-taxonomy faults across deployment profiles, and rollback restores verified operation in 72.4% of drift scenarios. Bounded-window aggregate-success gating proved insufficient against these fault operators; staged governance closes the gap at a modest, tunable cost.

Figures & tables

Figure 1: Governance profile comparison across five deployment dimensions (all axes oriented outer = better; values from the 15-seed evaluation of Section 8 ). Governed Upgrade (blue) covers the safety and recoverability dimensions while staying competitive on task success. The practice-adapted Canary baseline (orange) improves on Naïve only marginally: it aborts the 17.9% of faulty upgrades whose cohort success rate visibly collapses and promotes the rest. Naïve Upgrade (red) collapses on four of the five dimensions, retaining only partial policy compliance.

Panel of Figure 2 — Figure 2: Overview of governed capability evolution. Top: Naïve upgrade directly replaces the active capability version without governance. Bottom: Each new version is a candidate that must pass compatibility validation, which scores all four dimensions ( $\kappa_{I},\kappa_{P},\kappa_{B},\kappa_{R}$ ) from declared metadata before any execution, against the composite gate $\theta_{\mathrm{act}}$ , then the sandbox envelope $\theta_{\mathrm{sb}}$ , the shadow divergence bound $\Delta_{\mathrm{shadow}}$ , and gated activation, which adds supervisory approval in approval-requiring profiles. Online monitoring continues after activation; Figure 3 gives the full transition set, including demotion and rollback.

Figure 3: Governance state machine over candidate capability versions. Forward transitions (left to right) require passing the stage’s admission criterion; downward transitions capture rejection, demotion, and rollback. active remains monitored: drift returns a candidate to rolled-back or demoted , and demoted candidates re-enter at sandbox evaluation, the stage the experiments exercise.

Figure 4: Reference architecture and stage-to-component mapping. Top: the five subsystems; the persistent agent core is never modified by upgrades, and candidate versions stay under Upgrade Manager control until gated activation promotes them into the registry’s active view. Bottom: the seven formal pipeline stages of Section 5 , each labelled with the component that implements it. The evaluated artifact implements the stage components and the registry; the remaining boxes are architectural context.

Fault-operator family	Rejected / total	Rate	95% CI
Interface drift	243/270	90.0%	[85.8, 93.0]
Permission expansion	189/270	70.0%	[64.3, 75.2]
Recovery degradation	168/270	62.2%	[56.3, 67.8]
Behavioral regression	51/270	18.9%	[14.7, 24.0]
Benign (false alarms)	0/540	0.0%	[0.0, 0.7]
Out-of-taxonomy probes	0/405	0.0%	[0.0, 0.9]

Table 1: Compatibility-screening results over the generated pools (15 seeds; 72 in-taxonomy faulty and 36 benign candidates per seed; 405 out-of-taxonomy probes pooled). Rejection rates are pooled across seeds with Wilson 95% confidence intervals.

Metric	Static	Naïve	Canary	Blue-green	Governed
Final SR (%) $\uparrow$	66.2 $\pm$ 3.4	73.3 $\pm$ 5.2	73.2 $\pm$ 5.3	73.3 $\pm$ 5.2	68.7 $\pm$ 5.6
Final UAR (%) $\downarrow$	0.0 $\pm$ 0.0	73.3 $\pm$ 44.2	66.7 $\pm$ 47.1	73.3 $\pm$ 44.2	0.0 $\pm$ 0.0
Mean UAR (%) $\downarrow$	0.0	74.7	61.3	64.0	1.3
Final PVR (%) $\downarrow$	11.1 $\pm$ 3.3	22.6 $\pm$ 14.1	22.3 $\pm$ 14.3	22.6 $\pm$ 14.1	11.1 $\pm$ 3.2
Evaluation exposure (%) $\downarrow$	—	—	14.9	10.0	0.0

Table 2: Deployment outcomes across upgrade rounds (mean

\pm

std over 15 seeds; round-0 baseline plus five upgrade rounds). Mean UAR averages the per-round unsafe-activation indicator over the five upgrade rounds. Evaluation exposure is the mean share of upgrade-round traffic served by a faulty candidate during the strategy’s own evaluation phase, before its promote or revert rule fired; Static and Naïve have no evaluation phase, and Governed evaluates in sandbox and shadow rather than on served traffic.

Figure 5: Performance and deployment safety over upgrade rounds for the five strategies (mean, with

\pm 1

standard deviation bands over 15 seeds, clipped to the feasible 0–100% range; every strategy faces the same per-round candidate and noise stream, so round-0 values coincide by construction). (a) Task success rate. Naïve, Canary, and Blue-green track each other closely because all three promote most nominally attractive candidates; Governed tracks Static more closely because it rejects most candidates, benign ones included. (b) Unsafe activation rate. The success-signal strategies admit faulty candidates in 47–87% of rounds (per-round means across the three strategies), Canary and Blue-green sitting below Naïve in most rounds but far above zero, whereas Governed admits exactly one candidate, an out-of-taxonomy probe, across all 75 upgrade rounds (mean UAR 1.3%) and no in-taxonomy fault.

Figure 6: Sandbox versus shadow catch rates per fault-operator family (pooled over 15 seeds; error bars are Wilson 95% intervals). In-taxonomy families are shown left of the divider, out-of-taxonomy probes right. Sandbox evaluation absorbs the statistical bulk across families; shadow deployment’s unique contribution concentrates in latency inflation, where candidates that pass the sandbox envelope diverge visibly from the active version under live timing.

Variant	BADR (%) $\uparrow$	FAR (%) $\downarrow$	UAR (%) $\downarrow$	OOT act. (%) $\downarrow$
Full	100.0 $\pm$ 0.0	30.6 $\pm$ 6.3	0.0 $\pm$ 0.0	1.5 $\pm$ 2.3
$-$ Shadow	100.0 $\pm$ 0.0	28.0 $\pm$ 5.2	0.0 $\pm$ 0.0	9.9 $\pm$ 5.7
$-$ Sandbox	78.5 $\pm$ 5.0	3.0 $\pm$ 2.6	30.3 $\pm$ 5.0	34.3 $\pm$ 1.6
$-$ RecoveryCompat	100.0 $\pm$ 0.0	30.6 $\pm$ 6.3	0.0 $\pm$ 0.0	1.5 $\pm$ 2.3
CompatOnly	60.3 $\pm$ 4.7	0.0 $\pm$ 0.0	44.1 $\pm$ 2.9	100.0 $\pm$ 0.0
Naïve	0.0 $\pm$ 0.0	0.0 $\pm$ 0.0	66.7 $\pm$ 0.0	100.0 $\pm$ 0.0

Table 3: Screening-composition ablation, computed exactly per candidate over the generated pools (mean

\pm

std over 15 seeds). A candidate is activated under a variant iff it passes every included gate. OOT act. = out-of-taxonomy probes reaching activation.

\uparrow

= higher is better;

\downarrow

= lower is better.

AI Agent Governance

AI agent governance studies how to authorize, constrain, monitor, and hold autonomous and multi-agent systems accountable.

See all 442 papers

EmbodiedGovBench: A Benchmark for Governance, Recovery, and Upgrade Safety in Embodied Agent Systems
Apr 13, 2026Xue Qin, Simin Luan, John See +2AI Agent Governance AI Agent Safety Benchmarks
School of Software, Harbin Institute of Technology, Harbin, China · School of Computer Science and Technology, Harbin Institute of Technology, Harbin, China · School of Mathematical and Computer Sciences, Heriot-Watt University, Malaysia Campus, Putrajaya, Malaysia +1
Stop Shipping AI Agents on Faith: Capability Is Not Production Readiness
Jul 30, 2026Fouad BousetouaneAI Agent Reliability AI Agent Evaluation
The University of Chicago, USA
Beyond Component Testing: Validating Agentic AI Systems
Jul 31, 2026Fabio Orazio Mirto, Luca D'Agati, Giuseppe Tricomi +4AI Agent Evaluation AI Agent Safety
Department of Biomedical, Dental, and Morphological and Functional Imaging Sciences, University of Messina, A.O.U. Policlinico “G.Martino” - Via Consolare Valeria, Messina, 98125, Italy. · Department of Engineering, University of Messina, Contrada di Dio, Sant’Agata, Messina, 98158, Italy. · Institute for High Performance Computing and Networking of National Research Council of Italy, ICAR-CNR, Via Pietro Castellino 111, Naples, 80131, Italy. +2

Apr 13, 2026J/K move · Enter open · S save

EmbodiedGovBench: A Benchmark for Governance, Recovery, and Upgrade Safety in Embodied Agent Systems

Xue Qin, Simin Luan, John See, Cong Yang+1

School of Software, Harbin Institute of Technology, Harbin, China · School of Computer Science and Technology, Harbin Institute of Technology, Harbin, China · School of Mathematical and Computer Sciences, Heriot-Watt University, Malaysia Campus, Putrajaya, Malaysia+1

AI Agent Governance AI Agent Safety Benchmarks AI Agent Benchmarks Embodied AI

Open paper AskPDF arXiv

Recent progress in embodied AI has produced a growing ecosystem of robot policies, foundation models, and modular runtimes. However, current evaluation remains dominated by task success metrics such as completion rate or manipulation accuracy. These metrics leave a critical gap: they do not measure whether embodied systems are governable -- whether they respect capability boundaries, enforce policies, recover safely, maintain audit trails, and respond to human oversight. We present EmbodiedGovBench, a benchmark for governance-oriented evaluation of embodied agent systems. Rather than asking only whether a robot can complete a task, EmbodiedGovBench evaluates whether the system remains controllable, policy-bounded, recoverable, auditable, and evolution-safe under realistic perturbations. The benchmark covers seven governance dimensions: unauthorized capability invocation, runtime drift robustness, recovery success, policy portability, version upgrade safety, human override responsiveness, and audit completeness. We define a benchmark structure spanning single-robot and fleet settings, with scenario templates, perturbation operators, governance metrics, and baseline evaluation protocols. We describe how the benchmark can be instantiated over embodied capability runtimes with modular interfaces and contract-aware upgrade workflows. Our analysis suggests that embodied governance should become a first-class evaluation target. EmbodiedGovBench provides the initial measurement framework for that shift.

Abstract

AI Agent Governance

Explore similar work

EmbodiedGovBench: A Benchmark for Governance, Recovery, and Upgrade Safety in Embodied Agent Systems

Stop Shipping AI Agents on Faith: Capability Is Not Production Readiness

Beyond Component Testing: Validating Agentic AI Systems