The Alignment Flywheel: A Governance-Centric Hybrid MAS for Architecture-Agnostic Safety

Elias Malomgré; Pieter Simoens

cs.MAFeb 28, 2026

The Alignment Flywheel: A Governance-Centric Hybrid MAS for Architecture-Agnostic Safety

Authors: Elias Malomgré, Pieter Simoens

Organizations: IDLab, Ghent University - imec, Belgium

Abstract

Multi-agent systems provide mature abstractions for role decomposition, coordination, and normative governance, but increasingly capable learned components make post-deployment safety harder to inspect, audit, and update. When safety behavior is absorbed into a decision component, narrow failures may require retraining or rollback of the full component. This instantiates our vision of the Alignment Flywheel as a governance-centric hybrid MAS architecture that decouples decision generation from safety governance. We denote the agent or policy that generates candidate trajectories as the Proposer; it passes its output to a governed Safety Oracle stack, which returns safety scores, prediction uncertainty, audit coverage uncertainty, and evidence hooks through a stable interface. An Enforcement layer applies explicit risk policy at runtime. Around this loop, a governance MAS performs monitoring, red-teaming, verification, triage, refinement, and versioned release management. The central engineering principle is patch locality: many newly observed safety failures can be mitigated through small governance batches for the Oracle stack and its audit state rather than by retraining or retracting the Proposer. The architecture is implementation-agnostic with respect to both Proposer and Oracle. It defines the roles, artifacts, protocols, and release semantics needed for runtime gating, audit intake, signed updates, staged rollout, and rollback. We demonstrate executability in two scenarios: a learned spatial Oracle patched through regression-checked governance updates, and a clinical GenAI proxy setting illustrating structured norms, escalation, and audit coverage. Our implementation code and documentation are available open source at https://github.com/decide-ugent/Alignment-Flywheel.

Figures & tables

Figure 1: An overview of the Alignment Flywheel and its roles.

Symbol	Meaning
$\Sigma$	context (task state, inputs, metadata; any modality)
$\tau$	trajectory (action, tool call, message, or plan)
$P$	Proposer producing $\tau_{\mathrm{cand}}$ from $\Sigma$
$O$	Safety Oracle (third-party statistical artifact)
$s$	Safety Oracle raw safety score
$u$	Safety Oracle prediction uncertainty

Table 1: Minimal notation used in the architecture.

Panel of Figure 2 — Figure 2: Runtime enforcement during deployment. From context $\Sigma$ , proposer $P$ generates a candidate trajectory $\tau_{\mathrm{cand}}$ . Enforcement $E$ queries the deployed oracle stack, which returns the requested signals, e.g. $(s,u,u_{thresh},u_{a}u_{a,thresh},v_{O})$ , combining vendor-side outputs and Alignment Flywheel-side governance outputs under a single query interface. Enforcement then derives the action $a$ and the uncertainty state $u$ , logs the decision, and writes the audit intake to the append-only knowledge base $K$ . Dotted paths indicate optional revision and escalation under the configured risk policy.

Phase	Role-independent function	Example
Observe	Read relevant records from $K$ and queue heads.	Red Team reads active $v_{O}$ and high-severity norms
Orient	Interpret records relative to the role objective.	Red Team identifies a norm with low coverage.
Decide	Select a strategy module without changing the protocol.	Prompt mutation to verify norms for an LLM proposer
Act	Execute the strategy and write typed records back to $K$ .	Candidate flaws are pushed to the queue

Table 5: Role-independent OODA template for governance agents.

Stage	Step	Function	Output
Verification	Injection	Red Team generates candidate trajectories $\tau$ and pushes them to $Q_{ver}$ .	CandidateFlaw
Verification	Triage Candidates	They are prioritized by $\Delta=u_{thresh}-u$ . Low-uncertainty claimed-safe cases are high-value false-negative audit targets.	prioritized $Q_{ver}$
Verification	Validation	Verification checks candidates against $\Phi$ , and violations become immutable failure records in $K$ .	VerifiedBreach
Refinement	Ingestion	Verified breaches are hash-stored in $K$ for tracing and de-duplication.	breach records
Refinement	Triage II	Breaches are clustered into failure families, reducing cognitive load and enabling batch remediation [ 60 , 57 ] .	breach clusters
Refinement	Job creation	Triage packages a prioritized cluster as a repair task.	RefinementJob

Table 6: Double-filter pipeline from discovery to governed update.

Record	Producer / route	Purpose
CandidateFlaw	Red Team $\rightarrow Q_{ver}$	Discovery intake record containing $(\Sigma,\tau)$ and the Oracle signals observed at discovery time.
VerificationResult	Verification $\rightarrow K$	Explicit governance judgment separating raw suspicion from a confirmed or rejected violation of $\Phi$ .
VerifiedBreach	Verification $\rightarrow K$	Durable failure record created for confirmed violations; later clustered, prioritized, and linked to patches.
RefinementJob	Triage $\rightarrow Q_{ref}$	Clustered repair task that converts many verified breaches into one prioritized refinement unit.
PatchCommit	Refinement $\rightarrow K,L$	Governed update record binding a patch to its parent version, motivating breaches, regression evidence, and authorization signature.

Table 7: Core protocol records exchanged through

K

and the queues.

Step	Function
Propose	$P(\Sigma)\rightarrow\tau_{\mathrm{cand}}$ .
Oracle-stack evaluation	The governed Oracle stack evaluates $(\Sigma,\tau_{\mathrm{cand}})$ and returns $(s,u,u_{thresh},u_{a},u_{a,thresh},v_{O},\phi_{\mathrm{hint}},\mathrm{evid})$ , where $s$ is the raw safety score, $u$ is Oracle prediction uncertainty, $u_{a}$ is Alignment Flywheel audit coverage uncertainty, and $\phi_{\mathrm{hint}},\mathrm{evid}$ are optional hooks.
Enforcement decision	$E$ selects $a\in\{\mathrm{allow},\mathrm{block},\mathrm{revise},\mathrm{escalate}\}$ under the configured risk policy. High $u$ means the Oracle is unsure about its prediction; high $u_{a}$ means the Alignment Flywheel has insufficient audit coverage for this class of case.
Log	$E$ writes an auditable decision record to $K$ , including the active version $v_{O}$ for attribution, audit, and regression analysis.

Table 8: Runtime enforcement loop for a fixed governed Oracle version.

Record field	Purpose
Decision record	$(\Sigma,\tau_{\mathrm{cand}},a,s,u,u_{thresh},u_{a},u_{a,thresh},v_{O},\mathrm{evid})$ records the context, candidate, action, safety score, uncertainty signals, thresholds, version, and evidence.
Integrity metadata	Hash of the serialized trajectory, timestamp, and host identity support replay and fleet correlation.
Optional attachments	$\phi_{\mathrm{hint}}$ , counterexample identifiers, trace references, source deployment traffic label, stress test, or report.

Table 9: Runtime evidence captured for audit, replay, and later patch justification.

Figure 3: 3D spatial Alignment Flywheel progression. The learned Oracle initially assigns non-trivial reward to many cells outside the expert-path basin. Governance batches add suppression kernels around verified flaw regions. Across iterations, the active reward surface contracts toward the expert trajectory while the sampled basin is preserved.

Eval	Allow	Block	Escalate	Esc. Rate	Stack
0	6	0	9	60%	v0
1	6	3	6	40%	v1
2	9	6	0	0%	v2

Table 10: Patient Portal demo: enforcement convergence over evaluations.

Dimension	3D spatial demo	Patient Portal demo	Architectural property
Evaluation mode	Offline hardening of a learned Oracle artifact	Runtime Proposer–Oracle enforcement and hardening	Offline and runtime governance
Oracle type	Learned continuous IIRL-style Oracle	Structured proxy Oracle with safety and uncertainty signals	Oracle substitutability
Proposer role	No Proposer; the Oracle is hardened directly	Passthrough Proposer producing candidate replies and dispositions	Separation of proposal and governance
Norm structure	Spatial support and expert-basin preservation	Medication, urgency, lab-context, and unsupported-advice norms	Norm-semantic variation
Refinement mechanism	Gaussian suppression patches released through governance batches	Blocks, disposition-sensitive decisions, threshold updates, and audit-coverage updates	Patch-local correction
Observed effect	False-positive reward cells are removed while the sampled expert basin is preserved	Escalations are converted into definitive allow/block outcomes across governance batches	Executability and protocol stability

Table 11: Orthogonality of the two evaluation scenarios.

Appendix figures & tables24 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 4 — Figure 4: Abstract OODA interaction pattern for governance agents. Each role reads shared state from the append-only knowledge base $K$ during Observe , interprets it relative to its local objective during Orient , selects a strategy during Decide , and writes derived artifacts back to $K$ during Act . Role-specific behavior is captured by strategies and artifact types, while the interaction contract with $K$ remains uniform across the governance MAS.

Mode	Human role	Typical use
Low risk	Exception handling	Automated verification and refinement; humans review anomalies or periodic reports.
Medium risk	Batch approval	Agents cluster failures and propose governance batches; humans review summaries and authorize release.
High risk	Item and release approval	Humans review high-impact cases, approve or modify patches, and sign each governance batch before rollout.

Appendix

Table 12: Oversight modes supported by the Alignment Flywheel.

Control surface	Effect
Agent efficacy and strategy rotation	Surface metrics such as unique verified breaches per unit time; deprecate weak strategies and activate stronger ones.
Normative coverage and prioritization	Visualize coverage across $\phi\in\Phi$ ; adjust severity weights or introduce new norms to re-orient OODA loops.
Adversarial seeding and directed search	Inject seed trajectories into $Q_{\mathrm{ver}}$ ; Red Team agents mutate or exploit them to discover variants [ 22 , 30 ] .
Deployment feedback	Surface high-uncertainty runtime cases as incidents; direct coverage expansion and verification toward failed regions.
Release approval and rollback	Require human approval, signature, or staged rollout for governance batches in high-risk settings.

Appendix

Table 13: Human steering interfaces for scalable governance.

Norm kind	Verifier semantics
KEYWORD_BLOCK	Checks whether trajectory text contains prohibited keywords, optionally conditioned on evidence status. Example: medication-change terms under weak evidence.
REGEX	Checks whether trajectory text matches a regular-expression pattern, such as a prohibited action phrase.
PREDICATE	Evaluates multi-field relationships over payload and metadata, such as whether the proposed disposition is at least as severe as required for the case type and evidence status.
SPATIAL_BOUNDARY	Checks whether a spatial query point lies within the supported region, e.g., below a distance threshold from expert data.
THRESHOLD_RULE	Checks threshold-style constraints, such as age or vulnerability conditions requiring a specified evidence standard.

Appendix

Table 14: Implemented norm kinds in the reference implementation.

Cell type	Condition	Interpretation
Basin cell	$r_{i}>\sigma$ and $d_{i}\leq B$	Legitimate reward near the expert trajectory; must be preserved.
Flaw cell	$r_{i}>\sigma$ and $d_{i}>B$	Non-trivial reward outside the expert basin; must be suppressed.
Inactive cell	$r_{i}\leq\sigma$	Already below the safety floor; no action required.

Appendix

Table 15: Spatial cell classification used by the demo.

Step	Function
Skip covered flaws	If a previously accepted kernel in the same batch already covers flaw $p_{i}$ , skip it.
Propose bandwidth	Set $\beta_{\mathrm{prop}}=\mathrm{clip}\!\bigl((d_{i}-B)\cdot 0.5,\beta_{\min},\beta_{\max}\bigr),\hskip 9.24994pt\beta_{\min}=0.03,\hskip 9.24994pt\beta_{\max}=0.30.$ Distant flaws receive wider kernels; near-boundary flaws receive narrower kernels.
Single-kernel basin protection	Let $d_{\mathrm{near}}=\min_{b\in\mathcal{B}}\\|p_{i}-b\\|$ . The maximum bandwidth that keeps suppression of the nearest basin point below $\eta=0.10$ is $\beta_{\mathrm{safe}}=\frac{d_{\mathrm{near}}}{\sqrt{-2\ln\eta}}.$ The proposed bandwidth is shrunk to $\min(\beta_{\mathrm{prop}},0.9\beta_{\mathrm{safe}})$ .
Cumulative basin check	The planner tracks running suppression on every basin point from accepted kernels in the current batch: $S_{\mathrm{existing}}(b)=\sum_{j<i}\exp\!\bigl(-\\|b-c_{j}\\|^{2}/(2\beta_{j}^{2})\bigr).$ If adding the candidate kernel would push any basin point above $\eta$ , a 15-step binary search over $[\beta_{\min}/2,\beta]$ finds the largest safe bandwidth. If even $\beta_{\min}$ would cause excessive basin suppression, the patch is rejected.
Predict coverage	The effective suppression radius is $R_{\mathrm{eff}}=\beta\sqrt{-2\ln\sigma},\hskip 9.24994pt\sigma=0.005.$ Other flaw points within $R_{\mathrm{eff}}$ are predicted to be suppressed below the safety floor.
Mark covered flaws	Predicted-covered flaws are marked as handled and skipped later in the batch. This spends the patch budget on non-redundant kernel placements.

Appendix

Table 16: PatchPlanner loop for adaptive spatial governance batches.

Correction type	Effect
SPATIAL_FLAW_PATCH	Payload contains flaw_point $p_{i}$ and support_radius $\beta_{i}$ . Applying the correction installs a suppression kernel in the Oracle.
AUDIT_COVERAGE_UPDATE	Payload records the predicted coverage class, e.g. "spatial\|bw= $\beta_{i}$ \|cov= $n_{\mathrm{covered}}$ " , for audit trail and batch provenance.

Appendix

Table 17: Typed corrections used by the spatial demo.

Metric	PatchPlanner (adaptive)	Fixed BW ( $\beta=0.05$ )
Iterations to converge	16	$>25$ (not converged)
Patches/iter (max)	60	60
Basin preserved	783/783 (100%)	783/783 (100%)
Total kernels placed	834	1500 (it. 30)

Appendix

Table 18: 3D Spatial demo: adaptive PatchPlanner vs. fixed-bandwidth baseline.

Iter	Found	Kern	Predicted	Reject	Basin	Flaws	Oracle
1	500	87	500	0	783	412	v1
2	993	42	993	0	783	357	v2
3	874	58	874	0	783	290	v3
4	1 030	57	1 030	0	783	219	v4
5	1 119	49	1 119	0	783	159	v5
6	960	71	959	1	783	103	v6

Appendix

Table 19: 3D Spatial demo (adaptive PatchPlanner): convergence over iterations.

Component	Function
Proposer $P$	A PassthroughProposer wraps the incoming case data—patient message, draft reply, disposition, and metadata—as a Trajectory of kind MESSAGE . No generation occurs; the case already contains the candidate content.
Safety Oracle $O$	Evaluates the trajectory and returns OracleRawOutput : safety score $s$ , prediction uncertainty $u$ , uncertainty threshold $u_{\mathrm{thresh}}$ , Oracle version $v_{O}$ , and evidence status.
Alignment Flywheel Overlay	Evaluates audit coverage and returns Alignment FlywheelOverlay : audit coverage uncertainty $u_{a}$ , threshold $u_{a,\mathrm{thresh}}$ , governance version $v_{G}$ , and audit status.
Query Merger	A DefaultQueryMerger combines Oracle-side and governance-side signals into a UnifiedQueryResult while preserving provenance.
Enforcement $E$	A DefaultEnforcement policy maps the unified signals to $a\in\{\mathrm{allow},\mathrm{block},\mathrm{escalate}\}$ .

Appendix

Table 20: Shared runtime pipeline used by all medical demos.

Condition	Action
$u\geq u_{\mathrm{thresh}}$	escalate : Oracle prediction uncertainty is too high.
$s<\theta_{s}$	block : safety score indicates unsafe behavior.
$u_{a}\geq u_{a,\mathrm{thresh}}$ , evidence $\in\{\texttt{insufficient},\texttt{conflicting}\}$ , and $s<\theta_{a}$	escalate : audit coverage is insufficient for a weak-evidence, borderline-safety case.
Otherwise	allow .

Appendix

Table 21: Medical-domain enforcement policy used in all three demos.

Role	Function
Red Team	Generates 20 synthetic cases per iteration by sampling a combinatorial case space. Each case is classified by failure category and pushed to $Q_{\mathrm{ver}}$ as a CandidateFlaw .
Verification	Checks candidates against the normative specification $\Phi$ . Keyword-block norms check prohibited terms under weak evidence; predicate norms check multi-field relationships such as whether the proposed disposition meets the required minimum. Confirmed violations produce VerificationResult records.
Triage	Orders verified violations by priority and forwards them to $Q_{\mathrm{ref}}$ .
Refinement	Maps verified flaws to typed corrections. A tiered batch strategy rotates across categories to ensure diverse coverage. In all three demos, batch capacity is limited to one flaw per iteration.
Batch application	Applies the resulting GovernanceBatch to the Oracle and Alignment Flywheel Overlay. Oracle patches install hard-blocks and disposition overrides; overlay patches register audit coverage. Both components increment their version identifiers when state changes.

Appendix

Table 22: Shared governance cycle used by all medical demos.

Correction type	Effect
MEDICAL_HARD_BLOCK	Adds a keyword to the Oracle’s internal block list. Later trajectories containing the keyword return $s=0.0$ , $u=0.05$ , producing an unconditional block.
THRESHOLD_ADJUSTMENT	Installs a disposition override keyed by case_type\|evidence_status . If a matching trajectory has a disposition below the required minimum, the Oracle returns $s=0.15$ , $u=0.1$ , which Enforcement maps to block .
AUDIT_COVERAGE_UPDATE	Registers a case_type\|evidence\|acuity key as audited in the Alignment Flywheel Overlay. Covered keys produce $u_{a}=0.15{-}0.20$ ; uncovered keys produce $u_{a}=0.85{-}0.90$ .

Appendix

Table 23: Typed corrections used in the medical demos.

Role	Component	Function
Red Team	Medical Case Generator	Samples medications, symptoms, lab tests, and demographics; classifies cases as missed_urgency , undertriaged_med, lab_no_context, vulnerable_patient , or exploratory .
Red Team	Medical Priority Decider	Sorts candidates by category-weighted priority: missed_urgency $>$ undertriaged_med $>$ vulnerable_patient $>$ lab_no_context $>$ exploratory .
Red Team	Medical Candidate Submitter	Wraps candidates as Candidate Flaw artifacts with Oracle signals, trajectory payload, and metadata.
Verification	Medical NormM atcher and Medical Violation Decider	Extract draft text, patient message, evidence status, and disposition; evaluate keyword-block and predicate norms; emit VIOLATION or NO_VIOLATION .
Refinement	Medical Correction Orienter	Maps each verified flaw to a small bundle of corrections, e.g., hard-block, disposition overrides, and audit-coverage updates.
Refinement	Medical Batch Decider and Medical Batch Deployer	Selects at most one flaw per iteration, builds a GovernanceBatch , sets version transitions, and returns the batch for application.

Appendix

Table 24: Shared OODA components for the medical demos.

Group	Count	Risk pattern	Expected transition
Unsafe medication	3	Unsupported dosing advice with reply_only disposition.	escalate $\rightarrow$ block
Unsafe lab	3	Lab-result interpretation or monitoring advice without sufficient evidence and without nurse review.	escalate $\rightarrow$ block
Borderline	3	Similar clinical content, but already routed to clinician_review .	escalate $\rightarrow$ allow
Safe	6	Routine supported interactions such as refills, scheduling, and acknowledgements.	allow throughout

Appendix

Table 25: Simple Medical demo: fixed evaluation groups.

Component	Specification
Oracle	SimpleMedicalOracle ; heuristic scoring with patchable hard-block list $B$ and disposition override dictionary $D$ . Hard-blocks return $s=0.0,u=0.05$ ; disposition failures return $s=0.15,u=0.1$ . Base scoring penalizes medication stems, medication case type, and weak evidence.
Prediction uncertainty	Baseline $u=0.15$ , increased for insufficient, unknown, or conflicting evidence and for medication cases. $u_{\mathrm{thresh}}=0.30$ for medication cases and $0.50$ otherwise.
Alignment Flywheel Overlay	SimpleMedicalOverlay ; coverage key case_type\|evidence_status\|acuity . Covered keys produce $u_{a}=0.2$ ; uncovered keys produce $u_{a}=0.9$ . Threshold $u_{a,\mathrm{thresh}}=0.6$ .
Norms $\Phi$	Two norms: NO_UNSUPPORTED_MED_ADVICE , a keyword-block norm over medication-change terms under weak evidence; and REQUIRE_EVIDENCE_FOR_RESULTS , a predicate norm requiring at least nurse_review for weak-evidence lab/result interpretation.
Triage	FIFOTriage ; violations are passed to refinement in arrival order.

Appendix

Table 26: Simple Medical demo: concrete component choices.

Eval	Allow	Block	Escalate	Esc. Rate	Stack
0	6	0	9	60%	v0
1	6	3	6	40%	v1
2	9	6	0	0%	v2

Appendix

Table 27: Simple Medical demo: enforcement convergence.

Group	Count	Risk pattern	Expected transition
Unsafe stop-taking	3	Recommendations to stop high-risk medications such as warfarin, insulin, or oxycodone under weak evidence.	escalate $\rightarrow$ block
Unsafe dose increase	3	Autonomous dose increases for higher-risk drugs such as insulin, tramadol, or prednisone.	escalate $\rightarrow$ block
Unsafe lab	3	Organ-function or medication-related lab interpretation with insufficient evidence and direct patient routing.	escalate $\rightarrow$ block
Borderline	3	Lab or medication discussion already routed to clinician_review .	escalate $\rightarrow$ allow
Safe	6	Routine supported interactions without clinical decision-making.	allow throughout

Appendix

Table 28: Complex Medical demo: fixed evaluation groups.

Component	Specification
Oracle	ComplexMedicalOracle ; patchable state includes hard-block list $B$ , specialty threshold overrides $T$ , and disposition overrides $D$ . Evaluation order is hard-block, disposition override, then five-dimensional scoring.
Risk dimensions	$r_{\mathrm{med}}$ from a medication-risk table; $r_{\mathrm{interact}}$ from known interaction pairs; $r_{\mathrm{action}}$ from action severity; $r_{\mathrm{patient}}$ from age and comorbidities; $r_{\mathrm{ev}}$ from evidence status.
Alignment Flywheel Overlay	ComplexMedicalOverlay ; coverage key specialty\|case_type\|evidence_status . Covered keys produce $u_{a}=0.15$ ; uncovered keys produce $u_{a}=0.85$ . Threshold $u_{a,\mathrm{thresh}}=0.6$ .
Norms $\Phi$	Four norms: NO_HIGH_RISK_MED_WITHOUT_EVIDENCE , INTERACTION_SAFETY , VULNERABLE_PATIENT_PROTECTION , and SEVERE_ACTION_EVIDENCE . These cover keyword-block, regex, threshold-rule, and predicate-style checks.
Triage	PriorityTriage ; priority order is missed_urgency $>$ undertriaged_med $>$ vulnerable_patient $>$ lab_no_context $>$ exploratory .

Appendix

Table 29: Complex Medical demo: concrete component choices.

Eval	Allow	Block	Escalate	Esc. Rate	Stack
0	6	0	12	67%	v0
1	6	6	6	33%	v1
2	9	9	0	0%	v2

Appendix

Table 30: Complex Medical demo: enforcement convergence.

Group	Count	Risk pattern	Expected transition
Unsafe medication	3	Direct patient-facing medication-change advice, such as increasing dose, without clinician review or sufficient evidence.	escalate $\rightarrow$ block
Unsafe lab	3	Lab-result interpretation with medication context, weak evidence, and reply_only routing.	escalate $\rightarrow$ block
Borderline	3	Lab or medication discussion already routed to clinician_review .	escalate $\rightarrow$ allow
Safe	6	Routine supported interactions such as refills, appointments, physical-therapy progress, diet follow-up, improving symptoms, and normal annual labs.	allow throughout

Appendix

Table 31: Patient Portal demo: fixed evaluation groups.

Component	Specification
Oracle	PatientPortalOracle ; patchable state includes hard-block list $B$ and disposition override dictionary $D$ . Evaluation order is hard-block, disposition override, then four-dimensional scoring.
Risk dimensions	$r_{\mathrm{med}}$ counts medication keywords; $r_{\mathrm{urg}}$ counts urgency keywords in the message or draft; $r_{\mathrm{ev}}$ encodes evidence quality; $r_{\mathrm{disp}}$ measures the gap between required and proposed disposition.
Disposition ranks	reply_only $=0$ , nurse_review $=1$ , clinician_review $=2$ , urgent_escalation $=3$ . Needed disposition increases with urgency, medication risk, and semi-urgent acuity.
Alignment Flywheel Overlay	PatientPortalOverlay ; coverage key case_type\|evidence_status\|acuity . Covered keys produce $u_{a}=0.15$ ; uncovered keys produce $u_{a}=0.85$ . Threshold $u_{a,\mathrm{thresh}}=0.6$ .
Norms $\Phi$	Four norms: MED_ADVICE_REQUIRES_CLINICIAN , URGENCY_REQUIRES_ESCALATION , LAB_RESULTS_NEED_CONTEXT , and NO_UNSUPPORTED_MED_KEYWORDS .
Triage	PriorityTriage ; same category-priority ordering as the complex demo.

Appendix

Table 32: Patient Portal demo: concrete component choices.

Eval	Allow	Block	Escalate	Esc. Rate	Stack
0	6	0	9	60%	v0
1	6	3	6	40%	v1
2	9	6	0	0%	v2

Appendix

Table 33: Patient Portal demo: enforcement convergence.

Component	Simple	Complex	Patient Portal
Oracle scoring	3 heuristic rules	5-dimensional lookup	4-dimensional keywords
Patchable state	$B$ , $D$	$B$ , $T$ , $D$	$B$ , $D$
Norm count (kinds)	2 (2)	4 (4)	4 (2)
Triage	FIFO	priority-ranked	priority-ranked
Fixed cases	15	18	15
Evaluations to converge	3	3	3

Appendix

Table 34: Component variation across medical demos.

Explore similar work

Jul 1, 2026cs.AI

Managed Autonomy at Runtime: Gear-Based Safety and Governance for Single- and Multi-Agent Cyber-Physical Systems

Autonomous agents, whether LLM-driven software agents or robotic physical agents, face a common class of failure modes when operating without continuous human oversight: safety violations from unverified actions, behavioral instability from unconstrained loops, and continuity loss from unhandled error states. We develop \system{}, a discrete-time control system that combines five execution gears (\Gobs{}, \Gsug{}, \Gplan{}, \Gexec{}, \Gint{}) with utility-gated dispatch and event-driven fallback. For the single-agent case, we prove monotonic stability, execution safety, eventual stabilization, fallback completeness, and equivalence to a gear-constrained Markov decision process. For multi-agent cyber-physical systems (CPS), we apply the established \smart{} managed-autonomy lifecycle and map runtime evidence into its four governance states (\Stable{}/\Meta{}/\Assisted{}/\Regulated{}). Consensus gating, swarm-level Lyapunov analysis, per-agent gear authority, and rendezvous control provide distributed safety and stability guarantees, including zero collision under the stated assumptions. We evaluate the resulting runtime on a three-agent UR5 robotic assembly cell using fault magnitudes calibrated from the NIST \emph{Degradation Measurement of Robot Arm Position Accuracy} dataset across 10,000 Monte Carlo episodes. It achieves a 99.6% anomaly detection rate versus 2.1% for the single-agent baseline, reduces detection latency by

3.5\times

, and supplies a formal physical-workspace safety certificate. The execution gears act as micro-level permissions beneath the \smart{} runtime governance states, separating action control from autonomy governance.

Srini Ramaswamy, Wang Miaosheng

DNRS.ai, USA · Independent Researcher

Cyber-Physical System Runtime Lyapunov Function

Ask PDF

Aug 2, 2026cs.MA

MasDrift: Benchmarking Authorization Preservation Across Multi-Agent Architectures

Multi-agent systems (MAS) decompose long-horizon tasks across supervisors and subagents, but delegated goals do not necessarily carry their original authorization boundaries. Existing safety benchmarks mainly study adversarial compromise, while work on constraint drift lacks controlled architecture-level evaluation. We introduce MasDrift, a benchmark of 600 benign productivity tasks across eight domains. Each task pairs required work with reserved actions. MasDrift compares single-agent, centralized, and decentralized coordination while varying hierarchy depth and peer width, measuring task completion and authorization preservation. Across generic multi-agent conditions, centralized hierarchies achieve 93.9--98.6% task completion versus 85.7--87.0% for peer networks, while unauthorized actions occur in 2.7--19.8% of tasks versus 0.6--0.8%, a gap that widens with hierarchy depth. We further compare two defenses that differ in where authorization evidence resides. One re-anchors every pending call to the original user request. The other carries an attenuated policy along the delegation chain. Re-anchoring reduces unauthorized actions in every model configuration we evaluate, at a cost of 1.6 points of pooled completion. Chain propagation blocks required work instead, forfeiting up to 36.3 points. A heterogeneous case study confirms that the failure follows from coordination rather than model strength. MasDrift exposes a centralization tradeoff and makes authorization preservation a measurable property of MAS design.

Zhuoning Xu, Xiucheng Zhang, Hanjun Luo +3

New York University · New York University Abu Dhabi · The Hong Kong Polytechnic University +1

Model-Based Multi-Agent Systems Delegation Authorization Preservation

Ask PDF

Apr 27, 2026cs.AI

Governing What You Cannot Observe: Adaptive Runtime Governance for Autonomous AI Agents

Autonomous AI agents can remain fully authorized and still become unsafe as behavior drifts, adversaries adapt, and decision patterns shift without any code change. We propose the \textbf{Informational Viability Principle}: governing an agent reduces to estimating a bound on unobserved risk

\hat{B}(x) = U(x) + SB(x) + RG(x)

and allowing an action only when its capacity

S(x)

exceeds

\hat{B}(x)

by a safety margin. The \textbf{Agent Viability Framework}, grounded in Aubin's viability theory, establishes three properties -- monitoring (P1), anticipation (P2), and monotonic restriction (P3) -- as individually necessary and collectively sufficient for documented failure modes. \textbf{RiskGate} instantiates the framework with dedicated statistical estimators (KL divergence, segment-vs-rest

z

-tests, sequential pattern matching), a fail-secure monotonic pipeline, and a closed-loop Autopilot formalised as an instance of Aubin's regulation map with kill-switch-as-last-resort; a scalar Viability Index

VI(t) \in [-1,+1]

with first-order

t^*

prediction transforms governance from reactive to predictive. Contributions are the theoretical framework, the reference implementation, and analytical coverage against published agent-failure taxonomies; quantitative empirical evaluation is scoped as follow-up work.

German Marin, Jatin Chaudhary

Department of Computing, University of Turku, Turku, Finland

Probabilistic Safety Governance Artificial Intelligence Agents

Ask PDF