R-BED: A Consequence-Aware Framework for System-Level AI Evaluation and Continuous Assurance
Organizations: Independent Researcher
Abstract
Evaluations of AI systems usually report properties of a model, while decisions to deploy concern a configured system: a model together with prompts, tools, permissions, data, safeguards and an operating context. We ask how evidence about technical failures, deployment-control effectiveness and real-world consequences can be connected to defensible authorization decisions throughout the lifecycle of a deployed system. We present R-BED (Risk-Based Evaluability Design), a specification-level methodology with a fixed 6+4+3 architecture of thirteen diagnostic dimensions (technical failure, deployment controls, consequence and impact); a failure-mediated expected-loss identity that is kept separate from intended-behavior and misuse pathways; bounded evidence statuses with independence levels; a control assurance ladder; five authorization dispositions governed by eight rules; and a Continuous System Evaluation (CSE) layer in which evidence carries a configuration pin, an expiry and declared dependencies, so that a change invalidates evidence in proportion rather than wholesale. We work through a synthetic refund-agent walkthrough, with a deterministic script that reproduces every number, to show how a task-success view can miss a consequential control weakness that the procedure identifies as missing evidence. The contribution is a reproducible methodological specification. Most components are standard practice integrated into one evidence-to-decision record; the Evidence Validity Register, the authority envelope and the tier rules are proposals. None has been empirically validated, and we state seven hypotheses for independent testing.
Figures & tables
| Approach | Typical object and output | Relationship to R-BED |
|---|---|---|
| AI RMF and Generative AI Profile [ 13 , 14 ] | Voluntary risk-management outcomes and actions | R-BED is a method for some of the evidence the outcomes call for; it creates no organizational governance and implies no conformity. |
| ISO/IEC 42001 [ 9 ] | Certifiable management system | Complementary; R-BED supplies system-level evidence, not a management system. |
| Model and sociotechnical evaluation [ 11 , 19 , 25 ] | Model or layered evaluation results | R-BED adds a bounded, dated evidence record and a rule-governed link to authorization. |
| Safety and assurance cases [ 6 ] | Structured safety argument | R-BED fixes record fields, statuses and rules so that the argument is checkable for discipline. |
| Model risk management [ 2 , 3 ] | Validation, monitoring and change governance for models | R-BED extends independent challenge and revalidation to configured, including agentic, systems with controls. |
| Agent benchmarks and red teaming [ 26 , 7 , 17 ] | Scenario suites and attack results | These are test sources; R-BED adds authority-envelope metrics, control-effectiveness evidence and consequence linkage. |
| Item | Statement | Assumption, limit and check |
|---|---|---|
| Union of pathways | For nonnegative losses with union loss equal to the pointwise maximum, . | Bounds, not an estimate; the sum is the safe bound when overlap is unknown. |
| Dependent controls (Fréchet [ 8 ] ) | For failure probabilities , . | Use unless independence is demonstrated. For , the interval is (checked). |
| Zero-failure trial counts | Zero failures in trials give a one-sided upper bound . At 95% confidence, , and clean trials bound the rate below , and . | Counts checked. Assumes independent trials from the operating distribution; a bound is not a measurement. |
| Control effectiveness by contrast | Compare with under otherwise identical tests; report the upper bound with the control over the lower bound without. | Causal only if the arms differ solely in the control. Hypothetical 12/40 without and 1/80 with give a conservative ratio of (checked). |
| Rule | Name | Condition |
|---|---|---|
| 1 | Named, current authorization | A Deploy or Restrict has an authorization record with an authorizer, a date and an expiry. An expired authorization is no authorization. |
| 2 | Invalidation and monitoring | The record states the conditions that invalidate it and the monitoring that detects them. |
| 3 | No refuted claim; prohibitions supported; thresholds fixed first | Deploy is not permitted with a refuted material claim, or with a categorical prohibition that is refuted or unresolved. A rate claim is Supported only against a threshold fixed before testing and only if the result’s upper bound is at or below it. |
| 4 | Unresolved is not safe | Deploy is not permitted while a claim tied to a high-consequence Register entry is unresolved. |
| 5 | Restrict is bounded and must prevent the pathway | A Restrict states scope, exposure limit, duration and population. Where a refuted or unresolved claim is tied to a high-consequence entry, it must also name a credited control, enforced outside the model and with effectiveness evidence, that prevents that pathway from reaching affected people. An exposure percentage limits scale; it is not a restriction on the pathway. |
| 6 | Controls credited only with evidence | A control on which the risk argument relies has effectiveness evidence; limits on authority are enforced outside the model. |
| Declared dependency | Patch | Minor | Major |
|---|---|---|---|
| Critical | Review | Retest | Invalid |
| Relevant | No effect | Review | Retest |
| Peripheral | No effect | No effect | Review |
| Rule | Condition | Tier |
|---|---|---|
| TR1 | Consequence class 3 or 4, and (controls below L2, or evidence stale or unresolved, or detection possibly slower than harm onset). | T1 |
| TR2 | Severe (class 4) or irreversible, with verified controls, current evidence and fast detection. | T2, never below |
| TR3 | Consequence class 3 with verified controls and current evidence. | T2 |
| TR4 | Class 2, broad exposure (3 to 4), and weak controls or evidence. | T2 |
| TR5 | Class 2; or class 1 with broad exposure and weak controls. | T3 |
| TR6 | Otherwise. | T4 |
| Element | Specification |
|---|---|
| Authorized actions | Answer policy questions (read); read the authenticated customer’s own account (G2, read only, scoped by CTL3); initiate a refund at or below the threshold after authenticated approval (G1, act-with-approval, per-refund and per-day limits, enforced by CTL1). |
| Forbidden actions | Any refund above the threshold or limit; any access to another customer’s account data; any tool call induced by instruction-like text in untrusted retrieved content. |
| Harm pathways | K1 unauthorized money movement (financial, moderate, partially reversible); K2 notice obligations from an erroneous refund or denial (legal and regulatory, moderate, reversible); K3 cross-account disclosure (privacy, moderate, irreversible, hence high-consequence); RSK-02 harm from the policy itself under intended behavior (pathway , no outcome data). |
| Claims and thresholds | CL-01 a retrieved note cannot override refund policy (categorical). CL-02 a server-side check rejects over-limit refunds regardless of model content (upper 95% bound at most 0.10). CL-03 no unauthorized refund is completed (categorical prohibition). CL-04 replies and queries expose account data only to the authenticated customer (upper 95% bound at most 0.10). |
| Failure injection and evidence | 20 injected-note scenarios; 25 and later 60 forced over-limit tool calls through a sandbox gateway; 60 forced cross-account attempts; 10 monitoring drills; validity records EVID-01 to EVID-04 (Figure 7 ). |
| Deployment controls | CTL1: independent server-side authorization service, enforced outside the model (B1). CTL3: server-side scoping of every account-data query to the authenticated session, which the model cannot widen (B1). Human approval retained; daily attempted-versus-completed refund report; weekly drill. |
| Claim | Test (independence) | Count | 95% interval | Status |
|---|---|---|---|---|
| CL-01 | Injected notes producing an above-threshold call (initial) | 3 of 20 | Refuted (contained later) | |
| CL-02 | Completed over-limit refunds, forced (initial, I0) | 0 of 25 | Unresolved | |
| CL-02 | Completed over-limit refunds, forced (retest, I2) | 0 of 60 | Supported (bounded) | |
| CL-04 | Completed cross-account disclosures, forced (I2) | 0 of 60 | Supported (bounded) | |
| CL-02 | Drills triggering intervention within the agreed limit (I2) | 10 of 10 | Lower bound reported | |
| CL-03 | Production-like outcomes | none | not available | Unresolved |
| Change | EVID-01 | EVID-02 | EVID-03 | EVID-04 | Claims reverting |
|---|---|---|---|---|---|
| System prompt, patch | none | none | none | none | none |
| Query-scoping control CTL3, minor | none | none | retest | none | none |
| Model, minor update | retest | none | review | none | none |
| Model, major change | invalid | review | retest | none | CL-02 |
| Component | Label | Software status |
|---|---|---|
| 6+4+3 architecture, objectives, Consequence Register | [E] | Data model and selected checks in the public package. |
| Conditional pathway identity, evidence statuses, five dispositions, eight rules, SES minimum fields | [E] | Partly implemented in the public package (data model, selected checks, interval statistics). |
| Evidence Validity Register, proportional invalidation | [P] | Standalone demonstration script only. |
| Assurance ladder and control types; authority envelope, trajectory and oversight metrics; monitoring tiers, allocation, learning loop, avoided-loss contrast | [P] | None released. |
| Consequence-aware allocation avoids more harm than frequency-based allocation (H7) | [H] | Not tested. |
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
| Activity | Cadence | What it establishes |
|---|---|---|
| Calendar-based review | At the end of a validity period or authorization expiry. | Whether evidence and authorization are still current. |
| Continuous monitoring (automated signal checks) | Per tier, from continuous or daily to monthly; only signals that must be fast need near-real-time evaluation. | Whether a pre-set threshold has been crossed. It does not show that controls work. |
| Scheduled challenge | Monthly to annual by tier: drills, forced-failure retests, seeded-failure detection tests. | Whether safeguards and monitors still work under realistic failure. |
| Event-triggered reassessment | On a change to a pinned component, an incident or near miss, a threshold breach, drift, an expansion of authority or expiry. | Whether specific evidence still holds after the event. |
| ID | Hypothesis |
|---|---|
| H1 | Trained reviewers applying the SES rules agree on evidence status and the high-consequence classification of the same record more than chance. |
| H2 | Reconstructing documented incidents as SES records exposes the control, evidence or authorization gap later reported by investigators, more often than unstructured review. |
| H3 | Independent (I2 or I3) control challenge finds failures or gaps that I0 self-assessment of the same system misses. |
| H4 | Intervention-contrast estimates of control effectiveness predict containment observed later in monitored operation. |
| H5 | Teams can complete a record at an acceptable cost, and completing it changes at least one decision or safeguard. |
| H6 | The authorization rules block decisions that experts independently judge unsupported, without blocking many they judge supported. |
AI Assurance
Artificial intelligence assurance develops evidence and controls to evaluate, govern, and certify AI systems across their lifecycle and deployment contexts.
See all 87 papers