AI-GRACE: A Use-Case Operationalization Framework for Agentic AI: From Organizational Objectives and Obligations to Deployment Capabilities and Architecture
Organizations: University of Miami · Columbia University · Stanford University
Abstract
Organizations deploying agentic artificial intelligence must determine more than whether a model is trustworthy; they must establish what to validate, control, and observe for a use case to deliver its intended outcome while meeting applicable obligations. This paper proposes AI-GRACE (Agentic Intelligence-Governance, Risk, Assurance, Controls, and Evidence) as a use-case operationalization framework connecting organizational governance with technical implementation. The proposal draws on professional observations and a purposive synthesis of standards and literature, using design science to frame the method contribution and situational method engineering to guide contextual tailoring and reuse. The framework establishes objectives and obligations and then assesses risks in seven proposed domains, including mission and value realization. It derives requirements for assurance before deployment, runtime controls, and evidence, which guide capability qualification, gap assessment, and a logical architecture. An Agent Operating Envelope specifies permitted actions and escalation conditions, while Risk-Aligned Independence Levels (RAIL) summarize the authorized independence. A fictional retail banking application illustrates the method. The contribution is a traceable basis for deciding what an organization must implement, what it already supports, and what remains unresolved. Empirical evaluation must establish whether it improves deployment decisions, efficiency, and reuse.
Figures & tables
| Domain | Concern | Selected anchors |
|---|---|---|
| Mission & Value | Poor adoption, task failure, delay, resource use, or cost greater than expected benefit. | NIST MAP 1.3–1.4; CRI MP-1.4.1, MS-2.12.3 |
| AI Behavior & Authority | Without an attacker, the agent reasons poorly, mishandles ambiguous input, selects tools incorrectly, drifts from its goal, or attempts unauthorized action. | NIST MEASURE 2.5; ARC; delegation research |
| Safety & Human Impact | Harmful reliance, manipulation, unfair treatment, physical or psychological harm, or impaired access affects people or communities. | NIST MEASURE 2.6, 2.11; ISO/IEC 42005 clauses 6.7–6.9; Chan et al.,2023 |
| Cybersecurity | An adversary compromises or misuses the agent, identities, data, tools, dependencies, or connected systems. | NIST MEASURE 2.7; ARC; OWASP |
| Privacy | Personal information is improperly collected, accessed, inferred, retained, used, transferred, or disclosed, with or without an attacker. | NIST MEASURE 2.10; ISO/IEC 42005 |
| Operational Resilience | Availability, latency, capacity, recovery, dependencies, or maintainability cannot sustain the required service. | NIST MEASURE 2.3, 2.7; ISO/IEC 42001 Annex B.6.2.6; MeRGE quality phase |
| Level | Label | Authorized behavior |
|---|---|---|
| 0 | Assist | Retrieve, summarize, or present information within scope; no mandate to recommend a course of action or execute consequential external actions. |
| 1 | Recommend | Analyze and recommend; a human decides and performs consequential external actions. |
| 2 | Act with Approval | Each consequential external action requires explicit human approval. Scoped retrieval, reasoning, and preparation do not require approval at every step. |
| 3 | Bounded Autonomous | Execute a defined class of consequential actions within an approved mandate without individual approval; escalate at defined boundaries. |
| 4 | High-Authority Autonomous | Coordinate broader approved tasks or mandates under supervision rather than routine approval of individual consequential actions. All activity remains bounded. |
| ID | Source and applicability | Required outcome |
|---|---|---|
| G1 | Bank service charter: organizational objectives and commitments. | Correct completion, accessible human recourse, fewer interactions, and sustainable total cost, including failures and rework. |
| G2 | EU AI Act: Article 5(1)(a)–(b) prohibitions; Article 50(1), (5) AI interaction notice; Article 50(2) marking of generated content; recital 27’s nonbinding principles ( Regulation 2024/1689(2024)Regulation (EU) 2024/1689, EU ) . | Prevent prohibited manipulation or exploitation meeting the Act’s conditions; identify AI interaction. Record Article 50(2) applicability and any required marking of generated content. The bank adopts oversight, safety, fairness, and accountability as service requirements, including for vulnerable clients. |
| G3 | GDPR Article 5: lawful processing, purpose limitation, minimization, security, accountability ( Regulation 2016/679(2016)Regulation (EU) 2016/679, EU ) . | Constrain account access, retrieval, memory, model disclosure, and telemetry to the approved purpose; demonstrate these boundaries. |
| G4 | U.S. national bank: Gramm–Leach–Bliley Act safeguards through 12 CFR part 30, Appendix B, II–III ( Interagency Guidelines, 2026 ) . | Extend customer information safeguards to the agent, tools, and suppliers: threat assessment, access restrictions, protection, testing, and incident response. |
| G5 | EU credit institution: DORA Articles 8, 11, 17–19, 24–25, 28–30 ( Regulation 2022/2554(2022)Regulation (EU) 2022/2554, EU ) . | Map dependencies; test continuity and recovery; investigate and report qualifying ICT incidents through bank processes; assess suppliers and contracts. |
| G6 | U.S. Regulation E, 12 CFR 1005.11; national implementation of PSD2 Articles 64, 97–98 ( Procedures for Resolving Errors, 2026 ; Directive 2015/2366(2015)Directive (EU) 2015/2366, EU ) . | Preserve consent and applicable authentication, including transaction linking where required; reconstruct errors and disputes. The chosen design requires client approval of each payment alongside existing bank checks. |
| Risk / governance link | Failure and consequence | Required property and owner |
|---|---|---|
| R1: AI Behavior & Authority; G1, G6 | Invented balances, misread bills, wrong tools, or false completion produce incorrect advice or payment. | Ground answers and verify outcomes; service/payments owners require critical task and action tests. |
| R2: Mission & Value; G1 | Repeated calls and unresolved tasks increase cost and client effort. | Measure quality and total cost per resolved task; service owner rejects failure to meet value criteria. |
| R3: Cybersecurity; G2, G4, G5 | Injected bill content or compromised tools redirect the agent toward theft or altered payments. | Security owner requires adversarial tests, tool assessment, and independent enforcement. |
| R4: Privacy; G3, G4 | Retrieval, memory, model requests, or traces expose another client’s data or retain excess information. | Privacy owner requires client/entity isolation, limited collection, and access/leakage tests across data paths. |
| R5: Operational Resilience; G5, G6 | Model/tool failure stalls requests; retries duplicate payments or lose their status. | Operations owner requires consistent recovery, reconciliation, fallback, and workload/failure tests. |
| R6: Safety & Human Impact; G1, G2 | Manipulative guidance, harmful reliance, unequal service, or inaccessible assistance harms clients. | Service owner requires tests by affected group, safe responses, and recourse; critical harm blocks release. |
| Risk basis | Assurance: before release or change | Controls: runtime enforcement | Evidence: tests and outcomes |
|---|---|---|---|
| R1, R2, R6; G1–G2 | A1 Agent evaluations. Banking tasks: correctness, tool choice, completion, harmful advice, group differences, handoff, effort, cost. | C1 Quality and safety guardrails. Grounding, output checks, scope limits, AI notice, human recourse. | E1 Agent tracing and outcome review. Model/tool spans, sources, outputs, resolution, correction, handoff, cost; linked evaluation versions. |
| R3, R6; G2, G4–G5 | A2 Red teaming and MCP scanning. Model/agent attacks, injected content, tool descriptions, servers, permissions, dependencies. | C2 Runtime inspection. Inspect input, output, and tool results; permit only approved tool and server versions; contain or escalate suspicious activity. | E2 Security and safety findings. Test cases, scanner coverage, findings and disposition, guardrail events, policy versions, incidents. |
| R4; G3–G4 | A3 Access and privacy tests. Client/agent/entity permissions, session isolation, leakage, telemetry minimization. | C3 Identity-based enforcement. Scoped credentials, authorization at tool/data boundaries, isolated memory, protected/redacted telemetry. | E3 Access decisions. Principal, resource, policy, permit/deny result, retention/access configuration; exclude unnecessary payloads. |
| R1, R3, R5; G6 | A4 Payment integration tests. Approval tampering, revocation, replay, timeout, reconstruction through execution. | C4 Transaction enforcement. Approval bound to exact details; independent gate, durable duplicate suppression, reconciliation. | E4 Payment evidence. Proposal, client approval, authorization, committed transaction, reconciled status; linked identifiers. |
| R5, R2; G1, G5 | A5 Resilience and load tests. Outages, capacity, recovery, telemetry loss, incident routing. | C5 Operational constraints. Timeouts, retry/step/cost limits, circuit breakers, recoverable state, payment holds, fallback. | E5 System observability. Correlated logs, traces, metrics, dependency health, service objectives, incidents, recovery results. |
| Candidate capability | Illustrative fit | Required work and owner |
|---|---|---|
| Bank identity, read APIs, payment services | Existing bank services: sufficient. Assistant integration: not yet demonstrated. | Architecture/security owners verify the new agent and delegation paths. |
| Agent evaluations, red teaming, MCP scanning, access/privacy tests, payment integration tests (A1–A4) | Absent for this use case | Evaluation/security owners establish tasks, attacks, assessments, and acceptance evidence. |
| Runtime guardrails, agent access and approval enforcement (C1–C4) | Absent at agent boundary | Engineering/security/payments owners add inspection, scoped authorization, approval binding, and execution mediation; qualify through A1–A4. |
| Central logging and bank transaction records (E1–E5) | Partially sufficient | Observability/evidence owners add continuous tracing, correlated decisions, protected retention, outcome links, and incident integration; test reconstruction and collection failure. |
| Hosting, model serving, recovery, suppliers (A5/C5) | Not yet demonstrated for design | Platform/operations owners establish capacity, dependency, recovery, and supplier evidence. |
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
| Record | Required content |
|---|---|
| Use case charter | Objectives, impacts, comparator, acceptance conditions, affected parties, organizational and affected-party roles, workflow, data, tools, actions, exclusions. |
| Obligation register | Source and locator; legal, contractual, policy, or strategic status; applicability, jurisdiction, owner, requirements, conflicts, unresolved decisions. |
| Risk register | Scenario, causes, primary/linked domains, affected objectives and parties, likelihood, consequences, assumptions, uncertainty, treatment, residual risk, decision owner. |
| Capability requirement | Identifier, linked objective/obligation/risk, A/C/E function, scope, acceptance criterion, evidence method, placement, interfaces, dependencies, owner. |
| Capability fit | Candidate implementation/version, fit state, supporting evidence, qualification scope, limitations, required changes or validation. |
| Logical design | Agent/services, principals, data and execution paths, enforcement, evaluations, evidence, infrastructure, requirement links, gaps. |
AI Agent Governance
AI agent governance studies how to authorize, constrain, monitor, and hold autonomous and multi-agent systems accountable.
See all 445 papers