cs.AIApr 5, 2026
SaveQuantifying Trust: Financial Risk Management for Trustworthy AI Agents
Organizations: Microsoft Research · Columbia University · Google DeepMind · Stanford University ·
Abstract
Prior work on trustworthy AI emphasizes model-internal properties such as bias mitigation, adversarial robustness, and interpretability. As AI systems evolve into autonomous agents deployed in open environments and increasingly connected to payments or assets, the operational meaning of trust shifts to end-to-end outcomes: whether an agent completes tasks, follows user intent, and avoids failures that cause material or psychological harm. These risks are fundamentally product-level and cannot be eliminated by technical safeguards alone because agent behavior is inherently stochastic. To address this gap between model-level reliability and user-facing assurance, we propose a complementary framework based on risk management. Drawing inspiration from financial underwriting, we introduce the Agentic Risk Standard (ARS), a payment settlement standard for AI-mediated transactions. ARS integrates risk assessment, underwriting, and compensation into a single transaction framework that protects users when interacting with agents. Under ARS, users receive predefined and contractually enforceable compensation in cases of execution failure, misalignment, or unintended outcomes. This shifts trust from an implicit expectation about model behavior to an explicit, measurable, and enforceable product guarantee. We also present a simulation study analyzing the social benefits of applying ARS to agentic transactions. ARS's implementation can be found at https://github.com/t54-labs/AgenticRiskStandard.
Figures & tables
Figure 1: ARS is a transaction-layer assurance standard for agentic services that converts stochastic, outcome-level risk into explicit settlement rules. Without ARS , users must prepay agents (and in fund-moving tasks, also hand over execution capital), exposing them to non-delivery, misexecution, and downstream harms. With ARS , service fees are locked in an escrow vault and released only upon successful evaluation; for fund-moving tasks, users can optionally purchase underwriting coverage while the service provider posts collateral before principal is released. This shifts trust from model-internal reliability to auditable, enforceable guarantees over product-level outcomes.
| Industry / Context | Service Provider | Primary Risk | Risk Protection Mechanism | Who Bears Residual Risk |
|---|---|---|---|---|
| Construction | Contractor | Project failure, delay, quality defects | Performance bond | Insurer / surety provider |
| Professional services | Doctor, lawyer, accountant | Professional error, malpractice | Liability insurance | Insurance provider |
| E-commerce platforms | Online seller | Non-delivery, poor service | Escrow payment | Platform escrow account |
| Government procurement | Contractor | Contract non-performance | Bid bond / performance bond | Insurance / surety |
| Financial markets | Traders / brokers | Default and market volatility | Margin requirements + clearinghouse | Clearinghouse |
| DeFi / crypto lending | Borrower | Loan default | Collateralization | Smart contract protocol |
Table 1: Risk protection mechanisms across industries. High-stakes services rarely rely on trust alone. Instead, industries introduce financial mechanisms that allocate risk across participants. ARS applies the same principle to AI agents by combining escrow settlement, collateralization, and optional underwriting to provide outcome-level guarantees.
Figure 2: The requestor first sends a task specification to the business agent. Both parties may then enter a negotiation loop to refine the agreement. Once consensus is reached, the finalized agreement is recorded and a hash code is generated for future reference.
| Action | Sender | Required fields | Effect |
|---|---|---|---|
| SubmitRequest | Requestor | {job_id, task_spec, fee_terms, principal_terms?} | Creates job; enters REQUEST |
| AcceptRequest ; RejectRequest | Business agent | {job_id, decision, reason?} | Accept NEGOTIATION ; reject CANCELLED |
| ProposeAgreement | Either party | {job_id, agreement_draft} | Updates draft terms |
| SignAgreement | Requestor / Business agent | {job_id, agreement_hash} | Both signatures bind ; enter TRANSACTION |
| CancelJob | Either party | {job_id, agreement_hash, reason, signature} | Transitions to CANCELLED if permitted by |
Table 2: ARS actions for job creation and agreement binding.
Figure 3: Fee track in the transaction phase: The requestor locks the service fee in an escrow vault before execution. After the business agent submits the execution evidence, an evaluator checks its validity. If the evidence passes evaluation, the escrowed fee is released to the business agent; otherwise, the funds are refunded to the requestor.
| Action | Sender | Required fields | Effect |
|---|---|---|---|
| LockFeeEscrow | Requestor | {job_id, agreement_hash, lock_ref, signature} | Locks service fee in escrow |
| SubmitDeliverable | Business agent | {job_id, agreement_hash, deliverable_ref, signature} | Emits delivery signal; enables evaluation |
| SettleFeeEscrow | Settlement layer | {job_id, agreement_hash, release/refund, settlement_ref} | Releases fee to business agent or refunds requestor |
Table 3: ARS actions for the fee track (escrowed service compensation).
Figure 4: Principal Track in the transaction phase: When execution involves user funds (principal), additional financial risk is introduced. ARS therefore introduces an underwriter to provide protection. If the requestor agrees to pay the premium and the business agent locks the required collateral, the underwriter guarantees compensation to the user according to the protection policy in case the task fails.
| Action | Sender | Required fields | Effect |
|---|---|---|---|
| RequestUW | Business agent | {job_id, agreement_hash, coverage_request} | Initiates underwriting review |
| UWDecision | Underwriter | {job_id, agreement_hash, approve/reject, premium, collateral_required?} | Opens underwriting path or enables override/cancellation |
| PayPremium | Requestor (human) | {job_id, agreement_hash, premium, premium_ref, signature} | Adopts underwriting coverage |
| LockCollateral ; RefuseCollateral | Business agent | {job_id, agreement_hash, amount?, collateral_ref?, signature} | Locks collateral or escalates to override |
| OverrideDecision | Human authority | {job_id, agreement_hash, proceed/cancel, signature} | Explicitly accepts full pre-execution risk or terminates |
| ApproveRelease | Requestor-side signer | {job_id, agreement_hash, signature} | Contributes requestor-side approval for principal release |
Table 4: ARS actions for the principal track in fund-moving jobs.
| Action | Sender | Required fields | Effect |
|---|---|---|---|
| EvaluateOutcome | Evaluator/arbiter | {job_id, agreement_hash, pass/fail, trigger?, evidence_ref?} | Produces auditable outcome under |
| SettleCollateral | Settlement layer | {job_id, agreement_hash, slash/unlock, amount, settlement_ref} | Applies collateral policy under |
| FileClaim | Requestor | {job_id, agreement_hash, trigger, claimed_loss, evidence_ref} | Requests reimbursement under coverage terms |
| PayClaim | Underwriter (or payout vault) | {job_id, agreement_hash, payout, payout_ref, signature?} | Pays reimbursement up to coverage limit |
Table 5: ARS actions for evaluation, collateral settlement, and claims.
| State | Enabled action | Actor | Next state |
|---|---|---|---|
| FEE_AWAIT_LOCK | LockFeeEscrow | Requestor | FEE_ESCROW_LOCKED |
| FEE_ESCROW_LOCKED | SubmitDeliverable | Business agent | FEE_DELIVERED |
| FEE_DELIVERED | (no fee-track action; enables evaluation) | – | EVALUATION (phase transition) |
Table 6: Fee-track transaction states. The delivery signal ( SubmitDeliverable ) anchors auditable evidence used in EVALUATION to determine fee release/refund.
| State | Enabled action | Actor | Next state |
|---|---|---|---|
| UW_AWAIT_REQUEST | RequestUW | Business agent | UW_REVIEW |
| UW_REVIEW | UWDecision | Underwriter | approve PREMIUM_PENDING (if premium required) or COLLATERAL_REQUESTED ; reject OVERRIDE_PENDING (if allowed) else CANCELLED |
| PREMIUM_PENDING | PayPremium | Human requestor | COLLATERAL_REQUESTED |
| COLLATERAL_REQUESTED | LockCollateral ; RefuseCollateral | Business agent | lock APPROVAL_PENDING ; refuse OVERRIDE_PENDING |
| OVERRIDE_PENDING | OverrideDecision | Human authority | proceed APPROVAL_PENDING ; cancel CANCELLED |
| APPROVAL_PENDING | ApproveRelease | Requestor-side signer(s) | if then RELEASABLE |
Table 7: Principal-track transaction states for fund-involving jobs. Underwriting protection is enabled only when the requestor adopts coverage ( e.g. , by paying premium); otherwise, continuation requires explicit human override acknowledging full pre-execution risk.
Figure 5: AP2+ ARS . AP2 provides authorization evidence and bounded delegation, and ARS adds settlement semantics over the authorized transaction.
Figure 6: VI+ ARS : VI governs privacy-preserving authorization through layered credentials and selective disclosure; ARS governs settlement and compensation over the authorized transaction.
Figure 7: Loading factor sweep: adoption rate, loss reduction rate, failure reduction rate, and underwriter wallet final as a function of .
Figure 8: FP/FN sweep: adoption rate, loss reduction rate, failure reduction rate, and underwriter wallet final as functions of false-negative rate (x-axis) and false-positive rate (y-axis).
Figure 9: Sigmoid collateral sweep: adoption rate, loss reduction rate, failure reduction rate, and wallet final as functions of sigmoid midpoint (y-axis) and steepness (x-axis).
More in this topic
AI Risk Management
AI risk management studies how to identify, assess, reduce, and monitor harms from AI across development, deployment, and wider social systems.
See all 108 papers