cs.AINov 16, 2026

Which AI Use Cases Deserve a Pilot? A Risk and Value Framework for Funding, Scoping, and Scaling AI

Authors: Vishal Srivastava

Organizations: Whiting School of Engineering, Johns Hopkins University, Baltimore, MD 21218

Abstract

A composite score can rank AI use cases while leaving the permitted operating scope and the evidence required for expansion unspecified. We treat the unit of decision as a use case at a proposed scope (authority, human mediation, limits, volume, environment, population) and define three decisions: screen the scope, fund the experiment, and authorize expansion only when residual risk is acceptable and evidence of incremental net value applies at that scope. A risk track (R1-R4) is implemented as an explicit decision table with vetoes and documented precedence, and a value track follows four evidence stages with each transition needing its own evidence. Funding uses expected value of perfect information as an upper bound and the value of sample information for the specific experiment. A quantified expected-loss check, required where policy says so, computes the containment an expansion needs and credits a control only on evidence measured at the proposed scope; in a synthetic scenario the highest-value control design fails that test while a smaller scope passes. A reproducible Python package with 279 passing tests demonstrates the framework on synthetic refund, reconciliation and knowledge-assistant use cases, including a synthetic matched-scope fixture and a transferability stress test (value evidence is simulated, not realized), and compares it with a public AWS sample. Thresholds and appetite are prototype policy requiring field validation.

Figures & tables

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Real-World AI Evaluation: How FRAME Generates Systematic Evidence to Resolve the Decision-Maker's Dilemma

    Feb 28, 2026Reva Schwartz, Gabriella WatersHuman-Centered AIAI Governance

  2. Towards Apples to Apples for AI Evaluations: From Real-World Use Cases to Evaluation Scenarios

    May 8, 2026Yee-Yin Choong, Kristen Greene, Alice Qian +6Human-Centered AI

  3. The AI Evaluability Gap: The Missing Layer for Managing Risk and Sustaining Value

    Jun 19, 2026Vishal Srivastava, Tanmay SahAI AssuranceAI Risk Management