cs.AINov 6, 2026

Key Institutional Considerations for Designing Effective Independent Verification Organizations

Authors: Uma Kalkar, Rumman Chowdhury, Reva Schwartz, Gabriella Waters

Organizations: Independent AI Evaluation Foundation · Humane Intelligence PBC · Civitaas Insights · Virginia State University

Abstract

Independent verification organizations (IVOs) have emerged as a popular, bipartisan solution for facilitating high-quality independent evaluations of AI technologies. But momentum is outpacing design: there is emerging consensus on the necessary operating conditions for robust IVOs, but little discourse on institutional and policy specifications to ensure those conditions are met. IVOs are currently envisioned primarily as a mechanism operating at the AI frontier and model level to evaluate catastrophic risks. Little work has yet been done on how to establish or enforce an IVO ecosystem that is equipped to handle myriad risks (i.e., beyond catastrophic ones) and operate at the system and application level across the deployment and use stages of the AI lifecycle. This paper argues that IVO design should ensure the capacity for evaluating AI models, systems, and applications in context for both frontier companies and enterprise companies & AI-adopting organizations. Additionally, this paper provides early considerations to advance IVO discourse from normative to operative, and from existential to everyday risks. Drawing from existing institutional constructs in financial services, education, public health, and science, we outline key considerations on whom IVOs serve, what access they require, how they interact with external governing structures, how they are funded, and how they fit into the wider assurance ecosystem. Much of the structure and knowledge necessary to holistically design IVOs and their ecosystem does not yet exist, even as companies rush to ink agreements with their chosen evaluators.

Figures & tables

Explore similar work

CardsList
  1. Who Can Evaluate AI Independently? A Framework for Analysing Evaluation Infrastructure

    Nov 5, 2026Sumaya Nur Adan, Joanna Wiaterek, Nourhan Abdelrhim +4AI Safety EvaluationAI Governance

  2. The AI Evaluability Gap: The Missing Layer for Managing Risk and Sustaining Value

    Jun 19, 2026Vishal Srivastava, Tanmay SahAI AssuranceAI Risk Management

  3. Bounding the Black Box: A Statistical Certification Framework for AI Risk Regulation

    Apr 23, 2026Natan Levy, Gadi PerlAI AssuranceAI Risk Management