Key Institutional Considerations for Designing Effective Independent Verification Organizations
Organizations: Independent AI Evaluation Foundation · Humane Intelligence PBC · Civitaas Insights · Virginia State University
Abstract
Independent verification organizations (IVOs) have emerged as a popular, bipartisan solution for facilitating high-quality independent evaluations of AI technologies. But momentum is outpacing design: there is emerging consensus on the necessary operating conditions for robust IVOs, but little discourse on institutional and policy specifications to ensure those conditions are met. IVOs are currently envisioned primarily as a mechanism operating at the AI frontier and model level to evaluate catastrophic risks. Little work has yet been done on how to establish or enforce an IVO ecosystem that is equipped to handle myriad risks (i.e., beyond catastrophic ones) and operate at the system and application level across the deployment and use stages of the AI lifecycle. This paper argues that IVO design should ensure the capacity for evaluating AI models, systems, and applications in context for both frontier companies and enterprise companies & AI-adopting organizations. Additionally, this paper provides early considerations to advance IVO discourse from normative to operative, and from existential to everyday risks. Drawing from existing institutional constructs in financial services, education, public health, and science, we outline key considerations on whom IVOs serve, what access they require, how they interact with external governing structures, how they are funded, and how they fit into the wider assurance ecosystem. Much of the structure and knowledge necessary to holistically design IVOs and their ecosystem does not yet exist, even as companies rush to ink agreements with their chosen evaluators.
Figures & tables
| Framework | Status | Legislation Target | Designation Authority | AI Level Focus | Risk Focus | Notable Provisions |
|---|---|---|---|---|---|---|
| California SB 813 | Enacted, September 2026 | Voluntary basis: would-be IVOs | Government Operations Agency | Model (primarily); System | Safety; Risk | Notes that IVO designation requires “expertise, independence, and documented methods” |
| California AB 1405 | Enacted, September 2026 | AI auditors | Government Operations Agency | Not specified | Not specified | Establishes an AI Auditor Registry, with registration of AI auditors by 2029 |
| Connecticut HB 5222 | Enacted, May 2026 | Voluntary basis: AI systems | Department of Consumer Protection | System | State-set risk and harm standards | Pilot of up to five approved IVOs to assess AI system risks Verified companies can obtain “evidentiary support” in civil cases |
| Illinois SB 315 | Enacted, July 2026 | Mandatory, Annually: Large frontier developer organizations | Not specified (but auditors must be “competent” and “financially independent” from frontier companies) | Model | Catastrophic risks | Mandates independent third-party audits beginning in 2028 |
| Virginia HB 797 | Enacted, April 2026 | Not applicable | Not applicable (study by the Technology and Science Commission) | Not applicable | Not applicable | Commissions a study investigating whether IVOs are a useful governance mechanism for Virginia |
| Ohio HB 628 | Introduced, December 2025 | Voluntary: would-be IVOs | Ohio Attorney General | Not specified | Risks that the IVO chooses to assess | Obtaining a license requires independence from the AI industry and sufficiently rigorous methods Designation is revocable if independence is compromised or methods become obsolete |
| Model | Function | Funding mechanism | Lessons for IVOs |
|---|---|---|---|
| U.S. Public Company Accounting Oversight Board (PCAOB) | Oversees auditors of public companies. | Legally mandated funding provided by companies audited by PCAOB-registered firms, as per the Sarbanes-Oxley Act of 2002 and amendments in the Dodd-Frank Act of 2010. “Accounting support fees” collected from issuers and broker-dealers ( PCAOB n.d. ). Failure to pay by companies triggers legal action. | Mandatory, rule-based fee structure can direct companies to fund IVO activities via a pooled fund managed by a third-party organization. |
| U.K. Financial Conduct Authority (FCA) | Financial regulator that can require an independent ‘skilled person’ review of regulated firms. | Either the FCA or the regulated firm appoints the ‘skilled person’ reviewer. In both cases, the regulated firm pays for the assessment ( FCA 2026 ). | A regulator can direct a firm to pay for a similar ‘skilled IVO’ review to assess risk, although the threshold for requesting and carrying out the assessment would need to be defined and applied ex post. |
| Meta Oversight Board | External body where Meta product users can appeal content enforcement decisions sanctioned by the platform. | Meta funds the Board through an irrevocable trust, which initially received $130 million over six years to support managing, overseeing, and operating the Oversight Board ( Oversight Board 2022 ). Meta periodically provides “top-up” funding ( Carozza 2026 ). While the funding is separate from Meta, the Oversight Board is wholly reliant on Meta to provide funding in the trust to continue operations. The trust’s legal entity, not Meta, holds contracts with Board Members. A separate set of trustees, who serve fixed terms, must consent before a Board Member’s contract is ended ( Oversight Board Trust 2025 ). | Funding for IVOs from top frontier companies can be pooled in a trust model. However, this model is reliant on company goodwill to sustain the trust. |
| International Accord | Industry-driven agreement to improve workplace safety for textile and garment workers. | Signatory brands pay a fee and collectively fund the Secretariat and country programs. Fees are determined on a sliding scale and by the number of signatory parties ( International Accord 2026 ). | Collective, sliding-scale fees can be levied on those whose tools require IVO evaluation. However, this model is reliant on voluntary industry sign-on. |
| Health Effects Institute (HEI) | Research organization contributing to the science on health effects of air pollution. | Receives joint funding from the U.S. Environmental Protection Agency and the worldwide motor vehicle industry ( HEI n.d. ). Independent Research and Review Committees evaluate projects independently of sponsors and of one another to avoid conflict of interest ( HEI n.d. ). | Limited public co-funding can supplement industry contributions, but funders and project reviewers need to be separated to limit conflicts of interest. |
| Player | Role in the IVO Ecosystem | IVO Evidence Needed (Section 4.3) | Flow in Evaluation Ecosystem |
|---|---|---|---|
| Follows Principle 1: Findings are evidence, not verdicts | |||
| Regulators | Equipped with IVO findings, determine legal or corrective obligations in regulated settings | Full findings | Post-evaluation |
| Courts | Where applicable, consider IVO findings to adjudicate liability | Full findings | Post-evaluation |
| Insurers | Underwrite AI liability coverage informed by IVO-generated evidence | Deployment- and use-relevant findings | Pre- and post-evaluation |
| Communities & Public | Need insight into how AI deployment and use affect the services and institutions they depend on | Anonymous, aggregate insights Public assurance statement | Post-evaluation |
| Follows Principle 2: Those who set rules do not apply them | |||
AI Governance
Artificial intelligence governance studies how laws, institutions, and technical controls shape AI development, deployment, accountability, and social effects.
See all 412 papers