Who Can Evaluate AI Independently? A Framework for Analysing Evaluation Infrastructure

Sumaya Nur Adan; Joanna Wiaterek; Nourhan Abdelrhim; Sydney Reis; Robert F. Trager; Benjamin Rosman; Marina Jirotka

cs.CYNov 5, 2026

Who Can Evaluate AI Independently? A Framework for Analysing Evaluation Infrastructure

Authors: Sumaya Nur Adan, Joanna Wiaterek, Nourhan Abdelrhim, Sydney Reis, Robert F. Trager, Benjamin Rosman, Marina Jirotka

Organizations: University of Oxford · Centre for AI Security and Access · University of the Witwatersrand

Abstract

AI evaluation is increasingly central to AI safety and governance, yet taking part in an evaluation does not by itself confer authority over its design, its execution or the resources it depends on. This paper examines how infrastructure arrangements distribute that authority, and so shape who can evaluate AI systems independently. Drawing on Science and Technology Studies, political economy and applied evaluation standards, we introduce PAAI (Power, Access, Adequacy, Independence), a framework for examining who controls evaluation infrastructure, the terms on which it is available, whether it is adequate to the evaluation's purpose and what autonomy it supports. We apply PAAI to twelve African AI evaluation projects (2022-2025) and to the documented roles of all ten member institutes in the three published joint testing exercises of the international network of AI Safety Institutes. We identify three recurring configurations: extractive collaboration, dependent innovation and constrained independence. African actors often contribute data, language expertise and task design while external institutions retain authority over infrastructure and evaluation frameworks; design authority without infrastructure authority never yielded more than partial independence. The joint testing exercises show a similar division among mostly high-income members: the institutes that led each exercise also ran the models, while Kenya, the network's only African member, moved from annotation to task design with no documented model execution. Participation in evaluation does not by itself produce evaluative authority. For evaluation to serve as a credible governance mechanism, access must come with control over priorities, methods, publication and continuity.

Figures & tables

Figure 1: The PAAI framework. (a) Power has two parts. Design authority decides what is tested and how, and shapes whether the evaluation fits its setting. Infrastructure authority controls the compute, data, funding and model access an evaluation runs on, and sets the terms of Access. Together they determine Independence. Each box gives the question PAAI asks. (b) Who holds each part of Power gives four configurations, each shown with an example from our data.

Project	Type	Power (D/I)	Access	Adequacy	Independence	Configuration
AISI Multilingual Joint Testing	Ml. safety	Ext / Ext	Mediated, time-limited	Externally defined	None	Extractive
Kenya AISI Agentic Testing	Ml. safety	Part / Ext	Coordinated; renewal not established	Limited relevance	Minimal	Extractive
Masakhane Uhura	Ml. safety	AL / Ext	Provider credits; renewal not established	Translated benchmarks	Partial	Dependent
Sunbird AI ASR	Benchmarking	AL / AL	Grant-funded	Contextually apt	High, cost-limited	Constrained
Lelapa InkubaLM	Benchmarking	AL / Part	Provider credits	Rigorous coverage	Partial	Constrained
Wits/UJ SAfriSenti	Benchmarking	AL / AL	Own hardware + cloud	Sufficient for scope	High, volunteer-limited	Constrained

Table 1: Twelve African AI evaluation projects characterised through PAAI. Power = design authority / infrastructure authority (AL = African-led; Ext = external; Part = partial). Configuration: extractive collaboration, dependent innovation or constrained independence. † Meta-evaluative case; configuration provisional, since no models were run. ‡ No African author in a design or analysis role. Justifications are in Appendix C .

Figure 2: Who held design and infrastructure authority in each African project, and the independence of African participants, grouped by configuration. Darker cells indicate more African authority or independence. Design authority alone never raised independence above partial. † Meta-evaluative case (no models run); configuration provisional. ‡ No African author in a design or analysis role (Georgia Tech: no African-affiliated authors; Barcelona: African co-authors credited for image collection only).

Figure 3: Roles of the ten member institutes across the three published joint testing exercises. (a) Participation tier derived from coded design roles and documented model execution (n/p: not a participant; Undoc.: not documented, because the pilot report names only its leads). (b) Exercises with a lead or substantive design role against exercises in which the institute ran models; the dashed line marks equal counts.

Dimension	Diagnostic question	Design condition for joint testing and access programmes	Applied: Kenya, agentic exercise
Power: design authority	Who defines the harms, tasks and metrics, and how results are scored?	Each participant leads at least one harm category or task family, including its metrics and scoring.	Partly met: contributed new fraud and phishing tasks, but did not lead a strand or set scoring.
Power: infrastructure authority	Who controls the compute, data, funding and model access?	Each participant receives model access and compute directly, not only through the lead’s platform.	Not met: tests were run on the lead’s platform; no record of Kenya running tests itself.
Access	On what terms, and for how long?	Access terms, duration and renewal are written down, shared with participants and multi-year.	Not demonstrated in the public record: access was mediated through the exercise; duration and renewal terms are not reported.
Adequacy	Is the evaluation rigorous and fit for the contexts it informs?	Tasks reflect participants’ deployment risks, and automated judges are validated per language.	Partly met: regional tasks included, but automated judges were least reliable for Kiswahili.
Independence	Can participants publish, replicate and continue the evaluation?	Participants may publish separately and keep using the datasets and tools they helped build.	Partly met: credited co-authorship; no documented right to rerun or reuse.

Table 2: PAAI as a tool for diagnosis and design. The last column applies the questions to Kenya’s role in the agentic joint testing exercise.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Provider	Programme	Duration and renewal	Publication terms	Projects
OpenAI	Researcher Access Program	Project-based; duration not disclosed; reapplication at provider discretion	Not disclosed; usage policy applies	C.8, C.11, C.12
Anthropic	External Researcher Access Program	Project-based; reapplication required	Not disclosed; usage policy applies	C.8
Google	Cloud research credits; Gemma academic programme; Lacuna Fund co-funder	Credits expire per cycle; competitive reapplication	Not disclosed	C.11, C.12
Microsoft	Azure credits (direct provision)	Duration of credit agreement; reapplication required	Not disclosed	C.5
AWS	Research credits	12-month cycles; competitive reapplication	Not disclosed	None (C.7 used paid EC2)
Oracle	Oracle for Research credits	Project-based; application-based	Not disclosed	C.12

Appendix

Table 3: Infrastructure access programmes used by the African projects (public documentation, accessed December 2025). Numbers refer to Appendix C .

Institute	Roles	Led	Ran models	Design
Singapore	3	3	3	3
United Kingdom	3	3	3	3
United States	2	1	1	1
Japan	2	1	1	1
Australia	2	0	2	2
France	2	0	1	2

Appendix

Table 4: Institute totals across the three exercises.

Institute	Documented role	Design	Ran models	Access	Role	Credit	Evidence
Singapore	Exercise lead (multilingual); ran all tests; wrote annotation guide and metrics calculator; led analysis	Lead	Main tests	Direct	Lead	12 authors; First-listed author block	High
Japan	Co-lead (multilingual); translated dataset into ten languages; ran independent tests; Japanese annotation and language analysis; translated cyber benchmarks	Lead	Own tests	Direct	Lead	5 authors; Japanese language analysis	High
UK	Lead of cybersecurity strand; ran cyber evaluations	Lead	Main tests	Direct	Lead	4 authors	High
Australia	Ran independent tests in Mandarin Chinese; input on cyber analysis	New parts	Own tests	Direct	Executing	7 authors	High
France	Contributed internal cyber benchmark; French annotation and language analysis	New parts	None recorded	Via lead	Contributing	3 authors; French language analysis	High
Korea	Korean annotation and language analysis; Korean translation of cyber benchmarks	Language/ analysis	None recorded	Via lead	Contributing	7 authors; Korean language analysis	High

Appendix

Table 5: Second exercise (multilingual and cybersecurity, February 2025). Column meanings are given in Appendix E . Author counts are estimated from the order of names on the arXiv version.

Institute	Documented role	Design	Ran models	Access	Role	Credit	Evidence
Singapore	Lead of leakage/fraud strand; ran tests; annotation guides; led analysis	Lead	Main tests	Direct	Lead	12 authors; First-listed author block	High
UK	Lead of cybersecurity strand; ran evaluations; statistical framework	Lead	Main tests	Direct	Lead	11 authors; Second-listed author block	High
France	New tasks and tools; ran variant tests; alternate judge prompt; extra metrics; French translation and annotation	New parts	Own tests	Direct	Executing	6 authors; French analysis	High
Korea	Validated annotations; ran temperature/model-size variations; translation, annotation; analysis of cyber test records	New parts	Own tests	Direct	Executing	7 authors; Korean analysis	High
Australia	Validated annotations; added metrics; ran cyber evaluations; translation, annotation; analysis of test records	New parts	Own tests	Direct	Executing	14 authors	High
EU AI Office	Ran cyber evaluations; analysis of test records	Language/ analysis	Own tests	Direct	Executing	4 authors	High

Appendix

Table 6: Third exercise (agentic, July 2025). Column meanings are given in Appendix E . Author counts are estimated from the order of names on the arXiv version.

AI Safety Evaluation

AI Safety Evaluation studies how to test whether AI systems avoid harmful behavior and remain reliable across users, tasks, and deployment settings.

See all 152 papers

Beyond Task Success: An Evidence-Synthesis Framework for Evaluating, Governing, and Orchestrating Agentic AI
Apr 18, 2026Christopher Koch, Joshua Andreas WellbrockLLM Agent Orchestration AI Agent Evaluation
Independent Researcher
EEFA: A Framework for the Ethical Evaluation of AI Agents
Nov 2, 2026Swathi Pallikala, Yashas HariprasadAI Agent Evaluation AI Ethics
M.S. Candidate in Computer Science (AI and ML concentration), California State University, East Bay. This work was conducted independently and was not part of a university-sponsored thesis or research project. · Computer Science, California State University, East Bay.
Mapping General-Purpose AI Governance in Twenty AI Middle-Power Jurisdictions
Aug 18, 2026Josephine Schwab, Nathan Naidoo, Ferruccio Barazzutti +2AI Governance AI Accountability
Arcadia Impact AI Governance Taskforce

Apr 18, 2026J/K move · Enter open · S save

Beyond Task Success: An Evidence-Synthesis Framework for Evaluating, Governing, and Orchestrating Agentic AI

Christopher Koch, Joshua Andreas Wellbrock

Independent Researcher

LLM Agent Orchestration AI Agent Evaluation AI Agent Safety AI Agent Governance

Open paper AskPDF arXiv

Agentic AI systems plan, use tools, maintain state, and act across multi-step workflows with external effects, meaning trustworthy deployment can no longer be judged by task completion alone. The current literature remains fragmented across benchmark-centered evaluation, standards-based governance, orchestration architectures, and runtime assurance mechanisms. This paper contributes a bounded evidence synthesis across a manually coded corpus of twenty-four recent sources. The core finding is a governance-to-action closure gap: evaluation tells us whether outcomes were good, governance defines what should be allowed, but neither identifies where obligations bind to concrete actions or how compliance can later be proven. To close that gap, the paper introduces three linked artifacts: (1) a four-layer framework spanning evaluation, governance, orchestration, and assurance; (2) an ODTA runtime-placement test based on observability, decidability, timeliness, and attestability; and (3) a minimum action-evidence bundle for state-changing actions. Across sources, evaluation papers identify safety, robustness, and trajectory-level measurement as open gaps; governance frameworks define obligations but omit execution-time control logic; orchestration research positions the control plane as the locus of policy mediation, identity, and telemetry; runtime-governance work shows path-dependent behavior cannot be governed through prompts or static permissions alone; and action-safety studies show text alignment does not reliably transfer to tool actions. A worked enterprise procurement-agent scenario illustrates how these artifacts consolidate existing evidence without introducing new experimental data.

Abstract

AI Safety Evaluation

Explore similar work

Beyond Task Success: An Evidence-Synthesis Framework for Evaluating, Governing, and Orchestrating Agentic AI

EEFA: A Framework for the Ethical Evaluation of AI Agents

Mapping General-Purpose AI Governance in Twenty AI Middle-Power Jurisdictions