Actionable Meta-Requirements for AI Robustness Auditing

Joshua Owotogbe; Roya Nasiri; Indika Kumara; Damian Andrew Tamburri; Willem-Jan van den Heuvel

cs.AIOct 9, 2026

Actionable Meta-Requirements for AI Robustness Auditing

Authors: Joshua Owotogbe, Roya Nasiri, Indika Kumara, Damian Andrew Tamburri, Willem-Jan van den Heuvel

Organizations: Jheronimus Academy of Data Science, Netherlands · Tilburg University, Netherlands · University of Sannio, Italy

Abstract

AI-enabled systems must remain dependable when the conditions under which they operate change (e.g., due to input perturbations, distribution shifts, or changes in external dependencies). This makes robustness auditing of such systems important for judging whether the available evidence is sufficient to support a defined robustness conclusion. However, designing an AI robustness auditing framework is challenging. A key unresolved issue is determining the requirements such a framework should support; these requirements have not yet been consolidated. We address this gap through meta-requirements: higher-order requirements specifying what a class of auditing frameworks should support. In design science research, literature and domain experts are common sources of meta-requirements; accordingly, we derive the initial set from literature and validate and refine it with practitioners. Our systematic literature review retained 74 of 2,890 records and derived 43 meta-requirements across 16 categories. We evaluated them with 128 practitioners, obtaining 5,504 ratings and 586 substantive comments. Overall, 81.20% of ratings indicated agreement or strong agreement. In particular, 41 of the 43 literature-derived meta-requirements were classified as strongly supported. Practitioner feedback refined the requirements and identified an additional requirement concerning external components and supply chain dependencies. The final set comprises 44 meta-requirements across 16 categories, of which 43 received quantitative validation. These meta-requirements can help organizations implement AI robustness auditing frameworks, assess existing frameworks for completeness, and refine them.

Figures & tables

Concept	Meaning in this study
AI-enabled system	A software system in which at least one AI component contributes to the system’s functionality ( Martínez-Fernández et al., 2022 ) .
AI audit	A systematic assessment of an AI-enabled system, its development or use, and relevant evidence against specified technical, organisational, legal, or ethical criteria ( Mökander, 2023 ; Minkkinen et al., 2022 ) .
AI robustness	The ability of an AI-enabled system to maintain specified acceptable behaviour when relevant inputs or operating conditions change ( Freiesleben and Grote, 2023 ; Tocchetti et al., 2025 ; Wang et al., 2026 ) .
Robustness testing	Testing whether an AI-enabled system or its components maintain specified acceptable behaviour under defined changes to inputs or operating conditions. ( Wang et al., 2026 ; Yu et al., 2026 ; Håkansson et al., 2023 ; Bayram and Ahmed, 2025 ; Håkansson et al., 2023 ; Silva and De França, 2023 )
Adversarial robustness	A form of robustness concerned with adversarial attacks, that is, deliberate attempts to cause incorrect or otherwise undesired system behaviour, for example by manipulating model inputs ( Wang et al., 2026 ; Nolte et al., 2025 ; Liu et al., 2022 ) .
Non-adversarial robustness	A form of robustness concerned with naturally occurring changes, such as noise, image blur, lighting changes, sensor degradation, or distribution shift, where the data encountered during evaluation or deployment differ from those represented during model development ( Wang et al., 2026 ; Nolte et al., 2025 ; Freiesleben and Grote, 2023 ; Kemmerzell et al., 2025 ) .

Table 1. Key concepts and definitions used in this study.

Ref.	Year	Objective	Method	Robustness Focus	Audit/ Assur.	Req./MR Output	Pract. Eval.	Findings / Limitation
( Raji et al., 2020 )	2020	Develop an end-to-end internal framework for algorithmic auditing for development of AI systems.	Framework synthesis informed by auditing practices in adjacent disciplines; illustrative cases	◐	✓	-	✗	Provides a broad internal AI auditing framework with staged evidence collection and testing, but no robustness-specific meta-requirements.
( Zacharias et al., 2022 )	2022	Develop a design framework for explainable feature selection and instantiate it as a software artefact.	Design Science Research; literature and legal-document review; two expert focus groups	✗	✗	3 meta-requirements + design principles	✓( $n=12$ )	Derives and evaluates meta-requirements for explainable feature selection, not robustness auditing.
( Mäntymäki et al., 2023 )	2023	Derive meta-requirements for organisational AI governance frameworks.	Design Science Research; literature-derived premises; industry-academia collaboration with organisational design partners	✗	◐	7 meta-requirements	✓	Derives meta-requirements for AI governance with organisational design-partner input, but does not develop robustness-specific auditing meta-requirements.
( Freiesleben and Grote, 2023 )	2023	Develop a unifying conceptual account of ML robustness based on targets and modifiers.	Conceptual analysis	✓	◐	-	✗	Provides a conceptual robustness framework and distinguishes robustness testing from robustness improvement, but no robustness auditing requirements framework.
( Díaz-Rodríguez et al., 2023 )	2023	Connect trustworthy-AI principles, ethics, regulation, and technical requirements.	Multidisciplinary conceptual analysis	◐	◐	7 trustworthy-AI requirements analysed	✗	Analyses seven trustworthy-AI requirements, including robustness and safety, but does not derive robustness auditing meta-requirements.
( Mökander et al., 2024 )	2023	Develop a three-layered blueprint for auditing large language models.	Literature review; conceptual synthesis	◐	✓	7 design claims; no meta-requirements	✗	Proposes coordinated governance, model, and application audits and seven design claims, but no robustness-specific meta-requirements.

Table 2. Comparison of representative studies relevant to AI robustness auditing meta-requirements.

Figure 1. Overview of the four-phase research method used to derive, refine, validate, and synthesize the meta-requirements.

Research Question	Motivation
RQ1: What meta-requirements should an AI robustness auditing framework satisfy?	This question identifies the requirements that should guide the design and conduct of AI robustness auditing.
RQ2: What are practitioners’ perspectives on the proposed meta-requirements for AI robustness auditing?	This question examines practitioners’ support for, qualifications of, challenges to, and suggested refinements or additions to the proposed meta-requirements.

Table 3. Research questions and motivation.

Digital Library	Search String	Retrieved Articles Numbers
ACM	[[Abstract: "robustness"] OR [Abstract: resilien] OR [Abstract: reliab] OR [Abstract: dependab]] AND [[Abstract: "artificial intelligence"] OR [Abstract: "machine learning"] OR [Abstract: "algorithmic"] OR [Abstract: "ai"] OR [Abstract: "ml"]] AND [[Abstract: audit] OR [Abstract: assess] OR [Abstract: test] OR [Abstract: evaluat]] AND [[Abstract: practitioner] OR [Abstract: developer] OR [Abstract: engineer] OR [Abstract: stakeholder]] AND [[Abstract: interview] OR [Abstract: survey*]]	925
Scopus	TITLE-ABS-KEY (("robustness" OR "resilien" OR "reliab" OR "dependab") AND ("artificial intelligence" OR "machine learning" OR "algorithmic" OR "AI" OR "ML") AND ("audit" OR "assess" OR "test" OR "evaluat") AND ("practitioner" OR "developer" OR "engineer" OR "stakeholder") AND ("interview" OR "survey*"))	473
IEEE Xplore	("All Metadata":"robustness" OR "All Metadata":resilien* OR "All Metadata":reliab* OR "All Metadata":dependab) AND ("All Metadata":"artificial intelligence" OR "All Metadata":"machine learning" OR "All Metadata":"algorithmic" OR "All Metadata":"AI" OR "All Metadata":"ML") AND ("All Metadata":audit OR "All Metadata":assess* OR "All Metadata":test* OR "All Metadata":evaluat) AND ("All Metadata":practitioner OR "All Metadata":developer* OR "All Metadata":engineer* OR "All Metadata":stakeholder) AND ("All Metadata":interview OR "All Metadata":survey*)	1292
Total	-	2690

Table 4. Search Strings and Results

Inclusion Criteria	Exclusion Criteria
IC1: Studies must be written in English.	EC1: Studies not written in English.
IC2: Studies must be published between January 2019 and February 2026.	EC2: Studies published outside the selected time window.
IC3: Studies must be available in full text.	EC3: Studies whose full text is unavailable or inaccessible.
IC4: Studies must be academic research publications, including journal articles, conference papers, or workshop papers of at least six pages.	EC4: Duplicate records, papers shorter than six pages, books, editorials, abstracts, posters, proceedings summaries, presentation-only papers, or other publication types lacking sufficient research content.
IC5: Studies must examine AI, machine learning, deep learning, large language models, algorithmic systems, or AI-enabled software systems as the object of study.	EC5: Studies outside the AI/ML domain, or studies that use AI/ML only as an analytical tool for prediction, classification, diagnosis, forecasting, optimisation, sensing, or domain analysis without examining the AI/ML system itself.
IC6: Studies must address robustness or related dependability concerns of AI/ML systems, such as reliability, resilience, safety, fault tolerance, failure behaviour, distribution shift, out-of-distribution behaviour, adversarial robustness, or failure analysis.	EC6: Studies that use robustness, reliability, or resilience only in a non-AI sense, or only to describe general predictive performance, efficiency, usability, adoption, trust, sustainability, educational outcomes, physical-system performance, or organisational resilience.

Table 5. Inclusion and exclusion criteria.

Figure 4. Example of how extracted literature evidence is coded and consolidated into MR-3.2, Robustness Trade-off Assumptions.

Inclusion	I1:	The respondent must demonstrate relevant academic, professional, or practical experience in AI/ML systems, AI robustness assessment or auditing, robustness testing, model evaluation, adversarial evaluation, AI safety, model reliability, or related assurance activities.
Exclusion	E1:	A response is excluded if it contains inconsistent, apparently random, or meaningless answers.
	E2:	A response without informed consent is excluded.

Table 6. Inclusion and exclusion criteria for selecting valid responses.

ID	Sub-Category	Requirement	Source(s)
Human Dimension
Category 1: Standards and Governance
MR-1.1	Standardized Metrics	Robustness auditing should use standardised robustness metrics and benchmark definitions where available, so that reported robustness results are comparable across systems, audits, and regulatory contexts.	( Nolte et al., 2025 ; Khinvasara et al., 2024 ; Oala et al., 2020 ; Blood et al., 2023 ; Yu et al., 2026 ; Tocchetti et al., 2025 ; Yang et al., 2020 ; Agarwal and Nene, 2025 ; Chen and Das, 2023 ; Madaio et al., 2022 ; Freiesleben and Grote, 2023 ; Casper et al., 2024 ; Raji et al., 2020 ; Thelisson and Verma, 2024 ; Nanda et al., 2021 ; Hornischer, 2026 ; Javed et al., 2024 ; Liu et al., 2022 ; Li et al., 2023 ; Wei and Liu, 2025 ; Costanza-Chock et al., 2022 )
MR-1.2	Standardized Lexicon and Terminology	Robustness audits should use a shared vocabulary, taxonomy, and terminology for robustness and auditability concepts.	( Mohseni et al., 2022 ; Vadlapati, 2026 ; Yu et al., 2026 ; Ge et al., 2024 ; Tocchetti et al., 2025 ; Yang et al., 2020 ; Chen and Das, 2023 ; Freiesleben and Grote, 2023 ; Raji et al., 2020 ; Thelisson and Verma, 2024 ; Nanda et al., 2021 ; Nolte et al., 2025 ; Khinvasara et al., 2024 ; Hornischer, 2026 ; Liu et al., 2022 ; Li et al., 2023 ; Wei and Liu, 2025 ; Costanza-Chock et al., 2022 )
MR-1.3	Standardized Tools	Robustness audits should use common or comparable tools, checklists, templates, scorecards, libraries, or toolkits where they support consistent assessment.	( Raji et al., 2020 ; Momcilovic et al., 2026 ; Ojewale et al., 2025 ; Oala et al., 2020 ; Tocchetti et al., 2025 ; Dev et al., 2025 ; Yu et al., 2026 ; Ge et al., 2024 ; Yang et al., 2020 ; Agarwal and Nene, 2025 ; Casper et al., 2024 ; Kokina et al., 2025 ; Thelisson and Verma, 2024 ; Nolte et al., 2025 ; Khinvasara et al., 2024 ; Javed et al., 2024 ; Li et al., 2023 ; Costanza-Chock et al., 2022 )
MR-1.4	Actionable Guidance	Practitioners should receive practical guidance for selecting tests, thresholds, evidence, mitigations, and audit steps.	( Kokina et al., 2025 ; Khinvasara et al., 2024 ; Narula et al., 2025 ; Blood et al., 2023 ; Habbal et al., 2024 ; Thelisson and Verma, 2024 ; Cooper et al., 2022 ; Laux et al., 2024 ; Chen and Das, 2023 ; Saeed and Alsharidah, 2024 ; Xiong et al., 2022 ; McCormack et al., 2025 ; Koshiyama et al., 2024 ; Momcilovic et al., 2026 ; Chander et al., 2025 )

Table 7. Meta-requirements for AI robustness auditing: 43 literature-derived requirements and MR-8.4, added after the practitioner survey.

Figure 5. Final practitioner-informed meta-requirements for AI robustness auditing, comprising 44 meta-requirements across 16 categories and three dimensions.

Table 13

Rank	Category	Agree (%)	Rank	Category	Agree (%)
1	Workflow Integration	86.20	9	Communication	81.64
2	Configurability	83.59	10	Robustness Definition	81.25
3	Data Auditability	83.12	11	Human-in-the-Loop	81.25
4	Knowledge and Skills	82.81	12	Accessibility	81.25
5	Output Auditability	82.81	13	Efficiency	79.69
6	Monitoring	82.81	14	Standards and Governance	79.49

Table 11. Category-level agreement ranking across the original 16 survey categories.

Cat.	MR	Meta-requirement	$n$	Mean	SD	Median	Mode	IQR	Agree (%)	Neutral (%)	Disagree (%)	Quantitative status
1	MR-1.1	Should use standardized robustness metrics	128	4.23	1.12	5.0	5	1.0	82.81	6.25	10.94	Strongly supported
1	MR-1.2	Should use shared robustness terminology	128	4.15	1.04	4.0	5	1.0	76.56	16.41	7.03	Strongly supported
1	MR-1.3	Should use standardized robustness tools	128	3.90	0.92	4.0	4	0.0	78.12	12.50	9.38	Strongly supported
1	MR-1.4	Should provide actionable audit guidance	128	4.16	1.00	4.0	5	1.0	75.78	17.19	7.03	Strongly supported
1	MR-1.5	Should integrate robustness into AI lifecycle workflows	128	4.30	0.93	5.0	5	1.0	81.25	14.06	4.69	Strongly supported
1	MR-1.6	Should establish robustness governance protocols	128	4.31	0.97	5.0	5	1.0	83.59	10.16	6.25	Strongly supported

Table 12. Practitioner validation results for the 43 literature-derived meta-requirements (

n=128

respondents per meta-requirement).

AI Assurance

Artificial intelligence assurance develops evidence and controls to evaluate, govern, and certify AI systems across their lifecycle and deployment contexts.

See all 79 papers

May 7, 2026

Adaptive auditing of AI systems with anytime-valid guarantees

A major bottleneck in characterizing the failure modes of generative AI systems is the cost and time of annotation and evaluation. Consequently, adaptive testing paradigms have gained popularity, where one opportunistically decides which cases and how many to annotate based on past results. While this framework is highly practical, its extreme flexibility makes it difficult to draw statistically rigorous conclusions, as it violates classical assumptions: the number of observations is typically limited (often 10 to 50 cases) and decisions regarding sampling and stopping are made in the midst of data collection rather than based a pre-specified rule. To characterize what statistical inferences can be drawn from highly adaptive audits, we introduce a hypothesis testing framework from two "dueling" perspectives: (i) the model's null that asserts there is no failure mode with performance below a target threshold versus (ii) the auditor's null that asserts they have a sampling strategy that will uncover a failure mode. Leveraging Safe Anytime-Valid Inference (SAVI), we formalize the auditor as conducting "testing by betting," which translates into simultaneous e-processes for testing the dueling null hypotheses. Furthermore, if the auditor is sufficiently powerful, we prove that these two hypotheses are asymptotically inverses of each other, in that passage of such an audit certifies the AI system as being globally robust. Empirically, we demonstrate that our proposed testing procedures maintain anytime-valid Type-I error control, outperform pre-specified testing methods, and can reach statistically rigorous conclusions sometimes with as few as 20 observations.

Siyu Zhou, Patrick Vossler, Venkatesh Sivaraman +2

University of California, San Francisco · Stanford University

Algorithmic Auditing Anytime-Valid Inference Sequential Hypothesis Testing

Ask PDF

Aug 5, 2026

A Chain Is Only as Strong as Its Weakest Link: A Scoping Review of System Integration Audits in AI

As AI systems become increasingly integrated into diverse interfaces and applications, model-centric audits are insufficient to address risks arising from interactions among system components and deployment environments. System integration has long been central to software audits in safety-critical domains such as aerospace. However, its role in AI auditing remains underexplored. Scanning through 4,259 documents, we present a scoping review of AI audits that treat system integration as a core tenet of evaluation (n = 58). Using reflexive thematic analysis, we analyze their elements, actors, enablers, and constraints. We find that the corpus represents an emerging yet still fragmented form of AI auditing: few existing measures target integration-specific risks; large gaps remain in meeting traditional audit expectations; and access to necessary information and resources significantly influences audit design. Nonetheless, integration can be categorized across three sites (inter-component, system-environment, and multi-system), each serving the functions of risk exploration, risk determination, coordination, and procedural regularity. Deviating from other types of evaluations, these audits assess qualities specific to system integration, including compatibility, completeness, and oversight. This review calls on the AI community to prioritize system integration as a core strategy for addressing AI risk, and to develop audit practices capable of capturing failures across components, environments, and systems beyond the reach of component-level evaluation.

Leah Davis, Dominic Martin, AJung Moon

McGill University, Montr´eal, QC · Universit´e du Qu´ebec `a Montr´eal, Montr´eal, QC

AI Assurance Algorithmic Auditing

Ask PDF

Apr 24, 2026

Make Mechanistic Interpretability Auditable: A Call to Develop Guidelines via Continuous Collaborative Reviewing

While mechanistic interpretability (MI) has produced important insights into neural network internals, the field has yet to establish a standardized system to audit experiments. As such, many of its findings remain underutilized in safety-critical applications such as medical AI and autonomous systems, as stakeholders cannot certify their validity. Recent work demonstrates this concretely: two papers found conflicting conclusions for the same behavior, and a third study revealed that both were partially correct but incomparable due to methodological inconsistencies. Without standardized auditing, such ambiguities hinder adoption in high-stakes contexts requiring strong correctness guarantees. We call for the MI community to work towards developing a novel reviewing system that complements peer review via: (1) Continuous reviewing supported by a Collaborative Reviewing Platform where meta-science results and discussions (such as critiques, negative results, post-hoc extensions, reproductions, replications, and partial results) that fit outside of papers are organized and discussed, allowing for comments and revisions to be made at any time (2) Generalizing good practices found on this platform into expert-verified guidelines and protocols to improve auditing efficiency, and (3) Source-based auditing systems that track arguments which claims depend on. This position paper encourages constructive debate over the necessity, design and implementation of such a framework, providing early concrete examples to help catalyze these dialogues. Overall, we propose that auditing MI itself is essential for its application in AI safety, industry, and governance.

Michael Lan, Narmeen Fatimah Oozeer, Chaithanya Bandi +4

Martian · University of Delaware · MATS +2

Scholarly Peer Review Mechanistic Interpretability Scientific Reproducibility

Ask PDF

Abstract

AI Assurance

Explore similar work

Adaptive auditing of AI systems with anytime-valid guarantees

A Chain Is Only as Strong as Its Weakest Link: A Scoping Review of System Integration Audits in AI

Make Mechanistic Interpretability Auditable: A Call to Develop Guidelines via Continuous Collaborative Reviewing