cs.AIOct 9, 2026
SaveActionable Meta-Requirements for AI Robustness Auditing
Organizations: Jheronimus Academy of Data Science, Netherlands · Tilburg University, Netherlands · University of Sannio, Italy
Abstract
AI-enabled systems must remain dependable when the conditions under which they operate change (e.g., due to input perturbations, distribution shifts, or changes in external dependencies). This makes robustness auditing of such systems important for judging whether the available evidence is sufficient to support a defined robustness conclusion. However, designing an AI robustness auditing framework is challenging. A key unresolved issue is determining the requirements such a framework should support; these requirements have not yet been consolidated. We address this gap through meta-requirements: higher-order requirements specifying what a class of auditing frameworks should support. In design science research, literature and domain experts are common sources of meta-requirements; accordingly, we derive the initial set from literature and validate and refine it with practitioners. Our systematic literature review retained 74 of 2,890 records and derived 43 meta-requirements across 16 categories. We evaluated them with 128 practitioners, obtaining 5,504 ratings and 586 substantive comments. Overall, 81.20% of ratings indicated agreement or strong agreement. In particular, 41 of the 43 literature-derived meta-requirements were classified as strongly supported. Practitioner feedback refined the requirements and identified an additional requirement concerning external components and supply chain dependencies. The final set comprises 44 meta-requirements across 16 categories, of which 43 received quantitative validation. These meta-requirements can help organizations implement AI robustness auditing frameworks, assess existing frameworks for completeness, and refine them.
Figures & tables
| Concept | Meaning in this study |
|---|---|
| AI-enabled system | A software system in which at least one AI component contributes to the system’s functionality ( Martínez-Fernández et al., 2022 ) . |
| AI audit | A systematic assessment of an AI-enabled system, its development or use, and relevant evidence against specified technical, organisational, legal, or ethical criteria ( Mökander, 2023 ; Minkkinen et al., 2022 ) . |
| AI robustness | The ability of an AI-enabled system to maintain specified acceptable behaviour when relevant inputs or operating conditions change ( Freiesleben and Grote, 2023 ; Tocchetti et al., 2025 ; Wang et al., 2026 ) . |
| Robustness testing | Testing whether an AI-enabled system or its components maintain specified acceptable behaviour under defined changes to inputs or operating conditions. ( Wang et al., 2026 ; Yu et al., 2026 ; Håkansson et al., 2023 ; Bayram and Ahmed, 2025 ; Håkansson et al., 2023 ; Silva and De França, 2023 ) |
| Adversarial robustness | A form of robustness concerned with adversarial attacks, that is, deliberate attempts to cause incorrect or otherwise undesired system behaviour, for example by manipulating model inputs ( Wang et al., 2026 ; Nolte et al., 2025 ; Liu et al., 2022 ) . |
| Non-adversarial robustness | A form of robustness concerned with naturally occurring changes, such as noise, image blur, lighting changes, sensor degradation, or distribution shift, where the data encountered during evaluation or deployment differ from those represented during model development ( Wang et al., 2026 ; Nolte et al., 2025 ; Freiesleben and Grote, 2023 ; Kemmerzell et al., 2025 ) . |
Table 1. Key concepts and definitions used in this study.
| Ref. | Year | Objective | Method | Robustness Focus | Audit/ Assur. | Req./MR Output | Pract. Eval. | Findings / Limitation |
|---|---|---|---|---|---|---|---|---|
| ( Raji et al., 2020 ) | 2020 | Develop an end-to-end internal framework for algorithmic auditing for development of AI systems. | Framework synthesis informed by auditing practices in adjacent disciplines; illustrative cases | ◐ | ✓ | - | ✗ | Provides a broad internal AI auditing framework with staged evidence collection and testing, but no robustness-specific meta-requirements. |
| ( Zacharias et al., 2022 ) | 2022 | Develop a design framework for explainable feature selection and instantiate it as a software artefact. | Design Science Research; literature and legal-document review; two expert focus groups | ✗ | ✗ | 3 meta-requirements + design principles | ✓( ) | Derives and evaluates meta-requirements for explainable feature selection, not robustness auditing. |
| ( Mäntymäki et al., 2023 ) | 2023 | Derive meta-requirements for organisational AI governance frameworks. | Design Science Research; literature-derived premises; industry-academia collaboration with organisational design partners | ✗ | ◐ | 7 meta-requirements | ✓ | Derives meta-requirements for AI governance with organisational design-partner input, but does not develop robustness-specific auditing meta-requirements. |
| ( Freiesleben and Grote, 2023 ) | 2023 | Develop a unifying conceptual account of ML robustness based on targets and modifiers. | Conceptual analysis | ✓ | ◐ | - | ✗ | Provides a conceptual robustness framework and distinguishes robustness testing from robustness improvement, but no robustness auditing requirements framework. |
| ( Díaz-Rodríguez et al., 2023 ) | 2023 | Connect trustworthy-AI principles, ethics, regulation, and technical requirements. | Multidisciplinary conceptual analysis | ◐ | ◐ | 7 trustworthy-AI requirements analysed | ✗ | Analyses seven trustworthy-AI requirements, including robustness and safety, but does not derive robustness auditing meta-requirements. |
| ( Mökander et al., 2024 ) | 2023 | Develop a three-layered blueprint for auditing large language models. | Literature review; conceptual synthesis | ◐ | ✓ | 7 design claims; no meta-requirements | ✗ | Proposes coordinated governance, model, and application audits and seven design claims, but no robustness-specific meta-requirements. |
Table 2. Comparison of representative studies relevant to AI robustness auditing meta-requirements.
Figure 1. Overview of the four-phase research method used to derive, refine, validate, and synthesize the meta-requirements.
| Research Question | Motivation |
|---|---|
| RQ1: What meta-requirements should an AI robustness auditing framework satisfy? | This question identifies the requirements that should guide the design and conduct of AI robustness auditing. |
| RQ2: What are practitioners’ perspectives on the proposed meta-requirements for AI robustness auditing? | This question examines practitioners’ support for, qualifications of, challenges to, and suggested refinements or additions to the proposed meta-requirements. |
Table 3. Research questions and motivation.
| Digital Library | Search String | Retrieved Articles Numbers |
|---|---|---|
| ACM | [[Abstract: "robustness"] OR [Abstract: resilien*] OR [Abstract: reliab*] OR [Abstract: dependab*]] AND [[Abstract: "artificial intelligence"] OR [Abstract: "machine learning"] OR [Abstract: "algorithmic"] OR [Abstract: "ai"] OR [Abstract: "ml"]] AND [[Abstract: audit*] OR [Abstract: assess*] OR [Abstract: test*] OR [Abstract: evaluat*]] AND [[Abstract: practitioner*] OR [Abstract: developer*] OR [Abstract: engineer*] OR [Abstract: stakeholder*]] AND [[Abstract: interview*] OR [Abstract: survey*]] | 925 |
| Scopus | TITLE-ABS-KEY (("robustness" OR "resilien*" OR "reliab*" OR "dependab*") AND ("artificial intelligence" OR "machine learning" OR "algorithmic" OR "AI" OR "ML") AND ("audit*" OR "assess*" OR "test*" OR "evaluat*") AND ("practitioner*" OR "developer*" OR "engineer*" OR "stakeholder*") AND ("interview*" OR "survey*")) | 473 |
| IEEE Xplore | ("All Metadata":"robustness" OR "All Metadata":resilien* OR "All Metadata":reliab* OR "All Metadata":dependab*) AND ("All Metadata":"artificial intelligence" OR "All Metadata":"machine learning" OR "All Metadata":"algorithmic" OR "All Metadata":"AI" OR "All Metadata":"ML") AND ("All Metadata":audit* OR "All Metadata":assess* OR "All Metadata":test* OR "All Metadata":evaluat*) AND ("All Metadata":practitioner* OR "All Metadata":developer* OR "All Metadata":engineer* OR "All Metadata":stakeholder*) AND ("All Metadata":interview* OR "All Metadata":survey*) | 1292 |
| Total | - | 2690 |
Table 4. Search Strings and Results
| Inclusion Criteria | Exclusion Criteria |
|---|---|
| IC1: Studies must be written in English. | EC1: Studies not written in English. |
| IC2: Studies must be published between January 2019 and February 2026. | EC2: Studies published outside the selected time window. |
| IC3: Studies must be available in full text. | EC3: Studies whose full text is unavailable or inaccessible. |
| IC4: Studies must be academic research publications, including journal articles, conference papers, or workshop papers of at least six pages. | EC4: Duplicate records, papers shorter than six pages, books, editorials, abstracts, posters, proceedings summaries, presentation-only papers, or other publication types lacking sufficient research content. |
| IC5: Studies must examine AI, machine learning, deep learning, large language models, algorithmic systems, or AI-enabled software systems as the object of study. | EC5: Studies outside the AI/ML domain, or studies that use AI/ML only as an analytical tool for prediction, classification, diagnosis, forecasting, optimisation, sensing, or domain analysis without examining the AI/ML system itself. |
| IC6: Studies must address robustness or related dependability concerns of AI/ML systems, such as reliability, resilience, safety, fault tolerance, failure behaviour, distribution shift, out-of-distribution behaviour, adversarial robustness, or failure analysis. | EC6: Studies that use robustness, reliability, or resilience only in a non-AI sense, or only to describe general predictive performance, efficiency, usability, adoption, trust, sustainability, educational outcomes, physical-system performance, or organisational resilience. |
Table 5. Inclusion and exclusion criteria.
Figure 4. Example of how extracted literature evidence is coded and consolidated into MR-3.2, Robustness Trade-off Assumptions.
| Inclusion | I1: | The respondent must demonstrate relevant academic, professional, or practical experience in AI/ML systems, AI robustness assessment or auditing, robustness testing, model evaluation, adversarial evaluation, AI safety, model reliability, or related assurance activities. |
|---|---|---|
| Exclusion | E1: | A response is excluded if it contains inconsistent, apparently random, or meaningless answers. |
| E2: | A response without informed consent is excluded. |
Table 6. Inclusion and exclusion criteria for selecting valid responses.
| ID | Sub-Category | Requirement | Source(s) |
|---|---|---|---|
| Human Dimension | |||
| Category 1: Standards and Governance | |||
| MR-1.1 | Standardized Metrics | Robustness auditing should use standardised robustness metrics and benchmark definitions where available, so that reported robustness results are comparable across systems, audits, and regulatory contexts. | ( Nolte et al., 2025 ; Khinvasara et al., 2024 ; Oala et al., 2020 ; Blood et al., 2023 ; Yu et al., 2026 ; Tocchetti et al., 2025 ; Yang et al., 2020 ; Agarwal and Nene, 2025 ; Chen and Das, 2023 ; Madaio et al., 2022 ; Freiesleben and Grote, 2023 ; Casper et al., 2024 ; Raji et al., 2020 ; Thelisson and Verma, 2024 ; Nanda et al., 2021 ; Hornischer, 2026 ; Javed et al., 2024 ; Liu et al., 2022 ; Li et al., 2023 ; Wei and Liu, 2025 ; Costanza-Chock et al., 2022 ) |
| MR-1.2 | Standardized Lexicon and Terminology | Robustness audits should use a shared vocabulary, taxonomy, and terminology for robustness and auditability concepts. | ( Mohseni et al., 2022 ; Vadlapati, 2026 ; Yu et al., 2026 ; Ge et al., 2024 ; Tocchetti et al., 2025 ; Yang et al., 2020 ; Chen and Das, 2023 ; Freiesleben and Grote, 2023 ; Raji et al., 2020 ; Thelisson and Verma, 2024 ; Nanda et al., 2021 ; Nolte et al., 2025 ; Khinvasara et al., 2024 ; Hornischer, 2026 ; Liu et al., 2022 ; Li et al., 2023 ; Wei and Liu, 2025 ; Costanza-Chock et al., 2022 ) |
| MR-1.3 | Standardized Tools | Robustness audits should use common or comparable tools, checklists, templates, scorecards, libraries, or toolkits where they support consistent assessment. | ( Raji et al., 2020 ; Momcilovic et al., 2026 ; Ojewale et al., 2025 ; Oala et al., 2020 ; Tocchetti et al., 2025 ; Dev et al., 2025 ; Yu et al., 2026 ; Ge et al., 2024 ; Yang et al., 2020 ; Agarwal and Nene, 2025 ; Casper et al., 2024 ; Kokina et al., 2025 ; Thelisson and Verma, 2024 ; Nolte et al., 2025 ; Khinvasara et al., 2024 ; Javed et al., 2024 ; Li et al., 2023 ; Costanza-Chock et al., 2022 ) |
| MR-1.4 | Actionable Guidance | Practitioners should receive practical guidance for selecting tests, thresholds, evidence, mitigations, and audit steps. | ( Kokina et al., 2025 ; Khinvasara et al., 2024 ; Narula et al., 2025 ; Blood et al., 2023 ; Habbal et al., 2024 ; Thelisson and Verma, 2024 ; Cooper et al., 2022 ; Laux et al., 2024 ; Chen and Das, 2023 ; Saeed and Alsharidah, 2024 ; Xiong et al., 2022 ; McCormack et al., 2025 ; Koshiyama et al., 2024 ; Momcilovic et al., 2026 ; Chander et al., 2025 ) |
Table 7. Meta-requirements for AI robustness auditing: 43 literature-derived requirements and MR-8.4, added after the practitioner survey.
Figure 5. Final practitioner-informed meta-requirements for AI robustness auditing, comprising 44 meta-requirements across 16 categories and three dimensions.
| Rank | Category | Agree (%) | Rank | Category | Agree (%) |
|---|---|---|---|---|---|
| 1 | Workflow Integration | 86.20 | 9 | Communication | 81.64 |
| 2 | Configurability | 83.59 | 10 | Robustness Definition | 81.25 |
| 3 | Data Auditability | 83.12 | 11 | Human-in-the-Loop | 81.25 |
| 4 | Knowledge and Skills | 82.81 | 12 | Accessibility | 81.25 |
| 5 | Output Auditability | 82.81 | 13 | Efficiency | 79.69 |
| 6 | Monitoring | 82.81 | 14 | Standards and Governance | 79.49 |
Table 11. Category-level agreement ranking across the original 16 survey categories.
| Cat. | MR | Meta-requirement | Mean | SD | Median | Mode | IQR | Agree (%) | Neutral (%) | Disagree (%) | Quantitative status | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | MR-1.1 | Should use standardized robustness metrics | 128 | 4.23 | 1.12 | 5.0 | 5 | 1.0 | 82.81 | 6.25 | 10.94 | Strongly supported |
| 1 | MR-1.2 | Should use shared robustness terminology | 128 | 4.15 | 1.04 | 4.0 | 5 | 1.0 | 76.56 | 16.41 | 7.03 | Strongly supported |
| 1 | MR-1.3 | Should use standardized robustness tools | 128 | 3.90 | 0.92 | 4.0 | 4 | 0.0 | 78.12 | 12.50 | 9.38 | Strongly supported |
| 1 | MR-1.4 | Should provide actionable audit guidance | 128 | 4.16 | 1.00 | 4.0 | 5 | 1.0 | 75.78 | 17.19 | 7.03 | Strongly supported |
| 1 | MR-1.5 | Should integrate robustness into AI lifecycle workflows | 128 | 4.30 | 0.93 | 5.0 | 5 | 1.0 | 81.25 | 14.06 | 4.69 | Strongly supported |
| 1 | MR-1.6 | Should establish robustness governance protocols | 128 | 4.31 | 0.97 | 5.0 | 5 | 1.0 | 83.59 | 10.16 | 6.25 | Strongly supported |
Table 12. Practitioner validation results for the 43 literature-derived meta-requirements ( respondents per meta-requirement).
More in this topic
AI Assurance
Artificial intelligence assurance develops evidence and controls to evaluate, govern, and certify AI systems across their lifecycle and deployment contexts.
See all 79 papers