Scholarly work which aims to describe potential societal impacts (e.g., risks) of proliferating technology (especially related to artificial intelligence or other algorithmic systems) is likely to have an impact beyond the scientific communities it was written for, given that general society itself is a primary object of study. However, it is an open question whether the current practices of AI evaluation scholarship follow the principles and best practices established by risk science, which aims to systematically generate knowledge related to understanding, assessing, communicating, managing, and governing risk. In this work, we examine this in depth by conducting a literature review of scholarly works purporting to evaluate the bias or fairness of technological systems used for tasks related to hiring and employment. Through analysis of 22 common fairness evaluation metrics and studies using them, we find that most characterize the severity of bias- or fairness-related consequences but do not follow best practices to characterize the uncertainty around either the occurrence of these consequences or severity estimates. Next, we conduct a case study of fairness evaluation for an AI-mediated resume screening task and demonstrate how principles of risk science can be incorporated into such an evaluation. Finally, we propose the AI Risk Report Card, which facilitates the reporting and communication of risk assessment results to stakeholders in positions to act based on the predicted risks. The outcomes of these activities suggest that further research at the convergence of risk science and AI evaluation can lead to advancements in AI assessments of societal impact by enabling shared frameworks to evaluate and discuss AI risks both within and outside of the scientific community.
Figures & tables
Figure 1: Illustration of how both the expected severity of a consequence and associated uncertainties play a role in risk assessment. In the two leftmost panels, the expected severity μ (0.85) is identical, but the probability that an event is harmful (any severity less than 0.8) is over 60% greater when the variability σ increases. If the expected severity falls within the range of harmful consequences (as in the two rightmost panels where severity is 0.75), increasing the variability has the opposite effect and decreases the probability of a severe outcome. Thus, considering both the severity and uncertainty or variability of possible consequence(s) is essential to understanding and effectively controlling or mitigating it.
Table 1: Summary of whether the AI hiring fairness metrics listed in Fabris et al. (2025) describe severity or uncertainty of harms. If a metric is not a valid probability (all but one), the constraints it violates are given in parentheses (corresponding to criteria in Equations I and II ). The colors of rows correspond to the different kinds of fairness identified by Fabris et al. (2025) : Outcome , Accuracy , Impact , Representational , and Process .
Figure 2: Among the 28 evaluations in our analysis set, only seven provided variability estimates for their metric results. Of those, four provided variability estimates for all results: one used a confidence interval, and three used a combination of standard deviations or errors and visualizations of distributions. Two papers provided confidence intervals for some but not all results, and one provided variability estimates for some results but did not specify the type.
Figure 3: Examples of risk characterizations given in the AI resume screening evaluation conducted by Wilson and Caliskan (2024) .
Figure 4: [A] The distribution of observed Disparate Impact (DI) scores when evaluating white men’s vs. Black men’s resumes using GritLM. The red line marks the sample average; the two dashed black lines represent different possible DI thresholds at 0.666 and 0.800. [B] The number of resume screening scenarios that are classified as discriminatory vs. non-discriminatory when the DI threshold is 0.800 (based on the 80% rule). [C] The number of resume screening scenarios which are classified as discriminatory vs. not when the DI threshold is 0.666 (based on statistical significance).
Figure 5: Example of a proposed AI Risk Report Card to facilitate communication between researchers conducting AI evaluations of societal impact and stakeholders beyond the scientific community.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Uncertainty
Strength of Knowledge
Criteria
Conditions
The phenomena involved are well understood; the models used are known to give predictions with the required accuracy
Much reliable data are available
The assumptions made are seen as very reasonable
Low
Strong
All conditions are fulfilled
There is broad agreement among experts
Example: The phenomena involved are well understood, but the models used are considered simple
Medium
Moderate
Conditions between low and high are fulfilled to varying degrees
Example: Some reliable data are available
Appendix
Table 7
Rank
Criteria
Very high risk
Potential for extreme consequences, relatively large associated probability of such consequences and/or significant uncertainty (relatively weak background knowledge)
High risk
The potential for extreme consequences, relatively small associated probability of such consequences and moderate or weak background knowledge
Moderate risk
Between low and high risk. For example, the potential for moderate consequences, and weak background knowledge.
( Ali et al. 2019 ) ( Imana, Korolova, and Heidemann 2021 ) ( Pena et al. 2020 )
True-Positive Rate Difference (TPRD)
TPRg=Pr(y^=1∣y=1,s=g)\tprd=TPRg−TPRgc
( De-Arteaga et al. 2019 ) ( Hemamou and Coleman 2022 )
Appendix
Table 2: Summary of the AI hiring fairness metrics listed in Fabris et al. (2025) and the studies which used them. The colors of rows correspond to the different kinds of fairness identified by Fabris et al. (2025) : Outcome , Accuracy , Impact , Representational , and Process .
Figure 6: [A] The distribution of observed Disparate Impact (DI) scores when evaluating white men’s vs. Black men’s resumes using e5. The red line marks the sample average; the two dashed black lines represent different possible DI thresholds at 0.666 and 0.800. [B] The number of resume screening scenarios that are classified as discriminatory vs. non-discriminatory when the DI threshold is 0.800 (based on the 80% rule). [C] The number of resume screening scenarios which are classified as discriminatory vs. not when the DI threshold is 0.666 (based on statistical significance).
Figure 7: [A] The distribution of observed Disparate Impact (DI) scores when evaluating white men’s vs. Black men’s resumes using SFR. The red line marks the sample average; the two dashed black lines represent different possible DI thresholds at 0.666 and 0.800. [B] The number of resume screening scenarios that are classified as discriminatory vs. non-discriminatory when the DI threshold is 0.800 (based on the 80% rule). [C] The number of resume screening scenarios which are classified as discriminatory vs. not when the DI threshold is 0.666 (based on statistical significance).
The society and emerging risk-based regulatory frameworks for AI underscore the need for rigorous risk assessment to ensure safe and reliable AI systems. In response to this imperative, this paper presents an overview of AI risk assessment (identification and analysis) and management methodologies. It begins by reviewing the worldwide regulatory landscape that drives the need for systematic AI risk assessment. Then we characterize the spectrum of AI-related risks identified in the literature, from technical failures to ethical and social impacts. Subsequently, it reviews key risk assessment methodologies proposed for AI systems, focusing on general frameworks. The paper highlights best practices and illuminates methodological gaps, highlighting areas for further research on AI risk assessment.
Javier Irigoyen, Roberto Daza, Aythami Morales +5
BiometricsAI, Universidad Aut´onoma de Madrid (UAM), Spain · GHIA, Universidad Aut´onoma de Madrid (UAM), Spain · Universidad de Las Palmas de Gran Canaria (ULPGC), Spain
This paper addresses a rapidly emerging policy challenge: how to generate and interpret credible evidence about the biological capabilities and risks of AI scientists, or agentic AI systems capable of autonomously or collaboratively performing multi-step scientific tasks. As these systems enter real research workflows, decision-makers increasingly face evaluation results whose meaning depends on underlying design choices that are often implicit or under-documented. We synthesize current evidence on AI-enabled biological risks and introduce biological agentic evaluations as a promising, but interpretation-sensitive, tool for assessing these systems. Our central contribution is a set of practical, experience-grounded considerations -- drawing from our own evaluations -- that show how choices around defining, designing, running, scoring, and documenting evaluations materially shape what results do and do not imply about risk. The analysis is intended to help policymakers interpret biological evaluation outputs with appropriate caution; guide public and private funders toward high-leverage investments in AI-biology evaluation research; and support biosecurity practitioners assessing emerging AI systems. A secondary audience includes researchers designing or conducting agentic evaluations within frontier AI labs, AI providers, scientific institutions, and third-party evaluation organizations.
We present PopResume, a population-representative resume dataset for causal fairness auditing of LLM- and VLM-based resume screening systems. Unlike existing benchmarks that rely on manually injected demographic information and outcome-level disparities, PopResume is grounded in population statistics and preserves natural attribute relationships, enabling path-specific effect (PSE)-based fairness evaluation. We decompose the effect of a protected attribute on resume scores into two paths: the business necessity path, mediated by job-relevant qualifications, and the redlining path, mediated by demographic proxies. This distinction allows auditors to separate legally permissible from impermissible sources of disparity. Evaluating four LLMs and four VLMs on PopResume's 60.8K resumes across five occupations, we identify five representative discrimination patterns that aggregate metrics fail to capture. Our results demonstrate that PSE-based evaluation reveals fairness issues masked by outcome-level measures, underscoring the need for causally-grounded auditing frameworks in AI-assisted hiring.
Sumin Yu, Juhyeon Park, Taesup Moon
ECE, Seoul National University · IPAI, Seoul National University · ASRI / INMC / AIIS, Seoul National University