Evaluating Human-AI Safety: A Framework for Measuring Harmful Capability Uplift

Michelle Vaccaro; Jaeyoon Song; Abdullah Almaatouq; Michiel A. Bakker

cs.CYMar 6, 2026

Evaluating Human-AI Safety: A Framework for Measuring Harmful Capability Uplift

Authors: Michelle Vaccaro, Jaeyoon Song, Abdullah Almaatouq, Michiel A. Bakker

Organizations: Massachusetts Institute of Technology, Cambridge, USA

Abstract

Current frontier AI safety evaluations emphasize static benchmarks, third-party annotations, and red-teaming. In this position paper, we argue that AI safety research should focus on human-centered evaluations that measure harmful capability uplift: the marginal increase in a user's ability to cause harm with a frontier model beyond what conventional tools already enable. We frame harmful capability uplift as a core AI safety metric, ground it in prior social science research, and provide concrete methodological guidance for systematic measurement. We conclude with actionable steps for developers, researchers, funders, and regulators to make harmful capability uplift evaluation a standard practice.

Figures & tables

Figure 1: Approaches to AI Safety Evaluation. (Left) Current evaluations focus on isolated AI model outputs using static benchmarks, with human judges occasionally assessing the output from external observation points. (Right) Our proposed approach evaluates the human-AI system, measuring what malicious tasks a human-AI combination can accomplish using the harmful capability uplift metric.

Figure 2: The Harmful Capability Uplift Framework. (Left) The harmful capability uplift metric $U$ quantifies how much frontier models amplify the ability of people to perform malicious tasks. (Middle) Novel capability acquisition occurs when AI assistance enables previously impossible tasks. (Right) Hypothetical biosecurity analysis demonstrates how harmful capability uplift can vary across task dimensions.

Figure 3: The Proxy Task Challenge. (Left) Direct assessment of harmful capabilities raises ethical concerns, necessitating safe proxy tasks. (Middle) A formal task similarity framework quantifies the predictive relationship between proxy and target tasks. (Right) Empirical validation studies establish when proxy task performance reliably predicts capabilities on target tasks of concern, enabling evidence-based safety assessment without conducting harmful experiments.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Study	Task(s)	Design	Sample Size	Evaluation Criteria	Results	Data
[ 4 ]	Draft bioweapons acquisition plans	Between-subjects	Not provided	Task score	$U=2.1$ (significant)	No
[ 6 ]	Draft bioweapons acquisition plans	Between-subjects	Not provided	Task score	$U=2.5$ (Opus 4) (significant) $U=1.7$ (Sonnet 4) (not significant) $U=1.5$ (Sonnet 3.7) (not significant)	No
[ 47 ]	Plan bioweapons attacks	Between-subjects	15 teams (4-6 per condition)	Viability score	$U=0.94$ (not significant)	No
[ 53 ]	Research tasks for biological-threat creation	Between-subjects	100 participants (25 per condition)	Accuracy Completeness Innovation Time Difficulty	$U=1.15$ (not significant)	Yes
[ 7 ]	Reconstruct a virus	Not provided	Not provided	Task score	$U=1.97$ (Opus 4.5) $U=1.82$ (Opus 4) $U=1.32$ (Sonnet 3.7)	No
[ 5 ]	Answer CBRN risk-relevant questions	Between-subjects	30 participants (10 per condition)	Accuracy	No data (not significant)	No

Appendix

Table 1: Public evaluations of large language models on harmful tasks.

AI Safety Evaluation

AI Safety Evaluation studies how to test whether AI systems avoid harmful behavior and remain reliable across users, tasks, and deployment settings.

See all 141 papers

Harmonizing AI Safety Thresholds
Jul 17, 2026Wilber Sean Anterola, Matthew Ball, Luis F. Lafuerza +1AI Risk Management AI Governance
1Brown University · Centre pour la Sécurité de l’Intelligence Artificielle (CeSIA)
Safeguard-Conditioned Uplift: Measuring Utility-Risk Frontiers for Dual-Use Biology Assistants
Jun 12, 2026Dipesh Tharu MahatoAI Safety Evaluation LLM Safety Evaluation
New York University
A Translational Note on AI Safety Evaluation
Sep 6, 2026Madhava GaikwadLanguage Model Safety Evaluation AI Safety Evaluation
Independent Researcher, Bengaluru, India

Jul 17, 2026J/K move · Enter open · S save

Harmonizing AI Safety Thresholds

Wilber Sean Anterola, Matthew Ball, Luis F. Lafuerza, Markov Grey

1Brown University · Centre pour la Sécurité de l’Intelligence Artificielle (CeSIA)

AI Risk Management AI Governance AI Safety

Open paper AskPDF arXiv

Frontier AI companies have published capability thresholds that differ substantially, making it difficult for third parties to verify whether a threshold has been crossed or to compare requirements across companies. Moreover, without common minimum thresholds, risk mitigation may be inconsistent, creating a potential race to the bottom in safety standards. We develop a methodology for deriving harmonized thresholds across three risk domains. For misuse risks (cyber and biological), we take expected harm as the key primitive and use an explicit risk-modeling approach that accounts for risk channels and model release conditions. For automated AI R&D, we base our proposed threshold on the observed rate of AI progress rather than expected harm. Our analysis expands upon prior work and highlights existing empirical gaps and limitations.

Abstract

AI Safety Evaluation

Explore similar work

Harmonizing AI Safety Thresholds

Safeguard-Conditioned Uplift: Measuring Utility-Risk Frontiers for Dual-Use Biology Assistants

A Translational Note on AI Safety Evaluation