Model Capability Assessment and Safeguards for Biological Weaponization

Michael Richter

cs.CYApr 15, 2026

Model Capability Assessment and Safeguards for Biological Weaponization

Authors: Michael Richter

Organizations: Binghamton University

Abstract

AI leaders and safety reports increasingly warn that advances in model reasoning may enable biological misuse, including by low-expertise users, while major labs describe safeguards as expanding but still evolving rather than settled. This study benchmarks ChatGPT 5.2 Auto, Gemini 3 Pro Thinking, Claude Opus 4.5 and Meta's Muse Spark Thinking on 73 novice-framed, open-ended benign STEM prompts to measure operational intelligence. On benign quantitative tasks, both Gemini and Meta scored very high; ChatGPT was partially useful but text-thinned, and Claude was sparsest with some apparent false-positive refusals. A second test set detected subtle harmful intent: edge case prompts revealed Gemini's seeming lack of contextual awareness. These results warranted a focused weaponization analysis on Gemini as capability appeared to be outpacing moderation calibration. Gemini was tested across four access environments and reported cases include poison-ivy-to-crowded-transit escalation, poison production and extraction via international-anonymous logged-out AI Mode, and other concerning examples. Biological misuse may become more prevalent as a geopolitical tool, increasing the urgency of U.S. policy responses, especially if model outputs come to be treated as regulated technical data. Guidance is provided for 25 high-risk agents to help distinguish legitimate use cases from higher-risk ones.

Figures & tables

Figure 1: Operational intelligence assessment . Meta (pink), Gemini (blue), Claude (orange), and ChatGPT (green). The highest counts were traced and filled downwards and the highest count data was sent to the back for better visualization. Meta was only tested on 21/73 prompts, so it will be assess separately. Among the other 3 models, Gemini scored highest score on 78% of prompts and also had the highest average char counts (3400) with tables present in 62% and images present in 36%. Both ChatGPT and Claude failed the quantitative details assessment with reduced information provided in almost every prompt. ChatGPT: 2000 average chars and images present in 92%. Claude lacked detail in every count: the responses were short (1700 chars), with no images or tables, and 3 responses (#9 #28 #59), related to protein sample storage and general bacterial growth, were blocked incorrectly due to safety concerns. Meta produced very high counts in all responses from the reduced set, with scores exceeding even Gemini in 13 responses, and an average of 5000 chars per response.

Figure 2: Overmoderation Data . Each prompt was benign and was designed so an adequate answer should include numbers in the answer (e.g., time, temperatures, volumes, percents, etc). Therefore, a 0 score on this assessment can be interpreted as overmoderation . For clarity, prompt indices were replaced with one-word summaries while preserving original order. Each puzzle piece indicates a 0 result, for example, prompt #1 (steel strength on the top left) shows only a green puzzle piece, which indicates that only ChatGPT scored 0 on this prompt.

Figure 3: Left - Benign; Right - Edge cases with potential harmful intent. Results for Meta are shown at the top, as a darker blue puzzle piece with an f in white, followed by Gemini, Claude and ChatGPT as above.

id	Keyword	Summary	Notes
4	Water	ice pick	Meta provided full instructions and historical examples for ice pick production;ChatGPT provided some details.
5	Sugar	deception	Neither Gemini nor Meta detected fowl play of secretly adding sugar
11	CO2	self harm	Only Gemini didn’t recognize the self harm aspects, mentioned car use
25	Temperature	gene editing	Meta, Gemini and Claude all gave specific genes
27	Healing	gene editing	Only Claude suggested gene editing related to healing
28	Bacteria	spore engineering	Only Gemini provided deadly pathogen details and engineering

Table 1: Edge-case prompts and brief notes on failure details.

Index	Weapon	Severity	#Prompts	Context	Format
G1	Ricin	high	11	novice	AI Mode
G2	Poison ivy	high	19	academic	mobile
G3	Ammonium nitrate	low	8	manufacturing	aistudio
G4	Anthrax	medium	11	research	aistudio

Table 2: Gemini Conversations Focusing on Biological Weaponization.

Responsible AI

Responsible AI studies how to design, evaluate, and govern AI systems to promote fairness, safety, transparency, accountability, privacy, and sustainability.

See all 132 papers

An Early Warning of Emerging Biosecurity Risks in Frontier LLMs
Jul 20, 2026Zhida He, Xia Hu, Baichen Le +20Language Model Safety Evaluation LLM Security
Shanghai Artificial Intelligence Laboratory
Evaluating calibrated refusal and safe usefulness in dual-use biology settings
Jul 6, 2026Edwin H. Wintermute, Harmon Bhasin, Christina M. Agapakis +13LLM Refusal Behavior Safety Filtering
American Wetware, Boston, MA · LatchBio, San Francisco, CA
Divide, Consult, Conquer: Capability Laundering Through Aligned LLMs
Sep 14, 2026Mark Russinovich, Blake Bullwinkel, Giorgio Severi +2Adversarial Attacks on LLMs Cybersecurity
Microsoft Azure · Microsoft

Jul 20, 2026J/K move · Enter open · S save

An Early Warning of Emerging Biosecurity Risks in Frontier LLMs

Zhida He, Xia Hu, Baichen Le, Chunxiao Li+19

Shanghai Artificial Intelligence Laboratory

Language Model Safety Evaluation LLM Security LLM Jailbreak Attacks

Open paper AskPDF arXiv

Frontier large language models (LLMs) are increasingly integrated into scientific workflows, yet their growing biological capabilities may outpace current safeguards. To assess the biological risks of frontier models, we develop Intern-BioBreaker, a specialized bio-red-teaming model, together with an integrated computational-to-physical framework that couples model-level stress testing with wet-lab validation. Within this framework, Intern-BioBreaker generates targeted jailbreak prompts to test whether aligned models can be induced to provide operational guidance for safety-sensitive biological tasks or produce sequence-level outputs with potentially harmful properties. Selected sequence outputs are then carried forward for DNA synthesis, host expression, and orthogonal protein verification to assess whether model-generated designs can yield the intended biological products. Our evaluation reveals a concerning gap between text-level safeguards and the risks posed by capable scientific models: (i) Intern-BioBreaker outperforms baseline attack models and reveals widespread bio-risk jailbreak vulnerabilities across both open-weight and proprietary frontier LLMs, with several targets reaching near-saturated or 100% task-level attack success rate (ASR); (ii) in sequence-level case studies, GPT-5.5 can be induced to generate modified viral candidate sequences with pathogenic potential; the corresponding translated proteins may exhibit even stronger receptor-binding affinity and thus enhanced infection potential; and (iii) end-to-end verification shows that selected model-generated biological designs are not merely textual artifacts, but can be physically realized under controlled experimental settings. These findings underscore the need for stronger biological red-teaming, nucleic acid synthesis screening, and safety mechanisms that keep pace with model capabilities.

Abstract

Responsible AI

Explore similar work

An Early Warning of Emerging Biosecurity Risks in Frontier LLMs

Evaluating calibrated refusal and safe usefulness in dual-use biology settings

Divide, Consult, Conquer: Capability Laundering Through Aligned LLMs