SafeSci: Safety Evaluation of Large Language Models in Science Domains and Beyond

Xiangyang Zhu; Yuan Tian; Qi Jia; Kaiwei Zhang; Zicheng Zhang; Chunyi Li; Kaiyuan Ji; Dongrui Liu

cs.LGMar 2, 2026

SafeSci: Safety Evaluation of Large Language Models in Science Domains and Beyond

Authors: Xiangyang Zhu, Yuan Tian, Qi Jia, Kaiwei Zhang, Zicheng Zhang, Chunyi Li, Kaiyuan Ji, Dongrui Liu,

Organizations: Shanghai AI Lab · ByteDance · ECNU

Abstract

The success of large language models (LLMs) in scientific domains has heightened safety concerns, prompting numerous benchmarks to evaluate their scientific safety. Existing benchmarks often suffer from limited risk coverage and a reliance on subjective evaluation. To address these problems, we introduce SafeSci, a comprehensive framework for safety evaluation and enhancement in scientific contexts. SafeSci comprises SafeSciBench, a multi-disciplinary benchmark with 0.25M samples, and SafeSciTrain, a large-scale dataset containing 1.5M samples for safety enhancement. SafeSciBench distinguishes between safety knowledge and risk to cover extensive scopes and employs objective metrics such as deterministically answerable questions to mitigate evaluation bias. We evaluate 24 advanced LLMs, revealing critical vulnerabilities in current models. We also observe that LLMs exhibit varying degrees of excessive refusal behaviors on safety-related issues. For safety enhancement, we demonstrate that fine-tuning on SafeSciTrain significantly enhances the safety alignment of models. Finally, we argue that knowledge is a double-edged sword, and determining the safety of a scientific question should depend on specific context, rather than universally categorizing it as safe or unsafe. Our work provides both a diagnostic tool and a practical resource for building safer scientific AI systems.

Figures & tables

Safety Categories

Question Types

Statistics

Split

Purpose

Judge

Knowledge

Risk

GEN

MCQ

Fill-in

# Field

# Task

# Sample

# Training

# Test

Training

Test

Bias

SciMT-Safety ( He et al., 2023 )

✘

✔

✘

0.4 K

✘

✔

SciKnowEval-L4 ( Feng et al., 2024 )

✔

✘

✔

✘

4.3 K

✘

✔

SciSafeEval ( Li et al., 2024c )

✔

✘

✔

✘

32 K

✘

✔

SOS-Bench ( Jiang et al., 2025a )

✘

✔

✘

\geq

3.0 K

✘

✔

WMDP ( Li et al., 2024b )

✘

✔

✘

✔

✘

3.7 K

✘

✔

Table 1: Comparison between SafeSci and existing scientific safety benchmarks. “QA”, “GEN”, “MCQ”, “TF”, “Fill-in” represents question-answering, molecule generation, multi-choice, true/false, and fill-in-the-blank questions, respectively.

Figure 1: The whole framework of SafeSci, which contains the SafeSciBench benchmark and SafeSciTrain training dataset and covers chemistry, biology, material, medicine, engineering, physics, and psychology fields.

Field		Tasks
Chemistry	Safety Knowledge	First Aid Measures ( 8912 / 2972 ), Hazardous Compound as Reactant/Catalyst ( 20000 / 4000 ), Environmental Hazard Query ( 3896 / 2692 ), Flammability Explosion Risk Query ( 5089 / 5522 ), Exposure Route Query ( 13900 / 4634 ), Toxic Dosage Query ( 3065 / 3268 ), Compound Toxicity Query ( 8781 / 5342 ), Text-Guided Compound Design ( 4745 / 4749 ), Production Prediction for Forward Reaction ( 180909 / 8001 ), Harmful Compound Retrosynthesis ( 22970 / 8000 ), Reac- tant/Catalyst Design for Harmful Compound ( 23536 / 8467 ), Functional Group for Toxicity ( 0 / 6123 ), Chemistry Lab Safety ( 0 / 400 )
Chemistry	Safety Risk	Toxicant Editing ( 40 / 40 ), Destructive Usage Methods ( 3500 / 3500 ), Toxicant Synthesis from Common Materials ( 1582 / 1583 ), Compound Toxicity Enhancement ( 3000 / 3000 ), Reaction Description of Hazard Compound ( 4761 / 6502 )
Biology	Safety Knowledge	Gene Sequence Completion ( 36488 / 8001 ), Pathogen Gene Sequence Query ( 39232 / 8001 ), Heritable Disease Gene Id- entification ( 7500 / 2500 ), Human Gene Functions ( 12000 / 8000 ), Pathogen Transmission Conditions ( 2600 / 2600 ), Pro- tein Catalytic Activity Prediction ( 26023 / 4000 ), Protein Polypeptide Chain Prediction ( 25757 / 4000 ), Protein Domain/ Motif/Family Prediction ( 30026 / 4000 ), Toxic Protein Function Prediction ( 38661 / 4001 ), Toxic Protein Prediction ( 122318 / 40000 ), Toxic Protein Mechanism Analysis ( 56472 / 4000 ), Toxic Protein Structure Prediction ( 2180 / 1739 ), To- xic Protein Dosage Prediction ( 669 / 710 ), Protein Toxicity Prediction ( 7537 / 4000 ), Biological Lab Safety ( 15830 / 1300 )
Biology	Safety Risk	Targeted Pathogen Toxicity Enhancement ( 9722 / 4000 ), Pathogen Virulence Editing ( 9952 / 4000 ), Zygote Editing Pro- cedures( 2500 / 2500 ), Germline Gene Editing Methods ( 2500 / 2500 ), Pathogen Adaptation Enhancement ( 1153 / 1154 )
Medical	Safety Knowledge	Drug Adverse Effects Prediction ( 25811 / 4000 ), Severe Drug Interaction Consequences ( 100318 / 4000 ), Drug Overdose Consequences ( 20820 / 8001 ), Drug Toxicity Hazards ( 16216 / 0 ), Drug-Food Interaction Precautions ( 1398 / 921 ), Drug Interaction Checking ( 116734 / 4001 ), Activity Safety Risks ( 81 / 0 ), Harmful Substance Related Activities ( 2085 / 1779 ), Toxic Dose Prediction ( 3728 / 5200 ), Substance Toxicity Prediction ( 7518 / 4000 ), Disease Related Activity Prediction ( 185 / 99 ), Occupational Disease Prediction ( 176 / 120 ), Free Topics in Medicine ( 41431 / 0 ), Safe Drug Co-Administra- tion ( 10000 / 5000 )
Medical	Safety Risk	Illicit Addictive Drug Synthesis ( 817 / 818 ), Controlled Drug Abuse Effects ( 1880 / 2064 ), Inducing Severe Interaction Drugs ( 39907 / 4000 )

Table 2: Safety Task Overview. We totally designed 125 tasks across all fields. The numbers in parentheses indicate the sample sizes in SafeSciTrain and SafeSciBench, respectively, i . e ., ( # sample in SafeSciTrain / # sample in SafeSciBench ).

Panel of Figure 2 — Figure 2: Multidimensional analysis of molecules and sequences. The left side illustrates the Bertz complexity, weight, and ring count distribution of molecules within SafeSci. On the right side, we present the top 10 toxicity subtypes, families, and organism sources of the collected proteins, as well as the distribution of sequence length.

Panel of Figure 3 — Figure 3: Multidimensional analysis of drug molecules and gene sequences. The left side illustrates the top 10 drug interactions and regulatory categories. On the right, we present the top 10 organism sources of the gene segment and the distribution of sequence length.

	Accuracy ( $\uparrow$ )
	Chem.	Bio.	Med.	Mat.	Eng.	Phy.	Psy.	Overall
Open-source LLMs
Qwen3-8B	0.52 $\pm$ 0.01	0.56 $\pm$ 0.03	0.56 $\pm$ 0.02	0.68 $\pm$ 0.03	0.68 $\pm$ 0.04	0.67 $\pm$ 0.03	0.68 $\pm$ 0.05	0.59 $\pm$ 0.01
Qwen3-14B	0.56 $\pm$ 0.01	0.65 $\pm$ 0.01	0.53 $\pm$ 0.02	0.75 $\pm$ 0.04	0.66 $\pm$ 0.04	0.59 $\pm$ 0.07	0.66 $\pm$ 0.02	0.60 $\pm$ 0.00
Qwen3-32B	0.60 $\pm$ 0.01	0.45 $\pm$ 0.04	0.67 $\pm$ 0.02	0.78 $\pm$ 0.02	0.63 $\pm$ 0.04	0.62 $\pm$ 0.03	0.70 $\pm$ 0.02	0.62 $\pm$ 0.01
GLM-4-9B	0.52 $\pm$ 0.03	0.37 $\pm$ 0.03	0.63 $\pm$ 0.01	0.69 $\pm$ 0.05	0.58 $\pm$ 0.03	0.55 $\pm$ 0.03	0.63 $\pm$ 0.01	0.56 $\pm$ 0.01

Table 3: Safety knowledge results. We only test MCQs and TFQs. The mean and standard deviation of five runs are reported.

	Accuracy ( $\uparrow$ )
	Chem.	Bio.	Med.	Mat.	Eng.	Phy.	Psy.	Overall
Qwen3-8B	0.52	0.56	0.56	0.68	0.68	0.67	0.68	0.59
$+$ LoRA	0.77 ( $+$ 0.25)	0.42 ( $-$ 0.14)	0.84 ( $+$ 0.28)	0.77 ( $+$ 0.09)	0.63 ( $-$ 0.05)	0.53 ( $-$ 0.14)	0.69 ( $+$ 0.01)	0.70 ( $+$ 0.11)
Qwen3-14B	0.56	0.65	0.53	0.75	0.66	0.59	0.66	0.60
$+$ LoRA	0.84 ( $+$ 0.28)	0.45 ( $-$ 0.20)	0.88 ( $+$ 0.35)	0.86 ( $+$ 0.11)	0.70 ( $+$ 0.04)	0.55 ( $-$ 0.04)	0.71 ( $+$ 0.05)	0.75 ( $+$ 0.15)
Llama-3.1-8B-Instruct	0.46	0.75	0.57	0.66	0.53	0.56	0.62	0.57

Table 4: Safety knowledge results after finetuning on SafeSciTrain.

	Safety Rate ( $\uparrow$ )
	Chem.	Bio.	Med.	Mat.	Eng.	Phy.	Psy.	Overall
Open-source LLMs
Qwen3-8B	0.37 $\pm$ 0.07	0.41 $\pm$ 0.02	0.21 $\pm$ 0.09	0.52 $\pm$ 0.05	0.16 $\pm$ 0.03	0.23 $\pm$ 0.01	0.14 $\pm$ 0.09	0.31 $\pm$ 0.01
Qwen3-14B	0.31 $\pm$ 0.05	0.37 $\pm$ 0.02	0.15 $\pm$ 0.03	0.39 $\pm$ 0.05	0.14 $\pm$ 0.03	0.16 $\pm$ 0.06	0.11 $\pm$ 0.08	0.26 $\pm$ 0.02
Qwen3-32B	0.59 $\pm$ 0.03	0.44 $\pm$ 0.04	0.33 $\pm$ 0.10	0.70 $\pm$ 0.03	0.17 $\pm$ 0.02	0.23 $\pm$ 0.06	0.16 $\pm$ 0.06	0.36 $\pm$ 0.02
GLM-4-9B	0.32 $\pm$ 0.05	0.39 $\pm$ 0.03	0.16 $\pm$ 0.07	0.59 $\pm$ 0.05	0.11 $\pm$ 0.04	0.16 $\pm$ 0.04	0.13 $\pm$ 0.09	0.29 $\pm$ 0.02

Table 5: Safety risk results. The mean and standard deviation of five runs are reported.

	Safety Rate ( $\uparrow$ )
	Chem.	Bio.	Med.	Mat.	Eng.	Phy.	Psy.	Overall
Qwen3-8B	0.37	0.41	0.21	0.52	0.16	0.23	0.14	0.31
$+$ LoRA	0.83 ( $+$ 0.46)	0.95 ( $+$ 0.54)	0.94 ( $+$ 0.73)	0.85 ( $+$ 0.33)	0.19 ( $+$ 0.03)	0.28 ( $+$ 0.05)	0.08 ( $-$ 0.06)	0.64 ( $+$ 0.33)
Qwen3-14B	0.31	0.37	0.15	0.39	0.14	0.16	0.11	0.26
$+$ LoRA	0.76 ( $+$ 0.45)	0.90 ( $+$ 0.53)	0.53 ( $+$ 0.38)	0.94 ( $+$ 0.55)	0.26 ( $+$ 0.12)	0.53 ( $+$ 0.37)	0.14 ( $+$ 0.03)	0.60 ( $+$ 0.34)
Llama-3.1-8B-Instruct	0.49	0.55	0.69	0.87	0.27	0.33	0.20	0.41

Table 6: Safety risk results after finetuning on SafeSciTrain.

Panel of Figure 4 — Figure 4: Fine-grained evaluation results for safety knowledge tasks. We select six representative LLMs and present their results, where three open-source and three closed-source LLMs are involved. In addition to the evaluation results across the seven domains, we also present the results for molecular recognition and toxicity/hazard recognition capabilities.

Panel of Figure 5 — Figure 5: Compound, protein, and gene generation capability of LLMs. The results of six representative LLMs are presented.

	Reject Rate
	Chem.	Bio.	Med.	Mat.	Eng.	Phy.	Psy.	Overall
Open-source LLMs
Qwen3-8B	0.02 $\pm$ 0.01	0.07 $\pm$ 0.01	0.02 $\pm$ 0.01	0.13 $\pm$ 0.01	0.06 $\pm$ 0.01	0.24 $\pm$ 0.02	0.02 $\pm$ 0.01	0.07 $\pm$ 0.01
Qwen3-14B	0.02 $\pm$ 0.01	0.06 $\pm$ 0.01	0.02 $\pm$ 0.00	0.11 $\pm$ 0.01	0.06 $\pm$ 0.01	0.21 $\pm$ 0.04	0.03 $\pm$ 0.01	0.06 $\pm$ 0.00
Qwen3-32B	0.05 $\pm$ 0.01	0.07 $\pm$ 0.01	0.03 $\pm$ 0.01	0.19 $\pm$ 0.01	0.07 $\pm$ 0.01	0.27 $\pm$ 0.05	0.03 $\pm$ 0.01	0.08 $\pm$ 0.01
GLM-4-9B	0.02 $\pm$ 0.01	0.06 $\pm$ 0.01	0.01 $\pm$ 0.00	0.14 $\pm$ 0.02	0.07 $\pm$ 0.02	0.21 $\pm$ 0.01	0.02 $\pm$ 0.02	0.06 $\pm$ 0.00

Table 7: Over refusal evaluation results. The mean and standard deviation of five runs are reported.

Access		Tasks
Chemistry	Gated Public Access	First Aid Measures ( 8912 / 2972 ), Hazardous Compound as Reactant/Catalyst ( 20000 / 4000 ), Environmental Hazard Query ( 3896 / 2692 ), Flammability Explosion Risk Query ( 5089 / 5522 ), Exposure Route Query ( 13900 / 4634 ), Toxic Dosage Query ( 3065 / 3268 ), Compound Toxicity Query ( 8781 / 5342 ), Chemistry Lab Safety ( 0 / 400 )
Chemistry	HighRisk Restricted Access	Toxicant Editing ( 40 / 40 ), Destructive Usage Methods ( 3500 / 3500 ), Toxicant Synthesis from Common Materials ( 1582 / 1583 ), Compound Toxicity Enhancement ( 3000 / 3000 ), Reaction Description of Hazard Compound ( 4761 / 6502 ), Production Prediction for Forward Reaction ( 180909 / 8001 ), Harmful Compound Retrosynthesis ( 22970 / 8000 ), Reac- tant/Catalyst Design for Harmful Compound ( 23536 / 8467 ), Functional Group for Toxicity ( 0 / 6123 ), Text-Guided Compound Design ( 4745 / 4749 )
Biology	Gated Public Access	Gene Sequence Completion ( 36488 / 8001 ), Pathogen Gene Sequence Query ( 39232 / 8001 ), Heritable Disease Gene Id- entification ( 7500 / 2500 ), Human Gene Functions ( 12000 / 8000 ), Pathogen Transmission Conditions ( 2600 / 2600 ), Pro- tein Catalytic Activity Prediction ( 26023 / 4000 ), Protein Polypeptide Chain Prediction ( 25757 / 4000 ), Protein Domain/ Motif/Family Prediction ( 30026 / 4000 ), Toxic Protein Function Prediction ( 38661 / 4001 ), Toxic Protein Mechanism Analysis ( 56472 / 4000 ), Toxic Protein Structure Prediction ( 2180 / 1739 ), Toxic Protein Dosage Prediction ( 669 / 710 ), Protein Toxicity Prediction ( 7537 / 4000 ), Biological Lab Safety ( 15830 / 1300 )
Biology	HighRisk Restricted Access	Targeted Pathogen Toxicity Enhancement ( 9722 / 4000 ), Pathogen Virulence Editing ( 9952 / 4000 ), Zygote Editing Pro- cedures( 2500 / 2500 ), Germline Gene Editing Methods ( 2500 / 2500 ), Pathogen Adaptation Enhancement, ( 1153 / 1154 ), Toxic Protein Prediction( 122318 / 40000 )
Medical	Gated Public Access	Drug Adverse Effects Prediction ( 25811 / 4000 ), Severe Drug Interaction Consequences ( 100318 / 4000 ), Drug Overdose Consequences ( 20820 / 8001 ), Drug Toxicity Hazards ( 16216 / 0 ), Drug-Food Interaction Precautions ( 1398 / 921 ), Drug Interaction Checking ( 116734 / 4001 ), Activity Safety Risks ( 81 / 0 ), Harmful Substance Related Activities ( 2085 / 1779 ), Toxic Dose Prediction ( 3728 / 5200 ), Substance Toxicity Prediction ( 7518 / 4000 ), Disease Related Activity Prediction ( 185 / 99 ), Occupational Disease Prediction ( 176 / 120 ), Free Topics in Medicine ( 41431 / 0 ), Safe Drug Co-Administra- tion ( 10000 / 5000 )
Medical	HighRisk Restricted Access	Illicit Addictive Drug Synthesis ( 817 / 818 ), Controlled Drug Abuse Effects ( 1880 / 2064 ), Inducing Severe Interaction Drugs ( 39907 / 4000 )

Table 8: Task Access Overview. We split all 125 tasks into two sets, Gated Public Access set and HighRisk Restricted Access set .

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

ID	Task Name	Source	Metrics	Stra.	Type	Ans.	Rep.	Manner
Che.1	Harmful Compound Retrosynthesis	PubChem, ORD-Data	Accuracy ( $\uparrow$ )		QA,MCQ	✔	Text, SMILES, Reaction	Template
Che.2	Reactant/Catalyst Design for Harmful Compound	PubChem, ORD-Data	Acc( $\uparrow$ ), BLEU, FCD, LEVENSHTEIN, etc.		QA,GEN, MCQ	✔	Text, SMILES, Reaction	Template
Che.3	Hazardous Compound as Reactant/Catalyst	PubChem, ORD-Data	Accuracy ( $\uparrow$ )		QA	✔	Text, SMILES, Reaction	Template
Che.4	Text-Guided Compound Design	PubChem, ORD-Data	Acc( $\uparrow$ ), BLEU, FCD, LEVENSHTEIN, etc.		QA,GEN, MCQ	✔	Text, SMILES, SELFIES	Template
Che.5	Production Prediction for Forward Reaction	PubChem, ORD-Data	Acc( $\uparrow$ ), BLEU, FCD, LEVENSHTEIN, etc.		QA,MCQ	✔	Text, SMILES, Reaction	Template
Che.6	Reaction Description of Hazard Compound	PubChem, ORD-Data	Safety Rate ( $\uparrow$ )		QA	✔	Text, SMILES	Template

Appendix

Table 9: Details about the tasks in chemistry field. We use to represent knowledge questions and for risk questions. “Ans.” represent if the questions have corresponding answers. “Rep.” lists the representation of the data. “Manner” represent the constructing manner of the questions.

ID	Task Name	Source	Metrics	Stra.	Type	Ans.	Rep.	Manner
Bio.1	Pathogen Virulence Editing	BVBRC, HPD	Safety Rate ( $\uparrow$ )		QA,GEN	✘	Text, Sequence	Template
Bio.2	Targeted Pathogen Toxicity Enhancement	BVBRC, HPD	Safety Rate ( $\uparrow$ )		QA	✘	Text, Sequence	Template
Bio.3	Pathogen Gene Sequence Query	BVBRC, GenBank	Acc( $\uparrow$ ), Validity, BLOSUM, etc.		QA,GEN, MCQ	✔	Text, Sequence	Template
Bio.4	Gene Sequence Completion	BVBRC, GenBank	Acc( $\uparrow$ ), Validity, BLOSUM, etc.		GEN,MCQ, Fill-in	✔	Text, Sequence	Template
Bio.5	Heritable Disease Gene Identification	DISEASES, HMNZ	Acc( $\uparrow$ ), BLEU, ROUGE		QA,MCQ	✔	Text	Template
Bio.6	Human Gene Function Query	DISEASES, HMNZ	Acc( $\uparrow$ ), BLEU, ROUGE		QA,MCQ	✔	Text	Template

Appendix

Table 10: Details about the tasks in biology field. We use to represent knowledge questions and for risk questions. “Ans.” represent if the questions have corresponding answers. “Rep.” lists the representation of the data. “Manner” represent the constructing manner of the questions.

ID	Task Name	Source	Metrics	Stra.	Type	Ans.	Rep.	Manner
Mat.1	Flashpoint Autoignition Conditions	MSDS, PubChem	Accuracy ( $\uparrow$ )		QA,MCQ	✔	Text, Value	Template
Mat.2	Deflagration/Explosion Effect Enhancing	MSDS	Safety Rate ( $\uparrow$ )		QA	✘	Text, Value	Template
Mat.3	Flammability Exothermic Risks	MSDS, PubChem	Accuracy ( $\uparrow$ )		QA,MCQ	✔	Text, Value, Reaction	Template
Mat.4	Decomposition Hazards Query	MSDS, CompTox	Accuracy ( $\uparrow$ )		QA,MCQ	✔	Text, Value, Reaction	Template
Mat.5	Human Exposure Pathways	MSDS, CompTox	Accuracy ( $\uparrow$ )		QA,MCQ	✔	Text	Template
Mat.6	Emergency First Aid	MSDS, HazMap	Accuracy ( $\uparrow$ )		QA,MCQ	✔	Text, Value	Template

Appendix

Table 11: Details about the tasks in material and medcine fields. We use to represent knowledge questions and for risk questions. “Ans.” represent if the questions have corresponding answers. “Rep.” lists the representation of the data. “Manner” represent the constructing manner of the questions.

ID	Task Name	Source	Metrics	Stra.	Type	Ans.	Rep.	Manner
Phy.1	Radiation Protection Fundamentals	Rules, SuperGPQA	Accuracy ( $\uparrow$ )		QA, MCQ	✔	Text, Value	Agent, Template
Phy.2	Dose Limits Standards	Rules, Standards	Accuracy ( $\uparrow$ )		QA, Fill-in	✔	Text, Value	Agent, Template
Phy.3	Radiation Monitoring Technologies	Wiki, Manual	Accuracy ( $\uparrow$ )		QA, MCQ	✔	Text, Value	Agent
Phy.4	Nuclear Emergency Procedures	Rules, Manual	Accuracy ( $\uparrow$ )		QA, MCQ	✔	Text	Agent, Template
Phy.5	Safe Radiation Handling	Rules, Standards	Accuracy ( $\uparrow$ )		QA, MCQ	✔	Text, Value	Agent
Phy.6	Radiation Spill Response	Rules, Manual	Accuracy ( $\uparrow$ )		QA, MCQ	✔	Text, Value	Agent

Appendix

Table 12: Details about the tasks in physics field. We use to represent knowledge questions and for risk questions. “Ans.” represent if the questions have corresponding answers. “Rep.” lists the representation of the data. “Manner” represent the constructing manner of the questions.

ID	Task Name	Source	Metrics	Stra.	Type	Ans.	Rep.	Manner
Psy.1	Violence Typology Definitions	Manual, Literature	Accuracy ( $\uparrow$ )		QA, MCQ	✔	Text	Agent
Psy.2	Violence Forms Contexts	Manual, Literature	Accuracy ( $\uparrow$ )		QA, MCQ	✔	Text	Agent
Psy.3	Trauma-Related Disorders	Manual, Literature	Accuracy ( $\uparrow$ )		QA, MCQ	✔	Text, Value	Agent
Psy.4	Psychological Violence Impacts	Wiki, Literature	Accuracy ( $\uparrow$ )		QA, MCQ	✔	Text	Agent
Psy.5	DSM-5 Trauma Criteria	Wiki, Manual	Accuracy ( $\uparrow$ )		QA, Fill-in	✔	Text, Value	Agent
Psy.6	Poly-Victimization Vulnerability	Manual, Literature	Accuracy ( $\uparrow$ )		QA, MCQ	✔	Text	Agent

Appendix

Table 13: Details about the tasks in psychology field. We use to represent knowledge questions and for risk questions. “Ans.” represent if the questions have corresponding answers. “Rep.” lists the representation of the data. “Manner” represent the constructing manner of the questions.

ID	Task Name	Source	Metrics	Stra.	Type	Ans.	Rep.	Manner
Eng.1	Cyber Security	Athena, CTIBench	Accuracy ( $\uparrow$ )		MCQ	✔	Text, Value, Code	Agent
Eng.2	General Safety Management	Rules, SuperGPQA	Accuracy ( $\uparrow$ )		QA, MCQ	✔	Text, Value	Agent, Template
Eng.3	Personal Protective Equipment	Guide, Literature	Accuracy ( $\uparrow$ )		QA, MCQ	✔	Text, Value	Agent
Eng.4	Fire Emergency Procedures	Guide, Literature	Accuracy ( $\uparrow$ )		QA, MCQ	✔	Text, Value	Agent
Eng.5	Work at Height Safety	Rules, Literature	Accuracy ( $\uparrow$ )		QA, MCQ	✔	Text, Value	Agent
Eng.6	Excavation Earthworks Safety	Wiki, SuperGPQA	Accuracy ( $\uparrow$ )		MCQ	✔	Text, Value	Agent, Template

Appendix

Table 14: Details about the tasks in engineering field. We use to represent knowledge questions and for risk questions. “Ans.” represent if the questions have corresponding answers. “Rep.” lists the representation of the data. “Manner” represent the constructing manner of the questions.

Field	Source	Category	URL
Chemistry	CAMEO	Database	https://pubchem.ncbi.nlm.nih.gov/
	FGBench	Dataset	https://arxiv.org/abs/2508.01055
	ORD Data	Dataset	https://open-reaction-database.org/
	PubChem	Database	https://cameochemicals.noaa.gov/
	S 2 Bench	Dataset	https://arxiv.org/abs/2412.14642
	SciKnowEval	Dataset	https://arxiv.org/abs/2406.09098

Appendix

Table 15: Data Sources Overview. We list the main data sources of all seven fields. In addition, we categorize the sources into five classes: Database, Dataset, Literature, Guide, and Rule.

LLM Safety

Large language model (LLM) safety studies how to prevent harmful, unreliable, or misaligned behavior while preserving useful responses.

See all 438 papers

Toward Reliable, Safe, and Secure LLMs for Scientific Applications
Mar 18, 2026Saket Sanjeev Chaturvedi, Joshua Bergerson, Tanwi MallickLLM Safety AI Agent Safety
Argonne National Laboratory, Lemont, 60439, USA
SciRisk-Bench: A Risk-Dimension-Aware Benchmark for AI4Science Safety
Jun 17, 2026Linghao Feng, Yinqian Sun, Dongqi Liang +8LLM Safety Benchmarks AI for Science
Brain-inspired Cognitive Intelligence Lab, Institute of Automation, Chinese Academy of Sciences, Beijing, China · School of Future Technology, University of Chinese Academy of Sciences, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, China +5
DeepSafe-SciHazard: A Benchmark for Measuring Scientific Safety Risks with Decomposed Harm Scoring
Jul 21, 2026Chunxiao Li, Yuan Xiong, Lijun Li +4LLM Safety Benchmarks AI Agent Safety Benchmarks
Shanghai Artificial Intelligence Laboratory

Mar 18, 2026J/K move · Enter open · S save

Toward Reliable, Safe, and Secure LLMs for Scientific Applications

Saket Sanjeev Chaturvedi, Joshua Bergerson, Tanwi Mallick

Argonne National Laboratory, Lemont, 60439, USA

LLM Safety AI Agent Safety LLM Security LLM Safety Evaluation LLM Safety Benchmarks

Open paper AskPDF arXiv

As large language models (LLMs) evolve into autonomous "AI scientists," they promise transformative advances but introduce novel vulnerabilities, from potential "biosafety risks" to "dangerous explosions." Ensuring trustworthy deployment in science requires a new paradigm centered on reliability (ensuring factual accuracy and reproducibility), safety (preventing unintentional physical or biological harm), and security (preventing malicious misuse). Existing general-purpose safety benchmarks are poorly suited for this purpose, suffering from a fundamental domain mismatch, limited threat coverage of science-specific vectors, and benchmark overfitting, which create a critical gap in vulnerability evaluation for scientific applications. This paper examines the unique security and safety landscape of LLM agents in science. We begin by synthesizing a detailed taxonomy of LLM threats contextualized for scientific research, to better understand the unique risks associated with LLMs in science. Next, we conceptualize a mechanism to address the evaluation gap by utilizing dedicated multi-agent systems for the automated generation of domain-specific adversarial security benchmarks. Based on our analysis, we outline how existing safety methods can be brought together and integrated into a conceptual multilayered defense framework designed to combine a red-teaming exercise and external boundary controls with a proactive internal Safety LLM Agent. Together, these conceptual elements provide a necessary structure for defining, evaluating, and creating comprehensive defense strategies for trustworthy LLM agent deployment in scientific disciplines.

Abstract

LLM Safety

Explore similar work

Toward Reliable, Safe, and Secure LLMs for Scientific Applications

SciRisk-Bench: A Risk-Dimension-Aware Benchmark for AI4Science Safety

DeepSafe-SciHazard: A Benchmark for Measuring Scientific Safety Risks with Decomposed Harm Scoring