LLM Safety

LLM: Large Language Model

LLM safety studies how models can avoid harmful assistance, manipulation, privacy violations, and unreliable advice while still helping with legitimate requests. Approaches include safety training and data filtering, inference-time guardrails and steering, output monitoring, and mechanistic methods that probe or edit internal representations. Benchmarks increasingly test jailbreaks across languages and modalities, long conversations, tool use, and high-stakes settings such as health and cybersecurity. A central challenge is maintaining robustness against adaptive attacks and changing contexts without causing over-refusal or sacrificing utility.

Latest papers

442 papers68 papers in the last four weeks, up 70% on the four weeks before. 0.4% of all new papers.

Taiwan Safety Benchmark and Breeze Guard: Toward Trustworthy AI for Taiwanese Mandarin
Mar 7, 2026Po-Chun Hsu, Meng-Hsi Chen, Tsu Ling Chao +2LLM Safety Language Model Safety Evaluation
MediaTek Research · National Taiwan University
Ambiguity Collapse by LLMs: A Taxonomy of Epistemic Risks
Mar 6, 2026Shira Gur-Arieh, Angelina Wang, Sina FazelpourLLM Safety LLM Safety Alignment
Harvard University, United States · Cornell Tech, United States · Northeastern University, United States
Depth Charge: Jailbreak Large Language Models from Deep Safety Attention Heads
Mar 6, 2026Jinman Wu, Yi Xie, Shiqian Zhao +1Adversarial Attacks on LLMs LLM Jailbreak Attacks
Xidian University, Xi’an, China · Tsinghua University, Shenzhen, China · Nanyang Technological University, Singapore
Survive at All Costs: Exploring LLM's Risky Behaviors under Survival Pressure
Mar 5, 2026Yida Lu, Jianwei Fang, Xuyang Shao +7LLM Safety AI Agent Safety
The Conversational AI (CoAI) group, DCST, Tsinghua University · China Unicom Software Research Institute · University of Electronic Science and Technology of China +1
ThaiSafetyBench: Assessing Language Model Safety in Thai Cultural Contexts
Mar 5, 2026Trapoom Ukarapol, Nut Chukamphaeng, Kunat Pipatanakul +1Language Model Safety Evaluation LLM Safety
SCB DataX · Department of Computer Science and Technology, Tsinghua University · SCBX R&D +1
Alignment Backfire: Language-Dependent Reversal of Safety Interventions Across 16 Languages in LLM Multi-Agent Systems
Mar 5, 2026Hiroki FukuiLLM Safety LLM Alignment
Research Institute of Criminal Psychiatry / Sex Offender Medical Center; Department of Neuropsychiatry, Kyoto University
Efficient Refusal Ablation in LLM through Optimal Transport
Mar 4, 2026Geraldin Nanfack, Eugene Belilovsky, Elvis DohmatobAdversarial Attacks on LLMs Jailbreak Attacks
Concordia University, Montreal, Canada · Mila – Quebec AI Institute, Montreal, Canada
Internal Safety Collapse in Frontier Large Language Models
Mar 4, 2026Yutao Wu, Xiao Liu, Yifeng Gao +7LLM Safety Language Model Safety Evaluation
Deakin University · Institute of Trustworthy Embodied AI, Fudan University · Shanghai Key Laboratory of Multimodal Embodied AI +4
Residual Stream Analysis of Overfitting And Structural Disruptions
Mar 4, 2026Quan Liu, Han Zhou, Wenquan Wu +2Safety-Preserving Fine-Tuning of Language Models Refusal Behavior in Language Models
BUPT · Baidu
MUSE: A Run-Centric Platform for Multimodal Unified Safety Evaluation of Large Language Models
Mar 3, 2026Zhongxi Wang, Yueqian Lin, Jingyang Zhang +3LLM Safety Benchmarks LLM Safety
1Duke University · 2Virtue AI
SafeSci: Safety Evaluation of Large Language Models in Science Domains and Beyond
Mar 2, 2026Xiangyang Zhu, Yuan Tian, Qi Jia +14LLM Safety LLM Safety Evaluation
Shanghai AI Lab · ByteDance · ECNU
Inference-Time Safety For Code LLMs Via Retrieval-Augmented Revision
Mar 2, 2026Manisha Mukherjee, Vincent J. HellendoornLLM Safety Secure Code Generation
Carnegie Mellon University, Pittsburgh, PA, USA
Token-level Data Selection for Safe LLM Fine-tuning
Mar 1, 2026Yanping Li, Zhening Liu, Zijian Li +2Safety-Preserving Fine-Tuning of Language Models LLM Fine-Tuning
The Hong Kong University of Science and Technology · School of Data Science, Lingnan University
Enhancing Safety of Large Language Models via Embedding Space Separation
Mar 1, 2026Xu Zhao, Xiting Wang, Weiran ShenLLM Safety LLM Fine-Tuning
Gaoling School of Artificial Intelligence, Renmin University of China.
MIDAS: Multi-Image Dispersion and Semantic Reconstruction for Jailbreaking MLLMs
Feb 28, 2026Yilian Liu, Xiaojun Jia, Guoshun Nan +6Multimodal Jailbreak Attacks Adversarial Attacks on Unified Multimodal Models
Beijing University of Posts and Telecommunications, China · Nanyang Technological University, Singapore · Guilin University of Electronic Technology, China
LSR: Linguistic Safety Robustness Benchmark for Low-Resource West African Languages
Feb 27, 2026Godwin Abuh FarunaLLM Safety Benchmarks Language Model Safety Evaluation
Fagmart Lab
Multilingual Safety Alignment Via Sparse Weight Editing
Feb 26, 2026Jiaming Liang, Zhaoxin Wang, Handing WangLLM Safety Alignment LLM Safety
School of Artificial Intelligence, Xidian University.
Manifold of Failure: Behavioral Attraction Basins in Language Models
Feb 25, 2026Sarthak Munshi, Manish Bhatt, Vineeth Sai Narajala +4LLM Security LLM Safety
Amazon Web Services · Amazon Leo · Cisco +1
Enhancing Legal LLMs through Metadata-Enriched RAG Pipelines and Direct Preference Optimization
Feb 25, 2026Suyash Maniyar, Deepali Singh, Rohith ReddyLLM Grounding Language Modeling
Detoxifying LLMs via Representation Erasure-Based Preference Optimization
Feb 24, 2026Nazanin Mohammadi Sepahvand, Eleni Triantafillou, Hugo Larochelle +3LLM Safety Text Detoxification
McGill University, Canada · Mila, Canada · Google DeepMind +2
The Missing Red Line: How Commercial Pressure Erodes AI Safety Boundaries
Feb 24, 2026Nora Petrova, John BurdenLLM Safety LLM Safety Alignment
Prolific
Under the Influence: Evaluating and Improving LLM Vigilance in Agentic Tasks
Feb 24, 2026Sasha Robinson, Katherine M. Collins, Ilia Sucholutsky +1Persuasion in Language Models LLM Safety
McMaster University · Vector Institute · Princeton AI Lab +4
BarrierSteer: LLM Safety via Learning Barrier Steering
Feb 23, 2026Thanh Q. Tran, Arun Verma, Kiwan Wong +3LLM Safety Adversarial Robustness of Language Models
Department of Computer Science, National University of Singapore, Republic of Singapore · Singapore-MIT Alliance for Research and Technology Centre, Republic of Singapore · CSAIL, Massachusetts Institute of Technology, USA +1
Workflow-Level Design Principles for Trustworthy GenAI in Automotive System Engineering
Feb 23, 2026Chih-Hong Cheng, Brian Hsuan-Cheng Liao, Adam Molin +1LLM Safety Requirements Engineering
Carl von Ossietzky University of Oldenburg, Oldenburg, Germany · DENSO AUTOMOTIVE Deutschland Germany, Eching, Germany
MANATEE: Inference-Time Lightweight Diffusion Based Safety Defense for LLMs
Feb 21, 2026Chun Yan Ryan Kan, Tommy Tran, Vedant Yadav +4Jailbreak Defense LLM Safety
Learning to Stay Safe: Adaptive Regularization Against Safety Degradation during Fine-Tuning
Feb 19, 2026Jyotin Goel, Souvik Maji, Pratik MazumderSafety-Preserving Fine-Tuning of Language Models LLM Safety
Indian Institute of Technology Jodhpur Jodhpur, Rajasthan, India
DeepContext: Stateful Real-Time Detection of Multi-Turn Adversarial Intent Drift in LLMs
Feb 18, 2026Justin Albrethsen, Yash Datta, Kunal Kumar +1LLM Safety Jailbreak Defense
Improving LLM Reliability through Hybrid Abstention and Adaptive Detection
Feb 17, 2026Ankit Sharma, Nachiket Tapas, Jyotiprakash PatraLLM Guardrails LLM Reliability
Department of Computer Science & Engineering, Chhattisgarh Swami Vivekanand Technical University, Bhilai, Chhattisgarh, India
Exposing the Systematic Vulnerability of Open-Weight Models to Prefill Attacks
Feb 16, 2026Lukas Struppek, Adam Gleave, Kellin PelrineAdversarial Attacks on LLMs LLM Safety
Competition for attention predicts good-to-bad tipping in AI
Feb 16, 2026Neil F. Johnson, Frank Y. HuoLLM Safety Language Modeling
Physics Department and Dynamic Online Networks Laboratory, George Washington University, Washington D.C. 20052, U.S.A.

Mar 7, 2026J/K move · Enter open · S save

Taiwan Safety Benchmark and Breeze Guard: Toward Trustworthy AI for Taiwanese Mandarin

Po-Chun Hsu, Meng-Hsi Chen, Tsu Ling Chao, Chia Tien Han+1

MediaTek Research · National Taiwan University

LLM Safety Language Model Safety Evaluation LLM Safety Benchmarks Cultural Knowledge in Language Models Cross-Cultural Language Model Evaluation

Open paper AskPDF arXiv

Global safety models exhibit strong performance across widely used benchmarks, yet their training data rarely captures the cultural and linguistic nuances of Taiwanese Mandarin. This limitation results in systematic blind spots when interpreting region-specific risks such as localized financial scams, culturally embedded hate speech, and misinformation patterns. To address these gaps, we introduce TS-Bench (Taiwan Safety Benchmark), a standardized evaluation suite for assessing safety performance in Taiwanese Mandarin. TS-Bench contains 400 human-curated prompts spanning critical domains including financial fraud, medical misinformation, social discrimination, and political manipulation. In parallel, we present Breeze Guard, an 8B safety model derived from Breeze 2, our previously released general-purpose Taiwanese Mandarin LLM with strong cultural grounding from its original pre-training corpus. Breeze Guard is obtained through supervised fine-tuning on a large-scale, human-verified synthesized dataset targeting Taiwan-specific harms. Our central hypothesis is that effective safety detection requires the cultural grounding already present in the base model; safety fine-tuning alone is insufficient to introduce new socio linguistic knowledge from scratch. Empirically, Breeze Guard significantly outperforms the leading 8B general-purpose safety model, Granite Guardian 3.3, on TS-Bench (+0.17 overall F1), with particularly large gains in high-context categories such as scam (+0.66 F1) and financial malpractice (+0.43 F1). While the model shows slightly lower performance on English-centric benchmarks (ToxicChat, AegisSafetyTest), this tradeoff is expected for a regionally specialized safety model optimized for Taiwanese Mandarin. Together, Breeze Guard and TS-Bench establish a new foundation for trustworthy AI deployment in Taiwan.