Safety-Utility Trade-Off in Language Models

Latest papers

17 papers2 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

Safety Mid-Training: Internalizing LLM Safety as a Foundational Capability
Oct 19, 2026Zhengyue Zhao, Liwei Jiang, Yejin Choi +1Safety-Utility Trade-Off in Language Models LLM Safety Alignment
Johns Hopkins University · University of Washington · Stanford University
The Safety Operator: Modulating the Expression of Safety Instructions via Spectral Optimization
Sep 29, 2026Benoit Dherin, Michael Munn, Xavier Gonzalvo +14LLM Safety Alignment LLM Safety
Google Research · Cambridge University · Google DeepMind +2
Refuse without Refusal: A Structural Analysis of Safety-Tuning Responses for Reducing False Refusals in Language Models
Sep 4, 2026Minji Kim, Hyounghun KimRefusal Behavior in Language Models Refusal Mechanisms in Language Models
Graduate School of Artificial Intelligence, POSTECH · Department of Computer Science and Engineering, POSTECH
Arabic Safety Alignment as Selective Refusal: An Empirical Study of SFT, DPO, and Guard Calibration
Aug 29, 2026Mohamad Zbib, Ammar MohannaArabic Language Models Refusal Behavior in Language Models
American University of Beirut
Who Pays More for Safety? Measuring the Disparate Cost of Safety Alignment across Languages
Aug 23, 2026Chanwoong Yoon, Jungsoo Park, Alan RitterLLM Safety Alignment Language Model Safety Evaluation
Korea University · Georgia Institute of Technology
CLEAR: Continuous Latent Adapter Routing for Utility-Preserving LLM Safety Alignment
Aug 21, 2026Chengxiao Wang, Enyi Jiang, Xiaojing Liao +1Safety-Utility Trade-Off in Language Models LLM Safety
Siebel School of Computing and Data Science, University of Illinois at Urbana-Champaign · Computer Science, Stanford University
Judge-dependent safety gains and model-specific helpfulness costs of evidence-sufficiency prompting in clinical LLMs
Jul 20, 2026Koyar AfrasyabLLM-as-a-Judge LLM Prompting
Kinvectum AB, Sweden
Addressing Over-Refusal in LLMs with Competing Rewards
Jun 30, 2026Taeyoun Kim, Aviral KumarOver-Refusal LLM Safety
Carnegie Mellon University · SynthLabs
SHARD: Safe and Helpful Alignment via Self-Reframing Distillation
Jun 14, 2026Viswonathan Manoranjan, Amogh Gupta, Anvesh Rao Vijjini +2LLM Safety Alignment Language Model Distillation
UNC Chapel Hill
Inform, Coach, Relate, Listen: Auditing LLM Caregiving Support Roles
May 28, 2026Drishti Goel, Agam Goyal, Veda Duddu +8Healthcare LLM Auditing
University of Illinois Urbana-Champaign · University of Massachusetts Amherst · OSF HealthCare +1
Why Do Aligned LLMs Remain Jailbreakable: Refusal-Escape Directions, Operator-Level Sources, and Safety-Utility Trade-off
May 9, 2026Yu Chen, Yuanhao Liu, Qi CaoLLM Alignment Language Model Steering
Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences, Beijing, China
Self-Mined Hardness for Safety Fine-Tuning
May 4, 2026Prakhar Gupta, Garv Shah, Donghua ZhangRefusal Behavior in Language Models Hard Negative Mining
University of Michigan
Useless but Safe? Benchmarking Utility Recovery with User Intent Clarification in Multi-Turn Conversations
Apr 29, 2026Mingqian Zheng, Malia Morgan, Liwei Jiang +2LLM Safety Benchmarks Language Model Safety Evaluation
Carnegie Mellon University · Allen Institute for AI · University of Washington
Cat-DPO: Category-Adaptive Safety Alignment
Apr 19, 2026Tiankai Yang, Yi Nian, Xinyuan Li +3LLM Alignment Direct Preference Optimization
University of Southern California · 2Northwestern University
Why Agents Compromise Safety Under Pressure
Mar 16, 2026Hengle Jiang, Ke TangSafety-Utility Trade-Off in Language Models AI Agent Safety
Guangdong Provincial Key Laboratory of Brain-inspired Intelligent Computation, Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, China
The Straight and Narrow: Do LLMs Possess an Internal Moral Path?
Jan 15, 2026Luoming Hu, Jingjie Zeng, Liang Yang +1Moral Reasoning in Language Models Inference-Time Language Model Alignment
School of Future Technology, Dalian University of Technology, China · School of Computer Science and Technology, Dalian University of Technology, China · Key Laboratory of Social Computing and Cognitive Intelligence, Ministry of Education, China
Test-Time Detoxification without Training or Learning Anything
Jan 14, 2026Baturay Saglam, Dionysis KalogeriasLarge Language Model-Guided Optimization LLM Safety
Department of Electrical and Computer Engineering, Yale University, New Haven, CT.

Oct 19, 2026J/K move · Enter open · S save

Safety Mid-Training: Internalizing LLM Safety as a Foundational Capability

Zhengyue Zhao, Liwei Jiang, Yejin Choi, Chaowei Xiao

Johns Hopkins University · University of Washington · Stanford University

Safety-Utility Trade-Off in Language Models LLM Safety Alignment LLM Safety Safety-Preserving Fine-Tuning of Language Models

Open paper AskPDF arXiv

Safety alignment for large language models (LLMs) is primarily performed during post-training, while recent work has also begun incorporating safety data into pre-training. Yet safety can degrade when later training no longer includes safety data, and gains in safety often come at the cost of general capability. This tension suggests that safety and general reasoning should not be treated as separate training goals. We instead view safety as an LLM capability that can be learned through a reasoning structure shared with general tasks. Both require the model to understand intent or task goals, apply relevant knowledge, and reason about potential consequences. We introduce safety mid-training, a stage between pre-training and post-training that jointly develops safety and general reasoning through this shared structure. Given safety and utility prompts, the base model generates structured documents and reasoning traces that are then used for mid-training. Safety mid-training reduces average attack success rate (ASR) by 72% to 81% relative to Qwen3-1.7B, 4B, and 8B base models, while also improving general capability. The benefits persist through subsequent post-training. After general-only SFT, mid-trained models largely retain their safety, achieving lower ASR than the baselines without sacrificing general capability. They also retain safety under general-only GRPO without safety rewards, while subsequent RL reaches a better safety-utility frontier when initialized from the mid-trained models. Together, these results suggest that learning safety and general reasoning within a shared mid-training stage can reduce the safety-utility trade-off and make safety more robust to subsequent training.