Safety Layer

Safety layers are being developed to enhance the reliability and safety of various machine learning models, particularly large language models (LLMs) and reinforcement learning agents, by mitigating risks such as malicious inputs, exaggerated safety responses, and unsafe actions in real-world applications. Current research focuses on identifying and manipulating specific model layers crucial for safety, employing techniques like activation steering, partial parameter fine-tuning, and data-driven predictive control to achieve a balance between safety and functionality. These advancements are significant for improving the trustworthiness and deployment of AI systems in sensitive domains, ranging from robotics and energy management to human-robot interaction and autonomous systems.

Papers

October 23, 2024

Multi-Layered Safety of Redundant Robot Manipulators via Task-Oriented Planning and Control
Xinyu Jia, Wenxin Wang, Jun Yang, Yongping Pan, Haoyong Yu
External Control Task Planning Trajectory Planner Safety Layer Redundant Robot

October 21, 2024

Assisted Physical Interaction: Autonomous Aerial Robots with Neural Network Detection, Navigation, and Safety Layers
Andrea Berra, Viswa Narayanan Sankaranarayanan, Achilleas Santi Seisa, Julien Mellet, Udayanga G.W.K.N. Gamage, Sumeet Gajanan Satpute, Fabio Ruggiero, Vincenzo Lippiello, Silvia Tolu, Matteo Fumagalli, George Nikolakopoulos, Miguel Ángel Trujillo Soto, Guillermo Heredia
Unmanned Aerial Vehicle Neural Topic Target Detection Safety Layer

October 13, 2024

Targeted Vaccine: Safety Alignment for Large Language Models against Harmful Fine-Tuning via Layer-wise Perturbation
Guozhi Liu, Weiwei Lin, Tiansheng Huang, Ruichao Mo, Qi Mu, Li Shen
Fine Tuning Safety Alignment Vaccine Related Safety Layer

October 10, 2024

How Does Vision-Language Adaptation Impact the Safety of Vision Language Models?
Seongyun Lee, Geewook Kim, Jiyeon Kim, Hyunji Lee, Hoyeon Chang, Sue Hyun Park, Minjoon Seo
Vision Language Model Human SAFETY Safety Fine Tuning Safety Layer

August 30, 2024

Safety Layers in Aligned Large Language Models: The Key to LLM Security
Shen Li, Liuyi Yao, Lan Zhang, Yaliang Li
Language Model LLM Safety LLM Alignment Backdoor Data Safety Layer

August 21, 2024

Nothing in Excess: Mitigating the Exaggerated Safety for LLMs via Safety-Conscious Activation Steering
Zouying Cao, Yifei Yang, Hai Zhao
Large Language Model Safety Alignment Steering Vector Safety Layer Activation Steering

August 14, 2024

Real-world validation of safe reinforcement learning, model predictive control and decision tree-based home energy management systems
Julian Ruddick, Glenn Ceusters, Gilles Van Kriekinge, Evgenii Genov, Thierry Coosemans, Maarten Messagie
Reinforcement Learning Predictive Control Safe Reinforcement Learning Decision Relevant Information Energy Management Decision Tree Policy Safety Layer Home Energy Management

May 28, 2024

Defending Large Language Models Against Jailbreak Attacks via Layer-specific Editing
Wei Zhao, Zhe Li, Yige Li, Ye Zhang, Jun Sun
Large Language Model Jailbreak Attack Adversarial Prompt Defense Method Unsafe Input Safety Layer Layer Feature

September 18, 2023

A Concise Overview of Safety Aspects in Human-Robot Interaction
Mazin Hamad, Simone Nertinger, Robin J. Kirschner, Luis Figueredo, Abdeldjallil Naceri, Sami Haddadin
Human Robot Interaction Brief Introduction System Safety Safety Layer Safety Architecture

April 18, 2023

An adaptive safety layer with hard constraints for safe reinforcement learning in multi-energy management systems
Glenn Ceusters, Muhammad Andy Putratama, Rüdiger Franke, Ann Nowé, Maarten Messagie
Reinforcement Learning Algorithm Safe Reinforcement Learning Hard Constraint Safety Layer Multi Energy

December 12, 2022

Evaluating Model-free Reinforcement Learning toward Safety-critical Tasks
Linrui Zhang, Qin Zhang, Li Shen, Bo Yuan, Xueqian Wang, Dacheng Tao
Reinforcement Learning Safe Reinforcement Learning Safety Constraint Probabilistic Safety Safety Layer Safety Critical Task

November 20, 2022

Safe Reinforcement Learning using Data-Driven Predictive Control
Mahmoud Selim, Amr Alanwar, M. Watheq El-Kharashi, Hazem M. Abbas, Karl H. Johansson
Reinforcement Learning Data Driven Reinforcement Learning Algorithm Safe Reinforcement Learning Safety Layer

May 25, 2022

Power-based Safety Layer for Aerial Vehicles in Physical Interaction using Lyapunov Exponents
Eugenio Cuniato, Nicholas Lawrance, Marco Tognon, Roland Siegwart
Aerial Vehicle Autonomous Operation Physical Interaction Lyapunov Exponent Omnidirectional Micro Aerial Vehicle Safety Layer

November 22, 2021

BarrierNet: A Safety-Guaranteed Layer for Neural Networks
Wei Xiao, Ramin Hasani, Xiao Li, Daniela Rus
Neural Network Control Barrier Function Neural Network Controller Differentiable Convex Safety Layer

Safety Layer

Papers

Multi-Layered Safety of Redundant Robot Manipulators via Task-Oriented Planning and Control

Assisted Physical Interaction: Autonomous Aerial Robots with Neural Network Detection, Navigation, and Safety Layers

Targeted Vaccine: Safety Alignment for Large Language Models against Harmful Fine-Tuning via Layer-wise Perturbation

How Does Vision-Language Adaptation Impact the Safety of Vision Language Models?

Safety Layers in Aligned Large Language Models: The Key to LLM Security

Nothing in Excess: Mitigating the Exaggerated Safety for LLMs via Safety-Conscious Activation Steering

Real-world validation of safe reinforcement learning, model predictive control and decision tree-based home energy management systems

Defending Large Language Models Against Jailbreak Attacks via Layer-specific Editing

A Concise Overview of Safety Aspects in Human-Robot Interaction

An adaptive safety layer with hard constraints for safe reinforcement learning in multi-energy management systems

Evaluating Model-free Reinforcement Learning toward Safety-critical Tasks

Safe Reinforcement Learning using Data-Driven Predictive Control

Power-based Safety Layer for Aerial Vehicles in Physical Interaction using Lyapunov Exponents

BarrierNet: A Safety-Guaranteed Layer for Neural Networks