Novel Shielding Mechanism

Novel shielding mechanisms are being developed to enhance the safety and robustness of various AI systems, particularly in reinforcement learning and large language models. Current research focuses on techniques like prompt tuning for LLMs, adaptive shielding using contrastive autoencoders for RL agents, and incorporating human preferences to improve transparency and safety. These advancements are crucial for mitigating risks associated with adversarial attacks, unexpected behaviors, and resource constraints in real-world applications of AI, leading to more reliable and trustworthy systems.

Papers

October 7, 2024

Learning-Based Shielding for Safe Autonomy under Unknown Dynamics
Robert Reed, Morteza Lahijanian
Deep Reinforcement Learning Temporal Logic Unknown Dynamic Model Based Shielding Trustworthy Autonomy Novel Shielding Mechanism

July 3, 2024

Soft Begging: Modular and Efficient Shielding of LLMs against Prompt Injection and Jailbreaking based on Prompt Tuning
Simon Ostermann, Kevin Baum, Christoph Endres, Julia Masloh, Patrick Schramowski
Large Language Model Prompt Tuning Jailbreak Attack Natural Language Prompt Soft Prompt Prompt Injection Novel Shielding Mechanism

May 28, 2024

Safe Reinforcement Learning in Black-Box Environments via Adaptive Shielding
Daniel Bethell, Simos Gerasimou, Radu Calinescu, Calum Imrie
Reinforcement Learning Black Box Safe Reinforcement Learning Safe Exploration Novel Shielding Mechanism

November 28, 2023

Increasing Transparency of Reinforcement Learning using Shielding for Human Preferences and Explanations
Georgios Angelopoulos, Luigi Mangiacapra, Alessandra Rossi, Claudia Di Napoli, Silvia Rossi
Reinforcement Learning Human Robot Interaction Line by Line Explanation Transparency Index Human Preference Autonomous Decision Making Novel Shielding Mechanism

November 23, 2023

Acoustic Cybersecurity: Exploiting Voice-Activated Systems
Forrest McKee, David Noever
Attack Success Rate Voice Assistant Attack Vector Voice Activated Novel Shielding Mechanism Inaudible Voice Acoustic Eavesdropping

November 21, 2023

Scheduling Distributed Flexible Assembly Lines using Safe Reinforcement Learning with Soft Shielding
Lele Li, Liyong Lin
Reinforcement Learning Action Space Safe Reinforcement Learning Task Scheduling Scheduling Problem Assembly Line Novel Shielding Mechanism

August 8, 2023

Pelta: Shielding Transformers to Mitigate Evasion Attacks in Federated Learning
Simon Queyrut, Yérom-David Bromberg, Valerio Schiavoni
Adversarial Example Transformer Megatron Decepticons Evasion Attack Malicious Sample Novel Shielding Mechanism Malicious Attack

November 28, 2022

Shielding in Resource-Constrained Goal POMDPs
Michal Ajdarów, Šimon Brlej, Petr Novotný
Markov Decision Process Partially Observable Markov Decision Process Heuristic Search POMDP Planning Novel Shielding Mechanism

July 27, 2022

Dynamic Shielding for Reinforcement Learning in Black-Box Environments
Masaki Waga, Ezequiel Castellano, Sasinee Pruekprasert, Stefan Klikovits, Toru Takisaka, Ichiro Hasuo
Reinforcement Learning Black Box Cyber Physical System Learning Automaton Model Based Shielding Novel Shielding Mechanism Model Based Safe Reinforcement Learning

May 3, 2022

Don't sweat the small stuff, classify the rest: Sample Shielding to protect text classifiers against adversarial attacks
Jonathan Rusert, Padmini Srinivasan
Deep Learning Adversarial Attack Good Thing State of the Art Defense Text Classifier Rest RESTAD NAP Novel Shielding Mechanism

January 20, 2022

Sim-to-Lab-to-Real: Safe Reinforcement Learning with Shielding and Generalization Guarantees
Kai-Chieh Hsu, Allen Z. Ren, Duy Phuong Nguyen, Anirudha Majumdar, Jaime F. Fisac
Reinforcement Learning Autonomous System Safe Reinforcement Learning Based Policy Generalization Guarantee Bellman Equation Novel Shielding Mechanism Optimal Safe Policy