Not an Accident: Exploiting Safety Filter Inadequacy in Humanoid Robots

Tiffany Bao; Shangtao Li; Yifan Sun; Mengwen He; Bowei Li; Alden Wang; Changliu Liu; Ruoyu Wang

cs.CROct 22, 2026

Not an Accident: Exploiting Safety Filter Inadequacy in Humanoid Robots

Authors: Tiffany Bao, Shangtao Li, Yifan Sun, Mengwen He, Bowei Li, Alden Wang, Changliu Liu, Ruoyu Wang,

Organizations: Arizona State University · Carnegie Mellon University

Abstract

As humanoid robots move from research laboratories toward deployment in human-centered environments such as homes, hospitals, and warehouses, safe operation around nearby people and objects becomes a fundamental requirement. Safety filters have emerged as a promising technique to address this challenge by providing mathematical guarantees that generated control actions do not result in collisions. This paper shows that these guarantees hold only when the safety parameters are configured correctly. However, correct parameter configurations are task- and environment-specific, and no general method exists to determine them across diverse deployments. As a result, safety filters shipped with humanoid robots can fail after deployment in specific scenarios. We show that attackers can exploit these failures through only normal user interactions without any additional privileges or capabilities. To demonstrate this risk, we present goal insertion attacks and develop SIREN, an automated tool that identifies vulnerable inserting goals for a given task and environment. Experimental results show that SIREN identifies attacks in all vulnerable scenarios in the SPARK dataset. Compared with a baseline approach, SIREN discovers three times as many attacks. We also show that SIREN is able to generate attacks against a real humanoid and cause it to collide while executing tasks. These results demonstrate that current safety filters introduce exploitable vulnerabilities that can cause safety violations, which reveals that existing safety mechanisms can themselves become a source of safety risk for humanoid robots.

Figures & tables

Input: initial state $\mathbf{x}_{0}$ , original goal $\mathbf{G}_{1}$ , goal space $\mathcal{G}$ , filter $\psi$ , simulator $\mathsf{Sim}_{\psi}$ , budget $B$ , batch size $m$ , elite fraction $\alpha$
Output: ranked inserted goals and confirmed attacks
1	$(\mu,\sigma)\leftarrow\mathsf{InitGaussian}(\mathcal{G})$
2	$\mathcal{R}\leftarrow\emptyset$ ; $n\leftarrow 0$
3	while $n<B$ do
4	$\mathcal{B}\leftarrow\mathsf{Clip}_{\mathcal{G}}\!\left(\{\mathbf{g}_{i}\sim\mathcal{N}(\mu,\mathrm{diag}(\sigma^{2}))\}_{i=1}^{m}\right)$
5	$\mathcal{S}\leftarrow\emptyset$

Figure 1: Siren ’s CEM-based search algorithm.

Safety Filter	Valid Attacks	Min Original Distance (cm)	Initial Collision Distance (cm)
SSA	✓	0.0187	-0.0152
R-SSA	✓	0.0241	-0.0298
p-SSA	✓	0.0858	-0.0443
CBF	✓	0.2729	-0.0181
R-CBF	✓	0.0841	-0.0273
R-SSS	✓	0.0625	-0.0458

Table 1: Validated goal-insertion attacks found by enumeration in single-obstacle environments.

Figure 2: A goal insertion attack against SSA.

Figure 3: Original task success over SSA safety escaping rate.

Figure 4: Control authority of the nearest body part at each step.

Panel of Table 2 — Table 2: The deployment of goal insertion attacks to real humanoids.

Figure 5: Real-world attack demonstration. The original task succeeds without contacting the obstacle, while the insertion attack drives the humanoid through an intermediate point and causes a collision during the return to the original goal.

Method	# Attacked Scenarios	Scenario Hit Rate	# Total Attacks	Attack Hit Rate
Random	40/40	100.0%	1760	35.2%
Random-NI	40/40	100.0%	1751	35.0%
SIREN	40/40	100.0%	3277	65.5%
SIREN-NI	38/40	95.0%	3006	60.1%

Table 3: Search effectiveness across 35 scenarios.

Figure 6: Case study: searched points by Siren . Large red balls denote the obstacles. The blue point is the starting point, and the green point is the ending point of the original task.

Robot Safety

Robot safety studies how to prevent harmful physical actions while enabling robots to complete tasks around people and in uncertain environments.

See all 229 papers

Adversarial Stress Testing of SPARK Humanoid Safety Filters
May 18, 2026Saurav Ghosh, Abdou Sow, Luke ZhangAdversarial Robustness Robotics
aDepartment of Computer Science and Engineering, Washington University in St. Louis, Missouri, United States
ResSafe: Learning Safety Filtering with Residual Reinforcement Learning for Humanoids
Sep 14, 2026Gechen Qu, Tong Zhang, Bike Zhang +4Safety-Critical Control Humanoid Robot Control
University of California, Berkeley · University of California, Los Angeles
Filter-Aware Fine-Tuning for Safe Humanoid Whole-Body Tracking
Oct 1, 2026Pranit Mohnot, Christian Helten, Daniele Gammelli +1Humanoid Motion Tracking Robot Safety
Stanford University · Technical University of Munich · Italian Institute of Artificial Intelligence (AI4I) +1

May 18, 2026J/K move · Enter open · S save

Adversarial Stress Testing of SPARK Humanoid Safety Filters

Saurav Ghosh, Abdou Sow, Luke Zhang

aDepartment of Computer Science and Engineering, Washington University in St. Louis, Missouri, United States

Adversarial Robustness Robotics Robot Safety Safety Filtering AI Safety Evaluation Humanoid Robotics+1

Open paper AskPDF arXiv

Humanoid robots are difficult to deploy safely because they have high-dimensional bodies, many collision constraints, and must operate near people and obstacles. Safety filters help by modifying a nominal control action when it may violate collision-avoidance constraints. Still, nominal benchmark scores do not fully show how these filters behave in harder environments. In this work, we study the robustness of SPARK humanoid safety filters through replication and stress testing. We replicate the SPARK benchmark case G1SportMode_D1_WG_SO_v1 in MuJoCo and evaluate RSSA, RSSS, SSA, CBF, PFM, and SMA under controlled random seeds. We also built a post-processing pipeline that converts raw SPARK logs into goal-tracking, minimum-distance, and collision-step metrics. Our results show that some methods track the goal more closely, while others reduce collision steps more effectively. The stress tests further indicate that safety behavior can change under obstacle crowding, noisy distance estimates, and delayed obstacle information. These findings suggest that humanoid autonomy should be evaluated beyond nominal performance, using metrics that expose failure modes before deployment.

Abstract

Robot Safety

Explore similar work

Adversarial Stress Testing of SPARK Humanoid Safety Filters

ResSafe: Learning Safety Filtering with Residual Reinforcement Learning for Humanoids

Filter-Aware Fine-Tuning for Safe Humanoid Whole-Body Tracking