cs.CROct 22, 2026

Not an Accident: Exploiting Safety Filter Inadequacy in Humanoid Robots

Authors: Tiffany Bao, Shangtao Li, Yifan Sun, Mengwen He, Bowei Li, Alden Wang, Changliu Liu, Ruoyu Wang, 

Organizations: Arizona State University · Carnegie Mellon University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

As humanoid robots move from research laboratories toward deployment in human-centered environments such as homes, hospitals, and warehouses, safe operation around nearby people and objects becomes a fundamental requirement. Safety filters have emerged as a promising technique to address this challenge by providing mathematical guarantees that generated control actions do not result in collisions. This paper shows that these guarantees hold only when the safety parameters are configured correctly. However, correct parameter configurations are task- and environment-specific, and no general method exists to determine them across diverse deployments. As a result, safety filters shipped with humanoid robots can fail after deployment in specific scenarios. We show that attackers can exploit these failures through only normal user interactions without any additional privileges or capabilities. To demonstrate this risk, we present goal insertion attacks and develop SIREN, an automated tool that identifies vulnerable inserting goals for a given task and environment. Experimental results show that SIREN identifies attacks in all vulnerable scenarios in the SPARK dataset. Compared with a baseline approach, SIREN discovers three times as many attacks. We also show that SIREN is able to generate attacks against a real humanoid and cause it to collide while executing tasks. These results demonstrate that current safety filters introduce exploitable vulnerabilities that can cause safety violations, which reveals that existing safety mechanisms can themselves become a source of safety risk for humanoid robots.

Figures & tables

Input: initial state x0\mathbf{x}_{0} , original goal G1\mathbf{G}_{1} , goal space G\mathcal{G} , filter ψ\psi , simulator Simψ\mathsf{Sim}_{\psi} , budget BB , batch size mm , elite fraction α\alpha
Output: ranked inserted goals and confirmed attacks
1(μ,σ)←InitGaussian(G)(\mu,\sigma)\leftarrow\mathsf{InitGaussian}(\mathcal{G})
2R←∅\mathcal{R}\leftarrow\emptyset ; n←0n\leftarrow 0
3while n<Bn<B do
4B←ClipG ⁣({gi∼N(μ,diag(σ2))}i=1m)\mathcal{B}\leftarrow\mathsf{Clip}_{\mathcal{G}}\!\left(\{\mathbf{g}_{i}\sim\mathcal{N}(\mu,\mathrm{diag}(\sigma^{2}))\}_{i=1}^{m}\right)
5S←∅\mathcal{S}\leftarrow\emptyset
Figure 1: Siren ’s CEM-based search algorithm.
Safety FilterValid AttacksMin Original Distance (cm)Initial Collision Distance (cm)
SSA✓0.0187-0.0152
R-SSA✓0.0241-0.0298
p-SSA✓0.0858-0.0443
CBF✓0.2729-0.0181
R-CBF✓0.0841-0.0273
R-SSS✓0.0625-0.0458
Table 1: Validated goal-insertion attacks found by enumeration in single-obstacle environments.
Figure 2: A goal insertion attack against SSA.
Figure 3: Original task success over SSA safety escaping rate.
Figure 4: Control authority of the nearest body part at each step.
Table 2: The deployment of goal insertion attacks to real humanoids.
Figure 5: Real-world attack demonstration. The original task succeeds without contacting the obstacle, while the insertion attack drives the humanoid through an intermediate point and causes a collision during the return to the original goal.
Method# Attacked ScenariosScenario Hit Rate# Total AttacksAttack Hit Rate
Random40/40100.0%176035.2%
Random-NI40/40100.0%175135.0%
SIREN40/40100.0%327765.5%
SIREN-NI38/4095.0%300660.1%
Table 3: Search effectiveness across 35 scenarios.
Figure 6: Case study: searched points by Siren . Large red balls denote the obstacles. The blue point is the starting point, and the green point is the ending point of the original task.
More in this topic

Robot Safety

Robot safety studies how to prevent harmful physical actions while enabling robots to complete tasks around people and in uncertain environments.

See all 229 papers

Explore similar work

CardsList
May 18, 2026

Adversarial Stress Testing of SPARK Humanoid Safety Filters

Humanoid robots are difficult to deploy safely because they have high-dimensional bodies, many collision constraints, and must operate near people and obstacles. Safety filters help by modifying a nominal control action when it may violate collision-avoidance constraints. Still, nominal benchmark scores do not fully show how these filters behave in harder environments. In this work, we study the robustness of SPARK humanoid safety filters through replication and stress testing. We replicate the SPARK benchmark case G1SportMode_D1_WG_SO_v1 in MuJoCo and evaluate RSSA, RSSS, SSA, CBF, PFM, and SMA under controlled random seeds. We also built a post-processing pipeline that converts raw SPARK logs into goal-tracking, minimum-distance, and collision-step metrics. Our results show that some methods track the goal more closely, while others reduce collision steps more effectively. The stress tests further indicate that safety behavior can change under obstacle crowding, noisy distance estimates, and delayed obstacle information. These findings suggest that humanoid autonomy should be evaluated beyond nominal performance, using metrics that expose failure modes before deployment.

Saurav Ghosh, Abdou Sow, Luke Zhang

aDepartment of Computer Science and Engineering, Washington University in St. Louis, Missouri, United States

Adversarial RobustnessRoboticsRobot SafetySafety FilteringAI Safety EvaluationHumanoid Robotics+1
AskPDF
Sep 14, 2026

ResSafe: Learning Safety Filtering with Residual Reinforcement Learning for Humanoids

Safe control of humanoid robots remains challenging due to their high-dimensional dynamics, contact-rich interactions, and sensitivity to disturbances. Although reinforcement learning has enabled effective locomotion and motion tracking, learned policies can still generate unsafe actions that lead to instability or falls. In this work, we propose residual reinforcement learning as an implicit safety-filtering mechanism for safe humanoid control. Instead of relying on a single nominal policy to simultaneously balance performance, safety, and robustness, we decouple performance and safety. The nominal policy focuses solely on task performance, while a residual policy learns safety corrections. This decoupling leads to a better performance--safety Pareto trade-off and avoids the need for careful tuning of multiple competing reward terms within a single policy training. We show that the residual policy can act as an implicit safety filter.

Gechen Qu, Tong Zhang, Bike Zhang +4

University of California, Berkeley · University of California, Los Angeles

Safety-Critical ControlHumanoid Robot ControlRobot SafetySafety FilteringResidual Policy LearningSafe RL
AskPDF
Oct 1, 2026

Filter-Aware Fine-Tuning for Safe Humanoid Whole-Body Tracking

Safe whole-body motion is essential for deploying humanoid robots in unstructured environments. Modern humanoid control commonly separates reference specification from execution, with a planner, teleoperator, or motion generator providing a reference that a reinforcement-learning policy tracks through dynamically feasible whole-body control. Runtime safety filters, such as control barrier functions (CBFs), offer a promising approach for enforcing newly introduced constraints via interventions on the tracker's outputs. We show, however, that treating the tracking policy and safety filter independently induces fundamental mismatches, as filtering alters both the executed actions and the induced state distribution. We study this policy-filter interface through case studies that isolate dynamics, objective, and information mismatches, highlight their root causes, and use these insights to develop CoFiT (Constrained Filter-aware Tuning), a filter-aware fine-tuning method for pretrained trackers. Across diverse constraint scenes, CoFiT reduces violation time relative to filter-only training by 91% on TWIST2 and 21% on SONIC, while requiring smaller safety filter corrections. On Unitree G1 hardware, CoFiT reduces violation time by 83% for TWIST2 and completes every trial without operator intervention, whereas 50% of baseline trials require an operator stop. Together, these results provide actionable insights into policy-filter interactions and establish design principles for integrating learned trackers with runtime safety filters.

Pranit Mohnot, Christian Helten, Daniele Gammelli +1

Stanford University · Technical University of Munich · Italian Institute of Artificial Intelligence (AI4I) +1

Humanoid Motion TrackingRobot SafetyRobot Policy AdaptationRL for RoboticsControl Barrier Functions
AskPDF