Safety Filtering

Safety filtering studies how to detect and prevent unsafe behavior at runtime or during training, from robot collisions to harmful language-model outputs and risky tool calls. In control, common approaches use control barrier functions, quadratic-program action shields, reachability and backup policies, with learned models and conformal calibration addressing uncertainty. For language and multimodal systems, filters include trained guard classifiers, hidden-state probes, streaming checks, and pre-execution tool monitors. A central challenge is balancing reliable protection against unnecessary intervention, especially when attacks, environments, or user contexts differ from those seen in evaluation.