AudioGuard: Toward Comprehensive Audio Safety Protection Across Diverse Threat Models

Mintong Kang; Chen Fang; Bo Li

cs.SDApr 10, 2026

AudioGuard: Toward Comprehensive Audio Safety Protection Across Diverse Threat Models

Authors: Mintong Kang, Chen Fang, Bo Li

Organizations: University of Illinois Urbana-Champaign

Abstract

Audio has rapidly become a primary interface for foundation models, powering real-time voice assistants. Ensuring safety in audio systems is inherently more complex than just "unsafe text spoken aloud": real-world risks can hinge on audio-native harmful sound events, speaker attributes (e.g., child voice), impersonation/voice-cloning misuse, and voice-content compositional harms, such as child voice plus sexual content. The nature of audio makes it challenging to develop comprehensive benchmarks or guardrails against this unique risk landscape. To close this gap, we conduct large-scale red teaming on audio systems, systematically uncover vulnerabilities in audio, and develop a comprehensive, policy-grounded audio risk taxonomy and AudioSafetyBench, the first policy-based audio safety benchmark across diverse threat models. AudioSafetyBench supports diverse languages, suspicious voices (e.g., celebrity/impersonation and child voice), risky voice-content combinations, and non-speech sound events. To defend against these threats, we propose AudioGuard, a unified guardrail consisting of 1) SoundGuard for waveform-level audio-native detection and 2) ContentGuard for policy-grounded semantic protection. Extensive experiments on AudioSafetyBench and four complementary benchmarks show that AudioGuard consistently improves guardrail accuracy over strong audio-LLM-based baselines with substantially lower latency.

Figures & tables

Benchmark	Diverse Threat Models	Input & Output Safety	Audio-Specific Risk Taxonomy	Non-Speech Sound Events	Diverse Voices	Multi- Lingual
Nemotron-Content-Safety-Audio	✗	✗	✗	✗	✓	✗
Jailbreak-AudioBench	✗	✗	✗	✗	✓	✗
Omni-SafetyBench	✗	✗	✗	✗	✗	✗
AdvWave	✗	✗	✗	✓	✗	✗
AudioSafetyBench (Ours)	✓	✓	✓	✓	✓	✓

Table 1: Comparison between AudioSafetyBench and other audio safety benchmarks.

Figure 2: AudioGuard framework overview. Given an input audio waveform, SoundGuard detects audio-native safety cues directly from the signal (e.g., speaker identity, child voice, gunshot, sexual sounds) and outputs sound risk scores. In parallel, ContentGuard transcribes the audio via ASR and leverages TextGuard to predict the content risk scores (e.g., misinformation, fraud, harassment, sexual content). AudioGuard provides the end-to-end audio guardrail with interpretable predictions under diverse audio threat models.

Method	AudioSafetyBench			Nemotron- Audio	Jailbreak- AudioBench	Omni- SafetyBench	AdvWave	Avg	Latency (s)
Method	Speech	Non-Speech	ElevenLabs Red-Teaming Test	Nemotron- Audio	Jailbreak- AudioBench	Omni- SafetyBench	AdvWave	Avg	Latency (s)
Gemma 3	0.194	0.379	0.427	0.686	0.829	0.462	0.692	0.524	0.823
Qwen3-Omni	0.245	0.607	0.362	0.723	0.682	0.536	0.872	0.575	1.834
Audio Flamingo 3	0.235	0.597	0.103	0.800	0.430	0.550	0.780	0.499	1.042
Gemini 3	0.357	0.749	0.732	0.900	0.862	0.680	0.900	0.740	3.245
GPT-Audio	0.289	0.720	0.400	0.894	0.829	0.652	0.918	0.672	2.542

Table 2: Overall audio safety guardrail accuracy across benchmarks. We evaluate on AudioSafetyBench (speech, non-speech, and ElevenLabs red-teaming split) and four external audio safety benchmarks. A prediction is counted as correct only when both the sound-risk (audio-native cues) and the content-risk (transcript-level safety prediction) match the ground truth. We also report average performance across all benchmarks and end-to-end per-sample latency (seconds).

Panel of Figure 3 — Figure 3: Category-wise guardrail performance for audio-specific risks in AudioSafetyBench. Left: joint sound + content accuracy (higher the better) on representative severe voice–content compositional risks, where a high-risk voice attribute (child voice or celebrity/impersonation) co-occurs with a semantic risk category (e.g., Sexual, Self-Harm, Criminal, Misinformation, Unauthorized Advice, Terrorism); a prediction is correct only if both the voice/audio cue and transcript/content risk match the ground truth. Right: accuracy on representative non-speech harmful sound events (e.g., sexual moans, violence/struggle, breaking/forced entry, human distress, gunfire/explosions, crash sounds); since no transcript is involved, correctness depends only on the predicted audio risk category.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Method	AudioSafetyBench		Nemotron- Audio	Jailbreak- AudioBench	Omni- SafetyBench	AdvWave
Method	Speech	ElevenLabs Red-Teaming Test	Nemotron- Audio	Jailbreak- AudioBench	Omni- SafetyBench	AdvWave
Gemma 3n	0.801	0.828	0.614	0.723	0.960	0.834
Qwen3-Omni	0.824	0.623	0.734	0.623	0.523	0.803
Audio Flamingo 3	0.523	0.345	0.834	0.663	0.563	0.322
Gemini 3	0.934	0.902	0.723	0.902	0.849	1.000
GPT-Audio	0.846	0.876	0.605	0.903	0.916	0.988

Appendix

Table 3: Content guardrail accuracy across benchmarks. We evaluate the content guardrail accuracy (i.e., correctness of the ground-truth safe/unsafe tag for the audio content) of TextGuard in AudioGuard and other baselines. For AudioSafetyBench, we evaluate on the Speech split and the ElevenLabs Red-Teaming split (non-speech slices are excluded since they are not observable from transcripts).

Model	Speech	Non-Speech	ElevenLabs Red-Teaming
Gemma 3n	0.194	0.362	0.427
Qwen3-Omni	0.245	0.320	0.362
Audio Flamingo 3	0.235	0.230	0.103
Gemini 3	0.357	0.573	0.732
GPT-Audio	0.289	0.452	0.400
SoundGuard (Ours)	0.832	0.875	0.790

Appendix

Table 4: Sound guardrail accuracy on AudioSafetyBench. We report the audio-native (waveform-only) guardrail accuracy of SoundGuard across AudioSafetyBench splits: Speech , Non-Speech , and ElevenLabs Red-Teaming .

Language	Base	TextGuard	Language	Base	TextGuard
en	0.657	0.862	nl	0.466	0.618
es	0.523	0.689	cs	0.447	0.595
fr	0.547	0.692	ar	0.431	0.584
de	0.538	0.678	zh-cn	0.462	0.621
it	0.512	0.653	ja	0.428	0.573
pt	0.505	0.641	hu	0.439	0.589

Appendix

Table 5: Multilingual F1 of TextGuard. Per-language F1 on AudioSafetyBench (17 languages), comparing Base Gemma-1B vs. our finetuned TextGuard.

Threshold	Speech	Non-Speech	ElevenLabs RT	Avg
(0.1, 0.1)	0.215	0.532	0.327	0.358
(0.3, 0.3)	0.803	0.863	0.803	0.823
(0.5, 0.5)	0.832	0.876	0.890	0.866
(0.7, 0.7)	0.743	0.852	0.831	0.809
(0.9, 0.9)	0.332	0.629	0.251	0.404

Appendix

Table 6: Threshold sensitivity of end-to-end AudioGuard on AudioSafetyBench. We vary global compositional thresholds

(\tau_{k},\tau_{\ell})

while keeping trained SoundGuard and TextGuard fixed.

SoundGuard backbone	Speech	Non-Speech	ElevenLabs RT	Avg
ECAPA-TDNN + MLP	0.832	0.876	0.890	0.866
WavLM-Large + MLP	0.853	0.863	0.845	0.854

Appendix

Table 7: End-to-end AudioGuard accuracy with different SoundGuard encoders on AudioSafetyBench. ContentGuard and compositional integration remain fixed.

Setting	Speech	ElevenLabs Content Acc.	Avg
w/o TTS $\rightarrow$ ASR augmentation	0.902	0.932	0.917
w/ TTS $\rightarrow$ ASR augmentation	0.936	0.959	0.948

Appendix

Table 8: Content-only accuracy of TextGuard on speech-based AudioSafetyBench splits.

Condition	Gemini 3	GPT-Audio	AudioGuard
Clean	0.357	0.289	0.832
Noise / Overlapping	0.355	0.274	0.793

Appendix

Table 9: Robustness to noisy-overlap corruption on the AudioSafetyBench Speech split. We mix one of 10 background-noise wavs into the input audio with random overlap using ffmpeg .

LLM Safety Benchmarks

Large language model safety benchmarks test whether AI systems avoid harm while remaining helpful across realistic tasks, users, and deployment settings.

See all 284 papers

ARENA: Automated Red-Teaming for Large Audio Language Models
Aug 16, 2026Jiaming He, Zhicong Huang, Tian Jin +5Audio-Language Model Evaluation
Nanyang Technological University, Singapore · Ant Group, China · The Chinese University of Hong Kong, Shenzhen, China +2
Benign Fine-Tuning Breaks Safety Alignment in Audio LLMs
Apr 17, 2026Jaechul Roh, Virat Shejwalkar, Amir HoumansadrLanguage Model Safety Evaluation LLM Fine-Tuning
University of Massachusetts Amherst
Evaluation of Audio Language Models for Fairness, Safety, and Security
Feb 25, 2026Ranya Aloufi, Srishti Gupta, Soumya Shaw +2Audio-Language Model Evaluation Audio-Language Models

Aug 16, 2026J/K move · Enter open · S save

ARENA: Automated Red-Teaming for Large Audio Language Models

Jiaming He, Zhicong Huang, Tian Jin, Zhen Sun+4

Nanyang Technological University, Singapore · Ant Group, China · The Chinese University of Hong Kong, Shenzhen, China+2

Audio-Language Model Evaluation

Open paper AskPDF arXiv

Large audio-language models (LALMs) make it possible to interact with language models through speech, music, and environmental sound, but they also introduce a safety surface that is difficult to expose with text-only red-teaming. We study automated audio-grounded red-teaming, where a text query must remain safe in isolation while the joint text-audio input induces harmful target behavior. We propose ARENA, a closed-loop framework that trains a controller on an independent 2,000case text-audio dataset. MD-Judge supplies training rewards and adaptive search feedback, while a separate, non-adaptive Llama Guard 3 evaluator alone labels final outcomes. On 520 held-out AdvBench objectives, ARENA achieves FDR/PSR of 87.9/100.0%, 71.5/96.3%, 68.1/100.0%, and 75.4/98.5% on Audio Flamingo 3, Qwen2-Audio, MiMo-Audio, and GPTAudio, respectively. Ablations show that feedback-based refinement and audio-variant search substantially improve attack discovery.

Abstract

LLM Safety Benchmarks

Explore similar work

ARENA: Automated Red-Teaming for Large Audio Language Models

Benign Fine-Tuning Breaks Safety Alignment in Audio LLMs

Evaluation of Audio Language Models for Fairness, Safety, and Security