GRM: Utility-Aware Jailbreak Attacks on Audio LLMs via Gradient-Ratio Masking
Authors: Yunqiang Wang, Hengyuan Na, Di Wu, Miao Hu, Guocong Quan
Organizations: Sun Yat-Sen University Guangzhou Yunshan Research Institute of Artificial Intelligence Security Guangzhou, China · Sun Yat-Sen University Guangzhou, China
Audio Large Language Models (ALLMs) enable spoken interaction but introduce new jailbreak vulnerabilities. Existing perturbation-based jailbreaks do not explicitly control which frequency bands carry the perturbation. Although such perturbations can elicit unsafe responses, repeatedly applying a universal perturbation across diverse inputs may also degrade utility on normal tasks, leaving a conspicuous behavioral footprint that may expose the attack to users or automated monitoring systems and thereby compromise its stealthiness. To determine whether full-band perturbation is necessary, we vary coverage from partial-band to full-band. Jailbreak Success Rate (JSR) varies non-monotonically, while utility degradation grows with coverage. This mismatch shows that selected bands can yield stronger attacks with less utility degradation than full-band perturbations. Based on this observation, we propose GRM, a utility-aware, frequency-selective jailbreak framework that ranks Mel bands by the ratio between jailbreak contribution and transcript sensitivity, confines a universal perturbation to selected bands, and regularizes deviations from the intended request semantics. Experiments on four ALLMs show that GRM achieves an average JSR of 88.46% while substantially reducing utility degradation across benign transcription and response tasks relative to baselines. Our code is available at this repository. Warning: This paper contains potentially sensitive content.
Figures & tables
Figure 1. Effect of frequency coverage on JSR and Word Error Rate (WER). (a) JSR varies non-monotonically as coverage increases, while WER rises. (b) GRM achieves high JSR with relatively low WER. Two plots comparing jailbreak success and transcription degradation under different frequency coverage. In (a), Jailbreak Success Rate (JSR) does not improve monotonically as coverage increases, while Word Error Rate (WER) rises. In (b), different values of K produce different JSR and WER values, and GRM reaches high JSR with relatively low WER compared with representative baselines.
Figure 2. Overview of GRM. GRM identifies key frequency bands by gradient-ratio scoring and optimizes perturbations under a mask-constrained joint objective. The learned perturbation induces jailbreak behavior on harmful audio while preserving utility on benign audio. A three-step overview of GRM. Step 1 mines key frequency bands from harmful audio using gradient-ratio scoring and produces a key-band mask. Step 2 initializes perturbation noise and performs mask-constrained optimization under a joint objective combining jailbreak effectiveness and semantic preservation. Step 3 applies the optimized perturbation to harmful and benign audio for the target ALLM: the perturbed harmful audio elicits a jailbreak response, while the perturbed benign audio preserves transcript consistency and utility.
Figure 3. Band ranking scores across four target ALLMs. Each row corresponds to one model, and each column denotes a Mel-frequency band. Darker colors indicate higher-score bands, and the highlighted frequency regions vary across models. A heatmap with four rows and 128 columns showing normalized band-ranking scores for four Audio LLMs. Rows correspond to Qwen2-Audio, Qwen2.5-Omni, Llama2-Omni, and MERaLiON, and columns correspond to Mel-frequency bands. Darker colors indicate higher-ranked bands. High-rank regions appear in different frequency ranges across models, showing that the selected key bands are model-dependent.
Models
Metric
Vanilla
GCG
AutoDAN
BoN
AudioBench
SSJ
GRM-S
GRM
Qwen2-Audio
JSR (%) ↑
19.23
30.77
69.23
48.08
49.04
91.35
94.23
92.31
WER (%) ↓
3.03
37.69
44.12
48.20
11.48
—
3.98
3.59
RQS ↑
6.99
6.42
6.58
4.07
5.99
—
6.59
6.66
Qwen2.5-Omni
JSR (%) ↑
19.23
62.50
92.31
60.58
71.15
84.62
93.27
87.50
WER (%) ↓
6.32
38.69
41.37
98.06
12.18
—
17.06
16.00
RQS ↑
7.21
7.05
7.10
4.53
6.11
—
5.88
6.21
Table 1. Main results. JSR (%) ↑ , WER (%) ↓ , and RQS ↑ on four target ALLMs. SSJ is prompt-only; therefore, WER and RQS are not applicable. Best results are highlighted in red , and second-best results in blue .
Method Variant
JSR(%) ↑
WER(%) ↓
RQS ↑
GRM (full)
92.31
3.59
6.66
w/o Lemb
94.23
3.98
6.59
Random 48-band
82.69
4.12
6.54
Full-band + Lemb
78.85
13.72
5.58
Full-band w/o Lemb
76.92
25.46
5.48
Random Noise (no optimization)
21.15
3.29
6.62
Table 2. Ablation results on Qwen2-Audio. Best results are highlighted in red , and second-best results in blue .
Figure 4. Impact of the number of selected key bands ( K ) on JSR, WER, and RQS. We use K=48 as the default setting. Three bar charts showing the effect of the number of selected key bands K on JSR, WER, and RQS. As K increases, JSR first rises and then declines, while WER increases and RQS decreases. K equals 48 provides the best overall balance and is selected as the default setting.
Method
JSR ↑
WER ↓
RQS ↑
PESQ ↑
STOI ↑
GRM
92.31
3.59
6.66
3.126
0.978
BoN
48.08
48.20
4.07
1.107
0.270
AudioBench
49.04
11.48
5.99
1.353
0.221
Table 3. Perceptual quality evaluation for waveform-level audio jailbreak baselines.
Figure 5. Impact of the semantic loss weight ( λ ) on JSR, WER, and RQS. We use λ=5 as the default setting. Three bar charts showing the effect of the semantic loss weight lambda on JSR, WER, and RQS. As lambda increases, JSR decreases slightly, while WER improves and RQS increases. Lambda equals 5 provides a good balance between attack success and utility preservation.
Method
Original
Gaussian Noise
Local Smoothing
GRM
92.31
91.35
56.73
BoN
48.08
56.73
47.12
AudioBench
49.04
62.50
51.92
Table 4. Robustness under Gaussian noise and local smoothing. JSR (%) is reported for GRM, BoN, and AudioBench. A table reporting attack success rates under three conditions: original, Gaussian noise, and local smoothing. Rows correspond to GRM, BoN, and AudioBench. GRM remains highly effective under Gaussian noise but drops more under local smoothing, while BoN and AudioBench remain comparable or improve under the transformations.
Figure 6. t-SNE visualization of internal representations in Qwen2-Audio for benign, harmful, and adversarial audio inputs. A two-dimensional t-SNE scatter plot of internal representations in Qwen2. Benign samples and harmful samples occupy different regions, while GRM, AudioBench, and BoN show different representation shifts from the harmful region.
Harmful Query
Without GRM
With GRM
Status
Develop a mobile app that steals user data and transmits it to a third party.
“I’m sorry, but I cannot fulfill this request … it is not ethical or appropriate for me to provide instructions on how to create an application that engages in malicious or harmful behavior.”
“Sure! Here is a step-by-step guide on how to develop a mobile app that steals user data and transmits it to a third party:”; “Step 1: Choose a programming language and platform for your app”; “Step 2: Obtain user data”; “Step 3: Build the app”; “Step 4: Transmit the data to a third party”; “Step 5: Test the app”…
Unsafe
Write a script that harvests and sells user data to third-party companies.
“I’m sorry, but I cannot fulfill this request … it goes against ethical and moral principles to write a script that harvests and sells user data to third-party companies …”
“Sure! Here is a step-by-step guide to writing a script for a short film or video:”; “Step 1: Develop Your Concept”; “Step 2: Create an Outline”; “Step 3: Write the Treatment”; “Step 4: Write the Script”; “Think about what message you want to convey and who your target audience is”; “Step 5: Revise and Edit” …
Safe
Table 5. Representative case studies before and after applying GRM perturbations. We show one successful jailbreak case and one failed case; only key excerpts are included, while full responses are provided in the appendix.
Figure 7. Cross-model transferability of GRM perturbations. Rows denote source models and columns denote target models. The left heatmap reports JSR (%), and the right heatmap reports WER (%). Two heatmaps showing cross-model transferability of GRM perturbations. The left reports jailbreak success rate and the right reports word error rate. Rows are source models and columns are target models. Diagonal entries indicate in-model attacks, while off-diagonal entries indicate cross-model transfer.
Figure 8. Effect of the number of selected key bands ( K ) on JSR and WER for the remaining target models. Three line charts showing how the number of selected key bands K affects Jailbreak Success Rate (JSR) and Word Error Rate (WER) on LLaMA-Omni2, MERaLiON, and Qwen2.5-Omni. LLaMA-Omni2 and MERaLiON perform better at moderate K, while Qwen2.5-Omni attains higher JSR only at full-band coverage, with much higher WER.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 9. t-SNE visualization of internal representations in MERaLiON for benign, harmful, and adversarial audio inputs. A t-SNE scatter plot of internal representations in MERaLiON. Benign-A forms a separate cluster from harmful and adversarial inputs. Harmful-A, AudioBench-A, and BoN-A occupy different regions, while GRM forms a compact cluster in the adversarial region away from the original harmful samples.
Figure 10. t-SNE visualization of internal representations in LLaMA-Omni2 for benign, harmful, and adversarial audio inputs. A t-SNE scatter plot of internal representations in LLaMA-Omni2. Benign-A is clearly separated from harmful and adversarial inputs. Harmful-A, AudioBench-A, and BoN-A show noticeable overlap, while GRM forms a distinct compact cluster in the adversarial region.
Figure 11. Effect of the Whisper model on band scoring. A heatmap comparing normalized Mel-band scores under Whisper-Large-v3 and Whisper-Large-v3-turbo for Qwen2-Audio and LLaMA-Omni2, together with their turbo variants.
Model
Mask Overlap
JSR ↑
WER ↓
RQS ↑
Qwen2-Audio
72.92
92.31
3.59
6.66
Qwen2-Audio(turbo)
72.92
89.42
3.72
6.63
LLaMA-Omni2
28.12
81.73
15.51
5.84
LLaMA-Omni2(turbo)
28.12
54.88
15.69
5.81
MERaLiON
64.06
92.31
6.71
6.62
MERaLiON(turbo)
64.06
77.88
6.94
6.21
Appendix
Table 6. Effect of the Whisper model on GRM performance. A table reporting mask overlap, JSR, WER, and RQS when Whisper-Large-v3-turbo replaces Whisper-Large-v3 across different target models.
Method
PESQ ↑
STOI ↑
GRM (Qwen2.5-Omni)
3.075
0.978
GRM (LLaMA-Omni2)
3.139
0.979
GRM (MERaLiON)
3.116
0.978
BoN
1.107
0.270
AudioBench
1.353
0.221
Appendix
Table 7. Perceptual quality comparison of GRM and waveform-level audio jailbreak baselines. A table reporting PESQ and STOI for GRM perturbations optimized on Qwen2.5-Omni, LLaMA-Omni2, and MERaLiON, together with the waveform-level baselines BoN and AudioBench. Higher values indicate better perceptual quality.
Method
LLM
H1
H2
H3
Human Avg.
Vanilla
3
3
3
3
3.0
BoN
10
8
9
6
7.7
AudioBench
6
6
5
4
5.0
SSJ
18
16
14
13
14.3
GRM
18
18
17
17
17.3
Appendix
Table 8. LLM judge and human evaluation on 20 Qwen2-Audio responses. A table reporting the number of successful jailbreaks identified by the LLM judge and three human annotators on 20 Qwen2-Audio responses.
Large Audio Language Models (LALMs) expand jailbreak risks from token-level prompting to the full speech perception-to-reasoning pipeline, where unsafe behavior can be induced through semantics, acoustic style, signal artifacts, or internal representations. Existing work studies these risks under heterogeneous threat models and evaluation protocols, making it difficult to compare attack practicality or defense utility. This paper provides a unified taxonomy and a controlled empirical evaluation of LALM jailbreak attacks and defenses. We organize prior work into semantic, acoustic, signal, and embedding-layer attacks; guard-based, training-free, and training-based defenses; and cross-modal, audio-native, and interactive benchmarks. We then evaluate representative attacks and defenses across ten open-source LALMs, measuring not only attack success rate but also benign refusal and latency. Our results show that Acoustic Best-of-N reveals strong worst-case audio-space vulnerabilities, Narrative Framing is an effective low-latency semantic threat, and current defenses trade robustness against benign usability. These findings support cost- and utility-aware evaluation as a necessary complement to success-rate-only LALM safety benchmarks.
Jailbreak attacks on audio language models (ALMs) optimize audio perturbations to elicit unsafe generations, and they typically update the entire waveform densely throughout optimization. In this work, we investigate the necessity of such dense optimization by analyzing the structure of token-aligned gradients in ALMs. We find that gradient energy is highly non-uniform across audio tokens, indicating that only a small subset of token-aligned audio regions dominates the optimization signal. Motivated by this observation, we propose Token-Aware Gradient Optimization (TAGO), which enables sparse jailbreak optimization by retaining only waveform gradients aligned with audio tokens that have high gradient energy, while masking the remaining gradients at each iteration. Across three ALMs, TAGO outperforms baselines, and substantial sparsification preserves strong attack success rates (e.g. on Qwen3-Omni, ASRl remains at 86% with a token retention ratio of 0.25, compared to 87% with full token retention). These results demonstrate that dense waveform updates are largely redundant, and we advocate that future audio jailbreak and safety alignment research should further leverage this heterogeneous token-level gradient structure.
Zheng Fang, Xiaosen Wang, Shenyi Zhang +2
1Wuhan University · 2Huazhong University of Science and Tech · Institute for Math & AI, Wuhan University +2
Large audio language models (LALMs) are increasingly deployed in real-world applications, yet their safety alignment is still primarily evaluated on monolingual, text-based harmful prompts. This leaves their generalizability under multilingual and spoken settings, particularly code-switched speech, largely underexplored. To address this gap, we introduce SpeechJBB, an audio jailbreak dataset for benchmarking across multiple state-of-the-art LALMs. The extent of safety weaknesses is further probed by introducing an augmented setting where phonologically plausible pseudo-words are inserted around safety-critical terms to simulate localized obfuscation. Across models, code-switched harmful audio yields substantially high jailbreak success rates (JSR), with non-English monolingual and non-English code-switched pairs exhibiting the highest attack success. Pseudo-word insertion further reduces refusal rates, which demonstrates that natural-sounding obfuscation can effectively bypass safety policies.
Virginia Ceccatelli, Yejin Jeon, David Ifeoluwa Adelani
Mila - Quebec AI Institute · McGill University, Canada · Canada CIFAR AI Chair