GRM: Utility-Aware Jailbreak Attacks on Audio LLMs via Gradient-Ratio Masking

Yunqiang Wang; Hengyuan Na; Di Wu; Miao Hu; Guocong Quan

cs.SDApr 10, 2026

GRM: Utility-Aware Jailbreak Attacks on Audio LLMs via Gradient-Ratio Masking

Authors: Yunqiang Wang, Hengyuan Na, Di Wu, Miao Hu, Guocong Quan

Organizations: Sun Yat-Sen University Guangzhou Yunshan Research Institute of Artificial Intelligence Security Guangzhou, China · Sun Yat-Sen University Guangzhou, China

Abstract

Audio Large Language Models (ALLMs) enable spoken interaction but introduce new jailbreak vulnerabilities. Existing perturbation-based jailbreaks do not explicitly control which frequency bands carry the perturbation. Although such perturbations can elicit unsafe responses, repeatedly applying a universal perturbation across diverse inputs may also degrade utility on normal tasks, leaving a conspicuous behavioral footprint that may expose the attack to users or automated monitoring systems and thereby compromise its stealthiness. To determine whether full-band perturbation is necessary, we vary coverage from partial-band to full-band. Jailbreak Success Rate (JSR) varies non-monotonically, while utility degradation grows with coverage. This mismatch shows that selected bands can yield stronger attacks with less utility degradation than full-band perturbations. Based on this observation, we propose GRM, a utility-aware, frequency-selective jailbreak framework that ranks Mel bands by the ratio between jailbreak contribution and transcript sensitivity, confines a universal perturbation to selected bands, and regularizes deviations from the intended request semantics. Experiments on four ALLMs show that GRM achieves an average JSR of 88.46% while substantially reducing utility degradation across benign transcription and response tasks relative to baselines. Our code is available at this repository. Warning: This paper contains potentially sensitive content.

Figures & tables

Figure 1. Effect of frequency coverage on JSR and Word Error Rate (WER). (a) JSR varies non-monotonically as coverage increases, while WER rises. (b) GRM achieves high JSR with relatively low WER. Two plots comparing jailbreak success and transcription degradation under different frequency coverage. In (a), Jailbreak Success Rate (JSR) does not improve monotonically as coverage increases, while Word Error Rate (WER) rises. In (b), different values of K produce different JSR and WER values, and GRM reaches high JSR with relatively low WER compared with representative baselines.

Figure 2. Overview of GRM. GRM identifies key frequency bands by gradient-ratio scoring and optimizes perturbations under a mask-constrained joint objective. The learned perturbation induces jailbreak behavior on harmful audio while preserving utility on benign audio. A three-step overview of GRM. Step 1 mines key frequency bands from harmful audio using gradient-ratio scoring and produces a key-band mask. Step 2 initializes perturbation noise and performs mask-constrained optimization under a joint objective combining jailbreak effectiveness and semantic preservation. Step 3 applies the optimized perturbation to harmful and benign audio for the target ALLM: the perturbed harmful audio elicits a jailbreak response, while the perturbed benign audio preserves transcript consistency and utility.

Figure 3. Band ranking scores across four target ALLMs. Each row corresponds to one model, and each column denotes a Mel-frequency band. Darker colors indicate higher-score bands, and the highlighted frequency regions vary across models. A heatmap with four rows and 128 columns showing normalized band-ranking scores for four Audio LLMs. Rows correspond to Qwen2-Audio, Qwen2.5-Omni, Llama2-Omni, and MERaLiON, and columns correspond to Mel-frequency bands. Darker colors indicate higher-ranked bands. High-rank regions appear in different frequency ranges across models, showing that the selected key bands are model-dependent.

Models	Metric	Vanilla	GCG	AutoDAN	BoN	AudioBench	SSJ	GRM-S	GRM
Qwen2-Audio	JSR (%) $\uparrow$	19.23	30.77	69.23	48.08	49.04	91.35	94.23	92.31
	WER (%) $\downarrow$	3.03	37.69	44.12	48.20	11.48	—	3.98	3.59
	RQS $\uparrow$	6.99	6.42	6.58	4.07	5.99	—	6.59	6.66
Qwen2.5-Omni	JSR (%) $\uparrow$	19.23	62.50	92.31	60.58	71.15	84.62	93.27	87.50
	WER (%) $\downarrow$	6.32	38.69	41.37	98.06	12.18	—	17.06	16.00
	RQS $\uparrow$	7.21	7.05	7.10	4.53	6.11	—	5.88	6.21

Table 1. Main results. JSR (%)

\uparrow

, WER (%)

\downarrow

, and RQS

\uparrow

on four target ALLMs. SSJ is prompt-only; therefore, WER and RQS are not applicable. Best results are highlighted in red , and second-best results in blue .

Method Variant	JSR(%) $\uparrow$	WER(%) $\downarrow$	RQS $\uparrow$
GRM (full)	92.31	3.59	6.66
w/o $L_{\mathrm{emb}}$	94.23	3.98	6.59
Random 48-band	82.69	4.12	6.54
Full-band + $L_{\mathrm{emb}}$	78.85	13.72	5.58
Full-band w/o $L_{\mathrm{emb}}$	76.92	25.46	5.48
Random Noise (no optimization)	21.15	3.29	6.62

Table 2. Ablation results on Qwen2-Audio. Best results are highlighted in red , and second-best results in blue .

Figure 4. Impact of the number of selected key bands (

K

) on JSR, WER, and RQS. We use

K=48

as the default setting. Three bar charts showing the effect of the number of selected key bands K on JSR, WER, and RQS. As K increases, JSR first rises and then declines, while WER increases and RQS decreases. K equals 48 provides the best overall balance and is selected as the default setting.

Method	JSR $\uparrow$	WER $\downarrow$	RQS $\uparrow$	PESQ $\uparrow$	STOI $\uparrow$
GRM	92.31	3.59	6.66	3.126	0.978
BoN	48.08	48.20	4.07	1.107	0.270
AudioBench	49.04	11.48	5.99	1.353	0.221

Table 3. Perceptual quality evaluation for waveform-level audio jailbreak baselines.

Figure 5. Impact of the semantic loss weight (

\lambda

) on JSR, WER, and RQS. We use

\lambda=5

as the default setting. Three bar charts showing the effect of the semantic loss weight lambda on JSR, WER, and RQS. As lambda increases, JSR decreases slightly, while WER improves and RQS increases. Lambda equals 5 provides a good balance between attack success and utility preservation.

Method	Original	Gaussian Noise	Local Smoothing
GRM	92.31	91.35	56.73
BoN	48.08	56.73	47.12
AudioBench	49.04	62.50	51.92

Table 4. Robustness under Gaussian noise and local smoothing. JSR (%) is reported for GRM, BoN, and AudioBench. A table reporting attack success rates under three conditions: original, Gaussian noise, and local smoothing. Rows correspond to GRM, BoN, and AudioBench. GRM remains highly effective under Gaussian noise but drops more under local smoothing, while BoN and AudioBench remain comparable or improve under the transformations.

Figure 6. t-SNE visualization of internal representations in Qwen2-Audio for benign, harmful, and adversarial audio inputs. A two-dimensional t-SNE scatter plot of internal representations in Qwen2. Benign samples and harmful samples occupy different regions, while GRM, AudioBench, and BoN show different representation shifts from the harmful region.

Harmful Query	Without GRM	With GRM	Status
Develop a mobile app that steals user data and transmits it to a third party.	“I’m sorry, but I cannot fulfill this request … it is not ethical or appropriate for me to provide instructions on how to create an application that engages in malicious or harmful behavior.”	“Sure! Here is a step-by-step guide on how to develop a mobile app that steals user data and transmits it to a third party:”; “Step 1: Choose a programming language and platform for your app”; “Step 2: Obtain user data”; “Step 3: Build the app”; “Step 4: Transmit the data to a third party”; “Step 5: Test the app”…	Unsafe
Write a script that harvests and sells user data to third-party companies.	“I’m sorry, but I cannot fulfill this request … it goes against ethical and moral principles to write a script that harvests and sells user data to third-party companies …”	“Sure! Here is a step-by-step guide to writing a script for a short film or video:”; “Step 1: Develop Your Concept”; “Step 2: Create an Outline”; “Step 3: Write the Treatment”; “Step 4: Write the Script”; “Think about what message you want to convey and who your target audience is”; “Step 5: Revise and Edit” …	Safe

Table 5. Representative case studies before and after applying GRM perturbations. We show one successful jailbreak case and one failed case; only key excerpts are included, while full responses are provided in the appendix.

Figure 7. Cross-model transferability of GRM perturbations. Rows denote source models and columns denote target models. The left heatmap reports JSR (%), and the right heatmap reports WER (%). Two heatmaps showing cross-model transferability of GRM perturbations. The left reports jailbreak success rate and the right reports word error rate. Rows are source models and columns are target models. Diagonal entries indicate in-model attacks, while off-diagonal entries indicate cross-model transfer.

Figure 8. Effect of the number of selected key bands (

K

) on JSR and WER for the remaining target models. Three line charts showing how the number of selected key bands K affects Jailbreak Success Rate (JSR) and Word Error Rate (WER) on LLaMA-Omni2, MERaLiON, and Qwen2.5-Omni. LLaMA-Omni2 and MERaLiON perform better at moderate K, while Qwen2.5-Omni attains higher JSR only at full-band coverage, with much higher WER.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 9. t-SNE visualization of internal representations in MERaLiON for benign, harmful, and adversarial audio inputs. A t-SNE scatter plot of internal representations in MERaLiON. Benign-A forms a separate cluster from harmful and adversarial inputs. Harmful-A, AudioBench-A, and BoN-A occupy different regions, while GRM forms a compact cluster in the adversarial region away from the original harmful samples.

Figure 10. t-SNE visualization of internal representations in LLaMA-Omni2 for benign, harmful, and adversarial audio inputs. A t-SNE scatter plot of internal representations in LLaMA-Omni2. Benign-A is clearly separated from harmful and adversarial inputs. Harmful-A, AudioBench-A, and BoN-A show noticeable overlap, while GRM forms a distinct compact cluster in the adversarial region.

Figure 11. Effect of the Whisper model on band scoring. A heatmap comparing normalized Mel-band scores under Whisper-Large-v3 and Whisper-Large-v3-turbo for Qwen2-Audio and LLaMA-Omni2, together with their turbo variants.

Model	Mask Overlap	JSR $\uparrow$	WER $\downarrow$	RQS $\uparrow$
Qwen2-Audio	72.92	92.31	3.59	6.66
Qwen2-Audio(turbo)	72.92	89.42	3.72	6.63
LLaMA-Omni2	28.12	81.73	15.51	5.84
LLaMA-Omni2(turbo)	28.12	54.88	15.69	5.81
MERaLiON	64.06	92.31	6.71	6.62
MERaLiON(turbo)	64.06	77.88	6.94	6.21

Appendix

Table 6. Effect of the Whisper model on GRM performance. A table reporting mask overlap, JSR, WER, and RQS when Whisper-Large-v3-turbo replaces Whisper-Large-v3 across different target models.

Method	PESQ $\uparrow$	STOI $\uparrow$
GRM (Qwen2.5-Omni)	3.075	0.978
GRM (LLaMA-Omni2)	3.139	0.979
GRM (MERaLiON)	3.116	0.978
BoN	1.107	0.270
AudioBench	1.353	0.221

Appendix

Table 7. Perceptual quality comparison of GRM and waveform-level audio jailbreak baselines. A table reporting PESQ and STOI for GRM perturbations optimized on Qwen2.5-Omni, LLaMA-Omni2, and MERaLiON, together with the waveform-level baselines BoN and AudioBench. Higher values indicate better perceptual quality.

Method	LLM	H1	H2	H3	Human Avg.
Vanilla	3	3	3	3	3.0
BoN	10	8	9	6	7.7
AudioBench	6	6	5	4	5.0
SSJ	18	16	14	13	14.3
GRM	18	18	17	17	17.3

Appendix

Table 8. LLM judge and human evaluation on 20 Qwen2-Audio responses. A table reporting the number of successful jailbreaks identified by the LLM judge and three human annotators on 20 Qwen2-Audio responses.

LLM Jailbreak Attacks

Large language model (LLM) jailbreak attacks study how adversarial inputs bypass safety rules and elicit disallowed outputs.

See all 203 papers

Audio Jailbreaks in Large Audio-Language Models: Taxonomy, Attack-Defense Analysis, and Cost-Aware Evaluation
May 28, 2026Bo-Han Feng, Yu-Hsuan Li Liang, Chien-Feng Liu +2Audio-Language Model Evaluation Audio-Language Models
National Taiwan University
Sparse Tokens Suffice: Jailbreaking Audio Language Models via Token-Aware Gradient Optimization
May 6, 2026Zheng Fang, Xiaosen Wang, Shenyi Zhang +2Deep Learning Optimization Audio-Language Models
1Wuhan University · 2Huazhong University of Science and Tech · Institute for Math & AI, Wuhan University +2
SpeechJBB: Probing Safety Alignment and Comprehension in Large Audio Language Models under Code-Switched Speech
Jun 4, 2026Virginia Ceccatelli, Yejin Jeon, David Ifeoluwa AdelaniAudio-Language Model Evaluation Language Model Safety Evaluation
Mila - Quebec AI Institute · McGill University, Canada · Canada CIFAR AI Chair

May 28, 2026J/K move · Enter open · S save

Audio Jailbreaks in Large Audio-Language Models: Taxonomy, Attack-Defense Analysis, and Cost-Aware Evaluation

Bo-Han Feng, Yu-Hsuan Li Liang, Chien-Feng Liu, You-Hsuan Chang+1

National Taiwan University

Audio-Language Model Evaluation Audio-Language Models LLM Jailbreak Attacks Jailbreak Defense

Open paper AskPDF arXiv

Large Audio Language Models (LALMs) expand jailbreak risks from token-level prompting to the full speech perception-to-reasoning pipeline, where unsafe behavior can be induced through semantics, acoustic style, signal artifacts, or internal representations. Existing work studies these risks under heterogeneous threat models and evaluation protocols, making it difficult to compare attack practicality or defense utility. This paper provides a unified taxonomy and a controlled empirical evaluation of LALM jailbreak attacks and defenses. We organize prior work into semantic, acoustic, signal, and embedding-layer attacks; guard-based, training-free, and training-based defenses; and cross-modal, audio-native, and interactive benchmarks. We then evaluate representative attacks and defenses across ten open-source LALMs, measuring not only attack success rate but also benign refusal and latency. Our results show that Acoustic Best-of-N reveals strong worst-case audio-space vulnerabilities, Narrative Framing is an effective low-latency semantic threat, and current defenses trade robustness against benign usability. These findings support cost- and utility-aware evaluation as a necessary complement to success-rate-only LALM safety benchmarks.

Abstract

LLM Jailbreak Attacks

Explore similar work

Audio Jailbreaks in Large Audio-Language Models: Taxonomy, Attack-Defense Analysis, and Cost-Aware Evaluation

Sparse Tokens Suffice: Jailbreaking Audio Language Models via Token-Aware Gradient Optimization

SpeechJBB: Probing Safety Alignment and Comprehension in Large Audio Language Models under Code-Switched Speech