Audio-enabled large language models (AuLLMs) allow users to interact with them simply by talking to them, without the need for an intermediate transcription. This expands the model's accessibility and capabilities by encoding non-verbal cues. However, using non-verbal cues may lead to biased behaviour based on the attributes inferred (or assumed) from them. In this work, we focus on the gender attribute and show that models' perception of it can be unlearned using adversarial methods without a loss in performance on other benchmarked tasks. Such models are useful in situations where gender inference is unwanted. However, in other practical settings, gender perception (without bias) may be desired.As a less radical measure, we distil the gender-agnostic model, creating a model that perceives gender but responds similarly to the gender-agnostic teacher, unless explicitly asked.We apply our newly proposed methods to three AuLLMs and obtain satisfactory results in the gender-unlearning part and partially satisfactory results in the distilling part.
Figures & tables
Figure 1: Outline of the two proposed methods: adversarial-only (blue part) and adversarial/distillation pipeline (whole figure).
Figure 2: Fightin’ Words method applied to the open generation adjectives from the Nemotron-3-Nano-Omni model (before any training), sorted by log odds ratio (LOR) .
Figure 3: Training setup for the adversarial stage.
α
LR
γ
steps
acc \male\female
mmau
Qwen2-Audio-7B
81.5
42.1
pitch normalisation to median
50.3
36.9
0.05
1e-5
1.0
500
46.0
46.6
0.05
1e-5
0.5
375
45.2
45.2
0.05
1e-5
1.0
375
52.4
45.1
Qwen3-Omni-30B-A3B
99.9
75.0
Table 1: Top three hyperparameter configurations for each model that achieve gender detection accuracy below 0.6. Pitch normalisation was applied to the original model; the median was 144 Hz. The original performance is next to the model names. Coloured cells are in percentage.
Model
acc abs
acc NR
refuse
%\male
%\female
Qwen2-Audio-7B
base
81.5
90.7
10.1
31.4
58.4
GUM
46.0
64.5
28.8
12.6
58.7
dist.
70.4
87.7
19.7
20.8
59.7
Qwen3-Omni-30B-A3B
base
99.9
99.9
0.0
49.9
50.1
Table 2: Gender detection evaluation results for the model variants. Key: acc abs : absolute gender detection accuracy, acc NR : accuracy on non-refused samples, refuse: refusal rate, %\male / %\female : fraction of samples predicted as male/female. Values in percentages.
Model
bias \male
bias \female
LOR
Qwen2-Audio-7B
base
55.0
55.0
0.40
GUM
56.3
46.3
0.10
dist.
58.4
47.4
0.23
Qwen3-Omni-30B-A3B
base
56.1
49.9
0.24
Table 3: Gender bias evaluation results for the model variants. Key: bias \male /bias \female : percentage of samples where the model chose the stereotypical option for masculine/feminine voices (expected 50% for uniformly random selection), LOR: log odds ratio. Coloured values are in percentages.
Figure 4: Misgendering rate for each model variant and gender.
Figure 5: Average response lengths for each model variant, measured on the misgend dataset. For reference, we also show the sift-50m \neuter length (used to train the GUM models) and each \textscsift−50mM length (used in the distillation).
Figure 6: Gender bias log odds ratio (LOR) distribution for the distilled LoRA checkpoints for Qwen3-Omni .
LR
rank
step
acc \male\female
mmau
LOR
Qwen2-Audio-7B
81.5
42.1
0.40
5e-5
32
720
70.4
41.3
0.23
5e-5
16
640
70.1
41.9
0.24
5e-5
32
800
71.6
41.5
0.27
Qwen3-Omni-30B-A3B
99.9
75.0
0.24
5e-5
16
400
99.9
76.2
0.22
Table 4: Top three distilled LoRA checkpoints per model that satisfy the criteria. The original performance is next to the model names. Sorted by log odds ratio (LOR) , ascending. Coloured cells are in percentage.
Figure 7: Fightin’ Words method applied to the open generation adjectives from the Nemotron-3-Nano-Omni model after the final distilling stage. Stereotypical words are considerably less pronounced, compared to Figure 2 .
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Model
infer
train
Qwen2-Audio-7B-Instruct
48
95
Qwen3-Omni-30B-A3B
95
190
Nemotron-3-Nano-Omni-30B-A3B
95
190*
Appendix
Table 5: GPU memory use for each model. *Requires 2 × 95 GB (not 4 × 48 GB).
Figure 8: Fightin’ Words method applied to the open generation adjectives from the Qwen2-Audio model before any training.
Figure 9: Fightin’ Words method applied to the open generation adjectives from the Qwen2-Audio model after the final distilling stage. Compared to Figure 8 , the words are less stereotypically charged.
Figure 10: Fightin’ Words method applied to the open generation adjectives from the Qwen3-Omni model before any training.
Figure 11: Fightin’ Words method applied to the open generation adjectives from the Qwen3-Omni model after the final distilling stage. The improvement against Figure 10 is minor, but present in the power subplot and in the LOR values on the x axis.