AudioTrace: File-Level Membership Detection for Audio Large Language Models
Authors: Guangzhi Sun, Xiao Zhan, Shuai Wang, Jose Such, Phil Woodland
Organizations: University of Cambridge, Cambridge, United Kingdom · School of Intelligence Science and Technology, Nanjing University, Suzhou, P.R. China · State Key Lab. of Novel Software Technology, Nanjing University, Nanjing, P.R. China · INGENIO (CSIC-Universitat Polit`ecnica de Val`encia), Valencia, Spain
Significant privacy and copyright concerns are raised by the use of publicly available audio recordings to train audio large language models (LLMs), yet effective auditing methods remain limited. Existing membership inference attacks typically assume knowledge of complete training examples, including their associated textual supervision. In practice, however, audio recordings may be paired with privately generated transcripts, captions, or question-answer pairs that are unavailable to owners of the recordings. This motivates audio-file-level membership inference, which determines whether a recording was used for training without assuming knowledge of its training tasks and annotations. This paper provides the first systematic investigation of this challenge and introduces AudioTraceBench, a controlled benchmark comprising 1,285 audio recordings with explicit fine-tuning membership labels. Evaluations across three audio LLMs show that existing text- and video-based membership inference methods adapted to audio perform close to random guessing. To address this challenge, AudioTrace is proposed, where audio-conditioned proxy questions are generated using the target model, and membership evidence is extracted from its responses and aggregated at the recording level. Consistent improvements over existing baselines are observed, with AUROC reaching 0.60 and increasing further under increased fine-tuning exposure or adaptation capacity. Further experiments were conducted to examine the factors influencing membership signals for audio-file-level detection. These findings establish audio-file-level membership inference as a distinct and practically important auditing challenge, with AudioTrace providing an effective initial approach to this challenge.
Figures & tables
Fig. 1: AudioTrace membership inference method. AudioTrace comprises of 3 stages from left to right. It first generates proxy questions from the target audio LLM. Then, for each proxy question, the target LLM generates an answer and the membership signals, such as Min-K% and predictive entropy, are recorded. Finally, these signals are aggregated at audio-file-level using top-k average, producing one single membership score for each audio file.
Fig. 2: Data generation and curation pipeline for AudioTraceBench. We first collect YouTube videos from time-sensitive, information-rich categories and extract their audio tracks. We then filter the recordings based on content recency, information density, and duration. Finally, we divide the filtered recordings into a member set for supervised fine-tuning and a non-member set held out for membership auditing.
Fig. 3: Pie chart showing the distribution of the number of audio files across six different audio categories in AudioTraceBench.
Attack Method
Setting
Qwen2.5-Omni
SALMONN-2
Step-Audio-2-mini
AUROC
TPR@5%FPR
AUROC
TPR@5%FPR
AUROC
TPR@5%FPR
PPL
0.4957
0.0419
0.4970
0.0403
0.5207
0.0636
Entropy
0.4953
0.0419
0.4958
0.0372
0.5200
0.0698
Min-K%
Min-10%
0.4987
0.0279
0.4916
0.0388
0.5141
0.0589
Min-30%
0.4946
0.0481
0.4933
0.0310
0.5173
0.0667
Min-60%
0.4959
0.0459
0.4959
0.0372
0.5204
0.0636
TABLE I: Audio-file-level membership inference performance on AudioTraceBench , where each model is fine-tuned on the member set of AudioTraceBench. We report AUROC and TPR at 5% FPR for all metrics. Audio-SME and VideoMIA-Audio are adaptations of existing video-level MIA methods to audio introduced in § III-C .
Membership score
AUROC
TPR@5%FPR
Combined
0.5515
0.0574
ROUGE-L
0.5253
0.0731
BERT score
0.5441
0.0592
TABLE II: AudioTrace operating under black-box setting.
Method
Accuracy
Pitch and Duration
51.25%
Text Embeddings
52.77%
Acoustic Embeddings
53.32%
TABLE III: Blind baselines based on audio and content features. Binary classifiers are trained on each feature.
Setting
Metric
Qwen2.5-Omni
SALMONN-2
AUROC
TPR@5%FPR
AUROC
TPR@5%FPR
AudioTrace - Known Training Set QA for Member Audio
Original Audio
Min-60%
0.8104
0.4106
0.8216
0.4044
Max-60% Rényi-2
0.8092
0.4121
0.8206
0.3810
Combined
0.8152
0.4401
0.8234
0.4059
Original Audio + 0dB SNR Noise
Min-60%
0.5788
0.0529
0.5850
0.0871
TABLE IV: Comparison of known training set QA annotations against unknown annotation condition for Qwen2.5-Omni and SALMONN-2 models under corruptive noise and audio tokeniser reconstruction perturbations. AudioTrace is used across the table with the standard setting as before.
Model
Source of Query
Similarity
AUROC
Qwen2.5-Omni
self-gen.
0.4109
0.5515
Qwen2.5-Omni
Qwen3-Omni gen.
0.3870
0.5383
SALMONN-2
self-gen.
0.4166
0.5902
SALMONN-2
Qwen3-Omni gen.
0.3870
0.5562
TABLE V: Similarity to the training set questions and influence of the source of queries on auditing performance. The combined score is used to measure AUROC. We adopt the Qwen3-Omni-30BA3B-Instruct model to generate proxy questions as a comparison reference.
Model
M
Max-60% Rényi-2
Combined
Qwen2.5-Omni
3
0.5995
0.6209
5
0.5791
0.5973
10
0.5527
0.5669
20
0.5568
0.5527
30
0.5563
0.5515
SALMONN-2
3
0.6778
0.6822
TABLE VI: AUROC of different models using the M most similar proxy queries to the training set questions.
Fig. 4: Variation of AUROC against the number of proxy questions generated for Qwen2.5-Omni and SALMONN-2 using AudioTrace Combined score. The shaded area represents the single standard deviation range.
Fig. 5: Variation of AUROC and TPR@5%FPR with different training data composition, including QA, CapASR (caption and transcription) or the combination of the two, which is the default setting.
Fig. 6: Ablation studies on training configurations, including the number of repetitions of the samples (represented by the number of epochs), and the number of trainable parameters controlled by the LoRA rank, using (a) SALMONN-2 and (b) Qwen2.5-Omni.
Composition
Qwen2.5-Omni
SALMONN-2
Step-Audio-2-mini
Caption + ASR
0.5276
0.6037
0.5549
QA
0.5450
0.5723
0.5845
All
0.5515
0.5902
0.5952
TABLE VII: Impact of composition of the proxy questions in AudioTrace measured by AUROC across all three models. The combined score is used as the membership score. For caption + ASR, the average of both tasks is used.
Fig. 7: Variations of AUROC against different top- K aggregations on AudioTraceBench on Qwen2.5-Omni and SALMONN-2 using Max-60% Rényi-2 Entropy and the combined score.
Large language model auditing tests models and deployed systems for bias, safety, privacy, reliability, and provenance using behavioral and internal evidence.
We present the first systematic Membership Inference Attack (MIA) evaluation of LALMs. Using Multi-modal Blind Baselines based on textual, spectral and prosodic features, we demonstrate that common audio datasets exhibit near-perfect train/test separability (AUC ~ 1.0) even without model inference, thus MIA may primarily detect distribution shift. We therefore introduce a blind-baseline protocol to control for this confound. Under this protocol, we identify that the distribution-matched datasets enable reliable MIA evaluation without distribution-shift artifacts. We benchmark multiple MIA methods and conduct modality disentanglement experiments on these datasets. The results reveal that LALM memorization is cross-modal, arising only from binding a speaker's vocal identity with its text. These findings establish a principled standard for auditing LALMs beyond spurious correlations. Our codebase is available at https://github.com/snooow1029/ALM_MIA.
Jia-Kai Dong, Yu-Xiang Lin, Hung-Yi Lee
National Taiwan University · NTU Artificial Intelligence Center of Research Excellence
We introduce TRACE (Trustworthy Reasoning from Acoustic Cues and Evidence), a benchmark for diagnosing the reliability of large audio-language models (LALMs) by revealing when, and under what acoustic conditions these models behave untrustworthily. Existing benchmarks primarily measure whether a model produces the correct answer, but correctness alone cannot determine whether the answer is grounded in acoustic evidence or instead reflects language and contextual priors. TRACE evaluates model behavior along three complementary dimensions: (1) task performance, whether the answer is correct; (2) evidence alignment, whether the model is attending to the audio regions required to answer the question; and (3) robustness, whether answers and attributed evidence respond coherently to controlled changes such as event order, overlap, signal-to-noise ratio, or background content. TRACE targets temporal ordering, counting, and event existence, three tasks that remain challenging for current models. We synthesize contextually informed acoustic scenes that allow us to test whether models hallucinate events associated with the context but absent from the audio. This synthesis also allows us to obtain a scalable benchmark for which we know the answers by design. For each scene, we create controlled variants in a programmatic way to stress-test model robustness. We introduce attribution-based metrics that quantify model attention at scale on target events and non-target events. Evaluation across five state-of-the-art models shows that current LALMs often detect where relevant sound evidence occurs, but they do not reliably interpret it, integrate it into the answer, or respond to controlled changes in that evidence. Our benchmark also shows that models still struggle the most with ordering and counting questions.
Giovana Morais, Yujin Kim, Atilay Kucukoglu +1
Music and Audio Research Lab, NYU · Integrated Design and Media, NYU
Membership inference on large language models (LLMs) aims to determine whether a given text sample was included in an LLM's training data, without access to its training corpus. Despite recent progress, existing benchmarks suffer from three limitations: limited coverage of training stages, insufficient distributional alignment between members and non-members, and lack of rigorous filtering of non-members against the training corpus. To address these limitations, we propose OLMo-Detect, a multi-stage, confounder-controlled benchmark built upon the fully open OLMo 2 pipeline. OLMo-Detect spans pre-training, mid-training, and post-training, explicitly aligns members and non-members on three key axes, and rigorously filters non-members via infini-gram. To assess robustness to distribution shifts, we further introduce OLMo-Detect (Shifted), a variant where members are misaligned with non-members. We evaluate 15 unsupervised and 3 supervised membership inference attacks (MIAs) across the OLMo 2 family, finding that: (i) overall performance is limited: the best unsupervised and supervised MIAs both reach an AUC of only 0.68, and supervised MIAs degrade under cross-domain evaluation; (ii) MIA performance peaks at mid-training and is lower at pre-training and post-training, a pattern driven by data type rather than a stage effect: curated math data is far more detectable than other types; (iii) overall scores improve from 1B to 13B but plateau at 32B; and (iv) no unsupervised MIA is robust to distribution shifts, with AUCs shifting by up to 0.42. Finally, we find that our findings on OLMo 2 generalize to OLMo 3 and non-OLMo models.
Tao Shi, Chaoyi Xiang, Qiongkai Xu +1
The University of Melbourne · Macquarie University