cs.SDApr 14, 2026
SaveSpotSound: Enhancing Large Audio-Language Models with Fine-Grained Temporal Grounding
Organizations: Zhejiang University Hangzhou, Zhejiang, China · SAI, Shanghai Jiao Tong University Shanghai, China
Abstract
Large Audio-Language Models (ALMs) have recently demonstrated remarkable capabilities in holistic audio understanding, yet they remain unreliable for temporal grounding, i.e., the task of pinpointing exactly when an event occurs within long-form audio. This limitation stems from two factors: training data dominated by clip-level supervision lacking precise timestamps, and benchmarks that fail to simulate real-world scenarios where short events are obscured by dense background sounds. In this paper, we introduce SpotSound, an audio language model designed for grounding audio events. SpotSound incorporates a novel training objective, specifically designed to suppress hallucinated timestamps for events absent from the input. Additionally, we present SpotSound-Bench, a challenging temporal grounding benchmark where target events occupy less than ~10% of each clip, creating a rigorous 'needle-in-a-haystack' evaluation. Experiments demonstrate that SpotSound achieves state-of-the-art results on temporal grounding benchmarks while maintaining robust performance across general downstream audio-language tasks. Code, models and benchmark are released on https://loiesun.github.io/spotsound/
Figures & tables
Figure 1. Qualitative examples and performance comparison. (a) SpotSound accurately grounds timestamps relevant to the query. (b) SpotSound identifies sound events described in the query that are absent from the audio. (c) Quantitative comparison; blue denotes SpotSound , and red denotes previous top-tier models. We evaluate models on four benchmarks and report the mIoU and [email protected] for each.
Figure 2. Model architecture and dataset generation pipeline. (a) In SpotSound , we construct an interleaved sequence of timestamps and audio tokens and concatenate with a language query, and feed it to the LLM to predict the target temporal interval. (b) We employ LLMs to generate captions for foreground audio, then randomly mix foreground and background sounds to synthesise the final audio, preserving the insertion timestamp as ground-truth.
| Datasets | A_num | Q_num | Anno. | DUR. | Q_type |
| AudioGrounding | 3,770 | 8,935 | M. | 10s | Cap. |
| Clotho-Moment | 32,694 | 32,694 | A. | 60s | Cap. |
| UnAV-100 | 5,686 | 9,115 | M. | 43s | Label |
| ASSL | 5,000 | 16,896 | M. | 10s | Label |
| Ours | 10,000 | 10,000 | A. | 50s | Cap. |
Table 1. Statistics of the training set. ‘A_num’ and ‘Q_num’ denote the total number of audio clips and queries, respectively. ‘Anno’, ‘DUR’, and ‘Q_type’ represent the annotation method, average duration, and query type. Under annotation, ‘M.’ indicates manual annotation, while ‘A.’ indicates automatic annotation. Under query type, ‘Cap.’ indicates caption.
| Models | A_num | Q_num | DUR. | W_len | Q_type |
| AudioGrounding | 70 | 100 | 10s | 2.6s | Cap. |
| Clotho-Moment | 6,649 | 6,649 | 60s | 19.6s | Cap. |
| UnAV-100 subset | 492 | 997 | 42.4s | 14.6s | Cap. |
| SpotSound-Bench | 400 | 400 | 54.2s | 3.9s | Label |
Table 2. Benchmark statistics. ‘A_num’ and ‘Q_num’ denote the total number of audio clips and queries, respectively. ‘DUR’, ‘W_len’ and ‘Q_type’ represent the average duration, average window length and query type. Under query type, ‘Cap.’ indicates caption.
| Models | Clotho-Moment | UnAV-100 subset | SpotSound-Bench | AudioGrounding | ||||||||
| [email protected] | [email protected] | mIoU | [email protected] | [email protected] | mIoU | [email protected] | [email protected] | mIoU | [email protected] | [email protected] | mIoU | |
| Non-LLM Models | ||||||||||||
| WTATG ( Xu et al., 2024 ) | 12.1 | 6.3 | 9.1 | 53.0 | 37.0 | 38.4 | 53.0 | 34.5 | 38.4 | 72.5 | 53.7 | 51.4 |
| AM-DETR ( Munakata et al., 2025 ) | 89.8 | 88.0 | 80.9 | 59.0 | 46.0 | 42.8 | 25.5 | 15.0 | 19.5 | 52.5 | 15.6 | 30.2 |
| Proprietary Models | ||||||||||||
| Gemini-2.5-Flash ( Comanici et al., 2025 ) | 45.7 | 33.7 | 36.9 | 48.0 | 37.0 | 35.6 | 34.8 | 27.8 | 25.7 | 51.8 | 36.2 | 37.1 |
Table 3. Audio temporal grounding results. mIoU represents the mean IoU. [email protected] and [email protected] denote Recall@1 across IoU thresholds of 0.3 and 0.5, respectively. SpotSound-Q and SpotSound-A denotes the integration of the SpotSound method with Qwen2 Audio and Audio Flamingo 3, respectively. Best results are in bold, and second-best are underlined .
Figure 3. Qualitative comparison with other large audio-language models. (a) Example cases from the SpotSound-Bench , illustrating the temporal windows predicted by each model in response to the textual query. (b) Examples drawn from the Clotho-Moment, demonstrating cases where the models identify the non-existent sound events.
| Models | Clotho-Moment | AudioGrounding | ||
| Pos. | Neg. | Pos. | Neg. | |
| Kimi-Audio | 54.8 | 65.2 | 54.5 | 60.7 |
| TimeAudio | / | / | / | / |
| Qwen2-Audio | 72.2 | 43.1 | 57.6 | 55.1 |
| SpotSound-Q | 87.6 | 85.8 | 93.2 | 80.7 |
| Audio Flamingo 3 | 65.6 | 70.3 | 89.1 | 76.0 |
Table 4. Hallucinations for non-existent events evaluated in accuracy. ‘Pos.’ and ‘Neg.’ refer to the model prediction accuracy of positive and negative queries, respectively. ‘/’ denotes model hallucination, indicating the model is unable to determine whether the events in the query are present in the audio. Best results are in bold.
| Models | Clotho. | UnAV. | Spot. | Audio. |
| Kimi-Audio | 0.5 | 4.2 | 1.6 | 3.4 |
| TimeAudio | / | / | / | / |
| Qwen2-Audio | 5.4 | 11.1 | 3.4 | 41.6 |
| SpotSound-Q | 92.0 | 89.7 | 72.4 | 81.6 |
| Audio Flamingo 3 | 30.4 | 42.4 | 21.2 | 69.2 |
| SpotSound-A | 91.4 | 84.9 | 82.9 | 85.6 |
Table 5. Two-stage joint assessment results evaluated in F1-score. ‘Clotho.’, ‘UnAV.’, ‘Spot.’ and ‘Audio.’ refer to Clotho-Moment, Unav-100 subset, SpotSound-Bench and AudioGrounding, respectively. Best results are in bold.
| Models | TUT-Sound | DESED | ||
| [email protected] | mIoU | [email protected] | mIoU | |
| Kimi-Audio | 0.3 | 1.5 | 4.0 | 4.1 |
| TimeAudio | 18.0 | 22.5 | 4.0 | 19.3 |
| Qwen2-Audio | 0.7 | 2.6 | 28.2 | 33.8 |
| SpotSound-Q | 23.0 | 26.9 | 66.6 | 61.1 |
| Audio Flamingo 3 | 6.7 | 17.2 | 51.8 | 53.7 |
Table 6. Performance on sound event detection. Best results are in bold.
| Models | Clotho. | UnAV. | Spot. | Audio. |
| Qwen2-Audio | 5.7 | 9.7 | 4.4 | 37.0 |
| (+) FT | 59.2 | 50.7 | 23.8 | 62.5 |
| (+) unlock | 68.5 | 59.7 | 34.4 | 63.1 |
| (+) timestamps | 85.4 | 72.4 | 49.9 | 67.8 |
| Audio Flamingo 3 | 22.6 | 25.0 | 7.6 | 47.5 |
| (+) FT | 82.8 | 52.7 | 37.4 | 60.3 |
Table 7. Ablation on different modules evaluated in mIoU. ‘Clotho.’, ‘UnAV.’, ‘Spot.’ and ‘Audio.’ refer to Clotho-Moment, Unav-100 subset, SpotSound-Bench and AudioGrounding, respectively. ‘(+) FT’ denotes the standard fine-tuned version of baselines, ‘(+) unlock’ denotes the Qwen2-Audio without the 30-second encoder limitation. Best results are in bold.
| Settings | Clotho. | UnAV. | Spot. | Audio. |
| Granularity of Timestamps | ||||
| 0.2s | 85.8 | 69.7 | 57.2 | 72.7 |
| 1s | 85.6 | 69.8 | 57.9 | 70.3 |
| 2s | 87.2 | 69.6 | 56.5 | 69.7 |
| Quantity of ASSL and Synthetic Data | ||||
| 5k&10k | 85.6 | 69.8 | 57.9 | 70.3 |
Table 8. Ablation results of hyperparameters evaluated in mIoU. The experiments are based on SpotSound-A . ‘Clotho.’, ‘UnAV.’, ‘Spot.’ and ‘Audio.’ refer to Clotho-Moment, Unav-100 subset, SpotSound-Bench and AudioGrounding, respectively. Settings in our experiments are in bold.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Figure S1 . Category distributions for the synthetic dataset.
Figure S2 . Duration and window length distributions for SpotSound-Bench .
| Settings | Clotho. | UnAV. | Spot. | Audio. |
| Granularity of Timestamps | ||||
| 0.2s | 82.6 | 69.9 | 42.3 | 61.2 |
| 1s | 85.4 | 72.4 | 46.6 | 67.8 |
| 2s | 81.5 | 70.1 | 38.7 | 60.5 |
| Quantity of ASSL and Synthetic Data | ||||
| 5k:10k | 85.4 | 72.4 | 46.6 | 67.8 |
Table S1 . Ablation Results of Hyperparameters. ‘Clotho.’, ‘UnAV.’, ‘Spot.’ and ‘Audio.’ refer to Clotho-Moment, Unav-100 subset. Settings in our experiments are in bold.
| Load Model | Inference (1 Sample) | |
| SpotSound-A | 7.6s | 1.0s |
| SpotSound-A - 0.2s | 7.5s | 1.4s |
| SpotSound-A - 2s | 7.6s | 1.0s |
Table S2 . Latency in Inference Process. Settings in our experiments are in bold.
| Settings | [email protected] | [email protected] | [email protected] | mIoU |
| SpotSound-A | 76.5 | 59.9 | 40.0 | 57.9 |
| Temporal Event Shifting | ||||
| Target | 72.3 | 51.0 | 37.8 | 53.2 |
| Query Paraphrasing | ||||
| Synonyms | 75.0 | 57.8 | 39.5 | 56.3 |
| Questions | 74.8 | 55.0 | 37.8 | 54.9 |
Table S3 . Robustness test results. ‘Target’ and denotes randomly shifting the target sound event window. Settings in our experiments are in bold.
| Models | SpotSound-Bench | UnAV-100 subset | ||
| Pos. | Neg. | Pos. | Neg. | |
| Kimi-Audio | 51.8 | 60.5 | 50.0 | 41.0 |
| TimeAudio | / | / | / | / |
| Qwen2-Audio | 55.5 | 78.0 | 64.0 | 44.0 |
| SpotSound-Q | 85.3 | 88.3 | 93.0 | 96.0 |
| Audio Flamingo 3 | 85.5 | 95.0 | 80.0 | 92.0 |
Table S4 . Hallucinations for non-existent events evaluated in accuracy. ‘Pos.’ and ‘Neg.’ refer to the model prediction accuracy of positive and negative queries, respectively. ‘/’ denotes model hallucination, indicating the model is unable to determine whether the events in the query are present in the audio. Best results are in bold.
Figure S3 . Success and failure cases from SpotSound-Bench .
Figure S4 . Success and failure cases from AudioGrounding.
Figure S5 . Success and failure cases from UnAV-100 subset.