Organizations: College of Computer Science and Technology, Harbin Engineering University, Harbin, China · University of Technology Sydney, Ultimo, Australia · State Key Laboratory of Space Information System and Integrated Application, Beijing, China
Audio Moment Retrieval (AMR) aims to locate the temporal segment within a long audio recording that semantically corresponds to a natural-language query. This task is particularly challenging because query-related audio events are often sparsely distributed and embedded in complex acoustic environments, where background sounds and acoustically similar events can easily obscure the target moments. Existing methods mainly strengthen audio-text semantic interaction to enhance the query-related information in audio frames, but seldom refine the resulting audio representations before temporal localization, limiting their ability to accurately distinguish target events from irrelevant acoustic interference. To address this issue, we propose ETR-AMR, a post-fusion event-aware time-frequency refinement framework for AMR. Specifically, an event-aware time-frequency refinement (ETR) strategy is proposed after audio-text interaction to refine fused audio frame features before temporal moment localization. ETR consists of two complementary mechanisms: time frame gating and frequency refinement. The time frame gating mechanism estimates frame-wise event relevance to emphasize query-related audio frames and suppress redundant temporal responses, while frequency refinement applies learnable time-frequency reweighting to enhance discriminative event-related frequency patterns and reduce background-dominant interference. By jointly refining fused audio frame features in the temporal and frequency domains, ETR produces more discriminative event-aware features for accurate temporal boundary prediction. Experiments on four AMR datasets demonstrate that ETR-AMR achieves superior performance compared with existing methods, especially on the high-quality CASTELLA benchmark. Duration-wise analysis further confirms the effectiveness of ETR for medium and long audio moments.
Figures & tables
Figure 1: The overall framework of the proposed ETR-AMR method. ETR-AMR consists of three important stages: (1) audio-text interaction for fusing audio and text features, (2) the proposed event-aware time-frequency refinement to generate discriminative event-aware audio frame features through time-frequency refinement of audio features through the short-time Fourier transform (STFT), and (3) temporal moment localization to predict the start and end time of audio moments.
Dataset
Method
R1@50%
R1@70%
mAP@50%
mAP@70%
mAP-avg
Clotho-Moment [ 13 ]
AM-DETR [ 13 ]
87.50
81.86
91.39
82.51
76.66
QG-DETR [ 17 ]
88.64
83.2 7
93.61
83.42
77.35
UVCOM [ 12 ]
89.17
84.19
92.67
84.95
80.48
ETR-AMR (Ours)
87.89
82.58
91.70
83.50
78.71
UnAV-100 subset [ 8 ]
AM-DETR [ 13 ]
61.00
39.00
67.08
36.42
40.39
QG-DETR [ 17 ]
63.00
40.00
68.24
36.59
40.87
Table 1: Performance comparison between the proposed ETR-AMR method and existing AMR methods on four AMR datasets.
Figure 2: Visualization comparison between predictions without and with the proposed ETR strategy on the CASTELLA dataset [ 12 ] . Green bars denote ground-truth moments, blue bars denote predictions without ETR, and red bars denote predictions with ETR.
Figure 3: Duration-wise AMR performance comparison between the AMR model without ETR and the model with ETR on the CASTELLA dataset [ 12 ] . R1@50% and R1@70% are reported for different audio moment duration groups.
Activation
R1@50%
R1@70%
mAP@50%
mAP@70%
mAP-avg
Sigmoid
27.91
14.18
25.46
10.78
12.13
GELU
29.40
15.07
25.35
11.15
12.63
Table 2: Performance comparison on CASTELLA dataset [ 12 ] when using different activations for the time frame gating in our proposed ETR module.
Frequency Refinement
R1@50%
R1@70%
mAP@50%
mAP@70%
mAP-avg
✗
27.32
12.32
24.29
9.90
11.52
✓
29.40
15.07
25.35
11.15
12.63
Table 3: Performance comparison on the CASTELLA dataset [ 12 ] with and without frequency refinement in the proposed ETR module.
This paper presents an overview of the Detection and Classification of Acoustic Scenes and Events (DCASE) 2026 Challenge Task 6, Audio Moment Retrieval (AMR) from Long Audio. Given a several-minute-long audio recording and a free-form text query, AMR aims to retrieve temporal moments in the recording that match the query, where each moment is represented by a pair of start and end timestamps. This task requires effective cross-modal alignment and long-range temporal modeling. We describe the task definition, the evaluation metrics, the development and evaluation datasets, and a baseline system that combines a pre-trained MS-CLAP feature extractor with a Detection Transformer (DETR)-based moment-detection network. On the development data, the baseline trained on a manually annotated dataset and a synthetic dataset achieved [email protected] of 13.56%, indicating that AMR in long audio remains a challenging problem. The challenge attracted 21 teams, which submitted 59 systems in total. The three best systems achieved [email protected] of 48.59%, roughly 3.5 times the baseline score. The results show that strengthening the audio-text feature extractor and the moment-detection network led to substantial performance improvements. Furthermore, the top three teams boosted performance by applying confidence score calibration or ensembling across different temporal resolutions of features.
Large Audio-Language Models (LALMs) have substantially advanced general audio understanding, yet they remain limited in fine-grained temporal perception, particularly in precise event localization. Existing approaches primarily post-train LALMs to predict event boundaries as timestamp tokens. However, this generative formulation lacks explicit correspondence between the timestamp predictions and fine-grained acoustic evidence, limiting the precision and reliability of temporal localization. To address this issue, we augment the LALM with a dedicated frame-level grounding model while leveraging its semantic modeling capability to represent the event query. Specifically, the frozen LALM encodes the event query with audio as context, and the grounding model combines these query representations with fine-grained audio features to localize the target event at the frame level. Extensive experiments across diverse temporal grounding benchmarks demonstrate strong and consistent improvements over existing methods. Further evaluation shows that the grounding model can provide temporal evidence to support downstream reasoning.
Yanfeng Shi, Yan Song, Junhui Li +4
National Engineering Research Center of Speech and Language Information Processing, University of Science and Technology of China, Hefei, China · ICT Cluster, Singapore Institute of Technology, Singapore
This paper revisits frame-wise saliency for audio moment retrieval (AMR). We show that the frame-wise saliency sequence, conventionally used only as an auxiliary output in DETR-based AMR models, can itself serve as an effective source of moment predictions. We convert the saliency sequence into ranked moments using a simple SED-inspired segmentation rule with no learned parameters, enabling moment retrieval directly from frame-wise temporal information. On the CASTELLA dataset, saliency-based prediction consistently outperforms decoder-based prediction from the same model across all 18 runs of QD-DETR and CG-DETR. For QD-DETR, simply replacing the inference output improves [email protected] from 21.0 to 36.1. The advantage remains 7-15 points when the two outputs are evaluated at their independently selected best epochs. The same tendency extends to TaskWeave and UVCOM, whereas TR-DETR shows the opposite behavior, suggesting that how saliency construction may matter. The performance gap is especially pronounced for short moments: for queries whose annotated moments average at most 2 s, [email protected] improves from 6.3 to 27.8 with QD-DETR. Decoder supervision nevertheless benefits saliency-based prediction, indicating that its role during training differs from the utility of its inference output.