Organizations: School of Data Science, The Chinese University of Hong Kong, Shenzhen, China. · Microsoft Research Asia, Hong Kong, China. · Microsoft CoreAI, Redmond, WA, USA.
Audio language models are meant to answer questions from the audio itself, but an hour-long recording rarely fits in their context, and the answer often depends on a few seconds of it. Transcribing the recording first makes it fit but discards sounds and voices, while systems that return to the audio must be trained for this behavior. We propose ReListen, a training-free method in which a frozen audio language model finds its own evidence before answering. In a Jev-style scan, the model scores each 20-second window by the probability that it helps answer the question, and it then answers from the highest-scoring minute. This minute overlaps the annotated evidence far more often than transcript retrieval, for 88.5% of VoiceGiraffe questions. Without any change, the same scan also grounds events on LAT-Bench as well as a model trained for temporal grounding. Answering from it matches or exceeds hearing the whole recording with a small fraction of the audio, and on VoiceGiraffe it exceeds a transcript of the whole recording by 9.2 points. The selected evidence is also worth hearing rather than reading. Given the same excerpts, reading their transcript matches listening on speech content, but listening is clearly more accurate on questions about sounds and voices. Moreover, ReListen answers faster than every transcript-based system, and its memory stays constant even for recordings too long to hear whole on one GPU. Therefore, a frozen audio language model can search hour-long recordings by itself, without a transcript or additional training.
Figures & tables
Figure 1: ReListen on a test question about an 84-minute recording. All scores and answers come from the actual run. The frozen audio language model scores each 20-second window by the log-odds of a yes when asked whether the window helps to answer the question. The three best windows overlap the annotated evidence, and the answer from them is correct. From three evenly spaced windows, the same model answers 4, which is wrong.
Figure 2: Architecture of ReListen. One frozen reader scores every window in a Jev-style scan (1). It reads each yes/no decision from one forward pass as a probability, and it then answers from the selected audio (3). The native-question path selects by relevance without training (2a). The supervised occurrence extension uses trained acoustic detectors and a posterior over calibrated candidates (2b).
Figure 3: The three benchmarks, with real questions. Each bar is a whole recording drawn to scale, and the mark on it is the annotated evidence. Questions are shortened only with ellipses, and Appendix A gives full examples.
LongAudioSpan
VoiceGiraffe
Method
Training
GiB
s
Acc.
s
All
Sound
Content
Text-mediated
Reader transcript
✗
71.9
119
59.6
135
52.8
50.8
54.0
Whisper transcript
✗
68.4
27.6
58.1
28.4
–
–
–
Echo-style ( Wu et al., 2026 )
✗
72.8
157
58.1
169
58.1
51.0
62.2
Speaker-Reasoner ( Lin et al., 2026 )
✓
78.1
1124
53.0
1048
55.7
52.6
57.5
Table 1: Accuracy and cost per question. All rows use the same frozen reader to answer, and we report accuracy (%) on test LongAudioSpan and on VoiceGiraffe. Seconds (s) are means over the timed questions. GiB is the peak memory on any one GPU, including the reader’s 59 GiB of weights, and Training marks a model fine-tuned for the task. Uniform excerpts and ReListen give the reader 60 seconds of audio, and the full recording carries time marks. Rows use one GPU unless marked four GPUs, which reproduce the one-GPU selections and accuracy. The full recording uses two GPUs beyond about 62 minutes.
Figure 4: Accuracy against the audio given to the reader. Multiple-choice accuracy on test LongAudioSpan and on VoiceGiraffe. The full recording is placed at its mean duration up to 80 minutes, and the annotated evidence at its mean length.
Figure 5: Accuracy against seconds per question on VoiceGiraffe. Each point gives a method’s mean time over the timed questions and its test accuracy (Table 1 ). Filled marks use one GPU, and open marks divide the scan over four GPUs with the same selection. The horizontal axis is logarithmic, and every ReListen point lies left of all text-mediated systems.
LAS
VoiceGiraffe
Method
All
All
Sound
Uniform sampling
18.7
11.1
4.4
Learned audio retriever
32.6
16.5
5.8
BM25
45.2
58.8
45.6
BGE-M3
47.8
52.1
37.6
ReListen (ours)
69.2
88.5
76.5
Table 2: Evidence localization with a 60-second budget. The hit rate (%) is the fraction of questions for which any selected excerpt overlaps the annotated evidence. LAS is test LongAudioSpan.
Table 3: Event grounding on LAT-Bench. We report mean IoU and recall at IoU 0.3, 0.5 and 0.7 (%). ReListen returns the expanded region of its scan. Appendix G gives ablations. † Reported by Shao et al. (2026) .
LAS
VG content
VG sound
Excerpt source
Audio
Text
Audio
Text
Audio
Text
ReListen, 60 s
60.5
59.4
64.0
65.2
58.7
50.1
Annotated evidence
70.9
70.1
78.3
76.3
67.9
48.4
Table 4: Listening versus reading the same excerpts. The reader hears the excerpts (Audio) or reads its own transcript of them (Text), and we report accuracy (%). LAS uses test LongAudioSpan questions without time options, with audio sliced from whole-recording encoder outputs.
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
Test
Development
Method
Audio
All
Content
Time
S
M
L
Hit
All
Content
Time
Text-mediated
Whisper transcript
none
58.1
67.5
35.4
58.2
58.7
57.5
–
54.8
65.8
30.1
Whisper text + ReListen
60 s
59.8
63.1
51.9
61.6
60.8
56.9
69.2
58.9
65.0
45.2
Audio input
No audio
none
43.5
45.3
39.2
42.4
42.3
46.0
–
44.8
46.0
42.2
Appendix
Table 5: Native multiple-choice results on LongAudioSpan. We report accuracy (%) by option type and recording length, and the evidence hit rate (%) of the audio heard. Whisper text + ReListen adds the selected minute to the Whisper transcript. Annotated evidence is scored on the 487 test questions with annotated support ranges.
Audio input
Reference
Dimension
n
Time options
No audio
Uniform
ReListen
ReListen 600 s
Full recording
Transcript
Oracle
Perception
Recognition
48
0
58.3
56.3
75.0
77.1
68.8
72.9
79.5
Localization
80
78
43.8
40.0
40.0
31.3
38.8
41.2
73.2
Counting
52
0
48.1
34.6
36.5
46.2
51.9
48.1
55.3
Understanding
Appendix
Table 6: Native accuracy by question dimension. We report test accuracy (%), with n questions per dimension and the number of them whose options are clock times. The full recording carries time marks, the transcript is from Whisper, and the oracle gives the reader the annotated support ranges.
Half
Duration tier
Input
Test
Dev
S
M
L
(61)
(72)
(61)
(39)
(33)
No audio
49.2
55.6
49.2
61.5
48.5
ReListen, 60 s
55.7
62.5
62.3
56.4
57.6
ReListen, 180 s
67.2
68.1
68.9
64.1
69.7
ReListen, 300 s
68.9
69.4
72.1
64.1
69.7
Appendix
Table 7: Questions about the whole recording. We report native accuracy (%) on LongAudioSpan questions whose annotated evidence extends over at least half of the recording. Parentheses give the number of questions, and the duration tiers pool both halves. The full recording carries time marks, and the transcripts carry none.
Method
All
Aud. Ev.
Paraling.
Semantic
Temp. Loc.
Causal
Ev. Track.
En
Zh
Text-mediated
Transcript by the reader
52.8
46.0
53.6
85.5
38.4
46.0
52.4
51.1
54.6
Transcript + ReListen, 60 s
63.0
55.5
61.0
86.0
77.6
49.6
52.4
60.5
65.6
Audio input
No audio
34.5
35.5
38.4
32.0
34.0
38.0
27.2
33.9
35.1
Uniform, 60 s
40.7
46.0
49.9
38.5
36.4
45.2
25.2
40.9
40.5
Appendix
Table 8: VoiceGiraffe results by question family and language. We report accuracy (%) on all 1,499 questions and the evidence hit rate (%) on the 1,498 with annotated evidence. Transcript + ReListen adds the selected minute to the reader’s transcript. BM25 and dense retrieval rank chunks of the same transcript.
Table 9: Grounding ablations on LAT-Bench. The first row returns only the highest-scoring window instead of the expanded region. The second row replaces the reader, Qwen3-Omni-30B-A3B, with Qwen2.5-Omni-7B. We report mean IoU and recall at IoU thresholds 0.3, 0.5 and 0.7 (%).
Figure 6: Scan scores along a recording. The scan of an 87-minute development recording for the question “According to the lecture, what is the direct mechanical cause of the lens flattening when shifting focus to a distant object?” Circles mark the three windows that ReListen selects, and the grey band marks the support range that the benchmark annotates for the question.
Figure 7: Reader and detector evidence on planted events. On two development recordings, the reader’s presence score rises at each inserted sound but stays flat at each volume edit, whereas the acoustic detectors respond to both. Grey bars and triangles mark the true occurrences, and the circle marks the occurrence that ReListen selected.
Figure 8: Accuracy against the evidence heard. Native multiple-choice questions with content options on both halves of LongAudioSpan, with the scan at 60, 180 and 600 seconds pooled and grouped by the share of the annotated support seconds that the selected audio holds.
Test
Development
Method
Q1
Q2
Q3
Chain
Q1
Q2
Q3
Chain
Locating without the Q1 options
No audio
16.1
39.4
29.4
8.3
25.6
32.2
28.3
13.7
Uniform 60 s
22.8
28.9
34.4
10.0
33.3
28.9
27.8
15.0
Learned retrieval
18.3
39.4
32.8
9.6
27.2
42.8
27.2
13.5
Whisper transcript
22.8
29.4
32.8
10.2
31.1
23.3
26.1
13.7
Appendix
Table 10: Occurrence chains on both halves. We report accuracy (%) on each question and the chain score on 180 chains per half. Every audio selection totals 60 seconds, and the transcript is from Whisper. The chain-consistent row answers Q2 and Q3 at the time that ReListen chose for Q1.
Ins.
Noise
Sil.
Speed
Vol.
(60)
(31)
(29)
(26)
(34)
Time accuracy
Reader evidence only
61.7
90.3
82.8
3.8
2.9
Detector evidence only
35.0
87.1
82.8
88.5
73.5
ReListen
71.7
93.5
89.7
88.5
73.5
Q2 accuracy
Appendix
Table 11: Occurrence chains by edit type. We report time accuracy and Q2 accuracy (%) on the test half. Parentheses give the number of chains of each edit type.
Test
Dev
Variant
Windows
Time
Q1
Time
Q1
Removing one component
Reader evidence only
289
50.6
51.1
54.4
53.3
Detector evidence only
289
66.7
71.1
72.2
71.1
No calibration labels
289
60.0
58.3
59.4
59.4
Hard ordinal rule
289
80.0
72.8
79.4
71.7
Appendix
Table 12: Ablation of the occurrence path. We report time accuracy and Q1 accuracy (%). Windows is the mean number of windows that the reader scores per test recording. The detector-only row falls back to the reader’s scan when the detectors propose no region. The single-seed row averages three seeds.