Think2Listen: Evidence-Centric Learning For Agentic Audio Understanding With LALMS
Organizations: Xi’an University of Electronic Science and Technology · Ant International · Nanyang Technological University · Beijing University of Posts and Telecommunications ·
Abstract
Recent advances in large audio-language models (LALMs) have broadened audio understanding across speech, sound, and music, with tool-based agents further supporting these tasks by invoking predefined audio tools. However, a persistent gap remains between perceiving acoustic content and using it as evidence for reasoning: recognizing what is audible does not ensure that a model can identify which cues resolve an uncertain decision. To bridge this gap, we propose Think2Listen, an evidence-centric learning framework for agentic audio understanding. Guided by evidence before commitment, Think2Listen couples iterative reasoning with a targeted LALM interaction to actively seek acoustic cues that distinguish competing hypotheses and revise candidate judgments based on the observations returned. It learns what to ask, how to use evidence, and when to stop, using observation-induced changes in candidate-state to inform training credit. Extensive evaluations on two benchmarks spanning seven audio categories demonstrate that Think2Listen achieves (I) enhanced multi-step audio reasoning, with gains in overall accuracy of 5.08%--17.67% over competing audio agents; and (II) improved audio understanding and reasoning across speech, sound, and music, boosting overall accuracy by 2.29%--13.10% over the corresponding baselines. The source code is available at https://github.com/JinWang79/Think2Listen.
Figures & tables
| Metric | TaC | w/o TaC |
| Parsed ASK | 20,095/20,341 | 22,247/22,352 |
| Parse coverage | 98.79% | 99.53% |
| Singleton turns | 9,273 | 9,985 |
| Agrees with answer | 8,034 | 8,706 |
| Agreement | 86.64% | 87.19% |
| Human audit | 75% | 77% |
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
| Parameter | Value |
| Training epochs | 5 |
| Prompts per rollout batch | 32 |
| Trajectories per prompt, | 8 |
| Global batch size | 256 |
| Advantage estimator | GRPO |
| Optimizer | Adam |
| Modality | Prompt |
| Speech | Describe the audible speech content, speaker count, and relevant background scene. State only facts that can be heard. |
| Music | Describe the audible musical sources and texture: instruments, voices, timbre, accompaniment, and production effects. |
| Sound | Identify the audible sound sources, events, and background ambience. Distinguish foreground from background when possible. |
| Mixed | Inventory the audible speech, music, and non-speech sound sources, separating foreground and background. |
| Configuration | MMAR | MMAU test-mini |
| Qwen3-Omni (direct answering) | 66.07 | 74.18 |
| Untrained query | 60.00 | 68.82 |
| Initial description only (trained actor) | 62.83 | 73.03 |
| Learned query (trained actor) | 69.25 | 76.00 |
| Configuration | MMAR | MMAU test-mini |
| Untrained query | 58.00 | 73.50 |
| Independent descriptions + untrained-actor voting | 56.00 | 67.50 |
| Failure set | Original | Verified | Net gain | Residual |
| MMAR | 0.00 | 38.71 | 38.71 | 61.29 |
| MMAU test-mini | 4.17 | 37.50 | 33.33 | 62.50 |
| Combined | 1.82 | 38.18 | 36.36 | 61.82 |
| Text actor | Reasoning effort | MMAR | MMAU test-mini |
| Think2Listen (trained 9B) | – | 69 | 76 |
| GPT-6.1 Terra | medium | 69 | 76 |
| GPT-6.1 Terra | high | 70 | 77 |
| GPT-6.1 Terra | xhigh | 73 | 76 |
| GPT-6.1 Sol | medium | 73 | 77 |
| GPT-6.1 Sol | high | 75 | 74 |
| Metric | Result |
| Parse coverage on same-budget subset: original 8B | 98.93% 87.23% |
| Mean absolute advantage difference: overall | 0.0079 |
| Mean absolute advantage difference: groups with reward differences | 0.0544 |
| Advantage-sign changes | 0 |
| Model | MMAR | MMAU mini | Source |
| Think2Listen | 69.25 | 76.00 | Table 1 |
| Step-Audio-R1 | 69.15 | 71.10 | MMAR: MOSS-Audio , Table 1; MMAU: SKAD , Table 2 |
| SALMONN-2 9B | 64.50 | – | SALMONN-2 , Table VI |
| SALMONN-2 30B-A3B | 67.60 | – | SALMONN-2 , Table VI |
| AF-Next Instruct | 59.70 | – | Audio Flamingo Next , Table 1 |
| AF-Next Think | 61.00 | 74.50 | MMAR: Audio Flamingo Next , Table 1; MMAU: Audio-R1 , Table 1 |
Audio Understanding
Audio understanding enables machines to recognize speech, sounds, music, speakers, and events, then answer questions grounded in what they hear.
See all 225 papers