Audio Flamingo Next: Next-Generation Open Audio-Language Models for Speech, Sound, and Music
Organizations: NVIDIA, USA · University of Maryland, USA
Abstract
We present Audio Flamingo Next (AF-Next), the next-generation and most capable large audio-language model in the Audio Flamingo series, designed to advance understanding and reasoning over speech, environmental sounds and music. Compared to Audio Flamingo 3, AF-Next introduces: (i) a stronger foundational audio-language model that significantly improves accuracy across diverse audio understanding tasks; (ii) scalable strategies for constructing large-scale audio understanding and reasoning data beyond existing academic benchmarks; (iii) support for long and complex audio inputs up to 30 minutes; and (iv) Temporal Audio Chain-of-Thought, a new reasoning paradigm that explicitly grounds intermediate reasoning steps to timestamps in long audio, enabling fine-grained temporal alignment and improved interpretability. To enable these capabilities, we first conduct a systematic analysis of Audio Flamingo 3 to identify key gaps in audio understanding and reasoning. We then curate and scale new large-scale datasets totaling over 1 million hours to address these limitations and expand the existing AudioSkills-XL, LongAudio-XL, AF-Think and AF-Chat datasets. AF-Next is trained using a curriculum-based strategy spanning pre-training, mid-training and post-training stages. Extensive experiments across 20 audio understanding and reasoning benchmarks, including challenging long-audio tasks, show that AF-Next outperforms similarly sized open models by large margins and remains highly competitive with and sometimes surpasses, much larger open-weight and closed models. Beyond benchmark performance, AF-Next exhibits strong real-world utility and transfers well to unseen tasks, highlighting its robustness and generalization ability. In addition to all data, code and methods, we open-source 3 variants of AF-Next, including AF-Next-Instruct, AF-Next-Think and AF-Next-Captioner.
Figures & tables
| Dataset | Prior SOTA | Metrics | Results |
| MMAU-v05.15.25 (test) Sound | Music | Speech | Avg | Audio Flamingo 3 | ACC | 75.83 | 74.47 | 66.97 | 72.42 |
| AF-Next-Instruct | 78.80 | 74.23 | 69.57| 74.20 | ||
| AF-Next-Think | 78.70 | 74.73 | 71.5 | 75.01 | ||
| AF-Next-Captioner | 79.87 | 75.3 | 72.13 | 75.76 | ||
| MMAR | Audio Flamingo 3 | ACC | 58.5 |
| AF-Next-Instruct | 59.7 |
| Task | Model | AdvBench | AlpacaEval | CommonEval | OpenBookQA |
|---|---|---|---|---|---|
| VoiceBench | Qwen2.5-O | 99.62 | 4.33 | 3.84 | 79.12 |
| Audio Flamingo 3 | 98.26 | 4.19 | 3.40 | 66.81 | |
| AF-Next-Instruct | 98.84 | 4.43 | 3.96 | 80.9 | |
| CoVoST2 (Speech Translation, BLEU ) | |||||
| Lang. | Model | ZH | JA | AR | DE |
| EN X | Phi-4-mm | 38.0 | 31.9 | 9.9 | 35.3 |
Audio Understanding
Audio understanding enables machines to recognize speech, sounds, music, speakers, and events, then answer questions grounded in what they hear.
See all 211 papers