We present Audio Flamingo Next (AF-Next), the next-generation and most capable large audio-language model in the Audio Flamingo series, designed to advance understanding and reasoning over speech, environmental sounds and music. Compared to Audio Flamingo 3, AF-Next introduces: (i) a stronger foundational audio-language model that significantly improves accuracy across diverse audio understanding tasks; (ii) scalable strategies for constructing large-scale audio understanding and reasoning data beyond existing academic benchmarks; (iii) support for long and complex audio inputs up to 30 minutes; and (iv) Temporal Audio Chain-of-Thought, a new reasoning paradigm that explicitly grounds intermediate reasoning steps to timestamps in long audio, enabling fine-grained temporal alignment and improved interpretability. To enable these capabilities, we first conduct a systematic analysis of Audio Flamingo 3 to identify key gaps in audio understanding and reasoning. We then curate and scale new large-scale datasets totaling over 1 million hours to address these limitations and expand the existing AudioSkills-XL, LongAudio-XL, AF-Think and AF-Chat datasets. AF-Next is trained using a curriculum-based strategy spanning pre-training, mid-training and post-training stages. Extensive experiments across 20 audio understanding and reasoning benchmarks, including challenging long-audio tasks, show that AF-Next outperforms similarly sized open models by large margins and remains highly competitive with and sometimes surpasses, much larger open-weight and closed models. Beyond benchmark performance, AF-Next exhibits strong real-world utility and transfers well to unseen tasks, highlighting its robustness and generalization ability. In addition to all data, code and methods, we open-source 3 variants of AF-Next, including AF-Next-Instruct, AF-Next-Think and AF-Next-Captioner.
Figures & tables
Figure 1: Performance comparison of AF-Next against prior SOTA LALMs across key audio understanding and reasoning benchmarks.
Figure 2: Examples of new data types introduced to scale AF-Next training. More examples are shown in Figures – , and details are provided in Section 2.2.1 .
Figure 3: Training pipeline for AF-Next, curriculum learning stages, and illustration of sequence-parallel setup for long-context training. Example shown for 32 attention heads (H0–H31) and batch size 2 ( seq_0 – seq_1 ) across 2 GPUs. Before All-to-All : each GPU holds the full sequence shard with all attention heads. All-to-All (scatter heads, gather sequence) : heads are distributed across GPUs while sequence chunks are gathered -— each GPU now sees the full sequence but only a subset of heads. Flash attention is computed on the gathered sequence. After All-to-All (scatter sequence, gather heads) : the reverse exchange restores the original partitioning, after which FFN and layer norm are applied locally without communication.
Dataset
Prior SOTA
Metrics
Results
MMAU-v05.15.25 (test) Sound | Music | Speech | Avg
Audio Flamingo 3
ACC ↑
75.83 | 74.47 | 66.97 | 72.42
AF-Next-Instruct
78.80 | 74.23 | 69.57| 74.20
AF-Next-Think
78.70 | 74.73 | 71.5 | 75.01
AF-Next-Captioner
79.87 | 75.3 | 72.13 | 75.76
MMAR
Audio Flamingo 3
ACC ↑
58.5
AF-Next-Instruct
59.7
Table 1: Comparison of AF-Next with other LALMs on various benchmarks (WER ↓ (Word Error Rate), ACC ↑ (Accuracy), and GPT ↑ (GPT evaluation)). We report scores for only the top-performing prior LALM reproduced by us. We highlight closed source , open weights , and open source models.
Task
Model
AdvBench
AlpacaEval
CommonEval
OpenBookQA
VoiceBench
Qwen2.5-O
99.62
4.33
3.84
79.12
Audio Flamingo 3
98.26
4.19
3.40
66.81
AF-Next-Instruct
98.84
4.43
3.96
80.9
CoVoST2 (Speech Translation, BLEU ↑ )
Lang.
Model
ZH
JA
AR
DE
EN → X
Phi-4-mm
38.0
31.9
9.9
35.3
Table 2: Comparison of AF-Next-Instruct with open LALMs on VoiceBench and speech translation benchmarks.
We present Audio-Visual Flamingo (AV-Flamingo), a fully open state-of-the-art audio-visual large language model (AV-LLM) for joint understanding and reasoning over audio, images, and long-form videos. Unlike prior AV-LLMs that primarily focus on short clips, AV-Flamingo is designed for understanding and reasoning over long and complex real-world (audio-visual) videos. To support this, we make three key contributions: (i) Audio-Visual-Skills, a large-scale collection of real-world videos with ~7M caption and question-answer training instances designed to emphasize temporal, compositional, and cross-modal audio-visual reasoning; (ii) a novel three-stage curriculum that progressively trains the model from short-range perception to long-horizon multi-event reasoning; and (iii) Temporal Audio-Visual Interleaved Chain-of-Thought, a reasoning framework that explicitly grounds intermediate reasoning steps to timestamps in long audio-visual streams, improving temporal alignment and interpretability. Extensive experiments across 15+ audio-visual, omni-modal, audio, and vision benchmarks show that AV-Flamingo outperforms similarly sized open models by clear margins and remains highly competitive with, and in some cases surpasses, much larger open-weight and closed models, particularly on long and complex real-world audio-visual understanding and reasoning tasks. Beyond benchmark performance, AV-Flamingo exhibits strong real-world utility and transfers well to unseen tasks, highlighting its robustness and generalization ability.
General audio comprehension now covers speech, sound, and music over durations from seconds to hours, driven by large audio-language models (LALMs) that are increasingly omni-modal. Yet the benchmarks that test them still rely on clips of seconds, where scores saturate and models converge; recent long-form efforts extend duration but evaluate long audio much as short clips are. We introduce LongAudioSpan, a benchmark that spans both duration and depth: it pairs audio from 10 minutes to over 2 hours with 3,240 questions across three cognitive levels, namely perception, understanding, and reasoning. Two paths supply the questions, differing in how question content is sourced and how ground truth is obtained. Native QA extracts questions from the audio's content, posing each as a multiple-choice item and an open-ended one graded by detailed rubrics. Anchor QA instead injects ground truth, planting acoustic anchors into the audio and building a perception-to-reasoning chain scored only to the first error. A fully automated pipeline constructs every item through structured captioning, QA generation, and adversarial critic feedback. Evaluating 12 LALMs on LongAudioSpan, we find the hard part comes before reasoning: distilling a few relevant facts from a long, redundant signal. This difficulty grows with audio length and falls hardest on perception, especially temporal grounding. LongAudioSpan is available at https://huggingface.co/datasets/holvan/LongAudioSpan.
Wen Huang, Yunfei Chu, Meng Gao +2
Qwen Team, Alibaba Group · Tsinghua University · The Chinese University of Hong Kong
We introduce Long Listening Thoughts (LLT), a framework for constructing post-training datasets that improves audio reasoning by pairing deliberate perceptual grounding with explicit cognitive behaviors. On audio understanding and reasoning benchmarks, large audio-language models (LALMs) are limited primarily by perceptual errors, yet existing post-training datasets rely on audio descriptions of limited diversity. Through a four-stage pipeline, LLT (1) builds multi-aspect dense audio descriptions spanning semantic, acoustic, and temporal information, (2) generates more challenging questions, (3) synthesizes incorrect reasoning traces, and (4) extends short reasoning traces into long ones via a thought-continuation strategy that elicits cognitive behaviors such as verification and backtracking. Using LLT, we synthesize 1.6M reasoning traces and train Qwen2.5-Omni-7B with a two-stage pipeline of supervised fine-tuning (SFT) followed by direct preference optimization (DPO). This improves accuracy on MMAU Test-mini (71.5 to 78.2), MMAR (56.7 to 63.7), and MMSU (60.6 to 66.0).