Listen, Reason, and Segment: Aligning LALMs with Editorial Judgment for Media Chapterization

Tony Alex; Wish Suharitdamrong; Sara Atito; Armin Mustafa; Muhammad Awais; Philip J. B. Jackson; Jiankang Deng; Ismail Elezi

cs.SDAug 17, 2026

Listen, Reason, and Segment: Aligning LALMs with Editorial Judgment for Media Chapterization

Authors: Tony Alex, Wish Suharitdamrong, Sara Atito, Armin Mustafa, Muhammad Awais, Philip J. B. Jackson, Jiankang Deng, Ismail Elezi

Organizations: University of Surrey · Huawei Noah’s Ark Lab

Abstract

Large Audio Language Models (LALMs) have made rapid progress on standardized benchmarks, yet their deployment in practical media workflows, curation, archival indexing, and content distribution remains largely unrealized. We identify automated audio chapterization, the task of segmenting continuous audio streams into thematically coherent chapters, as a demanding and commercially consequential setting that exposes this gap. Chapterization is challenging because boundaries are defined less by objective acoustic events than by subjective editorial judgment, requiring models to reason sequentially over long acoustic contexts and approximate creator-authored boundary decisions. We present AudioChaps, a post-training framework for aligning end-to-end LALMs for this task via Group Relative Policy Optimization (GRPO) guided by Chain-of-Thought (CoT) reasoning. To support training and evaluation, we curate three datasets: AudioChaps-Alignment, derived from creator-annotated chapter boundaries on YouTube; AudioChaps-CoT, which provides structured supervision for well-formatted, high-quality, and evidence-grounded boundary reasoning; and AudioChaps-Eval, a held-out benchmark for audio chapterization. Applying GRPO directly without a Supervised Fine-Tuning (SFT) cold start, AudioChaps-R1-Zero already improves average F1 by 33 points over the state-of-the-art LALM Audio-Flamingo-3-Think. The AudioChaps framework produces our final aligned LALM, AudioChaps-R1, which improves average F1 by 49 points. These results demonstrate that GRPO-trained LALMs can reliably transform unstructured auditory streams into navigable, structured media. Our code, models, and dataset resources will be released upon acceptance at https://github.com/ta012/AudioChaps.

Figures & tables

Figure 1: Illustration of the audio chapterization task: the media’s audio is analysed in 60-s chunks via AudioChaps-R1-8B to determine the chapter boundaries, presented to the user as a timeline above the scrubber bar.

Figure 2: AudioChaps-CoT curation pipeline. A boundary-relevant pseudo-CoT is generated, sanitized into an acoustic perception log, and then converted into the final structured CoT target used for the SFT cold start.

	$\displaystyle J_{\mathrm{GRPO}}(\theta)$	$\displaystyle=\mathbb{E}_{q\sim P(Q),\;\{o_{i}\}_{i=1}^{G}\sim\pi_{\theta_{\mathrm{old}}}(O\mid q)}$		(1)
		$\displaystyle\left[\frac{1}{G}\sum_{i=1}^{G}\min\left(\frac{\pi_{\theta}(o_{i}\mid q)}{\pi_{\theta_{\mathrm{old}}}(o_{i}\mid q)}A_{i},\;\mathrm{clip}\left(\frac{\pi_{\theta}(o_{i}\mid q)}{\pi_{\theta_{\mathrm{old}}}(o_{i}\mid q)},1-\epsilon,1+\epsilon\right)A_{i}\right)-\beta D_{\mathrm{KL}}\left(\pi_{\theta}\parallel\pi_{\mathrm{ref}}\right)\right].$		(1)

Table 3

Model	sc	Acc	Pre	Rec	F1
	DM	60.2	55.3	62.7	58.8
Step-Audio-R1-32B	G	60.1	59.1	53.4	56.1
(Zero-Shot)	M	62.4	53.1	51.9	52.5
	SS	65.6	70.1	69.7	69.9
	Avg	62.1	59.4	59.4	59.3
	DM	55.8	47.5	23.7	31.6

Table 1: Performance comparison on AudioChaps-Eval across the four audio chapterization subcategories: Dynamic Media (DM), Gaming (G), Music (M), and Structured Speech (SS). Accuracy (Acc), precision (Pre), recall (Rec), and F1 are reported as percentages, together with the macro-average (Avg) over subcategories. Detailed comparison with additional multimodal and omni-modal LLMs is provided in Supplementary Table 7 .

Model	F1	Pre	Rec	dev-R2E
Fixed interval, 180 s	9.5	9.0	11.6	49.0 s
AF3-Think-8B	6.5	8.6	9.6	38.0 s
AudioChaps-R1-8B	37.6	34.1	45.7	10.0 s

Table 2: Full-length chapter-boundary detection on uncropped audio. Predictions are matched one-to-one with reference boundaries under a

\pm 10

-second tolerance. Precision, recall, and F1 are computed per recording, macro-averaged within each acoustic subtype, and then averaged equally across the four subtypes. dev-R2E denotes the pooled median temporal distance from each reference boundary to its nearest predicted boundary. Detailed results are provided in Supplementary Table 8 .

Model	DM	G	M	SS
AF3-Think-8B	31.6	49.9	6.0	27.0
AudioChaps-R1-Zero-8B	57.8	64.4	59.1	62.9
AudioChaps-SFT-8B	70.0	72.5	74.1	77.9
AudioChaps-R1-8B	73.4	75.5	84.6	77.8

Table 3: Ablation of the AudioChaps training stages. F1 scores are reported for Dynamic Media (DM), Gaming (G), Music (M), and Structured Speech (SS). Detailed results are provided in Supplementary Section B , Table 5 .

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 3 — Figure 3: The unified prompt template used in both the SFT cold-start and GRPO stages. The model is asked a binary single-choice question, instructed to reason with natural reflective markers inside <think> tags, and to emit a single option letter inside <answer> tags.

Hyperparameter	SFT	GRPO
Epochs	2	1
Learning rate	$1\mathrm{e}{-6}$	$1\mathrm{e}{-6}$
Per-device batch size	1	1
Gradient accumulation	4	1
Max completion length	—	768
Generations per prompt ( $G$ )	—	8

Appendix

Table 4: Training hyperparameters for the SFT cold-start and GRPO alignment stages.

Model	sc	Acc	Pre	Rec	F1
	DM	55.8	47.5	23.7	31.6
AF3-Think-8B	G	50.1	45.9	54.6	49.9
(Base-LALM)	M	62.3	64.0	3.2	6.0
	SS	44.7	46.4	19.1	27.0
	Avg	53.2	51.0	25.2	28.6
	DM	61.6	55.0	61.0	57.8

Appendix

Table 5: Detailed ablation results corresponding to Table 3 in the main paper across four audio chapterization subcategories (sc): Dynamic Media (DM), Gaming (G), Music (M), and Structured Speech (SS). AF3-Think-8B is the unaligned base LALM, which lacks a sufficiently strong reasoning prior and does not reliably follow the target structured output format. AudioChaps-R1-Zero-8B applies GRPO directly to AF3-Think-8B without an SFT cold start. AudioChaps-SFT-8B uses supervised fine-tuning to establish the target reasoning behavior and structured output format, while AudioChaps-R1-8B subsequently applies GRPO to calibrate the model’s chapter-boundary decisions. This ablation isolates the contribution of the SFT cold start and the subsequent GRPO alignment stage. The macro average (Avg) is reported over the four subcategories.

Model	sc	Acc	Pre	Rec	F1
	DM	55.8	47.5	23.7	31.6
AF3-Think-8B	G	50.1	45.9	54.6	49.9
(Base-LALM)	M	62.3	64.0	3.2	6.0
	SS	44.7	46.4	19.1	27.0
	Avg	53.2	51.0	25.2	28.6
	DM	75.9	70.0	77.2	73.4

Appendix

Table 6: Backbone-generalization results across audio chapterization subcategories (sc): Dynamic Media (DM), Gaming (G), Music (M), and Structured Speech (SS). Grey rows report the AF3-based models for reference. Across both AF3 and MOSS, GRPO alignment improves precision (Pre), accuracy (Acc), and F1 while reducing the high-recall behaviour of the preceding model. Avg denotes the macro-average over subcategories.

Model	sc	Acc	Pre	Rec	F1
	DM	72.5	66.7	78.8	72.2
Gemini-2.5-Flash	G	73.0	67.9	82.5	74.5
(Zero-Shot)	M	81.0	74.9	81.4	78.0
	SS	78.3	77.3	88.0	82.3
	Avg	76.2	71.7	82.7	76.8
	DM	73.1	68.6	69.2	68.9

Appendix

Table 7: Comparison with additional multimodal and omni-modal LLMs on AudioChaps-Eval across four audio chapterization subcategories: Dynamic Media (DM), Gaming (G), Music (M), and Structured Speech (SS). Accuracy (Acc), precision (Pre), recall (Rec), and F1 are reported as percentages, together with the macro-average (Avg) over the four subcategories. Gemini-2.5-Flash, Qwen3-Omni-30B-A3B-Thinking, MOSS-Think-8B, Step-Audio-R1-32B, and AF3-Think-8B are evaluated in the zero-shot setting. The two AudioChaps-R1 variants apply our alignment framework to the AF3-Think and MOSS-Think backbones, respectively.

Model	sc	Pre	Rec	F1	dev-R2E $\downarrow$
	DM	18.9	13.5	10.7	–
AF3-Think-8B	G	3.4	10.5	4.6	–
(Window)	M	0.4	1.7	0.7	–
	SS	11.8	12.7	10.0	–
	Overall	8.6	9.6	6.5	38.0 s
	DM	32.6	38.6	34.5	–

Appendix

Table 8: Full-length audio chapter-boundary detection across four acoustic subcategories: Dynamic Media (DM), Gaming (G), Music (M), and Structured Speech (SS). Predictions are matched one-to-one with reference boundaries under a

\pm 10

-second tolerance. Precision (Pre), recall (Rec), and F1 are computed per recording, macro-averaged within each subcategory, and then averaged equally across the four subcategories. dev-R2E denotes the pooled median distance from each reference boundary to its nearest prediction and is reported only in the Overall row; lower values indicate better localization. The window-run decoder merges contiguous positive windows into a single recording-level boundary estimate.

Figure 4: Per-subtype clip counts for the training (AudioChaps-Alignment) and held-out test (AudioChaps-Eval) splits, split by positive (boundary-spanning) and negative (within-chapter) clips. Both splits follow the same four-way stratification, with no source-video overlap between the training set and the held-out test set.

Panel of Figure 5 — Figure 5: Instructions shown to human evaluators. Each sample pairs a 1-minute clip and the boundary question with two anonymized, randomly ordered model outputs (Output A and Output B), each rated on a 1–5 scale.

Figure 6: Qualitative comparison of AF3-Think-8B and AudioChaps-R1-8B on human-evaluation (HE).

Figure 7: Qualitative comparison of AF3-Think-8B and AudioChaps-R1-8B on human-evaluation.

Figure 8: Qualitative comparison of AF3-Think-8B and AudioChaps-R1-8B on human-evaluation.

Figure 9: Qualitative comparison of AF3-Think-8B and AudioChaps-R1-8B on human-evaluation.

Audio-Language Models

Audio-language models connect audio and natural language for understanding, retrieval, dialogue, speech generation, and music or sound analysis.

See all 303 papers

Beyond Accuracy: ARIA-Rubrics for Evaluating Audio Reasoning in Large Audio Language Models
Sep 9, 2026Yupei Li, Qiyang Sun, Mohamed Mady +4Audio-Language Model Evaluation Audio Reasoning
Imperial College London, UK · Technische Universität München München, Germany · Mohamed bin Zayed University of Artificial Intelligence Abu Dhabi, AE +2
Listen-to-Reason: Listen with Experts, Retrieve over a Graph, Reason with LLMs
Oct 7, 2026Pooneh Mousavi, Mirco Ravanelli, Cem SubakanAudio-Language Models Audio Reasoning
Concordia University · Mila – Quebec AI Institute · Laval University
ALARM: Audio-Language Alignment for Reasoning Models
Mar 10, 2026Petr Grinberg, Hassan ShahmohammadiAudio-Language Models Audio Reasoning
EPFL, Switzerland · Sony Europe Ltd., Germany

Sep 9, 2026J/K move · Enter open · S save

Beyond Accuracy: ARIA-Rubrics for Evaluating Audio Reasoning in Large Audio Language Models

Yupei Li, Qiyang Sun, Mohamed Mady, Chenxi Wang+3

Imperial College London, UK · Technische Universität München München, Germany · Mohamed bin Zayed University of Artificial Intelligence Abu Dhabi, AE+2

Audio-Language Model Evaluation Audio Reasoning Audio-Language Models

Open paper AskPDF arXiv

Large Audio Language Models (LALMs) have shown strong performance on audio reasoning benchmarks, but accuracy alone cannot distinguish true reasoning from superficial pattern matching, often overestimating reasoning ability since high scores may result from guessing rather than genuine audio understanding. Evaluating the reasoning process itself is essential for improving LALMs' reasoning ability, yet remains challenging. Existing methods either rely on costly human annotation or opaque LLM-as-judge approaches, making them impractical, biased, and lacking transparency. Moreover, audio reasoning introduces unique challenges absent in text-based settings, perceptual hallucination and cross-modal alignment between audio understanding and textual inference, hence text-based evaluation frameworks cannot be directly applied. Therefore, we propose ARIA-Rubrics (Audio Reasoning Integrity Assessment), a lightweight, annotation-free gold reasoning chains, automatic and transparent framework comprising six complementary metrics that evaluate audio reasoning quality across perceptual grounding, reasoning coherence, and answer consistency. We use Chain-of-Thought prompting as an externalization mechanism to make the reasoning process observable. Experiments on 9 models across 2 benchmarks identify three reasoning modes of current LALMs with actionable directions for future development, with ARIA-Rubrics achieving high correlation with human judgments. The code is available at the Github Repository.

Abstract

Audio-Language Models

Explore similar work

Beyond Accuracy: ARIA-Rubrics for Evaluating Audio Reasoning in Large Audio Language Models

Listen-to-Reason: Listen with Experts, Retrieve over a Graph, Reason with LLMs

ALARM: Audio-Language Alignment for Reasoning Models