cs.CLNov 8, 2026

LongCap: Hour-Scale Timestamped Audio Captioning via Chunk-wise Processing

Authors: Jinchuan Tian, Sen Chen, Wenbin Huang, Chao Weng, Zhengwei Huang, Jiatong Shi, Yichen Lu, Haoran Wang, 

Organizations: Language Technologies Institute, Carnegie Mellon University · Independent Researcher

Abstract

Describing hours of audio requires accurate event timing and sustained descriptive detail throughout the recording. We present LongCap, a framework for timestamped audio captioning (TAC), with the human-annotated LongCapBench and three diarization-inspired metrics: Timeline Error Rate (TER), Granularity Error Rate (GER), and Caption Similarity (tcpCapSim). LongCap interleaves audio chunks with structured event captions, using relative timestamps and retained context to support length extrapolation within a 1M-token inference context window. Event-level reannotation restores caption detail, while systematic studies examine model scaling, dependency supervision, and self-distillation. LongCap-Diar achieves strong benchmark results and extrapolates to 16 hours of concatenated audio despite training on recordings no longer than 40 minutes. Our TAC models are competitive with Qwen3.5-Omni-Plus on short clips, outperform evaluated baselines in caption similarity and timeline accuracy in segment-level comparisons on LongCapBench-TAC-Long, and caption full recordings approaching two hours. On LongCapBench-TAC, 9B and 35B-A3B offer favorable size-performance tradeoffs, while 122B-A10B substantially improves event-dependency recovery. These findings support long-horizon general audio understanding and provide a foundation for always-on voice agents.

Figures & tables

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. TAC: Timestamped Audio Captioning

    Feb 17, 2026Sonal Kumar, Prem Seetharaman, Ke Chen +8Audio CaptioningTemporal Audio Grounding

  2. TAC2: Timestamped Audio Captioning as composable functions

    Oct 27, 2026Swapnil Bhosale, Prem Seetharaman, Ke Chen +3Audio-Language ModelsAudio Captioning

  3. Logbook: Extremely Long-form Audio Event Understanding

    Oct 5, 2026Kwanghee Choi, Suwon Shon, Dmitriy Serdyuk +7Audio-Language Model EvaluationAudio Understanding