Audio Language Models

Follow this field Ask about this field

Recent momentum

emerging

0 papers in the last 28 days · 0.0% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this field, kept on the site without email delivery.

Period ending 2026-09-21

25 new papers

A weekly snapshot of new work published in Audio Language Models.

Period ending 2026-09-14

25 new papers

A weekly snapshot of new work published in Audio Language Models.

Period ending 2026-09-07

8 new papers

A weekly snapshot of new work published in Audio Language Models.

Inside this field

Focused directions

Focused topic

Audio

Within Audio Language Models

167 papersExplore

Focused topic

Large Audio Language Models

Within Audio Language Models

135 papersExplore

Focused topic

Audio Latent

Within Audio Language Models

75 papersExplore

Focused topic

Audio Representation

Within Audio Language Models

69 papersExplore

Focused topic

Acoustic Representation

Within Audio Language Models

54 papersExplore

Focused topic

Music Editing

Within Audio Language Models

49 papersExplore

Focused topic

Audio Encoders

Within Audio Language Models

21 papersExplore

Focused topic

Audio Flow

Within Audio Language Models

20 papersExplore

478 papers

Latest in Audio Language Models

Browse all topics →

MelT: A Portable, Single-GEMM Mel Audio Frontend via Non-Uniform DFT with Measured Latency and Energy Gains on GPUs
May 31, 2026Augusto Camargo, Marcelo FingerMatrix Multiplication Audio
Quality Audio Prototyping: a prototype system for unified sound retrieval and procedural generation
May 30, 2026Nelly Garcia, Aditya Bhattacharjee, Gabryel Mason-Williams +3Audio Flow Audio Representation
UNISON: A Unified Sound Generation and Editing Framework via Deep LLM Fusion
May 29, 2026Zhaoqing Li, Haoning Xu, Jingran Su +9Audio Latent Unison
UniAudio-Token: Empowering Semantic Speech Tokenizers with General Audio Perception
May 29, 2026Yuhan Song, Linhao Zhang, Aiwei Liu +6Audio Tokens Tokenization
AnchorSteer: Self-Discovered Concept Injection for Structure-Preserving Music Editing
May 29, 2026Chih-Heng Chang, Keng-Seng Ho, Chih-Yu Tsai +3Music Editing Sheet Music
Towards Streaming Synchronized Spatial Audio Generation via Autoregressive Diffusion Transformer
May 29, 2026Ke Lei, Yu Zhang, Changhao Pan +4Spatial Audio Joint Audio-Video Generation
Audio Pirates: Black-box Audio Watermark Removal via Diffusion Priors
May 28, 2026Lingfeng Yao, Xincong Zhong, Chenpei Huang +6Watermarking Watermarks
3DAE: Binaural Quality Assessment for Audio Novel View Synthesis with Spatial Maps and Benchmark
May 28, 2026Jialu Xu, Yifan ZhouSpatial Audio Novel View Synthesis
Audio Jailbreaks in Large Audio-Language Models: Taxonomy, Attack-Defense Analysis, and Cost-Aware Evaluation
May 28, 2026Bo-Han Feng, Yu-Hsuan Li Liang, Chien-Feng Liu +2Large Language Model Jailbreaks Large Audio Language Models
COMET: Concept Space Dissection of the Modality Gap in Audio-Text Multimodal Contrastive Embeddings
May 28, 2026Yonggang Zhu, Liting Gao, Aidong Men +1Audio Representation Modality Gap
MusTBench: Benchmarking and Advancing Temporal Grounding in Music LLMs
May 28, 2026Daeyong Kwon, Qiyu Wu, Shinobu Kuriya +6Music Understanding Large Audio Language Models
Auditing Training Data in Generative Music Models via Black-Box Membership Inference
May 28, 2026Yi Chen Liu, Jiawei Yu, Kexin Cao +3Music Generation Membership Inference
Mind Your Tone: Does Tone Alter LLM Performance?
May 27, 2026Om Dobariya, Akhil KumarLarge Audio Language Models Multiple-Choice Questions
DEMON: Diffusion Engine for Musical Orchestrated Noise
May 27, 2026Ryan FosdickMusic Editing Instrument
Audio-Mind: An Auditable Agentic Framework for Audio Understanding
May 27, 2026Yucheng Wang, Jing Peng, Hanqi Li +6Audio Representation Voice Agents
EchoAvatar: Real-time Generative Avatar Animation from Audio Streams
May 27, 2026Bohong Chen, Yumeng Li, Yinglin Xu +3High-Fidelity Avatar Joint Audio-Video Generation
Unified Synthesis of Compositional Speech and Sound from Free-Form Text Prompts
May 27, 2026Yuyue Wang, Xihua Wang, Xin Cheng +2Zero-Shot Text-To-Speech Music Editing
KVoiceBench, KOpenAudioBench, and KMMAU: Agent-Driven Korean Speech Benchmarks for Evaluating SpeechLMs
May 27, 2026Haechan Kim, Seungjun Chung, Inkyu Park +2Multilingual Benchmark Korean
VoiceGiraffe: A Benchmark for Extreme Long-Context Audio-Language Understanding
May 27, 2026Jashin Ye, Dongxiao Wang, Yixuan Ye +10Large Audio Language Models Efficient Long-Context Inference
LoSATok: Low-dimensional Semantic-Acoustic Tokenizer for Cross-Domain Audio Understanding and Generation
May 27, 2026Zhisheng Zhang, Xiang Li, Yixuan Zhou +3Audio Tokens Audio Representation
Dasheng AudioGen: A Unified Model for Generating Coherent Audio Scenes from Text
May 27, 2026Jiahao Mei, Heinrich Dinkel, Yadong Niu +7Joint Audio-Video Generation Audio Latent
Do Audio LLMs Listen or Read? Analyzing and Mitigating Paralinguistic Failures with VoxParadox
May 26, 2026Jiacheng Pang, Ashutosh Chaubey, Mohammad SoleymaniParalinguistic Cues Large Audio Language Models
Escape the Language Prior: Mitigating Late-Stage Modality Collapse in Audio Reasoning via Modality-Aware Policy Optimization
May 26, 2026Cihan Xiao, Yiwen Shao, Chenxing Li +5Audio Representation Cross-Modal Learning
OmniMem: Perturbation-aware Memory Compression for Streaming Audio-Visual LLMs
May 26, 2026Guangzhi Sun, Yixuan Li, Yudong Yang +1Audio-Visual Reasoning Large Audio Language Models
MERIT: Learning Disentangled Music Representations for Audio Similarity
May 26, 2026Abhinaba Roy, Junyi Liang, Dorien HerremansMusic Understanding Audio Latent
Learning When to Think While Listening in Large Audio-Language Models
May 26, 2026Zhiyuan Song, Weici Zhao, Yang Xiao +3Large Audio Language Models Interaction Trajectories
FSA-GRPO: Teaching Auditory LLMs to Use Few-shot Demonstrations
May 26, 2026Haolong Zheng, Siyin Wang, Xulin Fan +2Large Audio Language Models Few-Shot Learning
Why Can't They Remember? Uncovering Representation and Retrieval Bottlenecks in Multi-Turn Acoustic Memory
May 26, 2026Yang Xiao, Siyi Wang, Han Yin +4Large Audio Language Models Large Language Model Memory
StreamSplit: Continuous Audio Representation Learning via Uncertainty-Guided Adaptive Splitting
May 26, 2026Minh K. Quan, Pubudu N. PathiranaAudio Latent Real-Time Inference
CosyEdit2: Speech-Editing-Oriented Reinforcement Learning Unlocks Better Zero-Shot TTS
May 25, 2026Junyang Chen, Yuhang Jia, Hui Wang +3Zero-Shot Text-To-Speech Music Editing
PitchBench: Measuring Pitch Hearing in Audio-Language Models
May 25, 2026Milan Liessens Dujardin, Song-Ze Yu, Craver Corbyn Thomas-Smith +2Large Audio Language Models Pitch
Baton: Explicit Semantic Blueprints for Joint Video-Audio Generation
May 24, 2026Shuyuan Tu, Qi Tian, Zihan Yang +9Joint Audio-Video Generation Audio Latent
Locality Matters for Training-Free Audio Token Compression in Audio-Language Models
May 24, 2026Jiale Luo, Xiaoyu Liang, Haoji HuAudio Representation Token Compression
Rethinking Continual Learning for Speech and Audio: A Representation-Centric Taxonomy and Open Problems
May 24, 2026Yang Xiao, Siyi Wang, Eun-Jung Holden +1Discrete Speech Representations Acoustic Representation
What Are We Actually Decoding? Source Attribution for Non-Invasive Brain-to-Language Retrieval
May 23, 2026Xinyu Zhang, Sichao Liu, Runhao Lu +2Neural Recordings Audio Representation
MixFake: Benchmarking and Enhancing Audio Deepfake Detection in Diverse Real-world Mixed Audio
May 22, 2026Qingcao Li, Yipeng Lin, Weichen Lian +3Audio Deepfake Detection Audio Flow
Eroding Trust in Real Speech: A Large-Scale Study of Human Audio Deepfake Perception
May 21, 2026Nicolas M. Müller, Wei Herng ChoongAudio Deepfake Detection Fake News
Direct content-based retrieval from music scores images
May 21, 2026Noelia Luna-Barahona, Antonio Ríos-Vila, Félix Fuentes-Hurtado +2Composed Image Retrieval Retrieval Task
Codec-Robust Attacks on Audio LLMs
May 19, 2026Jaechul Roh, Jean-Philippe Monteuuis, Jonathan Petit +1Neural Audio Codecs Large Audio Language Models
A conceptual framework for learning to listen by reward: Curiosity-driven search for novel sources
May 19, 2026Andreas Triantafyllopoulos, Jakub Šťastný, Alexios Terpinas +3Deep Reinforcement Learning Audio
Mega-ASR: Towards In-the-wild^2 Speech Recognition via Scaling up Real-world Acoustic Simulation
May 19, 2026Zhifei Xie, Kaiyu Pang, Haobin Zhang +4Large Audio Language Models Acoustic
Heterogeneity-Aware Dataset Scheduling for Efficient Audio Large Language Model Training
May 18, 2026Yanru Wu, Jianning Wang, Chongxin Gan +1Large Audio Language Models Audio Representation
A Survey of Large Audio Language Models: Generalization, Trustworthiness, and Outlook
May 18, 2026Kaiwen Luo, Zhenhong Zhou, Leo Wang +31Large Audio Language Models Large Language Model Safety
WavFlow: Audio Generation in Waveform Space
May 18, 2026Feiyan Zhou, Luyuan Wang, Shoufa Chen +6Joint Audio-Video Generation Audio Latent
InstructAV2AV: Instruction-Guided Audio-Video Joint Editing
May 18, 2026Haojie Zheng, Yixin Yang, Siqi Yang +2Joint Audio-Video Generation Video Editing
Acoustic Interference: A New Paradigm Weaponizing Acoustic Latent Semantic for Universal Jailbreak against Large Audio Language Models
May 18, 2026Yanyun Wang, Yu Huang, Zi Liang +2Large Audio Language Models Large Language Model Safety
Stable Audio 3
May 18, 2026Zach Evans, Julian D. Parker, Matthew Rice +4Audio Flow Autoregressive Diffusion
CounterFlow: A Two-Phase Inference-Time Sampling for Counterfactual Video Foley Generation
May 18, 2026Gyubin Lee, Junwon Lee, Juhan NamJoint Audio-Video Generation Audio Latent
Escaping the Linearity Trap: Manifold Detours for Black-Box Adversarial Attacks on Singing Audio Deepfake Detection
May 18, 2026Yifan Liao, Yule Liu, Zhen Sun +5Audio Deepfake Detection Singing Voice Conversion
Beyond Transcripts: Iterative Peer-Editing with Audio Unlocks High-Quality Human Summaries of Conversational Speech
May 17, 2026Kaavya Chaparala, Thomas Thebaud, Jesús Villalba López +3Accurate Transcription Summarization
Robust Audio Tagging under Class-wise Supervision Unreliability
May 17, 2026Yuanbo Hou, Zhaoyi Liu, Tong Ye +4Positive--Unlabeled Learning Audio
Omni-Customizer: End-to-End MultiModal Customization for Joint Audio-Video Generation
May 17, 2026Yuheng Chen, Qingdong He, Teng Hu +4Joint Audio-Video Generation Audio-Visual Consistency
Taming Audio VAEs via Target-KL Regularization
May 16, 2026Prem Seetharaman, Rithesh KumarAudio Latent Variational Autoencoder
Sound Sparks Motion: Audio and Text Tuning for Video Editing
May 14, 2026AmirHossein Naghi Razlighi, Aryan Mikaeili, Ali Mahdavi-Amiri +2Video Editing Audio Latent
From Text to Voice: A Reproducible and Verifiable Framework for Evaluating Tool Calling LLM Agents
May 14, 2026Md Tahmid Rahman Laskar, Xue-Yong Fu, Seyyed Saeed Sarfjoo +3Voice Agents Large Language Model Judges
SpeakerLLM: A Speaker-Specialized Audio-LLM for Speaker Understanding and Verification Reasoning
May 14, 2026KiHyun Nam, Jungwoo Heo, Siu Bae +2Speaker Verification Performance Speech Evaluation
AgentSteerTTS: A Multi-Agent Closed-Loop Framework for Composite-Instruction Text-to-Speech
May 14, 2026Bin Kang, Shaoguo Wen, Yang Fan +6Zero-Shot Text-To-Speech Acoustic Representation
AudioMosaic: Contrastive Masked Audio Representation Learning
May 14, 2026Hanxun Huang, Qizhou Wang, Xingjun Ma +3Audio Latent Audio Encoders
Masked Autoencoders with Limited Data: Does It Work? A Fine-Grained Bioacoustics Case Study
May 13, 2026Wuao Liu, Mustafa Chasmai, Subhransu Maji +1Passive Acoustic Monitoring Acoustic Representation
Direct Preference Optimization for English-Mandarin Code-Switching Speech Recognition in Audio LLMs
May 13, 2026Trung Nguyen Quang, Cheng Yi Lewis Won, Minh Duc Pham +3Code-Switching Large Audio Language Models

May 31, 2026 · cs.SDJ/K move · Enter open · S save

MelT: A Portable, Single-GEMM Mel Audio Frontend via Non-Uniform DFT with Measured Latency and Energy Gains on GPUs

Augusto Camargo, Marcelo Finger

Institute of Mathematics and Statistics (IME), University of São Paulo, São Paulo, Brazil

Modern neural audio models run on accelerators whose peak throughput comes from dense matrix multiplication, increasingly at the edge and in datacenters. The conventional acoustic frontend, however -- a Short-Time Fourier Transform (STFT) followed by sparse Mel aggregation -- remains a multi-stage pipeline centered on the Fast Fourier Transform (FFT), with execution overheads unlike the dense linear algebra dominating the inference stack. This work introduces MelT, a portable single-stage Mel frontend that precomputes Mel-spaced Non-Uniform Discrete Fourier Transform (NDFT) bases and applies them to time-domain frames through General Matrix Multiplication (GEMM). The contribution is a computational design principle: decoupling Mel feature extraction from vendor-specific FFT primitives and lowering it onto the matrix-multiplication substrate accelerators already optimize. It is not a new spectral operator. MelT's direct projection performs more arithmetic than the FFT pipeline. Yet in the compact-resolution regime of neural audio frontends, it achieves a 1.64-times to 3.29-times latency reduction and up to a 3.03-times reduction in measured active energy, from the Apple A18 Pro to the NVIDIA H100. All gains are within-platform comparisons, accompanied by task-level validation. Word error rate stays statistically equivalent to the native frontend's on frozen Whisper models of medium size and larger; speaker-attribute classification on VoxCeleb1 is non-inferior. The cepstral extension MFCCT preserves utility on a clinical respiratory-insufficiency classification task (SPIRA) while improving on the MFCC baseline. These results indicate that, in practical regimes on the accelerators evaluated here, hardware alignment rather than arithmetic count can govern the realized cost of feature extraction.

Matrix Multiplication Audio Fourier Low-Latency Systems Density Functional Theory

Open paper Ask about it PDF