Audio Representation Learning

Follow this topic Ask about this topic

Momentum

20 papers in the last four weeks, up 233% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 141

All topics

Alethia: A Foundational Encoder for Voice Deepfakes
Apr 30, 2026Yi Zhu, Brahmi Dwivedi, Jayaram Raghuram +1Audio Representation Learning Audio Deepfake Detection
Reality Defender Inc.
From Birdsong to Rumbles: Classifying Elephant Calls with Out-of-Species Embeddings
Apr 30, 2026Christiaan M. Geldenhuys, Thomas R. NieslerAudio Representation Learning Audio Understanding
Department of Electrical and Electronic Engineering University of Stellenbosch
Beyond the Baseband: Adaptive Multi-Band Encoding for Full-Spectrum Bioacoustics Classification
Apr 30, 2026Eklavya Sarkar, Marius Miron, David Robinson +6Audio Representation Learning Frequency-Domain Feature Learning
Earth Species Project
Multimodal LLMs are not all you need for Pediatric Speech Language Pathology
Apr 29, 2026Darren Fürst, Sebastian Steindl, Ulrich SchäferAudio Representation Learning Healthcare
Ostbayerische Technische Hochschule Amberg-Weiden
S-SONDO: Self-Supervised Knowledge Distillation for General Audio Foundation Models
Apr 27, 2026Mohammed Ali El Adlouni, Aurian Quelennec, Pierre Chouteau +2Model Compression Audio Representation Learning
LTCI, T´el´ecom Paris, Institut Polytechnique de Paris, Palaiseau, France
Enhancing Speaker Verification with Whispered Speech via Post-Processing
Apr 22, 2026Magdalena Gołębiowska, Piotr SygaAudio Representation Learning Speaker Verification
Department of Artificial Intelligence, Wroclaw University of Science and Technology, Wybrzeze Wyspianskiego 27, Wroclaw, 50-370, Poland
Audio Spoof Detection with GaborNet
Apr 21, 2026Waldek MaciejkoAudio Representation Learning Audio Deepfake Detection
Unsupervised Evaluation of Deep Audio Embeddings for Music Structure Analysis
Mar 28, 2026Axel MarmoretAudio Representation Learning Music Information Retrieval
BRAIN team, MEE, IMT Atlantique, Brest, France
Speech Codec Probing from Semantic and Phonetic Perspectives
Mar 11, 2026Xuan Shi, Chang Zeng, Tiantian Feng +3Audio Representation Learning Neural Audio Codecs
University of Southern California, USA · Dolby Laboratories, USA
An Empirical Analysis of Task-Induced Encoder Bias in Fréchet Audio Distance
Feb 27, 2026Wonwoo JeongText-to-Audio Generation Audio Representation Learning
Dept. of Computer Science and Engineering, Sogang University, South Korea
Scalable Music Cover Retrieval Using Lyrics-Aligned Audio Embeddings
Jan 16, 2026Joanne Affolter, Benjamin Martin, Elena V. Epure +2Audio Representation Learning Music Information Retrieval
Deezer Research - 24 Rue de Calais, 75009 Paris · EPFL DHLAB - Rte Cantonale, 1015 Lausanne
SITA: Learning Speaker-Invariant and Tone-Aware Speech Representations for Low-Resource Tonal Languages
Jan 14, 2026Tianyi Xu, Xuan Ouyang, Binwei Yao +4Audio Representation Learning Self-Supervised Speech Representation Learning
University of Wisconsin–Madison, Madison, WI, USA
Linear probing enables Ship-Radiated Noise recognition with pretrained audio embeddings
Jan 13, 2026Hilde I. Hummel, Sandjai Bhulai, Rob D. van der Mei +1Audio Representation Learning Linear Probing
Centre of Mathematics and Computer Science, Stochastics group, Science Park 123, Amsterdam, 1098 XG, Netherlands · Vrije Universiteit, Department of Mathematics, De Boelelaan 1111, Amsterdam, 1081 HV, Netherlands · Naturalis Biodiversity Center, Darwinweg 2, Leiden, 2333 CR, Netherlands
Categorize Early, Integrate Late: Divergent Processing Strategies in Automatic Speech Recognition
Jan 11, 2026Nathan Roll, Pranav Bhalerao, Martijn Bartelds +7Audio Representation Learning Hybrid CNN-Transformer Architectures
Stanford University
Harmonic-Percussive Disentangled Neural Audio Codec for Bandwidth Extension
Nov 26, 2025Benoît Giniès, Xiaoyu Bie, Olivier Fercoq +1Audio Representation Learning Neural Audio Codecs
LTCI, Télécom Paris, Institut Polytechnique de Paris Palaiseau, France
Revisiting Audio-language Pretraining for Learning General-purpose Audio Representation
Nov 20, 2025Wei-Cheng Tseng, Xuanru Zhou, Mingyue Huo +3Audio-Language Model Evaluation Audio Representation Learning
University of Texas at Austin · Tencent AI Lab Seattle · Zhejiang University +1
Perceptually Aligning Representations of Music via Noise-Augmented Autoencoders
Nov 7, 2025Mathias Rose Bjare, Giorgia Cantisani, Marco Pasini +2Audio Representation Learning Denoising Autoencoders
Johannes Kepler University, Linz, AT · STMS, CNRS, IRCAM, Sorbonne Universit´e, Paris, FR · Queen Mary University, London, UK +1
OpenBEATs: A Fully Open-Source General-Purpose Audio Encoder
Jul 18, 2025Shikhar Bharadwaj, Samuele Cornell, Kwanghee Choi +4Audio Representation Learning Audio QA
Carnegie Mellon University, USA · National Institute of Advanced Industrial Science and Technology (AIST), Japan
Structured-Noise Masked Modeling for Video, Audio and Beyond
Mar 20, 2025Aritra Bhowmik, Carlos Hinojosa, Fida Mohammad Thoker +2Audio Representation Learning Video Representation Learning
University of Amsterdam · King Abdullah University of Science and Technology
Audio-Language Models for Audio-Centric Tasks: A Systematic Survey
Jan 25, 2025Yi Su, Jisheng Bai, Qisheng Xu +2Audio Representation Learning Audio Understanding
College of Computer Science and Technology, National University of Defense Technology, Changsha, China · School of Communications and Information Engineering, Xi’an University of Posts and Telecommunications, Xi’an, China
ProPS: Prompted Profile Synthesis for Natural Language-Conditioned Speaker Embedding Distributions
Date pendingThomas Thebaud, Junhyeok Lee, Laureano Moro-Velazquez +2Audio Representation Learning Conditional Generative Modeling
Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, MD, USA · AMIAD, Palaiseau, France · Department of Signal Theory and Communications, Universidad Carlos III de Madrid (UC3M), Leganes, Spain

Apr 30, 2026, cs.SDJ/K move · Enter open · S save

Alethia: A Foundational Encoder for Voice Deepfakes

Yi Zhu, Brahmi Dwivedi, Jayaram Raghuram, Surya Koppisetti

Reality Defender Inc.

Audio Representation Learning Audio Deepfake Detection Speech Foundation Models

Open paper AskPDF arXiv

Existing voice deepfake detection and localization models rely heavily on representations extracted from speech foundation models (SFMs). However, downstream finetuning has now reached a state of diminishing returns. In this paper, we shift the focus to pretraining and propose a novel recipe that combines bottleneck masked embedding prediction with flow-matching based spectrogram reconstruction. The outcome, Alethia, is the first foundational audio encoder for various voice deepfake detection and localization tasks. We evaluate on

5

different tasks with

56

benchmark datasets, and note Alethia significantly outperforms state-of-the-art SFMs with superior robustness to real-world perturbations and zero-shot generalization to unseen domains (e.g., singing deepfakes). We also demonstrate the limitation of discrete targets in masked token prediction, and show the importance of continuous embedding prediction and generative pretraining for capturing deepfake artifacts.

Momentum

Related topics

Latest papers 141