CodecFlow: Efficient Bandwidth Extension via Conditional Flow Matching in Neural Codec Latent Space

Bowen Zhang; Junchuan Zhao; Ian McLoughlin; Ye Wang; A S Madhukumar

cs.SDMar 2, 2026

CodecFlow: Efficient Bandwidth Extension via Conditional Flow Matching in Neural Codec Latent Space

Authors: Bowen Zhang, Junchuan Zhao, Ian McLoughlin, Ye Wang, A S Madhukumar

Organizations: Singapore Institute of Technology, Singapore · Nanyang Technological University, Singapore · National University of Singapore, Singapore

Abstract

Speech Bandwidth Extension improves clarity and intelligibility by restoring/inferring appropriate high-frequency content for low-bandwidth speech. Existing methods often rely on spectrogram or waveform modeling, which can incur higher computational cost and have limited high-frequency fidelity. Neural audio codecs offer compact latent representations that better preserve acoustic detail, yet accurately recovering high-resolution latent information remains challenging due to representation mismatch. We present CodecFlow, a neural codec-based BWE framework that performs efficient speech reconstruction in a compact latent space. CodecFlow employs a voicing-aware conditional flow converter on continuous codec embeddings and a structure-constrained residual vector quantizer to improve latent alignment stability. Optimized end-to-end, CodecFlow achieves strong spectral fidelity and enhanced perceptual quality on 8 kHz to 16 kHz and 44.1 kHz speech BWE tasks.

Figures & tables

Figure 1: An overview of the proposed CodecFlow framework, including (a) the overall model pipeline, (b) the architecture of the voicing extractor, and (c) the architecture of the flow prediction network from the flow embedding converter (FEC).

Figure 2: V/UV segmentation and LR–HR embedding cosine similarity over time. Upper: spectrogram with word boundaries and V/UV labels. Lower: cosine similarity; orange dashed regions mark UV-aligned drops, blue dashed line shows the global mean.

Models	8 kHz $\rightarrow$ 16kHz						8 kHz $\rightarrow$ 44.1 kHz
Models	LSD $\downarrow$	LSD-LF $\downarrow$	LSD-HF $\downarrow$	VISQOL $\uparrow$	MOS $\uparrow$	COL $\uparrow$	LSD $\downarrow$	LSD-LF $\downarrow$	LSD-HF $\downarrow$	VISQOL $\uparrow$	MOS $\uparrow$	COL $\uparrow$
Input (LR Speech)	5.36	1.16	7.49	2.83	3.77	3.20	5.50	0.81	6.07	2.16	3.85	3.28
Target (HR Speech)	-	-	-	4.51	4.29	4.06	-	-	-	4.73	4.50	4.32
NU-Wave2 [ 13 ]	1.75	0.51	2.42	2.60	3.19	2.84	1.50	0.72	1.62	2.19	3.49	2.98
AP-BWE [ 11 ]	1.24	0.46	1.69	2.42	3.97	3.79	1.14	0.23	1.25	3.23	4.44	4.25
Fre-Painter [ 12 ]	1.35	0.34	1.87	2.64	3.76	3.69	1.30	0.62	1.40	3.31	4.01	3.99

Table 1: Bandwidth extension evaluation at 16 kHz and 44.1 kHz targets with multiple baseline models. Best results are highlighted in bold , and second-best results are underlined .

Figure 3: Spectrogram comparison between the 8 kHz input, the 44.1 kHz ground truth (GT), and model outputs, including NU-Wave2, AP-BWE, FlowHigh, and the proposed CodecFlow.

Figure 4: Spectrogram comparison between the 8 kHz input, the 16 kHz ground truth (GT), and model outputs, including NU-Wave2, AP-BWE, FlowHigh, and the proposed CodecFlow.

Figure 5: Gender-wise comparison of objective and perceptual metrics under two bandwidth-extension settings (8 kHz

\rightarrow

16 kHz and 8 kHz

\rightarrow

44.1 kHz).

Task: 8 kHz $\to$ 16 kHz
Method	LSD $\downarrow$	LSD-LF $\downarrow$	LSD-HF $\downarrow$	VISQOL $\uparrow$	MOS $\uparrow$	COL $\uparrow$
CodecReg (w/o FT)	1.19	0.67	1.53	2.46	3.77	3.63
CFM-Conf (w/o FT)	1.33	0.72	1.73	2.24	3.82	3.70
CFM-UConf (w/o FT)	1.21	0.82	1.48	2.29	4.05	3.90
CodecFlow	1.01	0.64	1.27	2.72	4.25	4.04
Task: 8 kHz $\to$ 44.1 kHz

Table 2: Ablation study of converter variants in CodecFlow, where CodecReg denotes direct latent residual regression, CFM-Conf and CFM-UConf denote conditional flow–based converters with Conformer and U-Conformer backbones, “w/o FT” indicates training without fine-tuning, and CodecFlow denotes the full fine-tuned model.

Figure 6: Comparison of continuous codec latent embeddings across LR input, CodecReg, CodecFlow and HR ( 16 kHz ) ground truth.

Figure 7: Comparison of continuous codec latent embeddings across LR input, CodecReg, CodecFlow, and HR ( 44.1 kHz ) ground truth.

Speech Enhancement

Speech enhancement improves intelligibility and listening quality by suppressing noise, reverberation, and competing speakers in recorded or live audio.

See all 127 papers

Multi-Rate Bandwidth Extension by Token Completion in Neural Audio Codecs
Sep 29, 2026Benoît Ginies, Olivier Fercoq, Gaël RichardAudio Representation Learning Neural Audio Codecs
LTCI, Télécom Paris, Institut Polytechnique de Paris, Palaiseau, France
Harmonic-Percussive Disentangled Neural Audio Codec for Bandwidth Extension
Nov 26, 2025Benoît Giniès, Xiaoyu Bie, Olivier Fercoq +1Audio Representation Learning Neural Audio Codecs
LTCI, Télécom Paris, Institut Polytechnique de Paris Palaiseau, France
BridgeCodec: Mamba Enhanced Neural Audio Codec with Schrödinger Bridge at Low Bitrate
Oct 11, 2026Zijian Lin, Jing Yang, Jinghao Luo +3Neural Audio Codecs
Shenzhen International Graduate School, Tsinghua University, China · Central Media Technology Institute, Huawei

Sep 29, 2026J/K move · Enter open · S save

Multi-Rate Bandwidth Extension by Token Completion in Neural Audio Codecs

Benoît Ginies, Olivier Fercoq, Gaël Richard

LTCI, Télécom Paris, Institut Polytechnique de Paris, Palaiseau, France

Audio Representation Learning Neural Audio Codecs Audio Tokenization

Open paper AskPDF arXiv

Bandwidth extension, the task of reconstructing the high-frequency components of an audio signal from its low-passed counterpart, is a long-standing problem in audio processing. In this work, we extend recent advances in neural architectures by framing bandwidth extension as an audio token prediction problem. Specifically, we train a transformer-based language model on the discrete representations produced by a disentangled neural audio codec, where the disentanglement is guided by a Harmonic-Percussive decomposition of the input signals, highlighting spectral structures particularly relevant for bandwidth extension. Our approach introduces a novel codec design that explicitly accounts for the downstream token prediction task, enabling a more effective coupling between codec structure and transformer modeling. This joint design yields high-quality reconstructions of the original signal, as measured by both objective metrics and subjective evaluations. These results highlight the importance of aligning codec disentanglement and representation learning with the generative modeling stage, and demonstrate the potential of global, representation-aware design for advancing bandwidth extension.

Abstract

Speech Enhancement

Explore similar work

Multi-Rate Bandwidth Extension by Token Completion in Neural Audio Codecs

Harmonic-Percussive Disentangled Neural Audio Codec for Bandwidth Extension

BridgeCodec: Mamba Enhanced Neural Audio Codec with Schrödinger Bridge at Low Bitrate