Route-DMD: Distribution Matching Distillation with Task-Routed Mechanism for Speech Foundation Models

Yanru Huo; Zhikang Niu; Wenming Tu; Junxi Liu; Tian Tan; Shan Yang; Yipeng Kang; Ziyang Ma

cs.SDNov 5, 2026

Route-DMD: Distribution Matching Distillation with Task-Routed Mechanism for Speech Foundation Models

Authors: Yanru Huo, Zhikang Niu, Wenming Tu, Junxi Liu, Tian Tan, Shan Yang, Yipeng Kang, Ziyang Ma,

Organizations: MoE Key Lab of Artificial Intelligence, X-LANCE Lab, Shanghai Jiao Tong University · State Key Laboratory of General Artificial Intelligence, BIGAI · Shanghai Innovation Institute · Tencent Hunyuan ·

Abstract

Speech foundation models built on flow matching handle generation, editing, and processing in one system, but every request costs 20-50 iterative steps, and classifier-free guidance (CFG) doubles each of them. Distribution matching distillation (DMD) trains a student that approaches the teacher's quality in a few steps, yet existing speech distillation work applies it to generation alone. We show that a distribution-matching objective does not serve every task: in separation, the noised states of erroneous student outputs deviate far from the teacher's training distribution, where its prediction is unreliable. Route-DMD therefore makes the objective task-dependent, routing the affected tasks to a regression loss and keeping decoupled distribution matching elsewhere. The student is initialized by consistency distillation, and teacher rollouts use adaptive projected guidance (APG) instead of CFG. Without CFG, the 4-step student matches the 32-step teacher on generation, editing, and processing tasks at a 4.5 $\times$ end-to-end speedup.

Figures & tables

Figure 1: Overview of Route-DMD. Stage 1 uses consistency distillation for initialization. Stage 2 routes separation samples to a regression loss and other samples to Decoupled DMD.

Dataset	System	Metrics	Results
SeedTTS-Eval en\|zh\|zh-hard\|avg	Qwen3-TTS [ 25 ]	SIM $\uparrow$ / WER $\downarrow$	0.717 / 1.23 \| 0.770 / 1.22 \| 0.748 / 6.76 \| 0.745 / 3.07
	VoxCPM 2 [ 26 ]		0.753 / 1.84 \| 0.795 / 0.97 \| 0.753 / 8.13 \| 0.767 / 3.65
	AuK (32 steps)		0.788 / 1.02 \| 0.814 / 1.02 \| 0.782 / 5.91 \| 0.795 / 2.65
	w/ Route-DMD (4 steps)		0.772 / 1.03 \| 0.814 / 1.10 \| 0.784 / 6.43 \| 0.790 / 2.85
InstructTTSEval DSD-ZH\|DSD-EN	Qwen3-TTS-VD	ACC $\uparrow$	81.1 \| 82.4
	AuK (32 steps)		83.4 \| 81.6

Table 1: Comparison with prior systems and the teacher on representative tasks. The best results are in bold , and the second-best results are underlined .

Figure 2: Ablation of the task-routed mechanism on Libri2Mix. Separation metrics vs. Decoupled DMD updates for the regression loss (w/ routing), Decoupled DMD with the teacher queried on a noised ground-truth target rather than a noised student output (w/ noised GT), and plain Decoupled DMD (w/o routing).

System	N-MOS $\uparrow$	I-MOS $\uparrow$	RTF $\downarrow$
AuK (32 steps)	4.23 $\pm$ 0.10	4.50 $\pm$ 0.09	0.4333
w/ Route-DMD (4 steps)	4.34 $\pm$ 0.10	4.57 $\pm$ 0.09	0.0958

Table 2: Results of the subjective evaluation are reported as mean ± 95% confidence interval (CI) over all ratings. End-to-end RTF is measured on a single NVIDIA H20 GPU.

Figure 3: Ablation of initialization on SeedTTS-Eval. Each curve corresponds to a student initialized before DMD by consistency distillation (CD), ODE regression (ODE), meanflow distillation, or no initialization.

Figure 4: Ablation of guidance in the DMD teacher rollouts. SIM and WER on SeedTTS-Eval versus DMD updates.

Speech Foundation Models

Speech foundation models are large pretrained systems that learn reusable audio representations for speech understanding, recognition, and generation.

See all 69 papers

Consistent Distribution Matching for Data-Free Diffusion Distillation
Oct 6, 2026Yuxiang Fu, Qi Yan, Zike Wu +4Diffusion Model Distillation Diffusion Model Inference Acceleration
University of British Columbia · Vector Institute for AI · Canada CIFAR AI Chair
DMAD: Distribution Matching as Adversarial Distillation for Fast Visual Generation
Oct 1, 2026Zhengming Yu, Junkun Yuan, Haotian Yang +8Diffusion Model Distillation Few-Step Diffusion Sampling
Texas A&M University · Intelligent Creation, ByteDance
sDMD: Sinkhorn Distribution Matching Distillation
Oct 29, 2026Jaihoon Kim, Lingxiao Li, Soshi Shimada +3Diffusion Model Distillation Distribution Matching Distillation
KAIST · Netflix

Oct 6, 2026J/K move · Enter open · S save

Consistent Distribution Matching for Data-Free Diffusion Distillation

Yuxiang Fu, Qi Yan, Zike Wu, Yongxing Zhang+3

University of British Columbia · Vector Institute for AI · Canada CIFAR AI Chair

Diffusion Model Distillation Diffusion Model Inference Acceleration Distribution Matching Distillation Data-Free Knowledge Distillation

Open paper AskPDF arXiv

Flow and diffusion models suffer from slow inference due to computationally expensive numerical integration. Distillation provides a promising way for a student model to learn from a teacher's dynamics, enabling one-step or few-step generation. However, existing methods often depend on curated distillation datasets, costly teacher rollouts, or auxiliary proxy networks, which complicate model training and scaling. In this work, we propose Consistent Distribution Matching, a simulation-free and data-free distillation method for accelerating diffusion and flow models while preserving strong generative capacity. Our key insight is to unify sample generation and score estimation with one student network. Thus, our framework uses only two models, a frozen teacher and a trainable student, and optimizes one objective. We prove that minimizing our objective indicates Wasserstein convergence of the student flow-map pushforwards to the teacher marginals. On ImageNet 256

\times

256, our method attains an FID of 2.04 with a single function evaluation (1-NFE) and a 4-NFE FID of 1.37 within 40 epochs of training, surpassing the state-of-the-art distillation baselines without data. Our code code and model are available at https://consistentdmd.github.io/.

Abstract

Speech Foundation Models

Explore similar work

Consistent Distribution Matching for Data-Free Diffusion Distillation

DMAD: Distribution Matching as Adversarial Distillation for Fast Visual Generation

sDMD: Sinkhorn Distribution Matching Distillation