X-OPD: Cross-Modal On-Policy Distillation for Capability Alignment in Speech LLMs

Di Cao; Dongjie Fu; Hai Yu; Siqi Zheng; Xu Tan; Tao Jin

eess.ASMar 6, 2026

X-OPD: Cross-Modal On-Policy Distillation for Capability Alignment in Speech LLMs

Authors: Di Cao, Dongjie Fu, Hai Yu, Siqi Zheng, Xu Tan, Tao Jin

Organizations: Tencent Hunyuan · Zhejiang University

Abstract

While the shift from cascaded dialogue systems to end-to-end (E2E) speech Large Language Models (LLMs) improves latency and paralinguistic modeling, E2E models often exhibit a significant performance degradation compared to their text-based counterparts. The standard Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) training methods fail to close this gap. To address this, we propose X-OPD, a novel Cross-Modal On-Policy Distillation framework designed to systematically align the capabilities of Speech LLMs to their text-based counterparts. X-OPD enables the Speech LLM to explore its own distribution via on-policy rollouts, where a text-based teacher model evaluates these trajectories and provides token-level feedback, effectively distilling teacher's capabilities into student's multi-modal representations. Extensive experiments across multiple benchmarks demonstrate that X-OPD significantly narrows the gap in complex tasks while preserving the model's inherent capabilities.

Figures & tables

Figure 1: Overview of the proposed Cross-Modal On-Policy Distillation (X-OPD) framework.

Model Series	BIG Bench Audio		Audio Multi-Chall.		xx Voice Bench xx		Avg. Drop (%)
Model Series	$S$	$T$	$S$	$T$	$S$	$T$	$\Delta S$	$\Delta T$
GPT-4o 1 1 1 Version: gpt-4o-2024-11-20 (Base, Text-modality)	–	90.27	–	32.13	–	92.61	–	–
GPT-4o-Audio 2 2 2 Version: gpt-4o-audio-preview-2025-06-03	88.50	–	24.57	–	89.69	–	9.55	–
Gemini-2.5-Flash	81.42	98.50	26.37	43.22	86.13	94.32	21.67	–
Gemini-3-Flash-Preview	80.77	95.28	42.70	49.21	92.21	95.32	10.57	–
Ministral-3-3B (Base, Text-modality)	–	59.70	–	22.67	–	76.03	–	–

Table 1: Performance comparison across different model series and training paradigms. We report performance scores across multiple benchmarks for both speech-input (S) and text-input (T) modalities. The Avg. Drop (%) measures the aggregate performance gap relative to each series’ original Base Model (for Gemini, its own text modality serves as the baseline).

Method	BIG Bench Audio		Audio Multi-Chall.		Voice Bench		Avg. Drop (%)
Method	$S$	$T$	$S$	$T$	$S$	$T$	$\Delta S$	$\Delta T$
Reference (Text-only Models)
Qwen3-A22B-Instruct	–	99.10	–	35.73	–	94.09	–	–
Qwen3-A3B-Instruct	–	97.72	–	29.31	–	91.00	–	–
Qwen3-Omni-A3B-Instruct	85.67	92.10	23.57	26.14	89.22	91.04	11.29	5.51
X-OPD	93.41	96.85	28.14	28.66	89.29	91.18	3.43	0.97

Table 2: Ablation Study of X-OPD. The results are presented in the same format as Table 1 . The default X-OPD configuration is based on the Qwen3-Omni-A3B-Instruct, utilizing Qwen3-A3B-Instruct as the teacher with

\lambda=0.5

Method	MMAR (%)	Drop
Original Model	71.3	-
SFT	60.3	11.0
Offline KD	59.9	11.4
GKD (Forward KL)	60.0	11.3
X-OPD ( $\lambda=0$ )	69.8	1.5
X-OPD ( $\lambda=0.5$ )	69.3	2.0

Table 3: Analysis of catastrophic forgetting on the MMAR benchmark. Total accuracy (

\uparrow

) is reported.

Explore similar work

Efficient Training for Cross-lingual Speech Language Models
Apr 13, 2026Yan Zhou, Qingkai Fang, Yun Hong +1Speech Language Models Cross-Modal Alignment
Key Laboratory of Intelligent Information Processing, Institute of Computing Technology, Chinese Academy of Sciences (ICT/CAS) · State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences, Beijing, China
X $^3$ -OPD: Distilling Reasoning into Large Audio-Language Models via On-Policy Alignment
Jul 23, 2026Dongjie Fu, Di Cao, Xize Cheng +6Cross-Modal Knowledge Distillation Audio Reasoning
1Tencent Hunyuan · 2Zhejiang University
TS-OPD: Reconciling ASR and QA in Speech Language Models via Task-Specific On-Policy Distillation
Sep 24, 2026Yujie Guo, Hongjie Chen, Jian Kang +3Automatic Speech Recognition On-Policy Distillation
College of Computer Science, Nankai University · Xingchen AGI Lab, China Telecom Artificial Intelligence Technology Co. Ltd

Apr 13, 2026J/K move · Enter open · S save

Efficient Training for Cross-lingual Speech Language Models

Yan Zhou, Qingkai Fang, Yun Hong, Yang Feng

Key Laboratory of Intelligent Information Processing, Institute of Computing Technology, Chinese Academy of Sciences (ICT/CAS) · State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences, Beijing, China

Speech Language Models Cross-Modal Alignment Cross-Lingual Representation Alignment

Open paper AskPDF arXiv

Currently, large language models (LLMs) predominantly focus on the text modality. To enable more natural human-AI interaction, speech LLMs are emerging, but building effective end-to-end speech LLMs remains challenging due to limited data and the difficulty in expanding to more languages. In this paper, we introduce Cross-lingual Speech Language Model (CSLM), an efficient training method for cross-lingual speech LLMs based on discrete speech tokens. We propose a novel alignment strategy that achieves cross-modal and cross-lingual alignment through continual pre-training. By conducting instruction fine-tuning following a speech-text interleaved chain-of-modality generation process, we enhance modal alignment at a finer granularity, thereby improving generation quality and reducing latency. CSLM aligns different modalities and languages simultaneously without the need for massive speech data, thus exhibiting good language scalability. Evaluations on cross-modal tasks, mono-lingual conversational tasks, and cross-lingual conversational tasks demonstrate CSLM's strong cross-modal alignment capabilities and general task abilities. (Code is available at: https://github.com/ictnlp/CSLM)

Abstract

Explore similar work

Efficient Training for Cross-lingual Speech Language Models

X3^33-OPD: Distilling Reasoning into Large Audio-Language Models via On-Policy Alignment

TS-OPD: Reconciling ASR and QA in Speech Language Models via Task-Specific On-Policy Distillation

X $^3$ -OPD: Distilling Reasoning into Large Audio-Language Models via On-Policy Alignment