OP-CAD: On-Policy Clean-Audio Distillation for Robust Audio-Visual Reasoning

Xingming Shui; Dapeng Chen; Bowei Liu; Jingqi Tian; Minfu Li; Kun Yi; Jiapeng Hong; Yansong Tang

cs.CVSep 30, 2026

OP-CAD: On-Policy Clean-Audio Distillation for Robust Audio-Visual Reasoning

Authors: Xingming Shui, Dapeng Chen, Bowei Liu, Jingqi Tian, Minfu Li, Kun Yi, Jiapeng Hong, Yansong Tang

Organizations: Tsinghua Shenzhen International Graduate School, Tsinghua University · Independent Researcher

Abstract

Omni-modal large language models deployed in real-world environments encounter external noise that can interfere with their perception and understanding of multimodal inputs. We study their robustness in audio-visual understanding, focusing on question answering under environmental noise and competing speech. The challenge is to resist acoustic interference while preserving useful audio evidence. On-policy distillation provides dense teacher feedback on student-generated responses, but uniform token weighting does not explicitly prioritize positions affected by acoustic interference. We introduce OP-CAD (On-Policy Clean-Audio Distillation), a curriculum-based privileged self-distillation framework for robust audio-visual understanding. Training progresses from mild to severe environmental noise and competing speech, with selective token-level supervision at each stage. The student generates responses from corrupted audio-visual input, while a frozen teacher uses clean audio and the verified answer to supervise the same response prefixes. To allocate this supervision, OP-CAD compares teacher predictions under clean, corrupted, and visual-only contexts without revealing the answer. These matched comparisons measure sensitivity to audio removal and corruption; a bounded weighting rule emphasizes positions identified by either signal while retaining supervision throughout the response. OP-CAD outperforms the compared methods across all evaluated noise conditions. Paired analyses further show improved preservation of clean-correct answers under strong interference, with no observed aggregate clean-accuracy penalty. These results demonstrate the value of directing clean-teacher supervision toward acoustically sensitive predictions for robust audio-visual reasoning.

Figures & tables

Panel of Figure 1 — Figure 1: Prediction examples at 0 dB: (a) environmental noise; (b) competing speech.

Figure 2: Overview of the OP-CAD training framework.

Figure 3: Token-level supervision allocation in OPSD and OP-CAD.

	Daily-Omni			OmniVideoBench
Method	Clean	Env.	Speech	Clean	Env.	Speech
Base	67.75	61.38	57.84	37.40	36.70	34.80
SFT warm-up	68.76	61.74	59.82	38.10	36.80	35.93
+ SFT	68.42 $\downarrow$ 0.34	61.88 $\uparrow$ 0.14	59.12 $\downarrow$ 0.70	39.00 $\uparrow$ 0.90	36.23 $\downarrow$ 0.57	36.20 $\uparrow$ 0.27
+ GRPO	68.34 $\downarrow$ 0.42	62.91 $\uparrow$ 1.17	59.93 $\uparrow$ 0.11	41.50 $\uparrow$ 3.40	37.87 $\uparrow$ 1.07	36.87 $\uparrow$ 0.94
+ OPSD	69.34 $\uparrow$ 0.58	63.49 $\uparrow$ 1.75	60.62 $\uparrow$ 0.80	41.40 $\uparrow$ 3.30	37.40 $\uparrow$ 0.60	35.13 $\downarrow$ 0.80

Table 1: Full test accuracy (%). Env./Speech: three-SNR averages. Bold: best result. Arrows: changes from SFT warm-up in percentage points, using displayed scores.

Daily-Omni
Cond.	Method	Question Type						Video Duration		Avg.
Cond.	Method	AV Align	Comp.	Ctx. Und.	Evt. Seq.	Infer.	Reas.	30s	60s	Avg.
Clean	Base	58.82	77.86	60.62	60.13	81.82	81.14	68.01	67.45	67.75
	+ SFT	61.34 $\uparrow$ 2.52	77.10 $\downarrow$ 0.76	64.77 $\uparrow$ 4.15	57.84 $\downarrow$ 2.29	83.77 $\uparrow$ 1.95	80.57 $\downarrow$ 0.57	68.62 $\uparrow$ 0.61	68.18 $\uparrow$ 0.73	68.42 $\uparrow$ 0.67
	+ GRPO	59.24 $\uparrow$ 0.42	79.39 $\uparrow$ 1.53	63.73 $\uparrow$ 3.11	59.80 $\downarrow$ 0.33	82.47 $\uparrow$ 0.65	80.00 $\downarrow$ 1.14	70.48 $\uparrow$ 2.47	65.82 $\downarrow$ 1.63	68.34 $\uparrow$ 0.59
	+ OPSD	56.72 $\downarrow$ 2.10	77.86 $\leftrightarrow$ 0.00	68.91 $\uparrow$ 8.29	60.78 $\uparrow$ 0.65	84.42 $\uparrow$ 2.60	82.29 $\uparrow$ 1.15	71.41 $\uparrow$ 3.40	66.91 $\downarrow$ 0.54	69.34 $\uparrow$ 1.59

Table 2: Daily-Omni accuracy (%) by question type and duration. Arrows: changes from Base within each condition in percentage points, using displayed scores. Bold: best per column.

OmniVideoBench
Cond.	Method	Audio Type			Video Duration				Avg.
Cond.	Method	Music	Sound	Speech	(0,1] min	(1,5] min	(5,10] min	(10,30] min	Avg.
Clean	Base	30.77	32.65	39.11	49.12	35.29	36.40	33.33	37.40
	+ SFT	32.97 $\uparrow$ 2.20	37.41 $\uparrow$ 4.76	40.03 $\uparrow$ 0.92	47.95 $\downarrow$ 1.17	39.41 $\uparrow$ 4.12	35.96 $\downarrow$ 0.44	35.25 $\uparrow$ 1.92	39.00 $\uparrow$ 1.60
	+ GRPO	36.26 $\uparrow$ 5.49	36.73 $\uparrow$ 4.08	43.04 $\uparrow$ 3.93	49.12 $\leftrightarrow$ 0.00	40.59 $\uparrow$ 5.30	39.47 $\uparrow$ 3.07	39.46 $\uparrow$ 6.13	41.50 $\uparrow$ 4.10
	+ OPSD	28.57 $\downarrow$ 2.20	40.82 $\uparrow$ 8.17	43.04 $\uparrow$ 3.93	48.54 $\downarrow$ 0.58	42.06 $\uparrow$ 6.77	40.79 $\uparrow$ 4.39	36.40 $\uparrow$ 3.07	41.40 $\uparrow$ 4.00

Table 3: OmniVideoBench accuracy (%) by audio type and duration. Arrows: changes from Base within each condition in percentage points, using displayed scores. Bold: best per column.

	Daily-Omni			OmniVideoBench
Variant	Clean	Env.	Speech	Clean	Env.	Speech
OPSD	69.34	63.49	60.62	41.40	37.40	35.13
Audio-rem. proxy	69.34	64.47	61.71	40.70	39.63	38.50
Corr.-sens. proxy	68.76	65.41	62.13	43.80	39.73	39.00
OP-CAD	70.09	66.97	63.55	41.60	40.23	38.93

Table 4: Accuracy (%) with single-signal and combined allocation. Bold marks the best result.

Training schedule	Daily-Omni			OmniVideoBench
Training schedule	Clean	Env.	Speech	Clean	Env.	Speech
Randomly mixed	69.93	65.16	62.67	42.30	38.27	37.13
Curriculum learning	70.09	66.97	63.55	41.60	40.23	38.93

Table 5: Curriculum order ablation: accuracy (%) under matched training budgets.

Figure 4: Prediction stability at 0 dB: harm rate (left) and clean-correct retention (right).

Figure 5: Token-level supervision allocation on an OP-CAD training rollout.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Source	Questions	10 dB	5 dB	0 dB	Total views
WorldSense	619	416	524	712	1,652
OmniBench	291	241	281	317	839
OmniInstruct-v1 (AVQA)	899	587	697	885	2,169
Total	1,809	1,244	1,502	1,914	4,660

Appendix

Table 6: Robustness-data partition after paired screening. SNR columns count corrupted acoustic views.

Source	Questions	10 dB	5 dB	0 dB	Total views
WorldSense	276	210	205	438	853
OmniBench	97	61	65	134	260
OmniInstruct-v1 (AVQA)	242	160	167	350	677
Total	615	431	437	922	1,790

Appendix

Table 7: Warm-up data after paired screening.

Setting	Value	Setting	Value
Backbone	Qwen3-Omni-Instruct	Model / data-order seed	42 / 42
Student initialization	Shared SFT warm-up checkpoint (1 epoch)	Gradient accumulation	1
Teacher	Frozen copy of the SFT warm-up checkpoint	Video sampling	1 FPS, at most 256 frames
Trainable adaptation	LoRA on the language backbone and multimodal projectors	Per-frame pixel budget	262,144
Frozen components	Acoustic and visual encoders	Audio-duration cap	None
LoRA rank / scale	64 / 128	Training objective	Counterfactual-weighted reverse-KL OPSD

Appendix

Table 15: Main training configuration.

Explore similar work

X $^3$ -OPD: Distilling Reasoning into Large Audio-Language Models via On-Policy Alignment
Jul 23, 2026Dongjie Fu, Di Cao, Xize Cheng +6Large Audio Language Models Reasoning Skills
1Tencent Hunyuan · 2Zhejiang University
OPOD: On-Policy Omni Distillation
Jul 23, 2026Tong Zhao, Yuyang Hu, Yutao Zhu +5Online-Policy Distillation Omni-Modal
1Renmin University of China · 2Tencent
EchoDistill: Robust Large Audio Language Models via Noisy-to-Clean Self-Distillation
May 11, 2026Kaiwen Luo, Chunxi Luo, Liang Lin +5Large Audio Language Models Audio Understanding
NTU · SHU · ICT, CAS +4

Jul 23, 2026, cs.LGJ/K move · Enter open · S save

X $^3$ -OPD: Distilling Reasoning into Large Audio-Language Models via On-Policy Alignment

Dongjie Fu, Di Cao, Xize Cheng, Zihan Zhang+5

1Tencent Hunyuan · 2Zhejiang University

Large Audio Language Models Reasoning Skills Tool-Grounded Reasoning Rp-Opsd On-Policy

Open paper AskPDF arXiv

While large audio-language models have achieved remarkable progress in auditory perception, they still lag behind text-based large language models in deep logical reasoning, primarily due to the scarcity of high-quality audio reasoning data. To bridge this gap, we propose X

^3

-OPD, a cross-modal on-policy distillation framework that transfers reasoning capabilities from a powerful text teacher to an audio-language student. During training, the student generates reasoning trajectories conditioned on its own acoustic perception, while the teacher provides token-level guidance using matched textual inputs and verified answers. We further construct a three-tier symmetric corpus covering textual reasoning rendered into speech, audio-event reasoning grounded in complex acoustic scenes, and spoken-dialogue reasoning involving paralinguistic cues. This design extends cross-modal distillation beyond textually recoverable content to reasoning grounded in non-linguistic events, prosody, and conversational context. Experiments on MMSU, MMAU, BIG Bench Audio, and MMAR demonstrate that X

^3

-OPD substantially improves audio-grounded reasoning and chain-of-thought quality while largely preserving the model's existing capabilities under domain shift.

Abstract

Explore similar work

X3^33-OPD: Distilling Reasoning into Large Audio-Language Models via On-Policy Alignment

OPOD: On-Policy Omni Distillation

EchoDistill: Robust Large Audio Language Models via Noisy-to-Clean Self-Distillation

X $^3$ -OPD: Distilling Reasoning into Large Audio-Language Models via On-Policy Alignment