Multi-Teacher On-Policy Distillation (MOPD) enables a single student to absorb the skills of several domain teachers by learning from its own generated responses. We present Multimodal Model Multi-Teacher On-Policy Distillation Reasoner (M3OPD-Reasoner), a fully open recipe for post-training vision-language models (VLMs) over three reasoning domains: image STEM, image perception, and video. We curate two datasets, M3OPD-SFT-1.35M for supervised fine-tuning (SFT) and M3OPD-RL-193K for reinforcement learning (RL) with verifiable rewards. On these data, we compare post-training recipes that start from Qwen3.5-9B: training one model on the mixed data of all domains (MixSFT and MixRL), and training one teacher per domain with Group Sequence Policy Optimization (GSPO), directly or after an SFT warm-up, and distilling the teachers into one student with MOPD. We evaluate on 21 image STEM, image perception, and video reasoning benchmarks after we decontaminate the training data against them. The comparison establishes MOPD of teachers trained with GSPO directly from Qwen3.5-9B as the best recipe: the student closely matches each teacher on its own domain and is on average above the mixed-training baselines on the same prompts. Since MOPD already merges the domain capabilities that MixRL would otherwise have to learn, its student is a stronger starting point for RL, and we ask whether the same data can carry the model further from there: we add MixRL on top of the MOPD student, on the same prompts and with the same settings, which, to our knowledge, no prior VLM recipe has studied. The same prompts still improve the MOPD student, with no collapse, and give our main model, M3OPD-Reasoner-9B. MOPD and MixRL can therefore work as consecutive stages that get more out of the same data. We release all data, the training code, and the checkpoints of every stage.
Figures & tables
Figure 2: Overview of the recipe. (1) GSPO on the prompts of one domain turns Qwen3.5-9B into the teacher of that domain, directly or after an SFT warm-up on the traces of the domain. (2) MOPD distills the domain teachers into one student that starts from Qwen3.5-9B, on the union of the prompts. (3) MixRL continues the MOPD student on the same prompts and gives the main model, M 3 OPD-Reasoner-9B. The baselines train one model on the mixed data with MixRL, after MixSFT or directly from Qwen3.5-9B, without teachers or MOPD.
Domain (avg@8, %)
Qwen3.8-Flash-Next traces
Self traces
Δ
∙ Image STEM
60.6
63.4
+2.8
∙ Image perception
77.2
77.7
+0.5
∙ Video
59.2
59.1
− 0.1
Table 1: SFT on Qwen3.8-Flash-Next traces and on self traces of Qwen3.5-9B (domain mean of the SFT model of each domain on the benchmarks of its domain, avg@8 accuracy, %). Δ is the change from the left to the right value.
Table 2: Results stage by stage (domain means and the mean of all 21 benchmarks, avg@8 accuracy, %). The check marks show the training stages of each row: SFT, GSPO on the prompts of one domain (Domain RL), MOPD, and MixRL. † Three models, one per domain, each evaluated on the benchmarks of its own domain. Bold marks the best in each column, without the † rows.
Table 3: Results on all 21 benchmarks (avg@8 accuracy, %). Each domain block starts with the domain mean, and “Overall” averages all 21 benchmarks. Δ is the change from Qwen3.5-9B. The mixed-training columns show MixRL directly from Qwen3.5-9B, MixSFT, and MixRL from MixSFT (the MixRL baseline). For the teachers with and without the SFT warm-up, we show the domain teachers, their MOPD student, and MixRL on top of that student. ⋆ marks M 3 OPD-Reasoner-9B. In each row, bold marks the best among all columns except the teachers. The domain teachers are three models, one per domain (colored bar), each evaluated only on its own domain, so their “Overall” combines the three models.
Figure 4: Gain over Qwen3.5-9B on each benchmark (avg@8 accuracy, points), with the domain mean first. Solid bars show MOPD of the teachers trained directly from Qwen3.5-9B (value on the right, which also sorts the benchmarks), and outlined bars show MOPD of the teachers with the SFT warm-up. An arrow with a value marks a gain outside the axis range. Both MOPD students improve on most benchmarks, and the MixRL baseline stays below them on most benchmarks.
Figure 5: Change of MixRL on top of MOPD over its MOPD student on each benchmark (avg@8 accuracy, points), with the domain mean first. Solid bars show M 3 OPD-Reasoner-9B over the MOPD student of the teachers trained directly from Qwen3.5-9B (value on the right, which also sorts the benchmarks), and outlined bars show MixRL on top of the MOPD student of the teachers with the SFT warm-up. For both students, MixRL raises most benchmarks, and the few drops are small.
Table 4: Comparison with published results on 16 image benchmarks (left) and 5 video benchmarks (right), in accuracy (%). The results of M 3 OPD-Reasoner-9B are avg@8. Best results are bold and the second-best are underlined . OpenMMReasoner, InternVL3.5 and MMFineReason use the values of MMVistaReason [ 37 ] , and Video-R1 on Video-Holmes that of the Video-Holmes paper [ 11 ] . On EMMA and We-Math, OneThinker uses the values of [ 27 , 78 ] , OpenMMReasoner those of [ 74 ] and [ 31 ] , and InternVL3.5 on We-Math its own.
Figure 6: MixRL in the two mixed-training runs, from MixSFT (the MixRL baseline) and from Qwen3.5-9B, and from the two MOPD students, of the teachers with and without the SFT warm-up (the latter gives M 3 OPD-Reasoner-9B). (a) Mean accuracy of the responses on the sampled training prompts, including the prompts that the group filter drops. (b) Share of the sampled prompts that the group filter drops: prompts with all eight responses correct (8/8, solid) and with no correct response (0/8, dashed). (c) Mean response length. (d) Mean token entropy of the policy. Training metrics are 10-step moving averages.
Table 5: Reasoning behavior on the benchmarks (mean over the benchmarks of each domain). Len: thinking characters per response (K). H/E: thinking length on hard over easy questions. Chk: self-checks per 1K thinking characters. Loop, Hdg: responses with a loop or a hedge (%). TS: cited timestamps per 1K thinking characters. † One model per domain.
Prompts
Size
STEM
Perc.
Video
Overall
MOPD student (start)
–
74.8
80.6
62.8
74.2
Full union
193,027
75.3
81.2
63.7
74.8
Drop h∈{0,8}
56,713
75.4
80.6
63.6
74.6
Keep h≤4
60,863
74.8
80.6
62.6
74.1
Table 6: MixRL on top of MOPD with and without a prompt filter, for the teachers with the SFT warm-up (domain means and overall mean, avg@8 accuracy, %). h is the number of solved responses ( Racc≥0.5 ) of the MOPD student among eight samples of a prompt.
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Source
SFT
RL
Image STEM
MMFineReason-SFT [ 36 ]
323,510
–
MMVistaReason-SFT (math, science, logic) [ 37 ]
30,990
–
FineVision [ 76 ] : TabMWP, ScienceQA, CLEVR-Math
13,802
–
DeepVision-103K [ 62 ]
–
23,486
Vero-600K, STEM subsets [ 58 ]
–
18,093
Appendix
Table 7: Data sources and sizes after deduplication and decontamination. “SFT” is the SFT set of the domain, which serves the SFT warm-up of the domain teacher that we compare and is a part of the MixSFT set, and “RL” the RL prompt set of the domain teacher. The union of the three RL sets is the prompt set of MOPD and MixRL. The image SFT sets carry self traces of Qwen3.5-9B, and the video SFT set carries traces of Qwen3.8-Flash-Next [ 56 ] . DeepVision-103K keeps math items with a pass rate in (0,0.5] and ViRL39K items with a pass rate of at most 0.25 (hard items). a The RL part holds out the DocVQA, TextVQA, ST-VQA, InfographicVQA questions of FineVision (15% of their images by hash).
Answer type
Image STEM
Image perception
Video
All
multiple choice
28,889
10,831
30,853
70,573
math
30,732
23,643
2,071
56,446
short answer
4,522
6,479
–
11,001
counting
164
7,484
14,000
21,648
OCR
–
9,979
–
9,979
grounding (box/point)
–
9,499
–
9,499
Appendix
Table 8: RL prompts by answer type. The answer type selects the format instruction and the answer matcher (Table 11 ).
Check
Rule
Text, exact
same normalized question
Text, n -gram
13-gram overlap ≥ 0.5 of the item (or ≥ 0.9 incl. templates)
Image, hash
same SHA-1
Image, pHash
64-bit distance ≤ 4, confirmed by 256-bit ≤ 10
Video, id
same video id (YouTube id of clips)
Video, frames
pHash of frames at 0.25, 0.5, 0.75 of the length, ≤ 4
Appendix
Table 9: Decontamination against the evaluation benchmarks. Questions shorter than 50 characters are not indexed for text matching, and n -grams shared by more than 20 training rows count as templates.
Type
Matching rule
Image STEM
Image perc.
Video
Math
Math-Verify, units, % = decimal
30.7K
23.6K
2.1K
Mult. choice
option letter or letter set
28.9K
10.8K
30.9K
Short
normalized text or number
4.5K
6.5K
–
Count
integer or number word
164
7.5K
14.0K
OCR
ANLS ≥ 0.5, digit guard
–
10.0K
–
Box/point
matched IoU, point radius
–
9.5K
–
Appendix
Table 10: Answer types of the RL prompt sets and the rule that scores each type. The accuracy term Racc is binary for the first four types and continuous in [0,1] for the last three.
Answer type
Format instruction (appended to the question)
Matcher
Details
multiple choice
Answer with the option letter only, inside \boxed{}.
option letter or set of letters
letter forms (“B”, “(B)”, “B. text”); multi-answer sets
math
Put your final answer within \boxed{}.
Math-Verify + numeric fallback
units must agree, % ↔ decimal, relations never match a value; rounding 6 digits
short answer
Put your final short answer within \boxed{}.
normalized string + numeric fallback
as math for numbers; multi-part answers part by part
counting
Put your final answer as a single integer within \boxed{}.
integer
number words, “about 7”, “7 birds”
OCR
Put the exact text of your final answer within \boxed{}.
ANLS
threshold 0.5; different digit sequences score 0
grounding (box/point)
Output the box as [x1, y1, x2, y2] in 0-1000 relative coordinates inside \boxed{}.
Hungarian-matched IoU
coordinates in [0,1000] ; sum IoU / max(#pred, #gold); point gold: within 20 units or inside the box
Appendix
Table 11: Answer types, format instructions and answer matchers of the rule-based reward (Math-Verify 0.9.0). The reward is R=0.9Racc+0.1Rfmt , and Racc counts only when Rfmt=1 .
Benchmark
Dataset
Subset
Split
n
Image STEM
DynaMath [ 110 ]
kcz358/DynaMath
–
test
5,010
EMMA [ 23 ]
lmms-lab/EMMA
All
test
2,788
LogicVista [ 81 ]
lscpku/LogicVista
–
test
448
MathVerse [ 104 ]
CaraJ/MathVerse-lmmseval
testmini
testmini
3,940
MathVista [ 44 ]
AI4Math/MathVista
–
testmini
1,000
Appendix
Table 12: Evaluation benchmarks. n is the number of scored items. DynaMath reports the accuracy over all generated variants.
Domain teachers
SFT warm-up c
GSPO
MOPD
MixRL a
Init
Qwen3.5-9B
Qwen3.5-9B c
Qwen3.5-9B
final MOPD
Data
domain SFT set
domain RL set
union (193K)
union (193K)
Learning rate
1e-5, cosine
1e-6, const.
1e-6, const.
1e-6, const.
Warmup
3%
20 steps
20 steps
20 steps
Weight decay / grad. norm
0.0/1.0
0.1/1.0
0.1/1.0
0.1/1.0
Appendix
Table 13: Overview of the settings of all stages. a MixRL on top of MOPD. The MixRL baseline starts from MixSFT and trains on the same union (Table 15 ). b Packed to 8192 tokens (domain SFT) or 16384 tokens (MixSFT). c Only for the teachers with the SFT warm-up, where GSPO starts from the SFT warm-up of the domain. Off-policy: maximum staleness in policy versions and the truncation of the token importance weight.
Setting
Image STEM
Image perception
Video
MixSFT
Data
Initialization
Qwen3.5-9B
SFT samples
368,302
625,993
357,304
1,351,599
Tokens (total / target)
396.8M / 284.2M
686.8M / 180.8M
539.3M / 73.3M
–
Trace generator
Qwen3.5-9B (self)
Qwen3.5-9B (self)
Qwen3.8-Flash-Next
as the domain runs
Chat template / thinking
Qwen3.5 native / on
Appendix
Table 14: SFT configuration (lmms-engine 0.1.4). The three domain SFT runs are the warm-ups of the domain teachers with the SFT warm-up. MixSFT trains on the union of the three domain SFT sets and is the start of the MixRL baseline. A single centered value applies to all runs. Prompt tokens are masked from the loss on every assistant turn.
Setting
Image STEM teacher
Image perception teacher
Video teacher
MixRL (baseline)
MixRL on top of MOPD
Per run
Initialization
Qwen3.5-9B
Qwen3.5-9B
Qwen3.5-9B
MixSFT
final MOPD
RL prompts
64,307
67,915
60,805
193,027
193,027
Prompts per step / mini-batch
256 / 64
256 / 64
256 / 64
252 / 63
256 / 64
Optimizer updates per step
4
4
4
4
4
Max. prompt tokens
2,560
2,048
5,632
5,632
5,632
Appendix
Table 15: GSPO configuration of the domain teachers trained directly from Qwen3.5-9B, the MixRL baseline, and MixRL on top of the MOPD student of these teachers (verl 0.9.1, rollout with vLLM 0.19.1). A single centered value applies to all runs. All runs use the reward of Section B and the media settings of the evaluation protocol (Section C ). A step fills one batch with prompts that pass the group filter, each with eight rollouts, and makes one optimizer update per mini-batch. For both sets of teachers, MixRL on top of MOPD starts from the final checkpoint of their MOPD student and trains on the same prompts as the MixRL baseline.
Setting
MOPD
Models and data
Student initialization
Qwen3.5-9B
Teachers
image STEM, image perception, video GSPO checkpoints
Routing
hard, by the domain label of the prompt
Prompts
193,027 (union of the three RL sets)
Objective
Appendix
Table 16: MOPD configuration (verl 0.9.1 with our distillation patches, vLLM 0.19.1 for student rollout and teacher scoring). The student rollout uses the sampling settings and the response limit of the GSPO runs of Table 15 , with one rollout per prompt.
Stage
Before
After
Domain RL Image STEM (DynaMath)
Qwen3.5-9B (wrong): “Wait, let’s look at the reflection again. …” The same check returns until the response limit, and no answer is given.
Domain teacher (correct): tries a rotation of the figure instead and checks it: “This matches the bottom text perfectly!”
Domain RL Video (Video-MME)
Qwen3.5-9B (correct): walks through the clip: “Around 00:10, the first dive sequence starts. It shows a solo dive. Wait, let me look closer. At 00:10, we see one diver. Then at 00:13, …”
Domain teacher (correct): lists only the relevant moments: “Around 00:43, the two athletes are diving together again.”
MOPD Video (Video-MME)
Domain teacher (correct): “The user wants to identify the location of the final scenes in the video. 1. Observe the video content from around 00:48 onwards.”
MOPD student (correct): “The user wants to identify the location of the final scenes in the video. 1. Look at the video from 00:48 onwards.”
MixRL on top of MOPD Video (Video-Holmes)
MOPD student (wrong): builds a timestamped chain on a misperceived event: “At 01:38, a man with a gun is seen firing.”
M 3 OPD-Reasoner-9B (correct): states the event that the question asks about: “The video shows the woman splitting the wall with a sickle, creating an opening.”
MixRL from MixSFT Image perception (CV-Bench)
MixSFT (wrong): “I need to count the umbrellas in the image. Looking at the scene, I see a group of colorful umbrellas …”
MixRL baseline (correct): same opening, then one recount: “I need to count the umbrellas in the image. Looking at the scene, … Wait, let me recount carefully.”
Appendix
Table 17: Excerpts from pairs of responses of two models to the same benchmark question. Each row compares the model before and after one stage, and (correct) and (wrong) give the benchmark verdict of the response.
Solved
Length
Model
∙
∙
∙
∙
∙
∙
Qwen3.5-9B
0.84
0.80
0.77
0.81
0.69
0.30
MixSFT
0.69
0.71
0.76
0.58
0.44
0.22
+ MixRL (MixRL baseline)
0.80
0.77
0.81
0.71
0.58
0.32
MOPD, with SFT warm-up
0.81
0.77
0.82
0.73
0.57
0.31
MOPD, without SFT warm-up
0.90
0.87
0.91
0.86
0.77
0.60
Appendix
Table 18: Per-question agreement with the domain teachers trained directly from Qwen3.5-9B on the benchmarks of each domain: Spearman correlation of the number of correct samples out of eight (Solved) and of the thinking length of one response (Length), mean over the benchmarks. Dots: image STEM ( ∙ ), image perception ( ∙ ), video ( ∙ ). ∗ Compared with its own teachers (with the SFT warm-up).