WeMM-Video: Pushing the Frontiers of Open-Source Omni-Modal Video Understanding
Organizations: WeChat Vision, Tencent Inc. · Tsinghua University
Abstract
Video understanding, particularly with accompanying audio, remains a central challenge in multimodal intelligence. In this report, we introduce WeMM-Video, a family of Chinese-English bilingual omni-modal foundation models spanning 2B, 4B, 9B, and 35B-A3B variants. Centering on joint audio-visual perception and reasoning, WeMM-Video unifies text, image, audio, and video processing within a hybrid-thinking paradigm, seamlessly supporting both a non-thinking instruct mode for fast, direct responses and a thinking mode for explicit step-by-step reasoning. The models are trained on a large-scale multimodal corpus of approximately 1.2 trillion tokens through progressive pre-training and a multi-stage post-training pipeline incorporating multi-teacher OPD, anti-repetition MrDPO, and specialist RL. Evaluated across more than 120 benchmarks, WeMM-Video demonstrates superior performance in omni-modal video understanding and reasoning, while maintaining highly competitive capabilities across general audio, silent video, image, and text tasks. Specifically, in omni-modal video evaluations, our flagship WeMM-Video-35B-A3B establishes a new open-source state of the art, substantially outperforming proprietary models such as Qwen3.5-Omni-Flash and Seed2.0-Mini-0428 while narrowing the gap with the strong Gemini-3.0-Flash, and our compact WeMM-Video-9B surpasses the larger open-source Qwen3-Omni-30B-A3B-Instruct. We publicly release the WeMM-Video family along with our evaluation and reproduction pipeline to foster further research and development in the open-source community.
Figures & tables
| Benchmark 1 | Gemini-3.0- | Seed2.0-Mini- | Qwen3.5-Omni- | WeMM-Video-35B-A3B | |
|---|---|---|---|---|---|
| Flash | 0428 | Flash | (Instruct) | (Thinking) | |
| General Audio-Visual Understanding | |||||
| Video-MME | 88.6 | 83.6 | 79.3 | 84.2 | 86.4 |
| Video-MME-v2 | 63.2 | 51.7 | 36.6 | 47.5 | 54.9 |
| Daily-Omni | 82.4 | 82.5 | 81.8 | 81.5 | 83.4 |
| WorldSense | 63.8 | 61.3 | 57.9 | 59.6 | 62.7 |
| Benchmark 1 | Qwen3-Omni- | MiniCPM-o | WeMM-Video- | ||
|---|---|---|---|---|---|
| 30B-A3B-Instruct | 4.5-9B | 2B (Thinking) | 4B (Thinking) | 9B (Thinking) | |
| General Audio-Visual Understanding | |||||
| Video-MME | 77.3 | 69.2 | 72.4 | 83.9 | 85.1 |
| Video-MME-v2 | 36.3 | 28.1 | 37.7 | 48.1 | 51.6 |
| Daily-Omni | 70.7 | 80.2 | 76.5 | 82.5 | 83.6 |
| WorldSense | 54.0 | 55.7 | 52.4 | 60.3 | 63.4 |
| Benchmark 1 | Gemini-3.0- | Seed2.0-Mini- | Qwen3.5-Omni- | Qwen3.6- | WeMM-Video-35B-A3B | |
|---|---|---|---|---|---|---|
| Flash | 0428 | Flash | 35B-A3B | (Instruct) | (Thinking) | |
| General Video Understanding | ||||||
| Video-MME | 86.4 | 78.1 | 77.0 | 82.5 | 78.0 | 80.4 |
| Video-MME-v2 | 55.6 | 44.4 | 33.7 | 45.3 | 39.8 | 45.2 |
| MVBench | 70.0 | 73.4 | 70.8 | 74.6 | 72.8 | 73.1 |
| Video-MMMU | 86.9 | 81.0 | 64.0 | 83.7 | 80.8 | 83.6 |
| Benchmark 1 | Qwen3-Omni- | Qwen3.5- | MiniCPM-o- | WeMM-Video- | ||
|---|---|---|---|---|---|---|
| 30B-A3B-Instruct | 9B | 4.5-9B | 2B (Thinking) | 4B (Thinking) | 9B (Thinking) | |
| General Video Understanding | ||||||
| Video-MME | 70.5 | 78.4 | 70.4 | 67.6 | 75.7 | 78.5 |
| Video-MME-v2 | 29.7 | 42.4 | 26.8 | 32.8 | 39.5 | 40.6 |
| MVBench | 68.8 | 74.4 | 63.4 | 64.6 | 70.0 | 71.4 |
| Video-MMMU | 54.7 | 78.9 | 52.9 | 65.1 | 79.6 | 79.2 |
| Benchmark | Qwen3.5-Omni- | Qwen3-Omni- | MiniCPM-o | WeMM-Video- | |||
|---|---|---|---|---|---|---|---|
| Flash | 30B-A3B-Instruct | 4.5-9B | 2B (Instruct) | 4B (Instruct) | 9B (Instruct) | 35B-A3B (Instruct) | |
| General Audio Understanding | |||||||
| MMAU | 80.4 | 77.5 | 76.9 | 72.4 | 75.2 | 77.0 | 78.0 |
| MMAU-Pro | 64.8 | 60.2 | 62.7 | 62.9 | 68.1 | 70.9 | 70.6 |
| MMAR | 74.0 | 73.2 | 66.3 | 56.9 | 67.3 | 72.8 | 73.6 |
| MMSU | 72.2 | 69.0 | 67.4 | 65.6 | 72.6 | 76.8 | 76.0 |
| Benchmark | Qwen3.5-Omni- | Qwen3.6- | Qwen3.5- | WeMM-Video- | |||
|---|---|---|---|---|---|---|---|
| Flash | 35B-A3B | 9B | 2B (Thinking) | 4B (Thinking) | 9B (Thinking) | 35B-A3B (Thinking) | |
| STEM/Puzzle | |||||||
| MMMU | 76.9 | 81.7 | 78.4 | 61.2 | 78.4 | 78.9 | 81.1 |
| MMMU-Pro | 68.2 | 75.3 | 70.1 | 49.4 | 64.7 | 69.4 | 72.9 |
| MathVision | 65.4 | 80.4 | 78.9 | 49.2 | 73.3 | 76.0 | 79.6 |
| MathVista (mini) | 82.9 | 86.4 | 85.7 | 78.8 | 85.2 | 86.1 | 86.6 |
| Benchmark | Qwen3.5-Omni- | Qwen3.6- | Qwen3.5- | WeMM-Video- | |||
|---|---|---|---|---|---|---|---|
| Flash | 35B-A3B | 9B | 2B (Thinking) | 4B (Thinking) | 9B (Thinking) | 35B-A3B (Thinking) | |
| Knowledge | |||||||
| MMLU-Pro | 79.9 | 85.2 | 82.5 | 63.7 | 78.3 | 81.6 | 84.7 |
| MMLU-Redux | 90.0 | 93.3 | 91.1 | 75.7 | 88.7 | 90.5 | 93.7 |
| SuperGPQA | 54.9 | 64.7 | 58.2 | 36.3 | 52.4 | 56.7 | 63.4 |
| C-Eval | 86.0 | 90.0 | 88.2 | 72.0 | 84.8 | 87.6 | 90.4 |
Multimodal Large Language Models
Multimodal large language models combine language models with image, video, audio, and other inputs for grounded understanding and generation.
See all 1,147 papers