Recent advancements in Multimodal Large Language Models (MLLMs) pursue omni-perception capabilities, yet integrating robust sensory grounding with complex reasoning remains a challenge, particularly for underrepresented regions. In this report, we introduce the research preview of MERaLiON2-Omni (Alpha), a 10B-parameter multilingual omni-perception tailored for Southeast Asia (SEA). We present a progressive training pipeline that explicitly decouples and then integrates "System 1" (Perception) and "System 2" (Reasoning) capabilities. First, we establish a robust Perception Backbone by aligning region-specific audio-visual cues (e.g., Singlish code-switching, local cultural landmarks) with a multilingual LLM through orthogonal modality adaptation. Second, to inject cognitive capabilities without large-scale supervision, we propose a cost-effective Generate-Judge-Refine pipeline. By utilizing a Super-LLM to filter hallucinations and resolve conflicts via a consensus mechanism, we synthesize high-quality silver data that transfers textual Chain-of-Thought reasoning to multimodal scenarios. Comprehensive evaluation on our newly introduced SEA-Omni Benchmark Suite reveals an Efficiency-Stability Paradox: while reasoning acts as a non-linear amplifier for abstract tasks (boosting mathematical and instruction-following performance significantly), it introduces instability in low-level sensory processing. Specifically, we identify Temporal Drift in long-context audio, where extended reasoning desynchronizes the model from acoustic timestamps, and Visual Over-interpretation, where logic overrides pixel-level reality. This report details the architecture, the data-efficient training recipe, and a diagnostic analysis of the trade-offs between robust perception and structured reasoning.
Figures & tables
Figure 1: MERaLiON2-Omni (Alpha) is a 10B omni model for Southeast Asia that extends MERaLiON-v2 with a stronger perception backbone aligning region-specific audio-visual signals to a multilingual LLM, and a Cognitive Reasoning Layer enabled by Chain-of-Thought transfer.
Method
Captioning
QA
Summ
MCQA
OCR
BERTScore
BERTScore
F1
BERTScore
Acc
CER
WER
WER*
SeaLion4VL-27B (2025)
77.64
85.16
14.71
82.92
83.32
0.40
0.45
0.094
Qwen3Omni-30B
78.90
84.63
14.51
82.86
86.21
0.29
0.31
0.102
Qwen2.5Omni-7B
80.63
83.58
12.79
81.86
72.26
0.36
0.41
0.237
M2-Omni-Base
80.41
83.28
12.93
81.54
72.32
0.98
1.03
0.675
M2-Omni-SEA
83.32
83.19
12.93
83.60
85.11
0.11
0.19
0.129
Table 1: Performance on the SEA-Image Benchmark. In WER*, we ignore insertions to handle verbose outputs. The best results are in bold while the second best are underlined.
Method
Summ
AQA
VQA
MCQA
BERTScore
BERTScore
F1
BERTScore
F1
Acc
Qwen3Omni-30B
84.75
86.19
18.75
85.50
17.02
83.07
Qwen2.5Omni-7B
85.13
85.77
16.82
85.68
16.50
49.83
M2-Omni-Base
85.22
86.05
17.99
85.92
17.07
24.35
M2-Omni-SEA
77.32
82.25
11.33
82.08
11.67
10.19
M2-Omni-Instruct
85.10
87.50
24.10
87.28
23.37
84.89
Table 2: Performance on the SEA-Video Benchmark.
Model
Video Benchmarks
Image Benchmarks
Video-MME
Video-MME-sub
MVBench
MME
MMB-V11
MMVet
Qwen3Omni-30B
61.48
66.22
52.11
2,043
86.18
73.90
Qwen2.5Omni-7B
54.19
58.63
53.96
1,756
81.97
61.10
M2-Omni-Instruct
52.52
56.67
36.98
1,669
62.03
39.30
M2-Omni-ML
50.26
56.00
37.08
1,667
58.02
36.40
Table 3: Performance comparison on 6 general-purpose multimodal benchmarks ( Fu et al., 2025 ; Li et al., 2024 ; Yu et al., 2024 ) . It shows a minor performance drop when expanding from the English-centric checkpoint to the multilingual backbone.
Dataset (cases)
M2O-ML (System 1)
M2O-ML-r1 (System 2)
AM_evol-zh (50)
0.2800
0.3000
AM_NuminaMath_1.5 (197)
0.2589
0.3706
AM_InfinityInstruct (1029)
0.4451
0.4898
AM_natural_reasoning (1101)
0.4087
0.4959
AM_OpenCoderStage2 (78)
0.3718
0.3974
AM_PRIME (31)
0.2258
0.1935
Table 4: Performance comparison between M2-Omni-ML and M2-Omni-ML-r1 across evaluation datasets. Results are reported as Consistency Scores (0-1) evaluated by a Qwen3-235B Judge against ground-truth. Best result per dataset is shown in bold.
Dataset
M2-AL
M2O-Inst
M2O-ML
M2O-ML-r1
M2O-ML-r1*
Qwen2.5Omni
Qwen3Omni*
English Datasets
Librispeech_test_clean
0.025
0.024
0.026
0.044
0.040 (0.4%) ↑
0.134
0.016
Librispeech_test_other
0.043
0.043
0.046
0.051
0.051 (0.2%) -
0.164
0.026
Commonvoice_15_en_test
0.084
0.084
0.092
0.092
0.091 (0.1%) ↑
0.290
0.061
Peoples_speech_test
0.195
0.207
0.226
0.249
0.247 (0.6%) ↑
0.491
0.153
Gigaspeech_test
0.094
0.097
0.105
0.100
0.098 (0.5%) ↑
0.318
0.088
Table 5: WER performance comparison on English, Multilingual (SEA), and Code-Switching datasets. M2O-ML-r1* refers to an ablation study where we count the number of ASR results that are translated into the reasoning language, and we report the performance with these language-drift generation removed to reveal their effects. Qwen3Omni* (30B) is included only for reference.
Channel
Dataset
MERaLiON Variants
Baseline
Reference
M2-AL
M2O-Inst
M2O-ML
M2O-ML-r1
Qwen2.5Omni
Qwen3Omni*
Spoken QA (English)
cn_college_listen_mcq
76.50
75.80
78.50
88.80 ↑
78.70
93.90
slue_p2_sqa5
89.51
90.34
92.45
90.39 ↓
82.55
91.42
dream_tts_mcq
71.90
74.80
77.05
85.70 ↑
72.00
87.10
public_sg_speech_qa
73.28
78.90
79.80
78.90 ↓
65.87
80.96
spoken_squad
89.04
87.70
90.96
88.42 ↓
77.46
82.24
Table 6: Detailed performance comparison on the standard AudioBench tasks ( Wang et al., 2025a ) . Colored arrows in the M2O-ML-r1 column indicate performance gain ( ↑ ) or loss ( ↓ ) compared to the M2O-ML model. We mainly compare our models against the comparable baseline Qwen2.5Omni (7B). Qwen3Omni* (30B) results are provided for reference only and are not included in the best-result comparison (bolded).
Model
PB-Read
Topic-Read
Daily Conv.
Code-Switch
Themed Conv.
Service Call
WER
WER*
WER
WER*
WER
WER*
WER
WER*
WER
WER*
WER
WER*
Qwen3Omni-30B
0.062
0.053
0.324
0.242
0.278
0.221
0.440
0.368
0.187
0.151
0.147
0.122
Qwen2.5Omni-7B
0.291
0.237
0.792
0.598
0.820
0.744
0.886
0.851
0.751
0.655
0.783
0.666
M2-AL ( He et al., 2025 )
0.059
0.048
0.144
0.130
0.263
0.210
0.342
0.268
0.199
0.154
0.160
0.123
M2O-Base
0.108
0.081
0.262
0.234
0.559
0.490
0.696
0.648
0.504
0.444
0.426
0.348
M2O-Instruct
0.055
0.046
0.112
0.101
0.284
0.239
0.338
0.279
0.199
0.160
0.156
0.128
Table 7: Performance comparison of different models on six Singlish-ASR datasets ( Wang et al., 2025b ) .
Figure 2: The cognitive gap. This radar chart illustrates the performance of our reasoning (M2O-ML-r1) and non-reasoning (M2O-ML) models on 6 representative AudioBench tasks.
Figure 3: The amplifier effect. The introduced reasoning capability benefits cognitive tasks but harms perceptual tasks.
Figure 4: Divergent impact of reasoning length. Visual accuracy improves with longer reasoning (the red line), while Audio accuracy degrades due to Temporal Drift (the green line).
Figure 5: The anatomy of temporal drift .
Figure 6: Anchor token analysis. In Audio QA tasks (where inputs comprise both audio and video frames), the reasoning model generates a significantly higher number of visual-anchor tokens (e.g., colors, shapes) compared to M2O-ML.
Figure 7: Success and failure modes in multimodal reasoning. (a) The M2O-ML-r1 model hallucinates invisible physiological processes in the chest X-ray to satisfy its logical narrative. (b) The M2O-ML-r1 model successfully disentangles speaker intent in a complex audio dialogue by explicitly tracking speaker states.
Improving joint audio-visual reasoning in Omni Large Language Models typically incurs substantial data construction and training costs. Our diagnostics reveal multi-hop reasoning difficulties despite correct answers to all corresponding single-hop questions and suggest partial decoupling in the local optimization of perception and reasoning objectives. This motivates post-training with different emphases on these capabilities. Text-only reasoning training yields gains across data sources, model scales, and families. With the best-performing text-only configuration, supervised fine-tuning followed by reinforcement learning (RL) raises Qwen2.5-Omni-7B's geometric mean of nine reasoning scores by 25.83% over the base model, outperforming the complete native audio-visual route with 56.6% fewer GPU-hours. Training on data synthesized entirely by a text-only LLM raises this geometric mean by 21.01% without audio-visual data in construction or training. However, text-only training degrades perception. We therefore propose a text-centric post-training paradigm: text-only training provides the main reasoning optimization, and reduced-data native audio-visual RL then refines perception. Refinement uses about 90% fewer input tokens than full-data audio-visual RL, restores perception above the base level, and retains 93.5% of the best-performing text-only pipeline's reasoning gain.
Reasoning in Multimodal Large Language Models (MLLMs) requires both fine-grained visual perception and rigorous logical deduction. Explicit text-based Chain-of-Thought (CoT) is computationally expensive and prone to visual hallucinations, while existing latent reasoning methods typically require costly training. Furthermore, directly adapting training-free LLM reasoning mechanisms to the multimodal setting yields unstable performance. We identify that this failure stems from their reliance on token-level entropy, which fundamentally conflates perceptual ambiguity (e.g., unclear visual details) with logical uncertainty (e.g., complex reasoning steps). To overcome this bottleneck, we present a novel training-free inference strategy for MLLMs that explicitly decouples perception and reasoning. We propose a novel metric, the vision-to-text attention ratio, to dynamically gauge the model's cognitive focus. Guided by this metric, our proposed framework, Attention-Guided Switching (AGS), adaptively triggers latent reasoning for perceptual tokens to preserve high-fidelity visual information in the continuous space, while enforcing explicit text generation for logical tokens to maintain structural anchoring. Extensive experiments demonstrate that our method achieves state-of-the-art performance, significantly improving both accuracy and inference efficiency by reducing autoregressive steps and latency. Code is released at https://github.com/swordAndSnow/MM26-AGS.
Haoqian Kang, Liupeng Li, Kuofeng Gao +5
Harbin Institute of Technology, Shenzhen Shenzhen, China · Tsinghua Shenzhen International Graduate School, Tsinghua University Shenzhen, China · Peng Cheng Laboratory Shenzhen, China +1
We find that current emotion-oriented Omni-MLLMs still lack reliable omni-modal perception: they (i) underutilize multimodal cues in their reasoning trajectories and (ii) exhibit unfaithful behavior, often hallucinating modality-specific statements from other modalities. Building on these insights, we propose OPPO (Omni-Perception Policy Optimization), a reinforcement learning framework that explicitly optimizes multimodal perception. First, an Omni-Perception Reward decomposes ground-truth reasoning into fine-grained visual, acoustic, and emotion cues and rewards trajectories that semantically recover these cues. Second, an Omni-Perception Loss compares the policy under full and unimodally masked inputs, applying a KL penalty only to modality-specific evidence tokens to suppress cross-modal hallucination. We further introduce MEP-Bench, a diagnostic benchmark that quantifies utilization and faithfulness. Experiments show that OPPO achieves state-of-the-art performance on MER-UniBench and MME-Emotion, while substantially improving utilization and faithfulness scores on MEP-Bench, highlighting the importance of sufficient and faithful omni perception for multimodal emotion reasoning.
Zhiyuan Han, Beier Zhu, Wenwen Tong +6
University of Science and Technology of China · SenseTime · Institute of Artificial Intelligence, Hefei Comprehensive National Science Center +1