Multimodal Mixture-of-Experts (MoE) models have achieved remarkable performance on vision-language tasks. However, we identify a puzzling phenomenon termed Seeing but Not Thinking: models accurately perceive image content yet fail in subsequent reasoning, while correctly solving identical problems presented as pure text. Through systematic analysis, we first verify that cross-modal semantic sharing exists in MoE architectures, ruling out semantic alignment failure as the sole explanation. We then reveal that visual experts and domain experts exhibit layer-wise separation, with image inputs inducing significant routing divergence from text inputs in middle layers where domain experts concentrate. Based on these findings, we propose the Routing Distraction hypothesis: when processing visual inputs, the routing mechanism fails to adequately activate task-relevant reasoning experts. To validate this hypothesis, we design a routing-guided intervention method that enhances domain expert activation. Experiments on three multimodal MoE models across six benchmarks demonstrate consistent improvements, with gains of up to 3.17% on complex visual reasoning tasks. Our analysis further reveals that domain expert identification locates cognitive functions rather than sample-specific solutions, enabling effective transfer across tasks with different information structures.
Figures & tables
Figure 1: Illustration of the Seeing but Not Thinking phenomenon. See Appendix B for details.
Figure 2: Overview of our work. We first conduct cross-modal concept intervention to verify semantic sharing in MoE architectures (left, § 3.1 ), then identify domain experts by comparing activation frequencies on domain-specific versus general data (middle, § 3.2 ), and finally analyze routing divergence across modalities and apply routing guidance to enhance domain expert activation (right, § 3.3 -§ 4 ).
Figure 3: Analysis of routing mechanisms in multimodal MoE models. (a) Cross-modal semantic sharing verification showing inverted U-shaped intervention success rates. (b) Expert specialization quantification using Gini coefficients. (c) Routing divergence across modalities for three image versions.
Figure 4: Layer-wise distribution of domain experts and visual experts. Left: Heatmap showing activation frequency differences (red: higher on math data; blue: higher on general data), with deep red concentrated in layers 6–42. Right: Expert counts per layer, where Overlap indicates experts identified as both math and visual experts.
Model
Intervention Layers
τ
λ
Kimi-VL-16B-A3B
[0, 20]
0.3
0.5
Qwen3-VL-30B-A3B
[6, 42]
0.3
0.5
Llama4-Scout-109B-A17B
[8, 40]
0.3
0.2
Table 1: Intervention configurations for each model. Intervention layers are specified as inclusive ranges.
Semantically Equivalent Scenarios
Natural Visual Scenarios
Method
Math
Chemistry
Physics
MathVerse
MATH-V
GSM8K-V
Average
Kimi-VL-16B-A3B-Instruct
Baseline
52.30
25.54
29.51
35.41
21.05
8.11
28.65
Random
51.64 -0.66
25.67 +0.13
28.78 -0.73
34.39 -1.02
19.74 -1.31
6.82 -1.29
27.84 -0.81
Hard
53.06 +0.76
25.81 +0.27
32.78 +3.27
35.79 +0.38
21.71 +0.66
7.13 -0.98
29.38 +0.73
Soft
54.54 +2.24
27.89 +2.35
32.49 +2.98
38.58 +3.17
23.36 +2.31
9.17 +1.06
31.01 +2.36
Table 2: Accuracy (%) across three multimodal MoE models and six benchmarks. Math, Chemistry, and Physics refer to image-rendered versions of MATH500 and GPQA-Diamond (chemistry and physics subsets) respectively. MATH-V denotes MATH-Vision.
Model
Text
Vision
w/ Guidance
RefMathVerse
RefGSM8K
Kimi-VL
45.94
35.41
38.58 +3.17
37.31 +1.90
Qwen3-VL
67.26
69.29
71.20 +1.91
69.54 +0.25
Llama4
63.32
56.09
57.11 +1.02
55.08 -1.01
Table 3: Text-only and vision-only accuracy on MathVerse, with guidance using experts identified from MathVerse text-only vs. GSM8K.
Model
Baseline
w/ Early
Middle
w/ Late
Kimi-VL
28.65
31.01
29.82
29.35
Qwen3-VL
59.24
59.76
60.72
58.97
Llama4
47.93
46.85
49.10
47.82
Table 4: Impact of intervention layer range on average accuracy. Middle: intervening only on middle layers; w/ Early: adding early layers; w/ Late: adding late layers. For Kimi-VL, middle layers refer to layers 6-20.
Figure 5: Effect of enhancement coefficient λ on reasoning accuracy gains across three models.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Baseline Response (Incorrect: Answer is 4)
Of course, let’s solve this problem step-by-step. The scenario involves a beanstalk that doubles in height every day and starts at 4 feet tall . We need to find out how many days it will take for the beanstalk to grow taller than a window located 20 feet off the ground . We can calculate the height of the beanstalk each day until its height exceeds 20 feet. - Day 1: The beanstalk is 4 feet tall. - Day 2: It doubles from Day 1: 4×2=8 feet tall. - Day 3: It doubles from Day 2: 8×2=16 feet tall. - Day 4: It doubles from Day 3: 16×2=32 feet tall. Now we compare these heights to the window: - On Day 3 , the beanstalk is 16 feet tall, which is less than 20 feet (the window). - On Day 4 , the beanstalk is 32 feet tall, which is greater than 20 feet. Therefore, on the fourth day , the beanstalk becomes taller than the window. It takes \boxed{{\color[rgb]{1,0,0}4}} days for the beanstalk to be taller than the window.
With Routing Intervention (Correct: Answer is 3)
Of course, let’s solve this problem step-by-step. The scenario involves a beanstalk that doubles in height every day and starts at 4 feet tall. We need to find out how many days it will take for the beanstalk to grow taller than a window located 20 feet off the ground. We can model the growth of the beanstalk using an exponential function: - Initial Height (Day 0): h0=4 feet. - Growth Rate: The plant doubles its size each day, so we multiply by 2 daily. - Height on Day n: hn=h0×2n Let’s calculate the height of the beanstalk over several consecutive days until it surpasses the 20-foot mark: — Step 1: Calculate the height after each day - After Day 1: h1=4 ft×21=8 ft - After Day 2: h2=4 ft×22=4 ft×4=16 ft - After Day 3: h3=4 ft×23=4 ft×8=32 ft — Step 2: Compare with the window height Now, compare these heights to the window’s height of 20 feet: - After Day 1 ( 8 ft), the beanstalk is shorter than the window ( 8<20 ). - After Day 2 ( 16 ft), the beanstalk is still shorter than the window ( 16<20 ). - After Day 3 ( 32 ft), the beanstalk is now taller than the window ( 32>20 ). Therefore, the first time the beanstalk exceeds the height of the window is on the third day. It takes 3 days for the beanstalk to be taller than the window.
Appendix
Table 5: Complete model responses for the problem shown in Figure 1 . Despite successfully extracting all necessary information, the baseline response incorrectly counts the initial value as a separate day (Day 1), leading to an off-by-one error. The response with routing-guided intervention correctly starts the count from Day 0 and produces the correct answer.
Figure 6: Examples of the three semantically equivalent image versions (v1, v2, v3 from top to bottom) for the same text problem.
Dataset
Acc
Acc (OCR first)
Perception Error
Reasoning Error
MATH500
92.8
-
-
-
MATH500-v1
89.0
87.4
31.8% (7/22)
68.2% (15/22)
MATH500-v2
88.2
86.8
26.9% (7/26)
73.1% (19/26)
MATH500-v3
87.4
86.8
31.0% (9/29)
69.0% (20/29)
Appendix
Table 6: Accuracy comparison and error analysis on MATH500 and its three image versions. “Acc (OCR first)” refers to performance when explicitly instructing the model to perform OCR before reasoning. Perception and reasoning error percentages are computed from samples where the model answered correctly on the text version but incorrectly on the image version.
Model
Layers
Routed Experts
Active Routed Experts
Shared Experts
Total Active Experts
Kimi-VL
27
64
6
2
8
Qwen3-VL
48
128
8
–
8
Llama4
48
16
1
1
2
Appendix
Table 7: Model architectures. All expert counts are specified per layer.
Multimodal mixture-of-experts (MoE) models combine sparse expert activation with visual-language capabilities, yet their inference remains costly because long visual-token sequences repeatedly incur attention, routing, dispatch, and expert-MLP computation. Existing methods typically compress either the token or expert dimension, leaving redundancy along the other. Our analysis reveals two complementary regularities: the depth required for visual propagation varies across inputs, while text-token routing exhibits concentrated and recurrent expert-importance patterns. Based on these observations, we propose DecoMoE, a two-dimensional structured compression framework that decouples visual propagation from expert computation. The Sample-Adaptive Visual Boundary (SAVB) predicts an input-dependent visual-exit layer at which the visual-token block is removed. The Routing-Calibrated Expert Prefix (RCEP) reorders experts offline using text-token routed mass and, from this predicted exit layer onward, retains at each MoE layer the shortest contiguous prefix covering a target routed-mass fraction. We evaluate DecoMoE on Qwen3-VL-MoE and InternVL3.5-30B-A3B across six benchmarks. On Qwen3-VL-MoE, DecoMoE retains 97.91% of dense-baseline performance while reducing computation from 27.06 to 16.73 TFLOPs and latency from 0.44 to 0.26 seconds, yielding a 1.69x speedup. Code will be available at https://github.com/ShawnTan86/DecoMoE.
Xudong Tan, Peng Ye, Ming Xie +4
Fudan University · The Chinese University of Hong Kong
Mixture-of-Experts (MoE) models are often interpreted by analysing which categories are routed to which experts. However, routing alone does not reveal what each expert actually encodes. We train sparsely-gated convolutional MoE models with a contrastive objective on natural images and characterise expert specialisation using tools from visual neuroscience. Extending from gating-level to expert-level analyses, we measure per-expert category separability, and per-expert tuning using the most exciting inputs. Extending from category-level to feature-level explanations, we interpret tuning via semantic dimensions derived from a dataset of human behavioural judgements (THINGS). Finally, we use tuning and representational similarity analysis to assess the stability of expertise-allocation across independent initialisations. We find that an animate-inanimate distinction dominates expert partitioning, apparent from gating through to expert readout, and is stable across independently trained models. Although routing statistics suggest relatively sparse, categorical preferences, expert analyses reveal broader tuning to continuous visual and semantic dimensions that extend beyond category boundaries. Experts exhibit similar category-separability to one another, despite distinct feature tuning, demonstrating the explanatory benefits of moving beyond category-level analyses. Together, these results show that expert specialisation in vision MoEs extends well beyond category routing and is better understood by probing fine-grained expert-level tuning and representational structure.
Gene Tangtartharakul, Katherine R. Storrs
School of Psychology · University of Auckland · Auckland, New Zealand
Large multimodal models have achieved strong reasoning on complex visual tasks, but their inference efficiency is often restricted by long chains of thought. A promising solution is to pair a small draft model with a large target model, enabling cooperative inference employing a routing signal that adaptively routes queries to either the draft or target model based on their difficulties for optimal efficiency and accuracy. Yet, the remaining bottleneck is to establish a reliable query difficulty signal under multimodal settings. Existing approaches designed for language models either rely on post-hoc token probabilities, which fall short in multimodal scenarios, or depend on supervised fine-tuning, which is a data-sensitive strategy. Both paradigms perform routing only after a complete output, and ignore whether the target model can actually solve the routed instances. To address this, we propose PRP, a Proactive Routing Paradigm that enables early decision-making by jointly evaluating the competence of both the draft and target models. Our Draft Rating Learning (DRL) equips the draft model with an internal confidence estimator, while Joint Rating Learning (JRL) predicts how well the target model can handle a given query, thereby prioritizing the allocation of samples it excels at rather than the hardest ones. These ratings enable fine-grained, instance-level Proactive Routing and substantially accelerate inference without compromising overall performance. Extensive experiments across multiple multimodal reasoning benchmarks validate our effectiveness and efficiency.
Yinan Zhou, Haokun Lin, Yichen Wu +7
Xi’an Jiaotong University · ARC Lab, Tencent IEG · City University of Hong Kong +3