SpikeMoE: Brain-Inspired Competitive Routing for Flexible Spiking Mixture-of-Experts

Xiaoli Liu; Yujie Liang; Jialin Li; Malu Zhang

cs.AIOct 1, 2026

SpikeMoE: Brain-Inspired Competitive Routing for Flexible Spiking Mixture-of-Experts

Authors: Xiaoli Liu, Yujie Liang, Jialin Li, Malu Zhang

Organizations: School of Computer Science and Engineering, University of Electronic Science and Technology of China

Abstract

Spiking Neural Networks (SNNs) enable event-driven computation through biologically inspired dynamics at the neuronal scale, while Mixture-of-Experts (MoE) perform conditional computation through expert selection at the model scale. Integrating their strengths offers potential for flexible neural architectures. A key challenge, however, lies in designing an expert selection mechanism based on spiking activity. To address this, we introduce a spike-based k-WTA Router inspired by competition-inhibition observed in the hippocampal CA1 region. The router incorporates lateral inhibition and refractory period to select Top-K experts according to discrete spike counts. Building on this, we present SpikeMoE, a framework that integrates neuronal-scale spiking dynamics with model-scale expert selection. To address incomplete multisensory inputs in multimodal tasks, we further equip SpikeMoE with a two-stage missing-modality modeling module that combines empirical prototypes from an observed-modality pool with modality-specific learnable embeddings to construct missing-modality representations. Experiments on vision, language, and multimodal benchmarks demonstrate that SpikeMoE achieves state-of-the-art performance among the SNN baselines, matches or exceeds the performance of ANN counterparts, and maintains robustness across diverse missing-modality conditions. These results demonstrate a favorable trade-off between performance and energy efficiency, validating the integration of spiking dynamics with sparse expert computation and highlighting SpikeMoE as a promising approach to energy-efficient brain-inspired computing.

Figures & tables

Figure 1: (a) Overall general framework; (b) Spike expert neuron.

Figure 2: (a) Common inhibitory patterns in the hippocampus; (b) Competitive selection in k-WTA Router; (c) Expert selection.

Figure 3: Two-stage missing modality modeling.

Figure 4: Visualization of routers as masks. The mask position (black) indicates router of 0 here. We show the dynamic sparsity for the selected experts (vertical) and time-steps (horizontal).

Methods	Architecture	Param (M)	Energy Con- sumption (mJ)	Time Step	Top-1 Acc (%)
Spikformer ( Zhou et al., 2022 )	Spikformer-8-384	16.81	12.43	4	70.24
Spikformer ( Zhou et al., 2022 )	Spikformer-8-512	29.68	18.82	4	73.38
Spikformer+SEMM ( Zhou et al., 2024 )	Spikformer-8-384	16.05	11.50*	4	72.86
Spikformer+SEMM ( Zhou et al., 2024 )	Spikformer-8-512	28.22	17.63*	4	75.93
	Spikformer-8-384	16.03	11.39	4	73.22
Spikformer+k-WTA Router	Spikformer-8-512	28.19	16.72	4	75.97

Table 1: Results on ImageNet. Model-

L

D

represents a model with

L

encoder blocks and

D

channels.

Model	Energy (mJ)	Time	MNLI -m/mm	QQP F1	QNLI	SST-2	CoLA	STS-B	MRPC F1	RTE	Avg.
BERT base ( Devlin et al., 2019 )	51.41	/	83.8/83.4	90.5	90.7	92.3	60.0	89.4	89.8	69.3	83.2
BERT 3L ( Devlin et al., 2019 )	12.90	/	77.1/77.1	85.2	85.8	88.1	31.7	85.7	86.4	66.4	75.9
Q2BERT ( Zhang et al., 2020 )	/	/	47.2/47.3	67.0	61.3	80.6	0.0	4.7	81.2	52.7	49.1
ELMo ( Peters et al., 2018 )	/	/	68.6/-	86.2	71.1	91.5	44.1	70.4	76.6	53.4	70.2
SpikeBERT ( Lv et al., 2023 )	14.30	4	71.4/71.0	68.2	66.4	85.4	16.9	18.7	82.0	57.5	59.7
LIF-BERT* ( Gerstner et al., 2014 ; Xing et al., 2024 )	/	4	35.4/35.2	0.0	50.5	50.9	0.0	0.0	81.2	52.7	34.6

Table 2: Performance comparison on GLUE benchmark.

Figure 5: UMAP visualization of multi-modal embedding space: CMU-MOSI.

Missing Ratio	Metric	ANNs				SNNs			Ours
Missing Ratio	Metric	ShaSpec ( Wang et al., 2023a )	TF ( Zadeh et al., 2017 )	mmFormer ( Zhang et al., 2022 )	FuseMoE ( Han et al., 2024 )	Weight Attention ( Liu et al., 2022 )	SCA ( Guo et al., 2023 )	S-CMRL ( He et al., 2025 )	SpikeMoE
0%	F1	41.78 $\pm$ 0.64	43.28 $\pm$ 0.36	50.22 $\pm$ 0.20	57.02 $\pm$ 0.39	50.41 $\pm$ 0.44	46.95 $\pm$ 0.21	49.01 $\pm$ 0.29	59.59 $\pm$ 0.23
0%	AUC	65.08 $\pm$ 0.39	64.48 $\pm$ 0.51	74.62 $\pm$ 0.49	77.26 $\pm$ 0.20	72.09 $\pm$ 0.38	70.33 $\pm$ 0.46	73.94 $\pm$ 0.50	78.48 $\pm$ 0.34
10%	F1	43.40 $\pm$ 0.74	40.14 $\pm$ 0.13	48.66 $\pm$ 1.20	55.81 $\pm$ 0.45	48.19 $\pm$ 0.52	46.81 $\pm$ 0.71	47.50 $\pm$ 0.06	56.51 $\pm$ 0.12
10%	AUC	65.45 $\pm$ 0.56	61.54 $\pm$ 0.66	73.21 $\pm$ 0.76	77.05 $\pm$ 0.93	74.68 $\pm$ 0.17	70.51 $\pm$ 0.02	72.12 $\pm$ 0.27	77.45 $\pm$ 0.67
20%	F1	39.15 $\pm$ 0.13	42.64 $\pm$ 0.30	48.51 $\pm$ 1.15	53.42 $\pm$ 0.58	47.46 $\pm$ 0.02	47.43 $\pm$ 0.29	47.35 $\pm$ 0.35	53.73 $\pm$ 0.92

Table 3: Experiment Setting I: Main results on CMU-MOSI. Top-3 results are highlighted as first , second and third .

Testing Modality			Metric	ANNs				SNNs			Ours
Video	Text	Audio	Metric	ShaSpec ( Wang et al., 2023a )	TF ( Zadeh et al., 2017 )	mmFormer ( Zhang et al., 2022 )	FuseMoE ( Han et al., 2024 )	Weight Attention ( Liu et al., 2022 )	SCA ( Guo et al., 2023 )	S-CMRL ( He et al., 2025 )	SpikeMoE
✓	✓		F1	41.27 $\pm$ 0.81	34.96 $\pm$ 0.72	48.25 $\pm$ 0.11	30.51 $\pm$ 2.53	25.75 $\pm$ 0.53	47.55 $\pm$ 0.56	48.86 $\pm$ 0.34	69.56 $\pm$ 0.23
			AUC	61.28 $\pm$ 0.88	59.37 $\pm$ 1.12	72.72 $\pm$ 0.08	55.35 $\pm$ 1.11	54.39 $\pm$ 0.12	71.98 $\pm$ 0.43	73.22 $\pm$ 0.06	74.29 $\pm$ 0.18
✓		✓	F1	36.99 $\pm$ 2.03	34.83 $\pm$ 0.54	35.79 $\pm$ 0.55	24.29 $\pm$ 4.23	22.18 $\pm$ 0.37	46.97 $\pm$ 0.08	23.82 $\pm$ 0.39	47.81 $\pm$ 0.61
			AUC	55.79 $\pm$ 2.53	57.87 $\pm$ 0.32	52.21 $\pm$ 0.31	56.24 $\pm$ 1.96	50.45 $\pm$ 0.08	68.56 $\pm$ 0.23	51.55 $\pm$ 0.28	69.04 $\pm$ 0.18
	✓	✓	F1	43.72 $\pm$ 1.64	37.25 $\pm$ 0.53	45.53 $\pm$ 0.33	20.68 $\pm$ 2.14	49.40 $\pm$ 0.23	19.18 $\pm$ 0.47	48.09 $\pm$ 0.21	49.47 $\pm$ 0.38

Table 4: Experiment Setting II: Main results on CMU-MOSI. Top-3 results are highlighted as first , second and third .

Methods	Missing Ratio=50%
Methods	F1	AUC
Fully k-WTA Router	$43.54\pm 0.03$	$67.30\pm 0.71$
w/o Lateral Inhibition	$41.37\pm 0.71$	$66.02\pm 0.49$
w/o Refractory Period	$42.50\pm 0.34$	$66.12\pm 0.56$
w/o Inhibition Strength	$40.41\pm 0.43$	$64.59\pm 0.80$

Table 5: Ablation on k-WTA Router.

Figure 6: Visualization of expert activation.

Figure 7: Ablation on two-stage strategy.

Figure 8: Sensitivity analysis on CMU-MOSEI (Missing ratio=50%).

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 9: UMAP visualization of multi-modal embedding space: CMU-MOSEI.

Metric	ShaSpec	TF	mmFormer	FuseMoE	WeightAttention	SCA	S-CMRL	SpikeMoE
	ANNs				SNNs			Ours
Ops (M)	21.73	48.42	1577.63	199.8	47.07	32.57	289.60	45.22
Params	2,177,406	4,211,331	11,373,315	243,543,180	131,585	264,192	2,685,729	2,472,370
AUC	55.60 $\pm$ 0.12	60.49 $\pm$ 0.80	61.87 $\pm$ 0.67	67.28 $\pm$ 0.28	65.81 $\pm$ 0.05	64.49 $\pm$ 0.43	65.71 $\pm$ 0.43	67.30 $\pm$ 0.71
Energy (mJ)	20.13	25.19	40.39	11.71	10.38	16.95	14.67	5.29

Appendix

Table 6: Complexity and Energy Consumption analysis

Parameter	Value
Number of Expert	4 for ImageNet-1K and GLUE, 6 for CMU-MOSI and CMU-MOSEI, 4 for UrbanSound8k-AV
Top-K Expert	2
Timesteps $T$	4 for ImageNet-1K, GLUE and UrbanSound8K-AV; 6 for CMU-MOSI and CMU-MOSEI
Learning Rate	5e-5
Hidden Size	128
MoE layers	2

Appendix

Table 7: Hyperparameter setting

Missing Ratio	Metric	ANNs				SNNs			Ours
Missing Ratio	Metric	ShaSpec ( Wang et al., 2023a )	TF ( Zadeh et al., 2017 )	mmFormer ( Zhang et al., 2022 )	FuseMoE ( Han et al., 2024 )	Weight Attention ( Liu et al., 2022 )	SCA ( Guo et al., 2023 )	S-CMRL ( He et al., 2025 )	SpikeMoE
0%	F1	47.48 $\pm$ 0.27	38.89 $\pm$ 0.54	56.75 $\pm$ 0.47	57.04 $\pm$ 0.15	59.40 $\pm$ 0.42	59.36 $\pm$ 0.06	58.09 $\pm$ 0.19	59.80 $\pm$ 0.17
0%	AUC	67.93 $\pm$ 0.56	64.26 $\pm$ 0.61	76.42 $\pm$ 0.24	78.54 $\pm$ 0.42	73.59 $\pm$ 0.08	74.41 $\pm$ 0.13	72.63 $\pm$ 0.21	78.63 $\pm$ 0.53
10%	F1	47.41 $\pm$ 0.18	34.94 $\pm$ 1.10	55.37 $\pm$ 0.47	55.90 $\pm$ 0.55	54.97 $\pm$ 0.35	56.81 $\pm$ 0.35	56.23 $\pm$ 0.32	56.83 $\pm$ 0.23
10%	AUC	69.26 $\pm$ 0.22	61.05 $\pm$ 0.34	75.36 $\pm$ 0.53	76.38 $\pm$ 0.39	72.18 $\pm$ 0.34	73.12 $\pm$ 0.23	73.17 $\pm$ 0.09	76.47 $\pm$ 0.49
20%	F1	46.54 $\pm$ 1.04	35.37 $\pm$ 0.94	52.91 $\pm$ 0.75	54.44 $\pm$ 0.27	53.14 $\pm$ 0.17	54.21 $\pm$ 0.11	52.84 $\pm$ 0.45	54.35 $\pm$ 0.57

Appendix

Table 8: Experiment Setting I: Main results on CMU-MOSEI. Top-3 results are highlighted as first , second and third .

Testing Modality			Metric	ANNs				SNNs			Ours
Video	Text	Audio	Metric	ShaSpec ( Wang et al., 2023a )	TF ( Zadeh et al., 2017 )	mmFormer ( Zhang et al., 2022 )	FuseMoE ( Han et al., 2024 )	Weight Attention ( Liu et al., 2022 )	SCA ( Guo et al., 2023 )	S-CMRL ( He et al., 2025 )	SpikeMoE
✓	✓		F1	40.75 $\pm$ 0.14	31.38 $\pm$ 0.61	56.02 $\pm$ 0.56	55.85 $\pm$ 0.69	33.34 $\pm$ 0.28	52.80 $\pm$ 0.76	55.62 $\pm$ 0.34	56.25 $\pm$ 0.32
			AUC	64.47 $\pm$ 0.10	56.98 $\pm$ 0.20	76.60 $\pm$ 0.39	75.81 $\pm$ 0.82	65.91 $\pm$ 0.45	71.60 $\pm$ 0.04	73.21 $\pm$ 0.45	77.09 $\pm$ 0.44
✓		✓	F1	38.15 $\pm$ 0.92	31.73 $\pm$ 0.16	37.12 $\pm$ 0.66	49.30 $\pm$ 0.67	22.00 $\pm$ 0.49	29.01 $\pm$ 0.04	22.10 $\pm$ 0.20	49.50 $\pm$ 0.36
			AUC	58.41 $\pm$ 0.10	58.41 $\pm$ 0.17	56.72 $\pm$ 0.84	56.32 $\pm$ 0.62	50.91 $\pm$ 0.57	51.26 $\pm$ 0.25	50.30 $\pm$ 0.06	57.08 $\pm$ 0.24
	✓	✓	F1	40.82 $\pm$ 0.12	31.86 $\pm$ 0.23	56.06 $\pm$ 0.29	57.02 $\pm$ 0.60	56.06 $\pm$ 0.48	33.90 $\pm$ 0.21	21.93 $\pm$ 0.83	58.53 $\pm$ 0.02

Appendix

Table 9: Experiment Setting II: Main results on CMU-MOSEI. Top-3 results are highlighted as first , second and third .

Figure 10: (a) Performance on UrbanSound8k-AV across Missing Ratios; (b) Ablation study on UrbanSound8k-AV (Experiment Setting I, Missing Ratio 50%).

Figure 11: (a) Latency Breakdown across Expert Counts; (b) Impact of wta-timesteps on Latency.

Explore similar work

May 22, 2026cs.NE

SpikingMoE: SDPrompt-Guided Dynamic Expert Fusion in Spiking Neural Networks

Spiking Neural Networks (SNNs) provide an energy-efficient paradigm for visual recognition. We present SpikingMoE, which integrates a spike-driven Transformer with a Mixture-of-Experts (MoE) framework for dynamic computation. Inspired by the lateral geniculate nucleus (LGN), a spike-driven prompt (SDprompt) enables input-dependent expert routing in a biologically plausible manner. By replacing standard MLPs with spike-compatible expert modules and enforcing binary spike communication, SpikingMoE is designed for neuromorphic hardware. Experiments on CIFAR-10 and CIFAR-100 achieve 94.09% and 74.54% top-1 accuracy, showing that modular expert routing can be incorporated while retaining reasonable performance. To our knowledge, SpikingMoE is the first open-source SNN framework that integrates MoE into a spike-driven Transformer with LGN-inspired routing.

Yukai Yang, Chenxi Qin, Jungang Li +3

Tianjin University, Medical School, Tianjin 300072, China · Southern University of Science and Technology, Department of Computer Science and Engineering, Shenzhen, Guangdong 518055, China · Chinese Academy of Medical Sciences, Institute of Biomedical Engineering, Tianjin 300192, China

Spiking Neural Networks Neuromorphic Computing Localized Lora-Moe Neural Network

Ask PDF

Aug 3, 2026cs.NE

SMM Transformer: Leveraging Spiking Neural Networks for Multimodal Tasks

Spiking Neural Networks (SNNs) enable event-driven computation with sparse activations, but building multimodal Transformers on SNNs is hindered by unstable training in deep spiking stacks and the mismatch between dense softmax attention and spike-based communication. We propose SMM Transformer, an SNN-based multimodal Transformer framework that combines (i)PLMP, a Parallel LIF with Multistage Learnable Parameters neuron and a tailored P-STBP algorithm for stable deep SNN training, (ii) SMSA, an attention-inspired spike-driven token-mixing module that replaces dense pairwise softmax attention with channel-wise spike co-activation and self-compensation, and (iii)SMoE, a spiking mixture-of-experts module for modality-aware fusion. Across visual and multimodal benchmarks, SMM Transformer achieves competitive accuracy compared to ANN baselines. Under a standard MAC/AC arithmetic model, SMSA reduces the estimated operator-level compute energy of the attention module by up to 97%, while whole-model profiling shows more moderate but consistent efficiency gains.

Xiubo Liang, Jinxing Han, Yuke Li +3

School of Software Technology, Zhejiang University, Ningbo, China · NetEase Yidun AI Lab, Hangzhou, China

Spiking Neural Networks Detection Transformer Transformer Architectures Multimodal Deep Learning Multimodal Understanding Neural Network

Ask PDF

May 15, 2026cs.CV

When Does Sparse MoE Help in Vision? The Role of Backbone Compute Leverage in Sparse Routing

Mixture-of-Experts (MoE) networks promise favorable accuracy-compute trade-offs, yet practical vision deployments are hindered by expert collapse and limited end-to-end efficiency gains. We study when sparse top-

k

routing with hard capacity constraints helps in vision classification, evaluated under multi-seed protocols on four benchmarks (CIFAR-10/100, Tiny-ImageNet, ImageNet-1K). We observe a \emph{compute-leverage pattern}: positive accuracy gaps require a substantial fraction

ρ

of total FLOPs to be routed; at ImageNet scale this is necessary but not sufficient, as multi-expert routing (

k \geq 2

) is additionally required. Two controlled experiments isolate these factors. A hidden-size sweep on CIFAR-10 yields both predicted sign reversals across standard and depthwise backbones, ruling out backbone family as the active variable. An ImageNet-1K ablation that varies only top-

k

-- holding architecture, initialization, and

ρ

fixed -- reverses the gap from positive to negative across all five seeds. A per-sample variant of Soft MoE that softmaxes over experts rather than the batch rescues CIFAR-100 above the dense baseline, identifying batch-axis dispatch as the dominant failure mode in per-sample CNN settings. Code and aggregate results: https://github.com/libophd/sparse-moe-vision-rho.

Libo Sun, Po-wei Harn, Peixiong He +1

Department of Computer Science and Software Engineering Auburn University, Auburn, AL 36849, USA · Department of Information Management National Central University, Taoyuan 320317, Taiwan · Department of Computer Science and Software Engineering Auburn University

Mixture-Of-Experts Architectures Mixture-Of-Experts Capability-Speed Trade-Offs Imagenet Role

Ask PDF