Mixture-of-Experts Models

Follow this topic Ask about this topic

Momentum

41 papers in the last four weeks, up 116% on the four weeks before. 0.4% of all new papers.

Jul 13Week of Sep 28

Latest papers 297

All topics

ConceptM $^3$ oE: Concept-Guided Multimodal Mixture of Experts for Interpretable Computational Pathology
May 23, 2026Xuan Wang, Zhongling Xu, Gopi Kannedhara +13Computational Pathology Concept Bottleneck Models
University of Texas at Austin, Austin, TX, USA · University of North Carolina at Chapel Hill, NC, USA · Dell Children’s Medical Center, Austin, TX, USA +2
Complete-muE: Optimal Hyperparameter Transfer and Scaling for MoE Models
May 22, 2026Hongwu Peng, Ohiremen Dibua, Yuanjun Xiong +3Transformer Neural Network Optimization
Adobe Research
Semantically Structured Mixture-of-Experts for Compositional Robotic Manipulation
May 22, 2026Chengyu Deng, Guanqi Chen, Yizhou Chen +4Robot Policy Learning Robot Skill Learning
The University of Hong Kong · Southern University of Science and Technology
Tackling Multimodal Learning Challenges with Mixture-of-Expert: A Survey
May 22, 2026Liangwei Nathan Zheng, Wei Emma Zhang, Olaf Maennel +2Efficient Multimodal Inference Cross-Modal Representation Learning
Adelaide University
SpikingMoE: SDPrompt-Guided Dynamic Expert Fusion in Spiking Neural Networks
May 22, 2026Yukai Yang, Chenxi Qin, Jungang Li +3Spiking Transformers Dynamic Neural Networks
Tianjin University, Medical School, Tianjin 300072, China · Southern University of Science and Technology, Department of Computer Science and Engineering, Shenzhen, Guangdong 518055, China · Chinese Academy of Medical Sciences, Institute of Biomedical Engineering, Tianjin 300192, China
GMENet: Generative Mixture of Experts Network for Multi-Center Glioma Diagnosis with Incomplete Imaging Sequences
May 22, 2026Pengfei Song, Fangjin Liu, Wenwen Zeng +5Missing-Modality Learning Medical Diagnosis
School of Biomedical Engineering and Technology Innovation, Fudan University · Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University · 3Intelligent Diagnosis and Treatment Laboratory for Brain Diseases, Joint Laboratory of Neurosurgery Department of Huashan Hospital and School of Information Science and Technology, Fudan University
FAME: Failure-Aware Mixture-of-Experts for Message-Level Log Anomaly Detection
May 21, 2026Huanchi Wang, Zihang Huang, Yifang Tian +3Fault Detection Mixture-of-Experts Models
Department of Electrical and Computer Engineering University of Toronto, Toronto, Ontario
Learning Emergent Modular Representations in Multi-modality Medical Vision Foundation Models
May 21, 2026Yuting He, Chenyu You, Shuo LiMedical Imaging Foundation Models Multimodal Pretraining
Case Western Reserve University · Stony Brook University
FedCoE: Bridging Generalization and Personalization via Federated Coordinated Dual-level MoEs
May 20, 2026Penglin Dai, Fulian Li, Xincao Xu +3Federated Learning Aggregation Non-IID Federated Learning
School of Computing and Artificial Intelligence, Southwest Jiaotong University, Chengdu 611756, China · Shenzhen Institute for Advanced Study, University of Electronic Science and Technology of China, Shenzhen 518110, China · School of Computer Science and Engineering, Northeastern University, Shenyang 110819, China
Task-Routed Mixture-of-Experts with Cognitive Appraisal for Implicit Sentiment Analysis
May 20, 2026Yaping Chai, Haoran Xie, Joe S. QinSentiment Analysis Multi-Task Learning
Division of Artificial Intelligence, Lingnan University, Hong Kong
HDMoE: A Hierarchical Decoupling-Fusion Mixture-of-Experts Framework for Multimodal Cancer Survival Prediction
May 20, 2026Huayi Wang, Haochao Ying, Yuyang Xu +5Cancer Survival Prediction Survival Analysis
Zhejiang University Hangzhou, China · Xinjiang University Urumqi, China · Hangzhou City University Hangzhou, China +1
Dynamic TMoE: A Drift-Aware Dynamic Mixture of Experts Framework for Non-Stationary Time Series Forecasting
May 20, 2026Jiawen Zhu, Shuhan Liu, Di Weng +1Concept Drift Mixture of Experts
School of Software Technology, Zhejiang University, Ningbo, China · State Key Lab of CAD&CG, Zhejiang University, Hangzhou, China
Beyond Routing: Characterising Expert Tuning and Representation in Vision Mixture-of-Experts
May 20, 2026Gene Tangtartharakul, Katherine R. StorrsVisual Representation Learning Mixture of Experts
School of Psychology · University of Auckland · Auckland, New Zealand
GEM: GPU-Variability-Aware Expert to GPU Mapping for MoE Systems
May 19, 2026Sourish Wawdhane, Avinash Kumar, Poulami DasExpert Load Balancing Mixture-of-Experts Inference
The University of Texas at Austin
DBES: A Systematic Benchmark and Metric Suite for Evaluating Expert Specialization in Large-Scale MoEs
May 18, 2026Jing Wang, Hongxuan Lu, Jazze Young +2Mixture-of-Experts Language Models Mixture-of-Experts Models
Patch-MoE Mamba: A Patch-Ordered Mixture-of-Experts State Space Architecture for Medical Image Segmentation
May 18, 2026Diego Adame, Fabian Vazquez, Jose A. Nunez +7Mamba Image Segmentation
University of Texas Rio Grande Valley, Edinburg, TX 78539, USA
Stable Routing for Mixture-of-Experts in Class-Incremental Learning
May 17, 2026Zirui Guo, Quan Cheng, Da-Wei Zhou +1Class-Incremental Learning Expert Routing
State Key Laboratory of Novel Software Technology, Nanjing University, Nanjing, China · School of Artificial Intelligence, Nanjing University, Nanjing, China
CHoE: Cross-Domain Heterogeneous Graph Prompt Learning via Structure-Conditioned Experts
May 15, 2026Peiyuan Li, Yongqi Huang, Jitao Zhao +3Heterogeneous GNNs Few-Shot Learning
School of Computer Science and Technology, Tianjin University, Tianjin, China · Department of Health Technology and Informatics, and Department of Data Science and Artificial Intelligence, The Hong Kong Polytechnic University, Kowloon, Hong Kong
When Does Sparse MoE Help in Vision? The Role of Backbone Compute Leverage in Sparse Routing
May 15, 2026Libo Sun, Po-wei Harn, Peixiong He +1Sparse Mixture-of-Experts Expert Routing
Department of Computer Science and Software Engineering Auburn University, Auburn, AL 36849, USA · Department of Information Management National Central University, Taoyuan 320317, Taiwan · Department of Computer Science and Software Engineering Auburn University
$φ$ -Balancing for Mixture-of-Experts Training
May 14, 2026Lizhang Chen, Jonathan Li, Qi Wang +5Expert Load Balancing Mirror Descent
University of Texas at Austin · 2Northwestern University.
RQ-MoE: Residual Quantization via Mixture of Experts for Efficient Input-Dependent Vector Compression
May 14, 2026Zhengjia Zhong, Shuyan Ke, Zaizhou Lin +3Residual VQ Vector Quantization
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, Xiamen, China.
MetaMoE: Diversity-Aware Proxy Selection for Privacy-Preserving Mixture-of-Experts Unification
May 14, 2026Weisen Jiang, Shuhao Chen, Sinno Jialin PanPrivacy-Preserving ML Mixture-of-Experts Models
Department of Computer Science and Engineering, The Chinese University of Hong Kong · Department of Computer Science and Engineering, Hong Kong University of Science and Technology · Department of Computer Science and Engineering, Southern University of Science and Technology
How to Scale Mixture-of-Experts: From muP to the Maximally Scale-Stable Parameterization
May 13, 2026Leena Chennuru Vankadara, Moritz Haas, Luke Hayward +2Mixture-of-Experts Models
1Gatsby Computational Neuroscience Unit, University College London · 2Amazon · University of Tübingen, Tübingen AI Center
Dense vs Sparse Pretraining at Tiny Scale: Active-Parameter vs Total-Parameter Matching
May 13, 2026Abdalrahman WaelLanguage Model Pretraining Mixture-of-Experts Language Models
Independent Researcher
EMO: Frustratingly Easy Progressive Training of Extendable MoE
May 13, 2026Linghao Jin, Chufan Shi, Huijuan Wang +4Sparse Mixture-of-Experts Efficient Language Model Training
USC-ISI · MBZUAI-IFM
Sparse Mixture-of-Experts Routing in Visual Diffusion Transformers:Diagnosis, Boundary Calibration and Evolutionary Roadmap from Routing Collapse to Selective Deadlock
May 12, 2026Haiying ShaVideo Diffusion Models Diffusion Transformer
Independent Researcher
Routers Learn the Geometry of Their Experts: Geometric Coupling in Sparse Mixture-of-Experts
May 12, 2026Sagi Ahrac, Noya Hochwald, Mor GevaExpert Load Balancing Parameter-Free Mixture-of-Experts Routing
Blavatnik School of Computer Science and AI, Tel Aviv University
Fast Training of Mixture-of-Experts for Time Series Forecasting via Expert Loss Integration
May 11, 2026Btissame El Mahtout, Florian ZielTime Series Forecasting Mixture-of-Experts Models
Chair of Data Science in Energy and Environment, University of Duisburg-Essen, Germany · Chair of Econometrics and Statistics, Technical University of Dortmund, Germany
Mixture of Layers with Hybrid Attention
May 10, 2026Ivan Ternovtsii, Yurii BilakMixture-of-Experts Models Hybrid Attention
Department of Software Systems, Uzhhorod National University Narodna sq. 3, Uzhhorod, Ukraine, 88000
DisagMoE: Computation-Communication overlapped MoE Training via Disaggregated AF-Pipe Parallelism
May 10, 2026Zhichen Zeng, Chi-Chih Chang, Jiayi Wang +10Expert Parallelism Communication-Efficient Distributed Training
ByteDance Seed · University of Washington · Cornell University

May 23, 2026, cs.AIJ/K move · Enter open · S save

ConceptM $^3$ oE: Concept-Guided Multimodal Mixture of Experts for Interpretable Computational Pathology

Xuan Wang, Zhongling Xu, Gopi Kannedhara, Joakim Nguyen+12

University of Texas at Austin, Austin, TX, USA · University of North Carolina at Chapel Hill, NC, USA · Dell Children’s Medical Center, Austin, TX, USA+2

Computational Pathology Concept Bottleneck Models Healthcare Interpretable ML Multimodal Classification Oncology+4

Open paper AskPDF arXiv

Healthcare models are transitioning from unimodal prediction toward multimodal reasoning over heterogeneous diagnostic inputs. In computational pathology, for complex tumor subtypes where morphology alone can be challenging to distinguish, pathology reports and molecular measurements may provide additional diagnostic evidence alongside whole-slide images, yet existing models often fail to clarify how diverse signals assemble into recognizable diagnostic concepts. We propose ConceptM

^3

oE (Concept Multimodal MoE), which embeds concept formation directly within interaction-aware mixture-of-experts (MoE) pathways. The architecture decomposes evidence into modality-specific, redundant, and synergistic experts, which are then projected into structured concept bottlenecks mapping latent features to a hierarchy of morphology and biomarker concepts. To prevent the information loss typical of interpretable bottlenecks, we utilize residual pathways within each expert to allow task-relevant signals to flow both through the concepts and directly to the final task prediction, so that high performance is maintained alongside interpretability. Across an institutional pediatric brain tumor cohort and a public glioma cohort, the framework delivers competitive performance to unconstrained models while producing reasoning traces validated by an independent neuropathologist. In data-limited regimes, ConceptM

^3

oE improves limited-data performance, increasing macro-F1 from 56.41% to 66.70% at small training sizes compared to non-concept-informed baselines, while also showing faster training convergence consistent with the regularizing effect of concept learning. This work offers a scalable path toward high-performance medical AI that is inherently verifiable and better aligned with the complex decision-making of clinical practice.

Momentum

Related topics

Latest papers 297