Mixture-of-Experts Models

Follow this topic Ask about this topic

Recent momentum

-34%

43 papers in the last 28 days · 0.7% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this topic, kept on the site without email delivery.

Period ending 2026-09-21

14 new papers

A weekly snapshot of new work published in Mixture-of-Experts Models.

Period ending 2026-09-14

2 new papers

A weekly snapshot of new work published in Mixture-of-Experts Models.

Period ending 2026-09-07

12 new papers

A weekly snapshot of new work published in Mixture-of-Experts Models.

419 papers

Latest in Mixture-of-Experts Models

Browse all topics →

Patch-MoE Mamba: A Patch-Ordered Mixture-of-Experts State Space Architecture for Medical Image Segmentation
May 18, 2026Diego Adame, Fabian Vazquez, Jose A. Nunez +7Medical Image Segmentation Convolutional Neural Networks
Mixture of Experts for Low-Resource LLMs
May 17, 2026Ori Bar Joseph, Smadar Arvatz, Noam Kayzer +2Mixture-Of-Expert Language Models Mixture-of-Experts Models
TAME: Test-Time Adversarial Prompt Tuning via Mixture-of-Experts for Vision-Language Models
May 17, 2026Xin Wang, Yixu Wang, Jiaming Zhang +6Vision-Language Model Adaptation Large Vision Language Models
Stable Routing for Mixture-of-Experts in Class-Incremental Learning
May 17, 2026Zirui Guo, Quan Cheng, Da-Wei Zhou +1Domain Incremental Learning Mixture-of-Experts Models
Dynamic Model Merging Made Slim
May 17, 2026Guodong Du, Wanyu LinContinual Model Merging Mixture-of-Experts Models
UB-SMoE: Universally Balanced Sparse Mixture-of-Experts for Resource-adaptive Federated Fine-tuning of Foundation Models
May 15, 2026Van-Tuan Tran, Hong-Hanh Nguyen-Le, Marco Ruffini +1Heterogeneous Federated Learning Mixture-of-Experts Models
Scalable Knowledge Editing for Mixture-of-Experts LLMs via Tensor-Structured Updates
May 15, 2026Roman Maksimov, Vladimir Aletov, Dmitry Bylinkin +3Multimodal Knowledge Editing Mixture-Of-Expert Language Models
MedMIX: Modality-Internal Expert Fusion for Multimodal Medical Diagnosis
May 15, 2026Seungik Cho, Anqi Li, Wei QiuMultimodal Clinical Data Missing Modalities
CHoE: Cross-Domain Heterogeneous Graph Prompt Learning via Structure-Conditioned Experts
May 15, 2026Peiyuan Li, Yongqi Huang, Jitao Zhao +3Graph Foundation Model Mixture-of-Experts Models
When Does Sparse MoE Help in Vision? The Role of Backbone Compute Leverage in Sparse Routing
May 15, 2026Libo Sun, Po-wei Harn, Peixiong He +1Mixture-of-Experts Models Imagenet
$φ$ -Balancing for Mixture-of-Experts Training
May 14, 2026Lizhang Chen, Jonathan Li, Qi Wang +5Mixture-of-Experts Models Balanced Learning
Eradicating Negative Transfer in Multi-Physics Foundation Models via Sparse Mixture-of-Experts Routing
May 14, 2026Ellwil Sharma, Arastu SharmaFluid Dynamics Mixture-of-Experts Models
Asymmetric Generative Recommendation via Multi-Expert Projection and Multi-Faceted Hierarchical Quantization
May 14, 2026Bin Huang, Xin Wang, Junwei Pan +6Large-Scale Recommendation Systems Recommendation
Reduce the Artifacts Bias for More Generalizable AI-Generated Image Detection
May 14, 2026Yiheng Li, Yang Yang, Zichang Tan +3Ai-Generated Image Detection Forgeries
BEAM: Binary Expert Activation Masking for Dynamic Routing in MoE
May 14, 2026Juntong Wu, Jialiang Cheng, Qishen Yin +5Mixture-Of-Expert Inference Mixture-of-Experts Models
MetaMoE: Diversity-Aware Proxy Selection for Privacy-Preserving Mixture-of-Experts Unification
May 14, 2026Weisen Jiang, Shuhao Chen, Sinno Jialin PanMixture-of-Experts Models Training Data
EnergyLens: Predictive Energy-Aware Exploration for Multi-GPU LLM Inference Optimization
May 14, 2026Zhiye Song, Kyungmi Lee, Eun Kyung Lee +3LLM Inference Efficiency Multi-Gpu Systems
How to Scale Mixture-of-Experts: From muP to the Maximally Scale-Stable Parameterization
May 13, 2026Leena Chennuru Vankadara, Moritz Haas, Luke Hayward +2Mixture-of-Experts Models Learning Rates
HodgeCover: Higher-Order Topological Coverage Drives Compression of Sparse Mixture-of-Experts
May 13, 2026Tao Zhong, Dongzhe Zheng, Christine Allen-BlanchetteMixture-of-Experts Models Context Compression
Dense vs Sparse Pretraining at Tiny Scale: Active-Parameter vs Total-Parameter Matching
May 13, 2026Abdalrahman WaelSparse Expert Activation Mixture-of-Experts Models
EMO: Frustratingly Easy Progressive Training of Extendable MoE
May 13, 2026Linghao Jin, Chufan Shi, Huijuan Wang +4Mixture-of-Experts Models Large Models
Bayesian Model Merging
May 13, 2026Kaiyang Li, Shaobo Han, Qing Su +1Continual Model Merging Model Merging
Sparse Mixture-of-Experts Routing in Visual Diffusion Transformers:Diagnosis, Boundary Calibration and Evolutionary Roadmap from Routing Collapse to Selective Deadlock
May 12, 2026Haiying ShaMixture-of-Experts Models Multimodal Diffusion Transformers
Routers Learn the Geometry of Their Experts: Geometric Coupling in Sparse Mixture-of-Experts
May 12, 2026Sagi Ahrac, Noya Hochwald, Mor GevaMixture-of-Experts Models Load Balancing
M $^4$ -SAM: Multi-Modal Mixture-of-Experts with Memory-Augmented SAM for RGB-D Video Salient Object Detection
May 12, 2026Jiyuan Liu, Jia Lin, Xiaofei Zhou +3Vision Foundation Models Mixture-of-Experts Models
Slicing and Dicing: Configuring Optimal Mixtures of Experts
May 12, 2026Margaret Li, Sneha Kudugunta, Danielle Rothermel +1Mixture-Of-Expert Language Models Mixture-of-Experts Models
Leveraging Multimodal Large Language Models for All-in-One Image Restoration via a Mixture of Frequency Experts
May 12, 2026Eunho Lee, Rei Kawakami, Youngbae HwangImage Restoration Multimodal Pretraining
DECO: Sparse Mixture-of-Experts with Dense-Comparable Performance on End-Side Devices
May 11, 2026Chenyang Song, Weilin Zhao, Xu Han +3Mixture-Of-Expert Inference Mixture-of-Experts Models
Fast Training of Mixture-of-Experts for Time Series Forecasting via Expert Loss Integration
May 11, 2026Btissame El Mahtout, Florian ZielMixture-of-Experts Models
HELLoRA: Hot Experts Layer-Level Low-Rank Adaptation for Mixture-of-Experts Models
May 11, 2026Jia Wei, Zhonghao Zhang, Ping Chen +5Mixture-Of-Expert Language Models Low-Rank Adaptation
Mixture of Layers with Hybrid Attention
May 10, 2026Ivan Ternovtsii, Yurii BilakMixture-of-Experts Models Mixtures
FLAME: Adaptive Mixture-of-Experts for Continual Multimodal Multi-Task Learning
May 10, 2026Xing Han, Shravan Chaudhari, Tanvi Ranade +2Mixture-of-Experts Models Unseen-Task Adaptation
DisagMoE: Computation-Communication overlapped MoE Training via Disaggregated AF-Pipe Parallelism
May 10, 2026Zhichen Zeng, Chi-Chih Chang, Jiayi Wang +10Mixture-Of-Expert Inference Mixture-of-Experts Models
TileQ: Efficient Low-Rank Quantization of Mixture-of-Experts with 2D Tiling
May 10, 2026Hongyaoxing Gu, Xinzhe Chen, Lijuan Hu +1Mixture-Of-Expert Inference Mixed-Precision Quantization
Sparse Layers are Critical to Scaling Looped Language Models
May 9, 2026Ryan Lee, Jacob Biloki, Edward J. Hu +1Transformer Layers Large Models
XPERT: Expert Knowledge Transfer for Effective Training of Language Models
May 9, 2026Chang Liu, Boyu Shi, Xu Yang +1Mixture-Of-Expert Language Models Mixture-of-Experts Models
SlimQwen: Exploring the Pruning and Distillation in Large MoE Model Pre-training
May 9, 2026Shengkun Tang, Zekun Wang, Bo Zheng +7Large Language Model Compression Large Language Model Pretraining
FLUX: Geometry-Aware Longitudinal Flow Matching with Mixture of Experts
May 9, 2026Josue Ortega Caro, Yongxu Zhang, Hannah M Batchelor +3Latent States Mixture-of-Experts Models
ReLibra: Routing-Replay-Guided Load Balancing for MoE Training in Reinforcement Learning
May 9, 2026Chao Jin, Xinming Wei, Yinmin Zhong +6Load Balancing Mixture-of-Experts Models
Uncovering Intra-expert Activation Sparsity for Efficient Mixture-of-Expert Model Execution
May 9, 2026Jongseok Park, Sunga Kim, Zhenyu Gu +2Sparse Expert Activation Mixture-of-Experts Models
SDG-MoE: Signed Debate Graph Mixture-of-Experts
May 8, 2026Stepan Kulibaba, Kirill Labzin, Artem Dzhalilov +4Mixture-of-Experts Models Deliberation
DIMoE-Adapters: Dynamic Expert Evolution for Continual Learning in Vision-Language Models
May 8, 2026Mengxin Qin, Xiang Zhang, Xi Wang +3Vision-Language Model Adaptation Continual Learning Methods
Hierarchical Mixture-of-Experts with Two-Stage Optimization
May 8, 2026Gleb Molodtsov, Alexander Miasnikov, Aleksandr BeznosikovMixture-of-Experts Models Load Balancing
When Are Experts Misrouted? Counterfactual Routing Analysis in Mixture-of-Experts Language Models
May 8, 2026Youngsik Yoon, Siwei Wang, Wei Chen +1Mixture-of-Experts Models Experts
Geometric Asymmetry in MoE Specialization: Functional Decorrelation and Representational Overlap
May 8, 2026Feilong LiuMixture-of-Experts Models Representation Space
Kernel Foundry: A Diagnosis-driven Evolutionary Kernel Optimizer with Multi-Experts
May 8, 2026Zixuan Huang, Da Chen, Kecheng Huang +5Graphics Processing Unit Kernels Code Optimization
UniPool: A Globally Shared Expert Pool for Mixture-of-Experts
May 7, 2026Minbin Huang, Han Shi, Chuanyang Zheng +5Mixture-of-Experts Models Experts
EMO: Pretraining Mixture of Experts for Emergent Modularity
May 7, 2026Ryan Wang, Akshita Bhagia, Sewon MinMixture-Of-Expert Language Models Mixture-of-Experts Models
E = T*H/(O+B): A Dimensionless Control Parameter for Mixture-of-Experts Ecology
May 7, 2026Qingjun ZhangMixture-of-Experts Models Exponents
Continuous Expert Assembly: Instance-Conditioned Low-Rank Residuals for All-in-One Image Restoration
May 7, 2026Haisen He, Xiangyu Zou, SongLin Dong +3Image Restoration Restoration
VisMMOE: Exploiting Visual-Expert Affinity for Efficient Visual-Language MoE Offloading
May 7, 2026Cheng Xu, Xiaofeng Hou, Jiacheng Liu +1Large Vision Language Models Mixture-of-Experts Models
MTL-MAD: Multi-Task Learners are Effective Medical Anomaly Detectors
May 7, 2026Bogdan Alexandru Bercean, Florinel Alin Croitoru, Vlad Hondru +3Medical Imaging Anomaly
ZAYA1-8B Technical Report
May 6, 2026Robert Washbourne, Rishi Iyer, Tomas Figliolia +15Deepseek Mixture-of-Experts Models
Expert Routing for Communication-Efficient MoE via Finite Expert Banks
May 6, 2026Mohammad Reza Deylam Salehi, Ali KhalesiMixture-Of-Expert Inference Mixture-of-Experts Models
Piper: Efficient Large-Scale MoE Training via Resource Modeling and Pipelined Hybrid Parallelism
May 6, 2026Sajal Dash, Feiyi WangMixture-Of-Expert Inference High-Performance Computing
Adaptive Inverted-Index Routing for Granular Mixtures-of-Experts
May 6, 2026Klaus-Rudolf Kladny, Maximilian Mordig, Bernhard Schölkopf +1Mixture-of-Experts Models
AxMoE: Characterizing the Impact of Approximate Multipliers on Mixture-of-Experts DNN Architectures
May 6, 2026Omkar B Shende, Marcello Traiola, Gayathri AnanthanarayananMixture-Of-Expert Inference Graph Neural Network Inference
SPHERE: Mitigating the Loss of Spectral Plasticity in Mixture-of-Experts for Deep Reinforcement Learning
May 6, 2026Lirui Luo, Guoxi Zhang, Hongming Xu +2Deep Reinforcement Learning Mixture-of-Experts Models
GEM: Graph-Enhanced Mixture-of-Experts with ReAct Agents for Dialogue State Tracking
May 6, 2026Ziqi Zhu, Adithya Suresh, Tomal Deb +1Multi-Turn Conversations Agent Framework
Unified Multimodal Visual Tracking with Dual Mixture-of-Experts
May 5, 2026Lingyi Hong, Jinglun Li, Xinyu Zhou +6Multi-Object Tracking Multimodal Fusion

May 18, 2026 · cs.CVJ/K move · Enter open · S save

Patch-MoE Mamba: A Patch-Ordered Mixture-of-Experts State Space Architecture for Medical Image Segmentation

Diego Adame, Fabian Vazquez, Jose A. Nunez, Huimin Li+6

University of Texas Rio Grande Valley, Edinburg, TX 78539, USA

CNN- and Transformer-based architectures have achieved strong performance in medical image segmentation, but CNNs are limited in modeling long-range dependencies, while Transformers often suffer from quadratic computational and memory complexity. State space models, especially Mamba-based networks, offer an efficient alternative with linear sequence complexity. However, existing Mamba segmentation models still face two limitations: pixel-wise directional scanning can disrupt local 2D spatial structure, and simple summation-based fusion of scan directions cannot adapt well to diverse object sizes, shapes, and boundaries. To address these issues, we propose \textit{Patch-MoE Mamba}, a patch-ordered mixture-of-experts state space architecture for medical image segmentation. It introduces a hierarchical patch-ordered scanning mechanism that preserves local spatial neighborhoods while capturing multi-scale context, and an MoE-based directional fusion module that adaptively combines multiple Mamba scanner outputs using four directional experts, a learnable concatenation expert, and residual directional aggregation. Experiments on five public polyp segmentation benchmarks and the ISIC 2017/2018 skin lesion segmentation datasets demonstrate the effectiveness and generality of Patch-MoE Mamba.

Medical Image Segmentation Convolutional Neural Networks Mixture-of-Experts Models

Open paper Ask about it PDF

Recent momentum

Neighbouring directions

Recent digests

14 new papers

2 new papers

12 new papers

Latest in Mixture-of-Experts Models