Mixture-Of-Experts

Follow this field Ask about this field

Recent momentum

emerging

0 papers in the last 28 days · 0.0% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this field, kept on the site without email delivery.

Period ending 2026-09-21

21 new papers

A weekly snapshot of new work published in Mixture-Of-Experts.

Period ending 2026-09-14

8 new papers

A weekly snapshot of new work published in Mixture-Of-Experts.

Period ending 2026-09-07

23 new papers

A weekly snapshot of new work published in Mixture-Of-Experts.

Inside this field

Focused directions

Focused topic

Mixture-of-Experts Models

Within Mixture-Of-Experts

419 papersExplore

Focused topic

Experts

Within Mixture-Of-Experts

167 papersExplore

Focused topic

Mixture-Of-Expert Language Models

Within Mixture-Of-Experts

72 papersExplore

Focused topic

Mixture-Of-Expert Inference

Within Mixture-Of-Experts

66 papersExplore

Focused topic

Expert Demonstrations

Within Mixture-Of-Experts

55 papersExplore

Focused topic

Load Balancing

Within Mixture-Of-Experts

18 papersExplore

584 papers

Latest in Mixture-Of-Experts

Browse all topics →

Patch-MoE Mamba: A Patch-Ordered Mixture-of-Experts State Space Architecture for Medical Image Segmentation
May 18, 2026Diego Adame, Fabian Vazquez, Jose A. Nunez +7Medical Image Segmentation Convolutional Neural Networks
Mixture of Experts for Low-Resource LLMs
May 17, 2026Ori Bar Joseph, Smadar Arvatz, Noam Kayzer +2Mixture-Of-Expert Language Models Mixture-of-Experts Models
TAME: Test-Time Adversarial Prompt Tuning via Mixture-of-Experts for Vision-Language Models
May 17, 2026Xin Wang, Yixu Wang, Jiaming Zhang +6Vision-Language Model Adaptation Large Vision Language Models
Stable Routing for Mixture-of-Experts in Class-Incremental Learning
May 17, 2026Zirui Guo, Quan Cheng, Da-Wei Zhou +1Domain Incremental Learning Mixture-of-Experts Models
Evaluating Deep Research Agents on Expert Consulting Work: A Benchmark with Verifiers, Rubrics, and Cognitive Traps
May 17, 2026Tanmay Asthana, Aman Saksena, Divyansh SahuDeep Research Verifier
Dynamic Model Merging Made Slim
May 17, 2026Guodong Du, Wanyu LinContinual Model Merging Mixture-of-Experts Models
UB-SMoE: Universally Balanced Sparse Mixture-of-Experts for Resource-adaptive Federated Fine-tuning of Foundation Models
May 15, 2026Van-Tuan Tran, Hong-Hanh Nguyen-Le, Marco Ruffini +1Heterogeneous Federated Learning Mixture-of-Experts Models
Scalable Knowledge Editing for Mixture-of-Experts LLMs via Tensor-Structured Updates
May 15, 2026Roman Maksimov, Vladimir Aletov, Dmitry Bylinkin +3Multimodal Knowledge Editing Mixture-Of-Expert Language Models
MedMIX: Modality-Internal Expert Fusion for Multimodal Medical Diagnosis
May 15, 2026Seungik Cho, Anqi Li, Wei QiuMultimodal Clinical Data Missing Modalities
CHoE: Cross-Domain Heterogeneous Graph Prompt Learning via Structure-Conditioned Experts
May 15, 2026Peiyuan Li, Yongqi Huang, Jitao Zhao +3Graph Foundation Model Mixture-of-Experts Models
When Does Sparse MoE Help in Vision? The Role of Backbone Compute Leverage in Sparse Routing
May 15, 2026Libo Sun, Po-wei Harn, Peixiong He +1Mixture-of-Experts Models Imagenet
$φ$ -Balancing for Mixture-of-Experts Training
May 14, 2026Lizhang Chen, Jonathan Li, Qi Wang +5Mixture-of-Experts Models Balanced Learning
Eradicating Negative Transfer in Multi-Physics Foundation Models via Sparse Mixture-of-Experts Routing
May 14, 2026Ellwil Sharma, Arastu SharmaFluid Dynamics Mixture-of-Experts Models
Discovering Physical Directions in Weight Space: Composing Neural PDE Experts
May 14, 2026Pengkai Wang, Pengwei Liu, Yuanyi Wang +7Neural Partial Differential Equation Solvers Partial Differential Equations
Asymmetric Generative Recommendation via Multi-Expert Projection and Multi-Faceted Hierarchical Quantization
May 14, 2026Bin Huang, Xin Wang, Junwei Pan +6Large-Scale Recommendation Systems Recommendation
Reduce the Artifacts Bias for More Generalizable AI-Generated Image Detection
May 14, 2026Yiheng Li, Yang Yang, Zichang Tan +3Ai-Generated Image Detection Forgeries
BEAM: Binary Expert Activation Masking for Dynamic Routing in MoE
May 14, 2026Juntong Wu, Jialiang Cheng, Qishen Yin +5Mixture-Of-Expert Inference Mixture-of-Experts Models
RQ-MoE: Residual Quantization via Mixture of Experts for Efficient Input-Dependent Vector Compression
May 14, 2026Zhengjia Zhong, Shuyan Ke, Zaizhou Lin +3Calibration-Free Low-Bit Vector Quantization Codebooks
MetaMoE: Diversity-Aware Proxy Selection for Privacy-Preserving Mixture-of-Experts Unification
May 14, 2026Weisen Jiang, Shuhao Chen, Sinno Jialin PanMixture-of-Experts Models Training Data
EnergyLens: Predictive Energy-Aware Exploration for Multi-GPU LLM Inference Optimization
May 14, 2026Zhiye Song, Kyungmi Lee, Eun Kyung Lee +3LLM Inference Efficiency Multi-Gpu Systems
How to Scale Mixture-of-Experts: From muP to the Maximally Scale-Stable Parameterization
May 13, 2026Leena Chennuru Vankadara, Moritz Haas, Luke Hayward +2Mixture-of-Experts Models Learning Rates
HodgeCover: Higher-Order Topological Coverage Drives Compression of Sparse Mixture-of-Experts
May 13, 2026Tao Zhong, Dongzhe Zheng, Christine Allen-BlanchetteMixture-of-Experts Models Context Compression
Dense vs Sparse Pretraining at Tiny Scale: Active-Parameter vs Total-Parameter Matching
May 13, 2026Abdalrahman WaelSparse Expert Activation Mixture-of-Experts Models
Children's English Reading Story Generation via Supervised Fine-Tuning of Compact LLMs with Controllable Difficulty and Safety
May 13, 2026Qian Shen, Fanghua Cao, Min Yao +3Llm-Generated Stories Large Language
EMO: Frustratingly Easy Progressive Training of Extendable MoE
May 13, 2026Linghao Jin, Chufan Shi, Huijuan Wang +4Mixture-of-Experts Models Large Models
Bayesian Model Merging
May 13, 2026Kaiyang Li, Shaobo Han, Qing Su +1Continual Model Merging Model Merging
Synthesizing the Expert: A Validated Multimodal Dataset for Trustworthy AI-Assisted Swimming Coaching
May 12, 2026Ahmad Al-Kabbany, Esraa KassemSports Science Multimodal Dataset
Sparse Mixture-of-Experts Routing in Visual Diffusion Transformers:Diagnosis, Boundary Calibration and Evolutionary Roadmap from Routing Collapse to Selective Deadlock
May 12, 2026Haiying ShaMixture-of-Experts Models Multimodal Diffusion Transformers
Routers Learn the Geometry of Their Experts: Geometric Coupling in Sparse Mixture-of-Experts
May 12, 2026Sagi Ahrac, Noya Hochwald, Mor GevaMixture-of-Experts Models Load Balancing
Online Learning-to-Defer with Varying Experts
May 12, 2026Dang Hoang Duy, Yannis Montreuil, Maxime Meyer +3Learning-Augmented Algorithms Experts
Composition of Memory Experts for Diffusion World Models
May 12, 2026Sebastian Stapf, Pablo Acuaviva Huertos, Aram Davtyan +1Discrete Diffusion Model Memory Retrieval
ROMER: Expert Replacement and Router Calibration for Robust MoE LLMs on Analog Compute-in-Memory Systems
May 12, 2026Wenyong Zhou, Yuannuo Feng, Yizhe Chen +6Mixture-Of-Expert Language Models In-Memory Computing
M $^4$ -SAM: Multi-Modal Mixture-of-Experts with Memory-Augmented SAM for RGB-D Video Salient Object Detection
May 12, 2026Jiyuan Liu, Jia Lin, Xiaofei Zhou +3Vision Foundation Models Mixture-of-Experts Models
Slicing and Dicing: Configuring Optimal Mixtures of Experts
May 12, 2026Margaret Li, Sneha Kudugunta, Danielle Rothermel +1Mixture-Of-Expert Language Models Mixture-of-Experts Models
Fast MoE Inference via Predictive Prefetching and Expert Replication
May 12, 2026Ankit Jyothish, Ali Jannesari, Aishwarya Sarkar +1Mixture-Of-Expert Inference LLM Inference Efficiency
Leveraging Multimodal Large Language Models for All-in-One Image Restoration via a Mixture of Frequency Experts
May 12, 2026Eunho Lee, Rei Kawakami, Youngbae HwangImage Restoration Multimodal Pretraining
DECO: Sparse Mixture-of-Experts with Dense-Comparable Performance on End-Side Devices
May 11, 2026Chenyang Song, Weilin Zhao, Xu Han +3Mixture-Of-Expert Inference Mixture-of-Experts Models
Fast Training of Mixture-of-Experts for Time Series Forecasting via Expert Loss Integration
May 11, 2026Btissame El Mahtout, Florian ZielMixture-of-Experts Models
HELLoRA: Hot Experts Layer-Level Low-Rank Adaptation for Mixture-of-Experts Models
May 11, 2026Jia Wei, Zhonghao Zhang, Ping Chen +5Mixture-Of-Expert Language Models Low-Rank Adaptation
Mixture of Layers with Hybrid Attention
May 10, 2026Ivan Ternovtsii, Yurii BilakMixture-of-Experts Models Mixtures
FLAME: Adaptive Mixture-of-Experts for Continual Multimodal Multi-Task Learning
May 10, 2026Xing Han, Shravan Chaudhari, Tanvi Ranade +2Mixture-of-Experts Models Unseen-Task Adaptation
DisagMoE: Computation-Communication overlapped MoE Training via Disaggregated AF-Pipe Parallelism
May 10, 2026Zhichen Zeng, Chi-Chih Chang, Jiayi Wang +10Mixture-Of-Expert Inference Mixture-of-Experts Models
TileQ: Efficient Low-Rank Quantization of Mixture-of-Experts with 2D Tiling
May 10, 2026Hongyaoxing Gu, Xinzhe Chen, Lijuan Hu +1Mixture-Of-Expert Inference Mixed-Precision Quantization
Sparse Layers are Critical to Scaling Looped Language Models
May 9, 2026Ryan Lee, Jacob Biloki, Edward J. Hu +1Transformer Layers Large Models
When (and How) to Trust the Expert: Diagnosing Query-Time Expert-Guided Reinforcement Learning
May 9, 2026Yann Berthelot, Philippe Preux, Riad AkrourExperts Execution Efficiency
XPERT: Expert Knowledge Transfer for Effective Training of Language Models
May 9, 2026Chang Liu, Boyu Shi, Xu Yang +1Mixture-Of-Expert Language Models Mixture-of-Experts Models
SlimQwen: Exploring the Pruning and Distillation in Large MoE Model Pre-training
May 9, 2026Shengkun Tang, Zekun Wang, Bo Zheng +7Large Language Model Compression Large Language Model Pretraining
FLUX: Geometry-Aware Longitudinal Flow Matching with Mixture of Experts
May 9, 2026Josue Ortega Caro, Yongxu Zhang, Hannah M Batchelor +3Latent States Mixture-of-Experts Models
ReLibra: Routing-Replay-Guided Load Balancing for MoE Training in Reinforcement Learning
May 9, 2026Chao Jin, Xinming Wei, Yinmin Zhong +6Load Balancing Mixture-of-Experts Models
Uncovering Intra-expert Activation Sparsity for Efficient Mixture-of-Expert Model Execution
May 9, 2026Jongseok Park, Sunga Kim, Zhenyu Gu +2Sparse Expert Activation Mixture-of-Experts Models
SDG-MoE: Signed Debate Graph Mixture-of-Experts
May 8, 2026Stepan Kulibaba, Kirill Labzin, Artem Dzhalilov +4Mixture-of-Experts Models Deliberation
DIMoE-Adapters: Dynamic Expert Evolution for Continual Learning in Vision-Language Models
May 8, 2026Mengxin Qin, Xiang Zhang, Xi Wang +3Vision-Language Model Adaptation Continual Learning Methods
Hierarchical Mixture-of-Experts with Two-Stage Optimization
May 8, 2026Gleb Molodtsov, Alexander Miasnikov, Aleksandr BeznosikovMixture-of-Experts Models Load Balancing
When Are Experts Misrouted? Counterfactual Routing Analysis in Mixture-of-Experts Language Models
May 8, 2026Youngsik Yoon, Siwei Wang, Wei Chen +1Mixture-of-Experts Models Experts
TAS-LoRA: Transformer Architecture Search with Mixture-of-LoRA Experts
May 8, 2026Jeimin Jeon, Hyunju Lee, Bumsub HamSelf-Supervised Vision Transformers Neural Architecture Search
Geometric Asymmetry in MoE Specialization: Functional Decorrelation and Representational Overlap
May 8, 2026Feilong LiuMixture-of-Experts Models Representation Space
Kernel Foundry: A Diagnosis-driven Evolutionary Kernel Optimizer with Multi-Experts
May 8, 2026Zixuan Huang, Da Chen, Kecheng Huang +5Graphics Processing Unit Kernels Code Optimization
UniPool: A Globally Shared Expert Pool for Mixture-of-Experts
May 7, 2026Minbin Huang, Han Shi, Chuanyang Zheng +5Mixture-of-Experts Models Experts
EMO: Pretraining Mixture of Experts for Emergent Modularity
May 7, 2026Ryan Wang, Akshita Bhagia, Sewon MinMixture-Of-Expert Language Models Mixture-of-Experts Models
Scene-Adaptive Continual Learning for CSI-based Human Activity Recognition with Mixture of Experts
May 7, 2026Wenhan Zheng, Yuyi Mao, Ivan Wang-Hei HoHuman Activity Recognition Cross-Sensor Generalization

May 18, 2026 · cs.CVJ/K move · Enter open · S save

Patch-MoE Mamba: A Patch-Ordered Mixture-of-Experts State Space Architecture for Medical Image Segmentation

Diego Adame, Fabian Vazquez, Jose A. Nunez, Huimin Li+6

University of Texas Rio Grande Valley, Edinburg, TX 78539, USA

CNN- and Transformer-based architectures have achieved strong performance in medical image segmentation, but CNNs are limited in modeling long-range dependencies, while Transformers often suffer from quadratic computational and memory complexity. State space models, especially Mamba-based networks, offer an efficient alternative with linear sequence complexity. However, existing Mamba segmentation models still face two limitations: pixel-wise directional scanning can disrupt local 2D spatial structure, and simple summation-based fusion of scan directions cannot adapt well to diverse object sizes, shapes, and boundaries. To address these issues, we propose \textit{Patch-MoE Mamba}, a patch-ordered mixture-of-experts state space architecture for medical image segmentation. It introduces a hierarchical patch-ordered scanning mechanism that preserves local spatial neighborhoods while capturing multi-scale context, and an MoE-based directional fusion module that adaptively combines multiple Mamba scanner outputs using four directional experts, a learnable concatenation expert, and residual directional aggregation. Experiments on five public polyp segmentation benchmarks and the ISIC 2017/2018 skin lesion segmentation datasets demonstrate the effectiveness and generality of Patch-MoE Mamba.

Medical Image Segmentation Convolutional Neural Networks Mixture-of-Experts Models

Open paper Ask about it PDF