Overview

Mixture-of-experts models divide computation among expert networks and use a router to select which experts process each input, often activating only a small subset. Common approaches include top-k sparse routing, shared or hierarchical experts, and parameter-efficient expert adapters; researchers also study load balancing to prevent some experts from being overused or ignored. These models are used in language, vision, multimodal learning, forecasting, and robotics, while systems work targets the memory, communication, and latency costs of serving and training them. A central tension is that specialization can improve capacity and efficiency, but balanced routing does not guarantee useful expertise, and routing overhead or weaker performance under changed conditions can offset the gains.

Latest papers

CardsList

489 papers48 papers in the last four weeks, down 27% on the four weeks before. 0.4% of all new papers.

  1. ReMoE: An Interpretable Mixture-of-Experts Framework for Human Mobility Recovery Forecasting Across Disasters

    Nov 17, 2026Yilin Lyu, Kang Liu, Wei Wang +3Spatiotemporal ForecastingMixture-of-Experts Models

  2. UNITE: Consolidating Embodied VLM Specialists via Shared-Anchor One-Bit Factorization

    Nov 16, 2026Tao Tao, Anke Tang, Enneng Yang +3vision-language model mergingVision-Language Models

  3. GEAR2^2: Pruned Experts Live On through Gated Reuse of Activated Experts and a Missing-Mass Residual

    Nov 16, 2026Linliang Chen, Yan Zhong, Wang Kang +3Mixture-of-Experts PruningMixture-of-Experts Language Models

  4. PMR: MoE Routing with Shared Prototypes and Congestion Pricing

    Nov 14, 2026Wending Wang, Binqi Huang, Ju'an Li +4Mixture-of-Experts ModelsExpert Load Balancing

  5. Supercharging Olmo-core for Efficient and Scalable MoE Training

    Nov 13, 2026Tianhua Tao, Akshita Bhagia, Dirk Groeneveld +8Mixture-of-Experts ModelsCommunication-Efficient Mixture-of-Experts

  6. From Local Prediction Error to Final Outputs: A Theory of Expert Sharing in Multi-Step Reasoning

    Nov 13, 2026Tiantong Wang, Zhiren Gong, Yixin Wang +2Mixture of ExpertsMixture-of-Experts Models

  7. Do Mixture-of-Experts Routers Benefit from Ensembling?

    Nov 12, 2026Viktor Veselý, Sebastian JitaruMixture-of-Experts ModelsEnsemble Learning

  8. DeltaForge: A Post-hoc Mixture-of-Experts VLA for Multi-Domain Autonomous Driving

    Nov 9, 2026Bingzhang Wang, Mo Poorsartep, Yasaman EsfandiariVLMs for Autonomous DrivingMixture-of-Experts Models

  9. Direct GPU Energy Measurements of Batch-Level Sharing in Mixture-of-Experts and Recurrent Models

    Nov 8, 2026Rafael Pina, Varuna De SilvaEnergy-Efficient MLMixture-of-Experts Inference

  10. OOM: Offload-Optimized Full-Parameter MoE Training via Memory-Execution Co-Design

    Nov 8, 2026Ruijia Yang, Shiyuan Lin, Chengxi Liao +1Memory-Efficient TrainingMixture-of-Experts Models

  11. Multi-GPU MoE Inference with Adaptive Expert Offloading

    Nov 6, 2026William X. Zheng, Zhongjie Chen, Shuo Yang +9Mixture-of-Experts InferenceExpert Offloading

  12. Rethinking MoE Load Balancing as Feedback Control

    Nov 6, 2026Yikun Hou, Alp YurtseverExpert Load BalancingMixture-of-Experts Models

  13. Multimodal Hierarchical Mixture-of-Experts for 3D Medical Vision-Language Modeling

    Nov 5, 2026Gizem Mert, Szymon Płotka, Pedro R. A. S. Bassi +5Medical VLMs3D VLMs

  14. LMGN: A Unified Policy Learning Model for Last-Mile Grasp-Navigation

    Nov 5, 2026Xuenmin Hu, Qingxue Qian, Wei Luo +3Robot Policy LearningRobotic Loco-Manipulation

  15. Server-Side Sparse Assignment for Federated Mixture-of-Experts

    Nov 4, 2026Xue Zheng, Aylin YenerSparse Mixture-of-ExpertsFederated Learning

  16. Combining Relevant Expert Knowledge Using Mixtures of Feedback Fuzzy Cognitive Maps

    Nov 3, 2026Olaoluwa Adigun, Akash Kumar Panda, Bart KoskoFuzzy Cognitive MapsMixture-of-Experts Models

  17. Provably Learning Mixtures of Linear Experts

    Nov 3, 2026Adrian Candocia, Aamod Varma, Mark DavenportMixture-of-Experts ModelsExpectation-Maximization

  18. Sonar-MoE: In-Backward Expert Updates for Memory-Efficient Mixture-of-Experts Training

    Nov 2, 2026Avni Mittal, Avinash Anand, Ashutosh Kumar +7Mixture-of-Experts ModelsMemory-Efficient Training

  19. NormMoX: A Normative Mixture of Experts for Explainable Autism Classification from Structural and Functional MRI

    Nov 2, 2026Muhammad Hasan Masrur, Fani DeligianniAutism Spectrum DisorderExplainable Medical Image Analysis

  20. VideoEditor-MoE: Scaling Complex Video Editing with Mixture of Experts

    Nov 2, 2026Xiao Cao, Fuchen Long, Yansong Qu +5Mixture-of-Experts ModelsInstruction-Based Video Editing

  21. When Does Sparse Routing Help? The Geometry of Expert Separation in Mixture of Experts

    Nov 1, 2026Huy Nguyen, Dung Le, Alessandro Rinaldo +1Sparse Mixture-of-ExpertsMixture-of-Experts Models

  22. MoEDDI: Semantic Mixtures of Experts for Long Tailed Drug Drug Interaction Prediction

    Oct 31, 2026Dinh Tan Nguyen, Hien Quang Kha, Minh-Toan Dinh +5Drug-Drug Interaction PredictionMixture-of-Experts Models

  23. Experts Exhibit Structured Redundancy: Structure-Preserving Expert Pruning for Mixture-of-Experts Large Language Models

    Oct 30, 2026Tian Wang, Yongding Tao, Yihong Dong +3Mixture-of-Experts PruningMixture-of-Experts Language Models

  24. Softmax Routing Dynamics in Mixture-of-Experts Models: Collapse and Recovery

    Oct 29, 2026Shengze Xu, Maolin Deng, Tieyong ZengMixture of ExpertsMixture-of-Experts Models