Mixture-of-Experts Models

Ask about this topic

Overview

Mixture-of-experts models divide computation among expert networks and use a router to select which experts process each input, often activating only a small subset. Common approaches include top-k sparse routing, shared or hierarchical experts, and parameter-efficient expert adapters; researchers also study load balancing to prevent some experts from being overused or ignored. These models are used in language, vision, multimodal learning, forecasting, and robotics, while systems work targets the memory, communication, and latency costs of serving and training them. A central tension is that specialization can improve capacity and efficiency, but balanced routing does not guarantee useful expertise, and routing overhead or weaker performance under changed conditions can offset the gains.

Latest papers

CardsList

461 papers75 papers in the last four weeks, up 97% on the four weeks before. 0.4% of all new papers.

  1. Geometric Routing Enables Causal Expert Control in Mixture of Experts

    Apr 15, 2026Ivan Ternovtsii, Yurii BilakMixture-of-Experts InterpretabilityMixture-of-Experts Expert Specialization

    Department of Software Systems, Uzhhorod National University, Narodna sq. 3, Uzhhorod, Ukraine, 88000 · HengeBytes

  2. Design and Behavior of Sparse Mixture-of-Experts Layers in CNN-based Semantic Segmentation

    Apr 15, 2026Svetlana Pavlitska, Haixi Fan, Konstantin Ditschuneit +1Sparse Mixture-of-ExpertsMixture-of-Experts Models

    FZI Research Center for Information Technology · Karlsruhe Institute of Technology (KIT)

  3. Awakening Dormant Experts:Counterfactual Routing to Mitigate MoE Hallucinations

    Apr 15, 2026Wentao Hu, Yanbo Zhai, Xiaohui Hu +6LLM Hallucination MitigationMixture-of-Experts Inference

    Xi’an Jiaotong University · Xingchen AGI Lab, China Telecom Artificial Intelligence Technology (Beijing) Co., Ltd. · Beijing Foreign Studies University +1

  4. Enhancing Mixture-of-Experts Specialization via Cluster-Aware Upcycling

    Apr 15, 2026Sanghyeok Chu, Pyunghwan Ahn, Gwangmo Song +3Mixture-of-Experts Expert SpecializationMixture-of-Experts Models

    ECE, Seoul National University · LG AI Research · University of Michigan +1

  5. Efficient Handwriting-Based Alzheimer,s Disease Diagnosis Using a Low-Rank Mixture of Experts Deep Learning Framework

    Apr 14, 2026Wu Wang, Yuang Cheng, Fouzi Harrou +1Alzheimer's Disease ClassificationMixture-of-Experts Models

    Center for Applied Statistics and School of Statistics Renmin University of China Beijing 100872, China · King Abdullah University of Science and Technology (KAUST) Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division Thuwal 23955-6900, Saudi Arabia

  6. EEG-Based Multimodal Learning via Hyperbolic Mixture-of-Curvature Experts

    Apr 14, 2026Runhe Zhou, Shanglin Li, Guanxiang Huang +5Multimodal LearningMultimodal Fusion

    Nanyang Technological University, Singapore · BIFOLD, Berlin Institute for the Foundations of Learning and Data, Berlin, Germany · ATR, Kyoto, Japan +2

  7. Learning How Much to Think: Difficulty-Aware Dynamic MoEs for Graph Node Classification

    Apr 13, 2026Jiajun Zhou, Yadong Li, Xuanze Chen +4Graph Neural NetworksMixture-of-Experts Models

    Jiajun Zhou, Yadong Li, Xuanze Chen, Chen Ma, Shanqing Yu and Qi Xuan are with the Institute of Cyberspace Security, Zhejiang University of Technology, Hangzhou 310023, China, with the Binjiang Cyberspace Security Institute of ZJUT, Hangzhou, 310056, China, and with the Soovar Technologies Co., Ltd., Hangzhou 310056, China. · Chuang Zhao is with the Department of Electronic and Computer Engineering, Hong Kong University of Science and Technology, Hong Kong, China.

  8. WaveMoE: A Wavelet-Enhanced Mixture-of-Experts Foundation Model for Time Series Forecasting

    Apr 12, 2026Shunyu Wu, Jiawei Huang, Weibin Feng +6Time Series ForecastingTime-Series Foundation Models

    Sun Yat-sen University · Xiaomi Corporation · National University of Singapore

  9. The Myth of Expert Specialization in MoEs: Why Routing Reflects Geometry, Not Necessarily Domain Expertise

    Apr 10, 2026Xi Wang, Soufiane Hayou, Eric NalisnickMixture-of-Experts Expert SpecializationMixture of Experts

    Johns Hopkins University

  10. Are Independently Estimated View Uncertainties Comparable? Unified Routing for Trusted Multi-View Classification

    Apr 10, 2026Yilin Zhang, Cai Xu, Haishun Chen +2Evidential Deep LearningUncertainty Quantification

    Xidian University, Xi’an, China

  11. Generalization and Scaling Laws for Mixture-of-Experts Transformers

    Apr 10, 2026Mansour Zoubeirou a MayakiMixture-of-Experts ModelsAI Scaling Laws

    Univ Lyon, Lyon 2, LIRIS UMR 5205 F-69676, Lyon, France

  12. MP-ISMoE: Mixed-Precision Interactive Side Mixture-of-Experts for Efficient Transfer Learning

    Apr 10, 2026Yutong Zhang, Zimeng Wu, Shangcai Liao +2Transfer LearningMemory-Efficient Fine-Tuning

    State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing, China · School of Computer Science and Engineering, Beihang University, Beijing, China · College of Information Technology, United Arab Emirates University, Al Ain, Abu Dhabi, United Arab Emirates

  13. Seeing but Not Thinking: Routing Distraction in Multimodal Mixture-of-Experts

    Apr 9, 2026Haolei Xu, Haiwen Hong, Hongxing Li +7VLM ReasoningExpert Routing

    Zhejiang University · Alibaba Group

  14. Symbiotic-MoE: Unlocking the Synergy between Generation and Understanding

    Apr 9, 2026Xiangyue Liu, Zijian Zhang, Miles Yang +3Cross-Modal Representation LearningUnified Multimodal Models

    HKUST · Tencent Hunyuan

  15. Region-Graph Optimal Transport Routing for Mixture-of-Experts Whole-Slide Image Classification

    Apr 8, 2026Xin Tian, Jiuliu Lu, Ephraim Tsalik +3Histopathology Image ClassificationWhole-Slide Image Analysis

    Centre for Human Genetics, University of Oxford, Oxford, UK · Beckman Coulter Diagnostics, Brea, CA, USA · Danaher Corporation, Washington, DC, USA

  16. Does a Global Perspective Help Prune Sparse MoEs Elegantly?

    Apr 8, 2026Zeliang Zhang, Nikhil Ghosh, Jiani Liu +2Mixture-of-Experts PruningNeural Network Pruning

    University of Rochester · Flatiron Institute · University of California, Berkeley +1

  17. Agentic AI-Based Joint Computing and Networking via Mixture of Experts and Large Language Models

    Apr 7, 2026Robert-Jeron Reifert, Alaa Alameer Ahmad, Hayssam Dahrouj +1LLM Agent Orchestration6G Networks

    Digital Communication Systems, Ruhr University Bochum, Bochum, Germany · Cariad SE, Wolfsburg, Germany · Department of Electrical Engineering, University of Sharjah, Sharjah, United Arab Emirates

  18. Mixture-of-Modality-Experts with Holistic Token Learning for Fine-Grained Multimodal Visual Analytics in Driver Action Recognition

    Apr 7, 2026Tianyi Liu, Yiming Li, Wenqian Wang +4Multimodal UnderstandingMultimodal Fusion

    Nanyang Technological University, 50 Nanyang Avenue, 639798, Singapore · Singapore University of Technology and Design, 8 Somapah Rd, 487372, Singapore · The Hong Kong Polytechnic University, 6/F, Core D, Lui Che Woo Building, Hung Hom, Hong Kong SAR, China

  19. QA-MoE: Towards a Continuous Reliability Spectrum with Quality-Aware Mixture of Experts for Robust Multimodal Sentiment Analysis

    Apr 7, 2026Yitong Zhu, Yuxuan Jiang, Guanxuan Jiang +4Multimodal Sentiment AnalysisMultimodal Robustness

    The Hong Kong University of Science and Technology (Guangzhou) · Tsinghua University · Aarhus University

  20. Do Domain-specific Experts exist in MoE-based LLMs?

    Apr 7, 2026Giang Do, Hung Le, Truyen TranMixture-of-Experts Language ModelsMixture-of-Experts Models

    Applied Artificial Intelligence Intiative (A2I2), Deakin University

  21. HI-MoE: Hierarchical Instance-Conditioned Mixture-of-Experts for Object Detection

    Apr 6, 2026Vadim Vashkelis, Natalia TrukhinaObject DetectionMixture-of-Experts Models

  22. Mixture-of-Experts in Remote Sensing: A Survey

    Apr 3, 2026Yongchuan Cui, Peng Liu, Lajiao ChenMixture-of-Experts ModelsRemote Sensing

    Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, China · School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences, Beijing 101408, China

  23. The Expert Strikes Back: Interpreting Mixture-of-Experts Language Models at Expert Level

    Apr 2, 2026Jeremy Herbst, Stefan Wermter, Jae Hee LeeMixture-of-Experts InterpretabilityMixture-of-Experts Models

    Department of Informatics, University of Hamburg, Hamburg, Germany.

  24. StanceMoE: Mixture-of-Experts Architecture for Stance Detection

    Apr 1, 2026Abdullah Al Shafi, Md. Milon Islam, Sk. Imran Hossain +1Mixture-of-Experts Models

    Department of Computer Science and Engineering, Khulna University of Engineering & Technology

  25. Cost-Penalized Fitness in FMA-Orchestrated Mixture of Experts: Experimental Evidence for Molecular Memory in Domain Adaptation

    Apr 1, 2026Martin JaraizMixture-of-Experts ModelsMixture of Experts

    Department of Electricity and Electronics, University of Valladolid, Spain

  26. Routing-Free Mixture-of-Experts

    Apr 1, 2026Yilun Liu, Jinru Han, Sikuan Yan +2Mixture-of-Experts ModelsExpert Load Balancing

    Ludwig Maximilian University of Munich · Munich Center for Machine Learning · University of California, Los Angeles

  27. Self-Routing: Parameter-Free Expert Routing from Hidden States

    Apr 1, 2026Jama Hussein Mohamud, Drew Wagner, Mirco RavanelliParameter-Free Mixture-of-Experts RoutingLLM Routing

    Mila – Quebec AI Institute · Universite de Montreal · Concordia University

  28. PASM: Population Adaptive Symbolic Mixture-of-Experts Model for Cross-location Hurricane Evacuation Decision Prediction

    Mar 31, 2026Xiao Qian, Shangjia DongDomain GeneralizationMixture-of-Experts Models

    Dept. of Civil, Environmental, and Construction Engineering, University of Delaware, Newark, DE, USA

  29. TrafficMoE: Heterogeneity-aware Mixture of Experts for Encrypted Traffic Classification

    Mar 31, 2026Qing He, Xiaowei Fu, Lei ZhangEncrypted Traffic ClassificationEncrypted Network Traffic Analysis

    School of Microelectronics and Communication Engineering, Chongqing University, Chongqing 400044, China.

  30. Divide and Restore: A Modular Task-Decoupled Framework for Universal Image Restoration

    Mar 30, 2026Joanna Wiekiera, Martyna ZurImage RestorationAll-in-One Image Restoration

    Faculty of Automatic Control, Electronics and Computer Science, Silesian University of Technology, Gliwice, Poland