Mixture-of-Experts Models

Follow this topic Ask about this topic

Momentum

41 papers in the last four weeks, up 116% on the four weeks before. 0.4% of all new papers.

Jul 13Week of Sep 28

Latest papers 297

All topics

PRISM: Synergizing Vision Foundation Models via Self-organized Expert Specialization
Jun 2, 2026Ying Tang, Dong Li, Youjia Zhang +3Vision Foundation Models Sparse Mixture-of-Experts
School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan, China.
When Model Merging Breaks Routing: Training-Free Calibration for MoE
Jun 2, 2026Canbin Huang, Tianyuan Shi, Xiaojun Quan +3Mixture of Experts LLM Routing
School of Computer Science and Engineering, Sun Yat-sen University, China · Shenzhen Loop Area Institute, China · Meituan, Inc., China +1
Pathway-Structured Privileged Distillation for Deployable Computational Pathology
Jun 1, 2026Yongxin Guo, Hao Lu, Onur Koyun +2Computational Pathology Oncology
1*School of Medicine, Wake Forest University, Winston-Salem, NC, USA.
CRAM: Centroid-Routing and Adaptive MoE for Multimodal Continual Instruction Tuning
Jun 1, 2026Jun-Tao Tang, Zhen-Hao Xie, Yu-Cheng Shi +1Mixture-of-Experts Models Multimodal Instruction Tuning
School of Artificial Intelligence, Nanjing University, China · State Key Laboratory of Novel Software Technology, Nanjing University, China
GC-MoE: Genomics-Guided Cell-Type-Specific Mixture of Experts for Histology-Based Single-Cell Spatial Transcriptomics
Jun 1, 2026Kaito Shiku, Ahtisham Fazeel Abbasi, Ryoma Bise +4Computational Pathology Spatial Transcriptomics
Kyushu University, Japan · German Research Center for Artificial Intelligence (DFKI GmbH), Germany · RPTU University Kaiserslautern-Landau, Germany +3
AdaWeather: Adaptively Mixing Probabilistic Weather Forecasts with Logarithmic Regret
Jun 1, 2026Saptarishi Dhanuka, Sarvesh Iyer, Manmeet Singh +5Probabilistic Forecasting Mixture-of-Experts Models
Ashoka University · Ashoka University, India · Western Kentucky University
Hierarchically Decoupled Mixture-of-Experts for Robust Traffic Sign Recognition in Complex Driving Scenarios
Jun 1, 2026Mingxiao Wang, Xiaozhen Qu, Bolin Gao +2Sparse Mixture-of-Experts Autonomous Driving Perception
School of Automotive and Traﬀic Engineering, Liaoning University of Technology, Jinzhou, Liaoning 121001, China. · State Key Laboratory of Intelligent Green Vehicles and Mobility, School of Vehicle and Mobility, Tsinghua University, Beijing 100084, China.
DOT-MoE: Differentiable Optimal Transport for MoEfication
Jun 1, 2026Udbhav Bamba, Arnav Chavan, Aryamaan Thakur +2LLM Inference Efficiency Mixture-of-Experts Models
Amazon
ProbMoE: Differentiable Probabilistic Routing for Mixture-of-Experts
Jun 1, 2026Heng Zhao, Zilei Shao, Guy Van den Broeck +1Sparse Mixture-of-Experts Expert Routing
Department of Computer Science, University of Virginia, Charlottesville, USA · Department of Computer Science, University of California, Los Angeles, Los Angeles, USA
Deft Scheduling of Dynamic Cloud Workflows with Varying Deadlines via Mixture-of-Experts
May 31, 2026Ya Shen, Gang Chen, Hui Ma +1Adaptive Model Routing Mixture-of-Experts Models
School of Engineering and Computer Science, Victoria University of Wellington, Wellington, New Zealand
DAG-MoE: From Simple Mixture to Structural Aggregation in Mixture-of-Experts
May 31, 2026Jiarui Feng, Hanqing Zeng, Karish Grover +11Mixture-of-Experts Language Models Sparse Mixture-of-Experts
Meta MRS · Washington University in St. Louis · Carnegie Mellon University +1
Beyond Task-Agnostic: Task-Aware Grouping for Communication-Efficient Multi-Task MoE Inference
May 31, 2026Zhiyao Xu, Aoxue Liu, Zhanjie Ding +3Mixture-of-Experts Inference Mixture-of-Experts Models
Tsinghua Shenzhen International Graduate School, Shenzhen, China · Pengcheng Laboratory, Shenzhen, China
Confidence-Adaptive SwiGLU for Mixture-of-Experts
May 30, 2026Shaohua Li, Xiuchao Sui, Xiaobing Sun +4Mixture-of-Experts Language Models Mixture-of-Experts Models
Institute of High Performance Computing, Agency for Science, Technology and Research, Singapore · Shanghai University of Engineering Science, China
Dialectics of Alignment: Harnessing Unsafe Knowledge for Dynamic Safety Routing
May 30, 2026Maryam Hashemzadeh, Jerry Huang, Minseon Kim +2LLM Safety Alignment LLM Safety
Chandar Research Lab · Mila – Quebec AI Institute · Université de Montréal +3
PithTrain: A Compact and Agent-Native MoE Training System
May 29, 2026Ruihang Lai, Hao Kang, Haozhan Tang +6Mixture-of-Experts Language Models Efficient Language Model Training
1Carnegie Mellon University · 3NVIDIA · 2Xlue
MoG: Mixture of Experts for Graph-based Retrieval-Augmented Generation
May 29, 2026Zheng Yuan, Chuang Zhou, Linhao Luo +4Retrieval-Augmented Generation Graph Retrieval-Augmented Generation
The Hong Kong Polytechnic University · Monash University · Tencent Youtu Lab
Eigenvectors of Experts are Training-free Non-collapsing Routers
May 29, 2026Giang Do, Hung Le, Truyen TranParameter-Free Mixture-of-Experts Routing Mixture of Experts
Applied Artificial Intelligence Intiative (A2I2), Deakin University, Victoria, Australia.
Graph-Conditioned Mixture of Graph Neural Network Experts for Traffic Forecasting
May 28, 2026Amirhossein Ghaffari, Saeid Sheikhi, Ekaterina GilmanAdaptive Model Routing Mixture-of-Experts Models
Future Computing Group, University of Oulu
DAMEL: Dual-Axis Multi-Expert Learning for Class-Imbalanced Learning
May 28, 2026Hyuck Lee, Taemin Park, Heeyoung KimClass-Imbalanced Learning Long-Tail Learning
AI Research, Krafton, Teheran-ro 231, Centerfield EAST, Gangnam-gu, Seoul, 06142, Republic of Korea · Department of Industrial and Systems Engineering, Korea Advanced Institute of Science and Technology (KAIST), 291 Daehak-ro, Yuseong-gu, Daejeon, 34141, Republic of Korea
A Minimal Bifurcation Model of Load Imbalance in a Softmax Mixture-of-Experts Router
May 27, 2026O. M. KiselevDynamical Systems Expert Load Balancing
Innopolis University, Innopolis, Russia
RA-MoE: Routing-Aligned Fine-Tuning for Multilingual Adaptation of Mixture-of-Experts Models
May 27, 2026Guanzhi Deng, Kuan Wu, Haibo Wang +6Mixture-of-Experts Language Models LLM Fine-Tuning
City University of Hong Kong, Hong Kong, China · Carnegie Mellon University, Pittsburgh, USA · The University of Hong Kong, Hong Kong, China
VidPrism: Heterogeneous Mixture of Experts for Image-to-Video Transfer
May 27, 2026Rui Lin, Chuanming Wang, Huadong MaVLM Adaptation Video Representation Learning
State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, China
Extracting Small Translation Specialists from LLMs by Aggressively Pruning Experts
May 27, 2026Liu O. Martin, Lucas Bandarkar, Nanyun PengMixture-of-Experts Pruning LLM Pruning
University of California, Los Angeles
NUCLEUS-MoE: Unified Model of Pool Boiling for Liquid Cooling
May 26, 2026Arthur Feeney, Xianwei Zou, Sheikh Md Shakeel Hassan +2Neural Surrogate Modeling PDE Surrogate Modeling
Department of Electrical Engineering and Computer Science University of California, Irvine Irvine, CA, USA
Towards Generalization-Oriented Models for Vehicle Routing Problems with Mixture-of-Experts
May 26, 2026Changhao Miao, Yuntian Zhang, Tongyu Wu +2Reinforcement Learning Adaptive Model Routing
State Key Laboratory of Autonomous Intelligent Unmanned Systems, Beijing Institute of Technology, Beijing 100081, China · Department of Computer, Control and Management Engineering “Antonio Ruberti”, Sapienza University of Rome, Rome, 00185, Italy · School of AI, Beijing Institute of Technology, Beijing 100081, China
BioFact-MoE: Biologically Factorized Mixture of Experts for Vision-Language Prognostic Modeling in Hepatocellular Carcinoma
May 25, 2026Junlin Yang, Tian Yu, Nicha C. Dvornek +6Radiology VLMs Cancer Survival Prediction
Department of Radiology & Biomedical Imaging, Yale University, New Haven, CT, 06510, USA · Department of Biomedical Engineering, Yale University, New Haven, CT, 06510, USA · Department of Electrical Engineering, Yale University, New Haven, CT, 06510, USA +1
VEN-VL: A Visual Ensemble MoE Framework for Effective and Efficient Multi-Modal Understanding
May 25, 2026Yinghao Wu, Zhuoyan Luo, Yiyao Yu +3Efficient Multimodal Inference Mixture-of-Experts Models
Tsinghua University
SP-MoMamba: Superpixel-driven Mixture of State Space Experts for Efficient Image Super-Resolution
May 25, 2026Wenbin Zou, Yawen Cui, Yi Wang +5Visual SSMs Mixture-of-Experts Models
Department of Electrical and Electronic Engineering, The Hong Kong Polytechnic University, Hong Kong SAR · School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, China
Courtroom Analogy: New Perspective on Uncertainty-Aware Classification
May 25, 2026Taeseong Yoon, Heeyoung KimUncertainty Quantification Interpretable ML
Department of Industrial and Systems Engineering, Korea Advanced Institute of Science and Technology, Daejeon, South Korea.
RotMoLE: Enhancing Mixture of Low-Rank Experts through Rotational Gating Mechanism
May 25, 2026Mengyang Sun, Maochuan Dou, Tao Feng +5LLM Fine-Tuning Low-Rank Adaptation
Tsinghua University · Beijing Information Science and Technology University · National University of Singapore +2

Jun 2, 2026, cs.CVJ/K move · Enter open · S save

PRISM: Synergizing Vision Foundation Models via Self-organized Expert Specialization

Ying Tang, Dong Li, Youjia Zhang, Zikai Song+2

School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan, China.

Vision Foundation Models Sparse Mixture-of-Experts Mixture-of-Experts Models

Open paper AskPDF arXiv

Unifying the complementary strengths of diverse Vision Foundation Models (VFMs) into a single efficient model is highly desirable but challenged by the negative transfer inherent in monolithic distillation. To address these feature conflicts, we introduce \textbf{PRISM}, a novel dual-stream Mixture-of-Experts (MoE) framework that synergizes VFMs via modular specialization. We propose a two-stage paradigm: (1) expertise deconstruction, where a teacher-conditional router guides experts to specialize in distinct representational subspaces to mitigate interference, followed by (2) dynamic recomposition, where the router learns to assemble these experts into tailored computational pathways for downstream tasks. Experiments on PASCAL-Context and NYUD-v2 show that \textbf{PRISM} establishes a new state of the art, validating that sparse, emergent specialization is a scalable approach for integrating diverse visual knowledge.

Momentum

Related topics

Latest papers 297