Mixture-of-Experts Models

Follow this topic Ask about this topic

Momentum

41 papers in the last four weeks, up 116% on the four weeks before. 0.4% of all new papers.

Jul 13Week of Sep 28

Latest papers 297

All topics

Spend Experts Where You Are Unsure: Confidence-Adaptive Routing for Mixture-of-Experts LoRA
Jul 28, 2026Tom Saliencro, Rohan Desai, Priya Nair +2Parameter-Free Mixture-of-Experts Routing LLM Inference Acceleration
University of California, Irvine · University of Washington
Balanced Soft mixture-of-expert model for Glaucoma Detection
Jul 28, 2026Sai Venkatesh Chilukoti, Krishna Rauniyar, Min Shi +1Multimodal Classification Mixture-of-Experts Models
School of Computing and Informatics, University of Louisiana at Lafayette, 301 East Lewis Street, Lafayette, 70504, LA, USA.
MMOE: Modernizing Diffusion Transformers with Efficient Expert Design
Jul 27, 2026Yanhao Jia, Jiepeng Wang, Haibin Huang +3Diffusion Transformer Diffusion Model Acceleration
College of Computing and Data Science, Nanyang Technological University, Singapore, 639798 · Institute of Artificial Intelligence of China Telecom (TeleAI), Shanghai, China, 200232
Hierarchical Copula-Gumbel-Top-\texorpdfstring{ $K$ }{K} Routing: Two-Sided Dependence Control for Frozen Mixture-of-Experts at Fixed Per-Token Routing Laws
Jul 25, 2026Richard Yi Da XuCopula Models Expert Load Balancing
Hong Kong Baptist University and TadReamk Limited
Emergent Compositional Skills in Mixture-of-Experts VLAs
Jul 22, 2026Shlok Shah, Rhiaan Jhaveri, Tharun Kumar Tiruppali Kalidoss +3Robot Policy Learning Robot Skill Learning
Department of Computer Science, Princeton University, Princeton, NJ, USA · Department of Electrical and Computer Engineering, Princeton University, Princeton, NJ, USA
MoAKE: Toward Unified All-in-One Action Quality Assessment via Mixture of Action Knowledge Experts
Jul 22, 2026Huangbiao Xu, Huanqi Wu, Xiao Ke +3Mixture-of-Experts Models Action Quality Assessment
Fujian Provincial Key Laboratory of Networking Computing and Intelligent Information Processing, College of Computer and Data Science, Fuzhou University, Fuzhou 350108, China · Engineering Research Center of Big Data Intelligence, Ministry of Education, Fuzhou 350108, China · School of Intelligence Science and Technology, University of Science and Technology Beijing, Beijing 100083, China
Continual Video-MLLM Adaptation over Evolving Domains
Jul 21, 2026Rui Cheng, Meixing Shi, Yuxiang Cai +3Video QA Domain-Incremental Learning
School of Software Technology, Zhejiang University, Ningbo, China · Zhejiang Key Laboratory of Digital-Intelligence Service Technology, China · Hong Kong Polytechnic University, Hong Kong SAR +1
Adaptive Multi-Expert Graph Transformer for Interpretable EEG-Based Diagnostics
Jul 20, 2026Maryam Rahimimovassagh, Md Elias Hossain, Ivan Garibay +1Electroencephalography Dynamic Graph Learning
Department of Industrial Engineering and Management Systems University of Central Florida, Orlando, FL, USA
HyBDM: Multi-Scale Hybrid Experts for Time Series Forecasting with Bidirectional Dependency Modeling
Jul 18, 2026Wenqiang Ma, Chen Cheng, Xue Cheng +1Multivariate Time Series Forecasting Multi-Scale Feature Fusion
Soul of Harlequin Lab
Can Experts Adapt Without Training? On Test-Time Modality Generalization in MVLMs
Jul 18, 2026Raza Imam, Darakshan Rashid, Yutong Xie +3VLM Robustness VLM Adaptation
Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE · Indian Institute of Technology Delhi, India · Khalifa University, Abu Dhabi, UAE
MixCompress: Mixture of Experts for Variable Rate Learned Image Compression
Jul 15, 2026Calvin-Khang Ta, Praneet Singh, Tong Shao +1Learned Image Compression Rate-Distortion Optimization
Dolby Laboratories, USA
Multi-Expert Routing for Multi-Domain Low-Resource OCR: A Manchu Case Study
Jul 15, 2026Zhan Chen, Jiqiao Ma, Chih-wen KuoMixture-of-Experts Models Optical Character Recognition
Institute of Advanced Studies, Beijing Normal University, Zhuhai, China · Faculty of Humanities and Social Sciences, Beijing Normal University–Hong Kong Baptist University United International College (UIC), Zhuhai, China · Independent Scholar +1
Less Experts, Faster Decoding: Cost-Aware Speculative Decoding for Mixture-of-Experts
Jul 14, 2026Jincheng Xie, Runheng Liu, Heyan Huang +4Mixture-of-Experts Inference Speculative Decoding
Department of Mathematical Sciences, Tsinghua University, Beijing, China · School of Computer Science and Technology, Beijing Institute of Technology, Beijing, China · JDT AI Infra +2
Understanding Structured Health Data through Interaction-Aware Mixture-of-Experts
Jul 14, 2026Ji Hwan Park, Ying Ding, Tianjin GuoMulti-View Learning Clinical Outcome Prediction
Department of Electrical and Computer Engineering, The University of Texas at Austin, Austin, Texas, USA · School of Information, The University of Texas at Austin, Austin, Texas, USA · Kelley School of Business, Indiana University, Bloomington, Indiana, USA
From Global to Factor-Wise Expert Composition in Discrete Diffusion Models
Jul 13, 2026Haozhe Huang, Yudong Xu, Abhijoy Mandal +1Expert Routing Mixture-of-Experts Models
Department of Computer Science, University of Toronto · Vector Institute for Artificial Intelligence · Department of Mechanical & Industrial Engineering, University of Toronto +1
HCRMap: Pressure-Aware Hot-Expert Residency Mapping for 3.5D MoE Chiplet Inference
Jul 13, 2026Yongqin ZhangAI Accelerator Inference Mixture-of-Experts Inference
Nanjing Vocational College of Information Technology, 99 Wenlan Road, Xianlin College Community, Nanjing, Jiangsu, 210023, P.R.C.
Mach-Mind-4-Flash Technical Report
Jul 10, 2026Foundation Model TeamAgentic RL Token Efficiency
Foundation Model, Li Auto Inc.
It Takes a MAESTRO To Prune Bad Experts
Jul 9, 2026Palaash Goel, Ayush Maheshwari, Tanmoy ChakrabortyMixture-of-Experts Pruning LLM Pruning
Indian Institute of Technology Delhi, India · NVIDIA, India
SHAP-Weighted Cross-Modal Expert Fusion for Emotion and Sentiment Recognition: Evidence and Limits
Jul 9, 2026Adis Alihodzic, Selma Skopljakovic HubljarSHAP Feature Attribution Multimodal Emotion Recognition
Department of Mathematical and Computer Sciences, Faculty of Science, University of Sarajevo, Sarajevo, Bosnia and Herzegovina.
On the Design of Mixture-of-Experts for Dynamic Gaussian Splatting
Jul 9, 2026In-Hwan Jin, Hyeongju Mun, Joonsoo Kim +23D Gaussian Splatting Dynamic Scene Reconstruction
Department of Electrical and Electronics Engineering, Pusan National University, Busan, Republic of Korea · Immersive Media Research Section, Electronics and Telecommunications Research Institute (ETRI), Daejeon, Republic of Korea · Department of Electrical and Electronics Engineering and JYS AI Nexus Institute, Pusan National University, Busan, Republic of Korea
Mixture of Enhanced-View Experts for Multi-Query Vehicle ReID and A Large-Scale Benchmark
Jul 9, 2026Aihua Zheng, Jie Zhen, Chenglong Li +2Multi-View Consistency Multi-View Learning
Information Materials and Intelligent Sensing Laboratory of Anhui Province, Anhui Provincial Key Laboratory of Multimodal Cognitive Computation, School of Artificial Intelligence, Anhui University. Hefei, 230601, China · Anhui Provincial Key Laboratory of Multimodal Cognitive Computation, School of Artificial Intelligence, Anhui University. Hefei, 230601, China · School of Artificial Intelligence, Anhui University of Science and Technology, Hefei 232001, China +1
Scaling Mixture-of-Experts Video Pretraining for Embodied Intelligence
Jul 8, 2026Shuailei Ma, Jiaqi Liao, Xinyang Wang +24Video Diffusion Models Self-Supervised Pre-Training
Image classification via a quantum-inspired strategy involving a mixture of experts
Jul 8, 2026Kumari Jyoti, Rohith Babu, Apoorva D. PatelHybrid Quantum-Classical ML Quantum-Inspired ML
Centre for High Energy Physics, Indian Institute of Science, Bangalore 560012
ProMoE-FL: Prototype-conditioned Mixture of Experts for Multimodal Federated Learning with Missing Modalities
Jul 7, 2026Aavash Chhetri, Bibek Niroula, Eduard Vazquez +4Multimodal Federated Learning Missing-Modality Learning
NepAl Applied Mathematics and Informatics Institute for research, Nepal · Fogsphere (Redev.AI Ltd), UK · University of Lausanne, Switzerland +4
UBEP: Re-architecting Expert Parallelism Communication Library for Production Superpods
Jul 7, 2026Yipeng Liu, Chang Liu, Si Shen +16Expert Parallelism High-Performance Computing
State Key Laboratory for Novel Software Technology, Nanjing University · Huawei Technologies Co., Ltd.
RoME: Robust Mixture of Low-Rank Experts against Multiple Adversarial Perturbations
Jul 7, 2026Woo Jae Kim, Kyle Min, Suhyeon Ha +2Adversarial Training Neural Network Robustness
KAIST, Daejeon, Korea · Oracle, Seattle WA, USA
NEST: Tackling Dataset-Level Distribution Shifts via Regime-Oriented Mixture-of-Experts
Jul 7, 2026Lanhao Li, Bingshu Xie, Lijun Sun +3Multivariate Time Series Forecasting Long-Term Time Series Forecasting
1Beihang University
MoP-JEPA: Hard-Assigned Predictor Mixtures for Stochastic JEPA World Models
Jul 6, 2026Zhi Song, Ximing Xing, Zhenchao Tang +8World Model Learning Latent Dynamics Modeling
City University of Hong Kong, China · Tencent, China
Localized LoRA-MoE: Block-wise Low-Rank Experts With Adaptive Routing
Jul 6, 2026Babak Barazandeh, Subhabrata Majumdar, Vinay Prithyani +1Adaptive Model Routing Low-Rank Adaptation
Fortinet · Indian Institute of Management Bangalore · Citadel Securities +1
Nemotron-Labs-3-Puzzle-75B-A9B: Compressing Hybrid MoE LLMs
Jul 5, 2026Akhiad Bercovich, Talor Abramovich, Daniel Afrimi +67Mixture-of-Experts Pruning LLM Pruning

Jul 28, 2026, cs.LGJ/K move · Enter open · S save

Spend Experts Where You Are Unsure: Confidence-Adaptive Routing for Mixture-of-Experts LoRA

Tom Saliencro, Rohan Desai, Priya Nair, Maya Lindqvist+1

University of California, Irvine · University of Washington

Parameter-Free Mixture-of-Experts Routing LLM Inference Acceleration Expert Routing LLM Uncertainty Estimation Mixture-of-Experts Models

Open paper AskPDF arXiv

Mixture-of-Experts (MoE) variants of Low-Rank Adaptation (LoRA) route every token to a fixed number of experts

k

. Tokens differ in how uncertain the model is about them, so a single k over-spends on easy tokens and under-serves hard ones. We observe that the router's output distribution is already a per-token uncertainty signal: peaked mass indicates confidence, while a flat distribution indicates ambiguity. We introduce CARE (Confidence-Adaptive Routing of Experts), which admits experts in a nucleus fashion. Experts are activated in decreasing router weight until their cumulative mass reaches a threshold, with a small extension when the admitted experts disagree. A budget thermostat calibrates the threshold so that the average number of active experts matches any target. CARE is a drop-in, single-forward-pass rule with no extra parameters. Across eight commonsense benchmarks on LLaMA-3.1-8B and Qwen2.5-7B, as well as math, code, and knowledge tasks, CARE improves over fixed top-k MoE-LoRA at matched compute and matches the fixed-k=4 baseline while activating fewer experts. The same confidence and disagreement signals also improve out-of-distribution detection over MSP, entropy, and multi-pass proxies. We support the design with nucleus fidelity, budget optimality, and an epistemic reading of disagreement, and we release code.

Momentum

Related topics

Latest papers 297