Mixture-of-Experts Models

Follow this topic Ask about this topic

Momentum

41 papers in the last four weeks, up 116% on the four weeks before. 0.4% of all new papers.

Jul 13Week of Sep 28

Latest papers 297

All topics

CTS-MoE: Implicit Terrain Adaptation via Mixture-of-Experts for Perceptive Locomotion
Jun 17, 2026Francisco Affonso, Matheus P. Angarola, Ana Luiza Mineiro +3Robot Policy Adaptation Terrain-Aware Robot Locomotion
University of Illinois Urbana-Champaign · University of São Paulo
FoMoE: Breaking the Full-Replica Barrier with a Federation of MoEs
Jun 17, 2026Lorenzo Sani, Zeyu Cao, Meghdad Kurmanji +5Expert Parallelism Communication-Efficient Distributed Training
Cambridge ML Systems Lab (CaMLSys), University of Cambridge · Flower Labs
SkillMoV: Mixture-of-View Routing with Prototype-Conditioned Gating for Unified Multi-View Proficiency Estimation
Jun 16, 2026Edoardo Bianchi, Antonio LiottaMulti-View Learning Mixture-of-Experts Models
Free University of Bozen-Bolzano, Faculty of Engineering, Via Bruno Buozzi 1, Bozen-Bolzano, 39100, Italy
Attribution-Guided and Coverage-Maximized Pruning for Structural MoE Compression
Jun 16, 2026Yifu Ding, Jiacheng Wang, Ge Yang +4Mixture-of-Experts Pruning LLM Pruning
State Key Laboratory of Complex & Critical Software Environment, Beihang University · School of Computer Science and Engineering, Beihang University · Nanyang Technological University +1
DynFS-MoE: Dynamic Functional-Structural Mixture-of-Experts for Post-Traumatic Epilepsy Diagnosis
Jun 15, 2026Jun-En Ding, Spencer Chen, Henry Noren +6Medical Diagnosis Neuroimaging
Department of Industrial and Systems Engineering, Rutgers University, Piscataway, NJ, USA · Department of Systems Engineering, Stevens Institute of Technology, Hoboken, NJ, USA
Task-Instructed Causal Routing of Vision Foundation Models for Multi-Task Learning
Jun 14, 2026Donghyun Han, Yuseok Bae, Jung Uk Kim +1Vision Foundation Model Adaptation Multi-Task Learning
Electronics and Telecommunications Research Institute (ETRI), Korea · Kyung Hee University, Korea · Chonnam National University, Korea
How to Score Experts for One-Shot MoE Expert Pruning: A Unified Formulation and Selection Principle
Jun 14, 2026Zongfang Liu, Jinghui Zhang, Zijian Ma +2Mixture-of-Experts Pruning LLM Pruning
1Zhejiang University · 2Westlake University · 3Mohamed bin Zayed University of Artificial Intelligence +1
Conflict-Aware Federated Fine-Tuning of Large Language Models with Mixture-of-Experts
Jun 14, 2026Yijun Lu, Zihan Fang, Pengpeng Qiao +6Federated Learning Aggregation LLM Fine-Tuning
Waseda University, Tokyo, Japan · City University of Hong Kong, China · Institute of Science Tokyo, Tokyo, Japan +4
MoECa: Aligning Feature Reuse with Expert Decomposition in Diffusion Transformers
Jun 14, 2026Maoliang Li, Haojing Chen, Jiayu Chen +4Diffusion Transformer Diffusion Model Caching
School of Computer Science, Peking University, Beijing, China · School of Software Engineering, University of Electronic Science and Technology of China, Chengdu, China
Towards a Unified Generative Model for Scarce Time Series with Domain Experts
Jun 13, 2026Zihao Yao, Qi Zheng, Jiankai Zuo +1Diffusion Transformer Time Series Generation
The Key Laboratory of Embedded System and Service Computing, Ministry of Education, Tongji University, Shanghai, China.
Director: Accelerating Distributed MoE Serving via Online Proactive Expert Placement
Jun 13, 2026Qianli Liu, Kaibin Guo, Zicong Hong +5Expert Parallelism LLM Serving
Department of Computer Science and Engineering, The Hong Kong University of Science and Technology, Hong Kong · School of Software Engineering, Sun Yat-Sen University, China · School of Cyber Science and Engineering, Xi’an Jiaotong University, China +1
Sticky Routing: Training MoE Models for Memory-Efficient Inference
Jun 12, 2026Ali KayyamMixture-of-Experts Inference Memory-Efficient Inference
BrainChip Inc., Laguna Hills, CA
From Self-Supervised Speech Models to Mixture-of-Experts for Robust Anti-Spoofing
Jun 12, 2026Hugo Daumain, Driss Matrouf, Khaled Khelif +1Audio Deepfake Detection Domain Generalization
LIA, Université d’Avignon, France · Airbus Defence & Space, France
Expert-Driven Survival Machines: Improving Stratification and Interpretability in Multiple Clinical Cohorts
Jun 12, 2026Farica Zhuang, Zixuan Wen, Christos Davatzikos +1Adaptive Model Routing Survival Analysis
University of Pennsylvania Philadelphia, Pennsulvania, USA
A theoretical model for task routing in mixture-of-expert transformers
Jun 12, 2026Vinoth Nandakumar, Yongli Xiang, Yunzhi Yao +2Transformer Expressivity Mixture of Experts
University of Sydney · Zhejiang University · Google Research
CausalMoE: A Billion-Scale Multimodal Foundation Model for Granger Causal Discovery with Pattern-Routed Heterogeneous Experts
Jun 11, 2026Bo Liu, Di Dai, Jingwei Liu +5Time-Series Causal Discovery Mixture-of-Experts Models
State Key Laboratory of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University Beijing, China · National Institute of Health Data Science, and Institute for Artificial Intelligence, Peking University Beijing, China
Redesign Mixture-of-Experts Routers with Manifold Power Iteration
Jun 10, 2026Songhao Wu, Ang Lv, Ruobing Xie +1Mixture-of-Experts Language Models Expert Routing
Gaoling School of Artificial Intelligence, Renmin University of China · Large Language Model Department, Tencent
Multi-Rate Mixture of Experts for Accelerating Liquid Neural Network Training
Jun 10, 2026Shilong Zong, Almuatazbellah Boker, Hoda EldardiryIrregular Time-Series Modeling Liquid Neural Networks
Department of Computer Science, Virginia Tech Blacksburg, VA 24061 USA · Department of Electrical and Computer Engineering, Virginia Tech Blacksburg, VA 24061 USA
MSUE: Multi-Modal Soccer Understanding Expert
Jun 10, 2026Litao Li, Yibo Yu, Yufeng Hu +4Visual Question Answering Video QA
South China University of Technology · Johns Hopkins University · University of Electronic Science and Technology of China +1
From Observation to Intervention: A Causal Audit of Expert Importance in Mixture-of-Experts Models
Jun 9, 2026Leonard Engmann, Christian Medeiros Adriano, Holger GieseMixture-of-Experts Pruning LLM Auditing
Hasso Plattner Institute, University of Potsdam, Germany.
MosaicIMU: Composing Carrier Experts for Generalizable Neural Inertial Odometry
Jun 8, 2026Junye Zou, Huiyi Yan, Xinning Xu +4Inertial Odometry Domain Adaptation
Department of Precision Instrument, Tsinghua University, Beijing 100084, China · School of Future Technology, Xi’an Jiaotong University, Xi’an 710049, China · College of Information Science and Technology, Beijing University of Chemical Technology, Beijing 100029, China +3
STAR: Rethinking MoE Routing as Structure-Aware Subspace Learning
Jun 7, 2026Sumin Park, Noseong ParkSparse Mixture-of-Experts Expert Routing
Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea.
LongMoE: Longitudinal Multimodal Learning via Trajectory-Aware Mixture-of-Experts
Jun 6, 2026Maxx Richard Rahman, Prakhar Kumar, Wolfgang MaassMissing-Modality Learning Multimodal Robustness
German Research Centre for Artificial Intelligence (DFKI) Saarbrücken, Germany
Value-and-Structure Alignment for Routing-Consistent Quantization of Mixture-of-Experts Models
Jun 4, 2026Hancheol Park, Geonho Lee, Tairen Piao +1LLM Quantization Mixture-of-Experts Quantization
Nota Inc., South Korea.
Less is MoE: Trimming Experts in Domain-Specialist Language Models
Jun 4, 2026Haoze He, Xinkai Zou, Xuan Jiang +4Mixture-of-Experts Pruning LLM Pruning
Carnegie Mellon University · UCSD · MIT
SHAPE: Coalition-Aware Expert Pruning for Sparse Mixture-of-Experts LLMs
Jun 3, 2026Yuhao ZhangMixture-of-Experts Pruning LLM Pruning
Beihang University
TENP: Trapezoidal Expert Neuron Pruning For Mixture-of-Experts
Jun 3, 2026Jiangyang He, Shaolin Zhu, Deyi XiongMixture-of-Experts Pruning LLM Pruning
TJUNLP Lab, School of Computer Science and Technology, Tianjin University, China
Treat Traffic Like Trees: A Semantic-Preserving Hierarchical Graph-Based Expert Framework for Encrypted Traffic Analysis
Jun 3, 2026Yuantu Luo, Jun Tao, Linxiao Yu +1Graph Neural Networks Encrypted Network Traffic Analysis
School of Cyber Science and Engineering, Southeast University, Nanjing 211189, China · Purple Mountain Laboratories, Nanjing 210096, China · Engineering Research Center of Blockchain Application, Supervision and Management (Southeast University), MoE, Nanjing 211189, Jiangsu, China +1
UltraEP: Unleash MoE Training and Inference on Rack-Scale Nodes with Near-Optimal Load Balancing
Jun 2, 2026Xinming Wei, Chao Jin, Tuo Dai +10Expert Parallelism Expert Load Balancing
School of Computer Science, Peking University · Xiaohongshu Inc. · Shanghai AI Laboratory +1
Expert-Aware Causal Tracing of Factual Recall in Sparse MoE Language Models
Jun 2, 2026Yuetian Lu, Ali Modarressi, Yihong Liu +1Mixture-of-Experts Language Models Sparse Mixture-of-Experts
Center for Information and Language Processing (CIS), LMU Munich

Jun 17, 2026, cs.ROJ/K move · Enter open · S save

CTS-MoE: Implicit Terrain Adaptation via Mixture-of-Experts for Perceptive Locomotion

Francisco Affonso, Matheus P. Angarola, Ana Luiza Mineiro, Aditya Potnis+2

University of Illinois Urbana-Champaign · University of São Paulo

Robot Policy Adaptation Terrain-Aware Robot Locomotion RL for Legged Locomotion Multi-Task RL Mixture-of-Experts Models

Open paper AskPDF arXiv

Perceptive legged locomotion over discontinuous terrain (e.g., stairs, gaps, and obstacles) requires adaptive behavior, as a single conservative gait cannot produce the anticipatory maneuvers needed for abrupt topology changes. Cast as multi-task reinforcement learning, this problem introduces a tension between sharing and separation. Tasks use a common locomotion base but have conflicting rewards, so a policy must share behavior while avoiding value interference. Prior work addresses only one side, with monolithic policies sacrificing specialization and hierarchical sub-policies sacrificing generalization across transitions and unseen terrain. We propose CTS-MoE, which combines a dense mixture-of-experts actor with perception-based gating to compose shared behaviors and a multi-critic with task-specific value heads to prevent interference. The model is trained end-to-end in a single-stage concurrent teacher-student setup that handles partial observability and avoids sequential distillation, with task labels used only during training. At deployment, routing depends solely on perception, allowing terrain adaptation without a high-level selector or terrain classifier. Experiments on a Unitree Go1 in simulation and on hardware across seen and unseen terrains show task-aware specialization, with lower tracking error and higher success rates than monolithic baselines. Project Website: https://cts-moe.github.io/ .

Momentum

Related topics

Latest papers 297