Communication-Efficient Mixture-of-Experts

Follow this topic Ask about this topic

Latest papers

14 papers3 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

MAEM: Mitigating Recency Bias via a Replay-Free, Capacity-Controlled Dynamic Mixture of Experts for Class-Incremental Graph Learning
Oct 14, 2026Junlin Chen, Wenhao Wang, Junchao Xu +5Communication-Efficient Mixture-of-Experts Class-Incremental Learning
Beihang University, Beijing, China · Renmin University of China, Beijing, China · Trine University, Indiana, USA +1
Democratizing MoE inference on commodity GPUs with CoMoE
Oct 7, 2026Ruwen Fan, Yuezhi Zu, Minhui Xie +6Communication-Efficient Mixture-of-Experts Mixture-of-Experts Inference
Jimmy · Tsinghua University · Nanjing University +1
Expert Coupling in MoE Pretraining: Reducing All-to-All Overhead with Correlated Placement and Token Shuffling
Oct 7, 2026Radha Gulhane, Quentin Anthony, Beren MillidgeCommunication-Efficient Mixture-of-Experts Mixture-of-Experts Language Models
Zyphra
Training Communication-Efficient Mixture-of-Experts Language Models with Layer Re-Configuration
Aug 28, 2026Simeng Sun, Roger WaleffeMixture-of-Experts Language Models Communication-Efficient Mixture-of-Experts
NVIDIA
Share First, Route What Remains: A Unified Framework for Token-Adaptive MoE Computation
Aug 11, 2026Gongli Zhang, Zhulin Liu, C. L. Philip ChenMixture of Experts Communication-Efficient Mixture-of-Experts
Guangdong Provincial Key Laboratory of Computational AI Models and Cognitive Intelligence, School of Computer Science and Engineering, South China University of Technology, Guangzhou 510006, China · Pazhou Lab, Guangzhou 510335, China · Engineering Research Center of the Ministry of Education on Health Intelligent Perception and Paralleled Digital-Human, Guangzhou 510641, China
Beyond Task-Agnostic: Task-Aware Grouping for Communication-Efficient Multi-Task MoE Inference
May 31, 2026Zhiyao Xu, Aoxue Liu, Zhanjie Ding +3Communication-Efficient Mixture-of-Experts Mixture-of-Experts Inference
Tsinghua Shenzhen International Graduate School, Shenzhen, China · Pengcheng Laboratory, Shenzhen, China
Dynamic TMoE: A Drift-Aware Dynamic Mixture of Experts Framework for Non-Stationary Time Series Forecasting
May 20, 2026Jiawen Zhu, Shuhan Liu, Di Weng +1Concept Drift Mixture of Experts
School of Software Technology, Zhejiang University, Ningbo, China · State Key Lab of CAD&CG, Zhejiang University, Hangzhou, China
Post-Trained MoE Can Skip Half Experts via Self-Distillation
May 18, 2026Xingtai Lv, Li Sheng, Kaiyan Zhang +12Mixture of Experts Communication-Efficient Mixture-of-Experts
Tsinghua University · Shanghai AI Lab · WeChat AI +1
SDG-MoE: Signed Debate Graph Mixture-of-Experts
May 8, 2026Stepan Kulibaba, Kirill Labzin, Artem Dzhalilov +4Mixture-of-Experts Language Models Mixture of Experts
Innopolis University, Innopolis · Sirius University, Sirius · Innopolis University, Innopolis; Sirius University, Sirius +1
Relay Buffer Independent Communication over Pooled HBM for Efficient MoE Inference on Ascend
May 7, 2026Tianlun Hu, Tiancheng Hu, Shengsheng Litang +8Communication-Efficient Mixture-of-Experts Mixture-of-Experts Inference
Huawei Technologies
Expert Routing for Communication-Efficient MoE via Finite Expert Banks
May 6, 2026Mohammad Reza Deylam Salehi, Ali KhalesiMixture of Experts Information-Theoretic Generalization Bounds
IEEE Graduate Member · Nice, France · Institut Polytechnique des Sciences Avancées (IPSA) +1
Scaling Multi-Node Mixture-of-Experts Inference Using Expert Activation Patterns
Apr 25, 2026Abhimanyu Bambhaniya, Geonhwa Jeong, Jason Park +6Expert Load Balancing Mixture-of-Experts Inference
1Georgia Institute of Technology, USA · 2Meta Platforms, Inc.
NCCL EP: Towards a Unified Expert Parallel Communication API for NCCL
Mar 13, 2026Amos Goldman, Nimrod Boker, Maayan Sheraizin +15Communication-Efficient Mixture-of-Experts Expert Parallelism
NVIDIA Corporation
Mixture-of-Experts under Finite-Rate Gating: Communication--Generalization Trade-offs
Feb 16, 2026Ali Khalesi, Mohammad Reza Deylam SalehiMixture-of-Experts Models Communication-Efficient Mixture-of-Experts
Institut Polytechnique des Sciences Avancées (IPSA) and LINCS Lab, Paris, France · IEEE member, Nice, France

Oct 14, 2026J/K move · Enter open · S save

MAEM: Mitigating Recency Bias via a Replay-Free, Capacity-Controlled Dynamic Mixture of Experts for Class-Incremental Graph Learning

Junlin Chen, Wenhao Wang, Junchao Xu, Qingsong Yao+4

Beihang University, Beijing, China · Renmin University of China, Beijing, China · Trine University, Indiana, USA+1

Communication-Efficient Mixture-of-Experts Class-Incremental Learning Graph Neural Networks Mixture of Experts

Open paper AskPDF arXiv

Graph Neural Networks (GNNs) excel on graph-structured data but suffer catastrophic forgetting when classes arrive sequentially. In class-incremental graph learning, predictions are often dominated by recently learned classes, causing severe recency bias. Dynamic expansion can isolate knowledge across sessions, but many methods assume task-incremental settings and require task identities at test time to select branches or heads---information unavailable in class-incremental learning. Learnable routers offer an alternative, yet continual fine-tuning the router on new data can bias routing toward recent components. We propose MAEM, a replay-free, capacity-controlled dynamic mixture-of-experts framework for class-incremental graph learning. MAEM builds representations with Simplified Graph Convolution (SGC) and trains a dedicated expert for each new session, then freezes it to protect prior knowledge. For routing without task IDs or a shared trainable router, each expert uses a masked autoencoder (MAE); test nodes are assigned to the expert with the lowest reconstruction error, reducing routing-level recency bias. When the expert pool reaches a fixed capacity, MAEM merges the least-used experts via Git Re-Basin alignment and further distills them into a fused expert using topology-aware pseudo data synthesized from MAE distributions (DeepInversion-style), without storing or revisiting historical graph data. On four class-incremental graph benchmarks, MAEM mitigates recency bias, outperforms strong continual graph learning baselines, and keeps model capacity bounded under strict replay-free learning.