Mixture-Of-Expert Language Models

Ask about this topic

Recent momentum

-33%

6 papers in the last 28 days · 0.1% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Also in Mixture-Of-Experts

Neighbouring directions

Mixture-of-Experts ModelsExpertsMixture-Of-Expert InferenceExpert DemonstrationsLoad Balancing

Weekly history

Recent digests

What was published in this topic, kept on the site without email delivery.

Period ending 2026-09-21

2 new papers

A weekly snapshot of new work published in Mixture-Of-Expert Language Models.

Period ending 2026-09-07

5 new papers

A weekly snapshot of new work published in Mixture-Of-Expert Language Models.

72 papers

Latest in Mixture-Of-Expert Language Models

Browse all topics →
CardsList

  1. Higher-order pruning of experts in mixture-of-experts language models

    Sep 16, 2026Alex M. Tseng, Prannay Kaul, Luca Zancato +2Mixture-Of-Expert Language ModelsMixture-of-Experts Models

  2. MoME: Mixture-of-Memory Embeddings for Context-Aware Sparse Lookup

    Sep 14, 2026Muchen Li, Leonid Sigal, Renjie LiaoMixture-Of-Expert Language ModelsLarge Language Model Memory

  3. Instella-MoE Technical Report

    Sep 1, 2026Jiang Liu, Sudhanshu Ranjan, Prakamya Mishra +10Mixture-Of-Expert Language ModelsLarge Models

  4. Residual Sparsification via Output Importance for Compressing Mixture-of-Experts LLMs

    Sep 1, 2026Seungwoo Jung, Dohyeok Kwon, Seungmin Cha +4Mixture-Of-Expert Language ModelsLarge Language Model Compression

  5. Q-Strata: Hierarchical Bit Allocation for Mixed-Precision Quantization of Mixture-of-Experts LLMs

    Aug 31, 2026Deokjae Lee, Sihun Chu, Hyun Oh SongMixed-Precision QuantizationMixture-Of-Expert Language Models

  6. A.X K2 Technical Report

    Aug 31, 2026Cheolseung Baek, Dhammiko Arya, Eunki Kim +40Mixture-Of-Expert Language ModelsMixture-of-Experts Models

  7. TradingMoE: Routing the Right Experts in Evolving Markets

    Aug 12, 2026Chang Zhou, Xingtong Yu, Minbin Huang +4Mixture-Of-Expert Language ModelsLarge Language Model Routing

  8. Motif 3: Technical Report

    Aug 10, 2026Junghwan Lim, Joon Son Chung, Sungmin Lee +24Mixture-Of-Expert Language ModelsTechnical Report

  9. Decided Upstream, Written Late: Locating and Pricing the Cross-Lingual Refusal Circuit of a Multilingual MoE

    Aug 8, 2026Ramakrishna P. Kompella, Aadit MahajanMultilingual SafetyLarge Language Model Safety

  10. Shape Mutating Expert Compression:LorExperts and BTExperts

    Aug 7, 2026Inesh Chakrabarti, Sourjya Roy, Bowen Bao +3Mixture-Of-Expert Language ModelsExperts

  11. LLaDA MoE v2: Scaling Mixture-of-Experts Diffusion Language Models

    Aug 4, 2026Fengqi Zhu, Shaoxuan Xu, Jingyang Ou +11Mixture-Of-Expert Language ModelsLarge Models

  12. MoEGen: Mixture-of-Experts for Instance-Adaptive LoRA Generation

    Aug 4, 2026Yiming Zeng, Lei Lu, Zexin Li +9Low-Rank AdaptationMixture-Of-Expert Language Models

  13. Qwen-CUA: Native Computer Use for (almost) Everything

    Aug 3, 2026Dunjie Lu, Shuai Bai, Tianyi Bai +43Computer-Use AgentsHigher Attack Success Rates

  14. MMOE: Modernizing Diffusion Transformers with Efficient Expert Design

    Jul 27, 2026Yanhao Jia, Jiepeng Wang, Haibin Huang +3Mixture-Of-Expert Language ModelsLarge Models

  15. MoLGE: Mixture of Language Group Experts for Efficient Scaling of Massively Multilingual Speech Recognition

    Jul 27, 2026Sangmin Lee, Woojin Chung, Woongjib Choi +1Multilingual Automatic Speech RecognitionMultilingual Language Models

  16. MoE2^2-LoRA: When MoE Models Meet MoE-style Low-Rank Adaptation

    Jul 24, 2026Qingyu Yang, Haonan He, Minglei Li +4Mixture-Of-Expert Language ModelsLow-Rank Adaptation

  17. Solar Open 2 Technical Report

    Jul 22, 2026Sungrae Park, Sanghoon Kim, Gyoungjin Gim +50Mixture-Of-Expert Language ModelsLong-Horizon Agents

  18. Half the Experts, All the Code: One-Shot Domain Pruning of Mixture-of-Experts LLMs for Coding

    Jul 18, 2026Anik JhaMixture-Of-Expert Language ModelsMixture-of-Experts Models

  19. PagedWeight: Efficient MoE LLM Serving with Dynamic Quality-Aware Weight Quantization

    Jul 17, 2026Yuchen Yang, Yifan Zhao, Anisha Dasgupta +1Mixture-Of-Expert Language ModelsGraphics Processing Unit Memory

  20. UMoE:Unlocking Every Expert in Domain-Specific Training

    Jul 13, 2026Xuefeng Li, Pengfei LiuMixture-Of-Expert Language ModelsExperts

  21. A Sovereign, Open-Source Foundation Model for German and English

    Jul 10, 2026The Soofi-Team, :, Benedikt Droste +29Mixture-Of-Expert Language ModelsWireless Foundation Models

  22. FlexMoE: One-for-All Nested Intra-Expert Pruning for MoE Language Models

    Jun 26, 2026Fan Mo, Yuxuan Han, Geng Zhang +2Mixture-Of-Expert Language ModelsMixture-of-Experts Models

  23. GeMoE: Gating Entropy is All You Need for Uncertainty-aware Adaptive Routing in MoE-based Large Vision-Language Models

    Jun 24, 2026Chaoxiang Cai, Minghe Weng, Jie Li +4Mixture-Of-Expert Language ModelsAdaptive Routing

  24. SoftMoE: Soft Differentiable Routing for Mixture-of-Experts in LLMs

    Jun 16, 2026Mikołaj Zasada, Łukasz Struski, Jacek Tabor +1Mixture-Of-Expert Language ModelsMixture-of-Experts Models

  25. Tying the Loop -- Tied Expert Layers in Mixture-of-Experts Language Models

    Jun 15, 2026Martin JaggiMixture-Of-Expert Language ModelsExperts

  26. MODE: Modality-Decomposed Expert-Level Mixed-Precision Quantization for MoE Multimodal LLMs

    Jun 15, 2026Yuanteng Chen, Peisong Wang, Zhilei Liu +9Mixture-Of-Expert Language ModelsMixed-Precision Quantization

  27. Sticky Routing: Training MoE Models for Memory-Efficient Inference

    Jun 12, 2026Ali KayyamMixture-Of-Expert Language ModelsMixture-of-Experts Models

  28. Decoupled Mixture-of-Experts for Parametric Knowledge Injection

    Jun 12, 2026Baoqing Yue, Weihang Su, Qingyao Ai +5Mixture-Of-Expert Language ModelsMixture-of-Experts Models

  29. Entropy-Aware Domain-Routed Mixture-of-Experts Speech-LLM Framework: A Case Study of Multi-Domain Child-Adult ASR

    Jun 9, 2026Mohan Shi, Kaiyuan Zhang, Zilai Wang +3Speech-LlmsMixture-Of-Expert Language Models

  30. Routing-Aware Expert Calibration for Machine Unlearning in Mixture-of-Experts Language Models

    Jun 9, 2026Jingyi Xie, Yijun Lin, Yinjiang Xiong +2Mixture-Of-Expert Language ModelsLarge Language Model Unlearning

  31. Sparse Subspace-to-Expert Sharing for Task-Agnostic Continual Learning

    Jun 5, 2026Fatema Siddika, Md Anwar Hossen, Tanwi Mallick +1Efficient Continual LearningContinual Learning Methods

  32. Reversible Foundations: Training a 120B Sparse MoE through State-Preserving Scaling

    Jun 5, 2026Rohan ShravanMixture-Of-Expert Language ModelsLarge Models

  33. TENP: Trapezoidal Expert Neuron Pruning For Mixture-of-Experts

    Jun 3, 2026Jiangyang He, Shaolin Zhu, Deyi XiongMixture-Of-Expert Language ModelsMixture-of-Experts Models

  34. LoopMoE: Unifying Iterative Computation with Mixture-of-Experts for Language Modeling

    Jun 3, 2026Wenkai Chen, Tianshu Li, Wenyong Huang +3Mixture-Of-Expert Language ModelsMixture-of-Experts Models

  35. DLLG: Dynamic Logit-Level Gating of LLM Experts

    Jun 3, 2026Bingnan Li, Zhaoyang Zhang, Xiaoze Liu +6Mixture-Of-Expert Language ModelsLarge Language

  36. Expert-Aware Refusal Steering

    Jun 2, 2026Anna C. Marbut, Daniel R. Olson, Travis J. WheelerRefusalsMixture-Of-Expert Language Models

  37. Expert-Aware Causal Tracing of Factual Recall in Sparse MoE Language Models

    Jun 2, 2026Yuetian Lu, Ali Modarressi, Yihong Liu +1Mixture-Of-Expert Language ModelsMixture-of-Experts Models

  38. MESA: Improving MoE Safety Alignment via Decentralized Expertise

    May 30, 2026Yitong Sun, Yao Huang, Teng Li +5Safety AlignmentMixture-Of-Expert Language Models

  39. PithTrain: A Compact and Agent-Native MoE Training System

    May 29, 2026Ruihang Lai, Hao Kang, Haozhan Tang +6Mixture-Of-Expert Language ModelsMixture-of-Experts Models

  40. Mellum2 Technical Report

    May 29, 2026Marko Kojic, Ivan Bondyrev, Aral de Moor +6Mixture-Of-Expert Language ModelsMixture-of-Experts Models

  41. Understanding Safety-Sensitive Expert Behavior in Mixture-of-Experts LLMs

    May 28, 2026Zhibo Zhang, Yuxi Li, Zhen Ouyang +2Safety AlignmentMixture-Of-Expert Language Models

  42. ConMoE: Expert-Pool Consolidation via Prototype Reassignment for MoE Compression

    May 28, 2026Yilun Yao, Jiaming Pan, Elsie Dai +3Mixture-Of-Expert Language ModelsMixture-of-Experts Models

  43. Routing-Aligned Fine-Tuning for Multilingual Downstream Tasks in Mixture-of-Experts Models

    May 27, 2026Guanzhi Deng, Kuan Wu, Haibo Wang +3Mixture-Of-Expert Language ModelsMixture-of-Experts Models

  44. How Far Can Disaggregation Go? A Design-Space Exploration of Attention-FFN Disaggregation for Efficient MoE LLM Serving

    May 27, 2026Hanjiang Wu, Abhimanyu Rajeshkumar Bambhaniya, Sarbartha Banerjee +9Mixture-Of-Expert InferenceMixture-Of-Expert Language Models

  45. Pruning and Distilling Mixture-of-Experts into Dense Language Models

    May 27, 2026Junhyuck Kim, Jihun Yun, Haechan Kim +3Mixture-Of-Expert Language ModelsMixture-of-Experts Models

  46. Extracting Small Translation Specialists from LLMs by Aggressively Pruning Experts

    May 27, 2026Liu O. Martin, Lucas Bandarkar, Nanyun PengMixture-Of-Expert Language ModelsMachine Translation Models

  47. FPMoE: A Sparse Mixture-of-Experts Approach to Functional Code Generation

    May 27, 2026Loc Pham, Lang Hong Nguyet Anh, Thanh Le-CongMixture-Of-Expert Language ModelsCode Generation

  48. Laguna M.1/XS.2 Technical Report

    May 26, 2026Julien Abadji, Marah Abdin, Connor Adams +93Mixture-Of-Expert Language ModelsMle-Bench Lite

  49. MobileMoE: Scaling On-Device Mixture of Experts

    May 26, 2026Yanbei Chen, Hanxian Huang, Ernie Chang +5Mixture-Of-Expert Language ModelsMixture-Of-Expert Inference

  50. The MiniMax-M2 Series: Mini Activations Unleashing Max Real-World Intelligence

    May 26, 2026MiniMax, :, Aili Chen +204Mixture-Of-Expert Language ModelsMinimax

  51. RotMoLE: Enhancing Mixture of Low-Rank Experts through Rotational Gating Mechanism

    May 25, 2026Mengyang Sun, Maochuan Dou, Tao Feng +5Mixture-Of-Expert Language ModelsMixture-of-Experts Models

  52. RouteScan: A Non-Intrusive Approach to Auditing MoE LLMs Safety via Expert Routing Telemetry

    May 24, 2026Bo Lv, Zhiheng Xu, KeDong Xiu +4Large Language Model SafetyMixture-Of-Expert Language Models

  53. Safety-Oriented Routing Analysis of Mixtral MoE Under Benign and Harmful Prompts

    May 22, 2026Md Nurul Absar SiddikyMixture-Of-Expert Language ModelsLarge Language Model Routing

  54. BitsMoE: Efficient Spectral Energy-Guided Bit Allocation for MoE LLM Quantization

    May 22, 2026Jiayu Zhao, Zihan Teng, Minhao Fan +4Large Language Model QuantizationMixture-Of-Expert Language Models

  55. GEMQ: Global Expert-Level Mixed-Precision Quantization for MoE LLMs

    May 21, 2026Jianing Deng, Song Wang, Dongwei Wang +4Mixed-Precision QuantizationMixture-Of-Expert Language Models

  56. A Data-Efficient Path to Multilingual LLMs: Language Expansion via Post-training PARAMΔΔ Integration into Upcycled MoE

    May 18, 2026Hao Zhou, Tianhao Li, Zhijun Wang +6Multilingual Large Language ModelsMixture-Of-Expert Language Models

  57. CP-MoE: Consistency-Preserving Mixture-of-Experts for Continual Learning

    May 18, 2026Yang Liu, Toan Nguyen, Flora D. SalimEfficient Continual LearningMixture-Of-Expert Language Models

  58. Mixture of Experts for Low-Resource LLMs

    May 17, 2026Ori Bar Joseph, Smadar Arvatz, Noam Kayzer +2Mixture-Of-Expert Language ModelsMixture-of-Experts Models

  59. Scalable Knowledge Editing for Mixture-of-Experts LLMs via Tensor-Structured Updates

    May 15, 2026Roman Maksimov, Vladimir Aletov, Dmitry Bylinkin +3Multimodal Knowledge EditingMixture-Of-Expert Language Models

  60. ROMER: Expert Replacement and Router Calibration for Robust MoE LLMs on Analog Compute-in-Memory Systems

    May 12, 2026Wenyong Zhou, Yuannuo Feng, Yizhe Chen +6Mixture-Of-Expert Language ModelsIn-Memory Computing