Mixture-of-Experts Fine-Tuning

Latest papers

CardsList

21 papers7 papers in the last four weeks, against 1 the four weeks before. 0.1% of all new papers.

  1. PMR: MoE Routing with Shared Prototypes and Congestion Pricing

    Nov 14, 2026Wending Wang, Binqi Huang, Ju'an Li +4Mixture-of-Experts ModelsExpert Load Balancing

  2. CoPre-MoE: Consolidating and Preserving MoE for Multimodal Continual Instruction Tuning

    Nov 13, 2026Tao Hu, Ling-Xiang Zhang, Da-Wei ZhouMixture-of-Experts Fine-TuningContinual Learning for VLMs

  3. Sonar-MoE: In-Backward Expert Updates for Memory-Efficient Mixture-of-Experts Training

    Nov 2, 2026Avni Mittal, Avinash Anand, Ashutosh Kumar +7Mixture-of-Experts ModelsMemory-Efficient Training

  4. Characterizing Recoverable Expert-Utilization Signatures in Fine-Tuned Mixture-of-Experts LLMs

    Oct 29, 2026Fei Wang, Baochun Li, Bo LiMixture-of-Experts Language ModelsMemorization in Language Models

  5. Upcycling Language Models for Multiple Specialized Domains

    Oct 29, 2026Mingzi Cao, Xi Wang, Nikolaos AletrasDomain Adaptation for LLMsDomain Adaptation

  6. DuplexTune: Coupled-State Replay for Memory-Efficient Mixture-of-Experts Adaptation

    Oct 26, 2026Paul Raj, Simon Caldwell, Daniel Whitmore +1Mixture-of-Experts Fine-TuningMemory-Efficient Fine-Tuning

  7. An Empirical Study of Parameter-Efficient Fine-Tuning for Mixture-of-Experts LLMs

    Oct 19, 2026Minglei Li, Peng Ye, Haonan He +3Mixture-of-Experts Fine-TuningParameter-Efficient Fine-Tuning

  8. MaskCoFT: Masked Co-Adaptive Fine-Tuning for Memory-Efficient MoE Inference

    Sep 28, 2026Junfeng Wu, Zehao Fan, Hadjer Benmeziane +3Expert OffloadingMixture-of-Experts Language Models

  9. ACE: Adapter Consolidation across Experts for Parameter-Efficient Fine-Tuning of MoE LLMs

    Sep 5, 2026Ahin Lee, Sehyun Yun, Joonha Park +1Mixture-of-Experts Fine-TuningParameter-Efficient Fine-Tuning

  10. Router Sensitivity Under Lightweight Fine-Tuning Identifies Prunable Experts in Mixture-of-Experts Models

    Aug 8, 2026Ali Janati, Kaoutar El Maghraoui, Chengke Zou +2Mixture-of-Experts PruningMixture-of-Experts Models

  11. MoEGen: Mixture-of-Experts for Instance-Adaptive LoRA Generation

    Aug 4, 2026Yiming Zeng, Lei Lu, Zexin Li +10LLM Fine-TuningLow-Rank Adaptation

  12. TEXAS: Task-Expert-Aware Supervision for Downstream Mixture-of-Experts LLM Adaptation

    Jul 31, 2026Guanzhi Deng, Haibo Wang, Kuan Wu +5Mixture-of-Experts Language ModelsMixture of Experts

  13. MoE2^2-LoRA: When MoE Models Meet MoE-style Low-Rank Adaptation

    Jul 24, 2026Qingyu Yang, Haonan He, Minglei Li +4Mixture-of-Experts Language ModelsLow-Rank Adaptation

  14. FRAME: Learning the Adaptation Domain with a Mixture of Fractional-Fourier Experts

    Jun 30, 2026Tom Saliencro, Maya Lindqvist, Rohan Desai +2Low-Rank AdaptationMixture-of-Experts Models

  15. HELLoRA: Hot Experts Layer-Level Low-Rank Adaptation for Mixture-of-Experts Models

    May 11, 2026Jia Wei, Zhonghao Zhang, Ping Chen +5Mixture-of-Experts Language ModelsFine-Tuning

  16. VLA-GSE: Boosting Parameter-Efficient Fine-Tuning in VLA with Generalized and Specialized Experts

    May 7, 2026Yuhua Jiang, Junjie Lu, Xinyao Qin +4Fine-TuningVision-Language-Action Models

  17. Preserving Long-Tailed Expert Information in Mixture-of-Experts Tuning

    Apr 24, 2026Haoze He, Xingyuan Ding, Xuan Jiang +5Supervised Fine-TuningFine-Tuning

  18. Train Separately, Merge Together: Modular Post-Training with Mixture-of-Experts

    Apr 20, 2026Jacob Morrison, Sanjay Adhikesaven, Akshita Bhagia +3Continual Learning for LLMsMixture-of-Experts Language Models

  19. DR-LoRA: Dynamic Rank LoRA for Fine-Tuning Mixture-of-Experts Models

    Jan 8, 2026Guanzhi Deng, Bo Li, Ronghao Chen +5Mixture-of-Experts Language ModelsFine-Tuning