Experts

Follow this topic Ask about this topic

Recent momentum

-48%

15 papers in the last 28 days · 0.2% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this topic, kept on the site without email delivery.

Period ending 2026-09-21

6 new papers

A weekly snapshot of new work published in Experts.

Period ending 2026-09-14

2 new papers

A weekly snapshot of new work published in Experts.

Period ending 2026-09-07

4 new papers

A weekly snapshot of new work published in Experts.

167 papers

Latest in Experts

Browse all topics →

$φ$ -Balancing for Mixture-of-Experts Training
May 14, 2026Lizhang Chen, Jonathan Li, Qi Wang +5Mixture-of-Experts Models Balanced Learning
Discovering Physical Directions in Weight Space: Composing Neural PDE Experts
May 14, 2026Pengkai Wang, Pengwei Liu, Yuanyi Wang +7Neural Partial Differential Equation Solvers Partial Differential Equations
RQ-MoE: Residual Quantization via Mixture of Experts for Efficient Input-Dependent Vector Compression
May 14, 2026Zhengjia Zhong, Shuyan Ke, Zaizhou Lin +3Calibration-Free Low-Bit Vector Quantization Codebooks
Synthesizing the Expert: A Validated Multimodal Dataset for Trustworthy AI-Assisted Swimming Coaching
May 12, 2026Ahmad Al-Kabbany, Esraa KassemSports Science Multimodal Dataset
Routers Learn the Geometry of Their Experts: Geometric Coupling in Sparse Mixture-of-Experts
May 12, 2026Sagi Ahrac, Noya Hochwald, Mor GevaMixture-of-Experts Models Load Balancing
Online Learning-to-Defer with Varying Experts
May 12, 2026Dang Hoang Duy, Yannis Montreuil, Maxime Meyer +3Learning-Augmented Algorithms Experts
Composition of Memory Experts for Diffusion World Models
May 12, 2026Sebastian Stapf, Pablo Acuaviva Huertos, Aram Davtyan +1Discrete Diffusion Model Memory Retrieval
Slicing and Dicing: Configuring Optimal Mixtures of Experts
May 12, 2026Margaret Li, Sneha Kudugunta, Danielle Rothermel +1Mixture-Of-Expert Language Models Mixture-of-Experts Models
HELLoRA: Hot Experts Layer-Level Low-Rank Adaptation for Mixture-of-Experts Models
May 11, 2026Jia Wei, Zhonghao Zhang, Ping Chen +5Mixture-Of-Expert Language Models Low-Rank Adaptation
When (and How) to Trust the Expert: Diagnosing Query-Time Expert-Guided Reinforcement Learning
May 9, 2026Yann Berthelot, Philippe Preux, Riad AkrourExperts Execution Efficiency
FLUX: Geometry-Aware Longitudinal Flow Matching with Mixture of Experts
May 9, 2026Josue Ortega Caro, Yongxu Zhang, Hannah M Batchelor +3Latent States Mixture-of-Experts Models
When Are Experts Misrouted? Counterfactual Routing Analysis in Mixture-of-Experts Language Models
May 8, 2026Youngsik Yoon, Siwei Wang, Wei Chen +1Mixture-of-Experts Models Experts
TAS-LoRA: Transformer Architecture Search with Mixture-of-LoRA Experts
May 8, 2026Jeimin Jeon, Hyunju Lee, Bumsub HamSelf-Supervised Vision Transformers Neural Architecture Search
UniPool: A Globally Shared Expert Pool for Mixture-of-Experts
May 7, 2026Minbin Huang, Han Shi, Chuanyang Zheng +5Mixture-of-Experts Models Experts
EMO: Pretraining Mixture of Experts for Emergent Modularity
May 7, 2026Ryan Wang, Akshita Bhagia, Sewon MinMixture-Of-Expert Language Models Mixture-of-Experts Models
Scene-Adaptive Continual Learning for CSI-based Human Activity Recognition with Mixture of Experts
May 7, 2026Wenhan Zheng, Yuyi Mao, Ivan Wang-Hei HoHuman Activity Recognition Cross-Sensor Generalization
VLA-GSE: Boosting Parameter-Efficient Fine-Tuning in VLA with Generalized and Specialized Experts
May 7, 2026Yuhua Jiang, Junjie Lu, Xinyao Qin +4Large Vision Language Models Parameter-Efficient Fine-Tuning
Detecting Time Series Anomalies Like an Expert: A Multi-Agent LLM Framework with Specialized Analyzers
May 7, 2026Hyeongwon Kang, Jeongseob Kim, Jinwoo Park +1Time-Series Anomaly Detection Anomaly Detection
Expert Routing for Communication-Efficient MoE via Finite Expert Banks
May 6, 2026Mohammad Reza Deylam Salehi, Ali KhalesiMixture-Of-Expert Inference Mixture-of-Experts Models
YOTOnet: Zero-Shot Cross-Domain Fault Diagnosis via Domain-Conditioned Mixture of Experts
May 6, 2026Zesen Wang, Zihao Wu, Yue Hu +2Fault Diagnosis Multimodal Domain Generalization
Ortho-Hydra: Orthogonalized Experts for DiT LoRA
May 5, 2026Seunghyun JiMultimodal Diffusion Transformers Low-Rank Adaptation
Prediction-powered Inference by Mixture of Experts
Apr 30, 2026Yanwu Gu, Linglong Kong, Dong XiaMixture-Of-Expert Inference Probabilistic Inference
Path-Lock Expert: Separating Reasoning Mode in Hybrid Thinking via Architecture-Level Separation
Apr 29, 2026Shouren Wang, Wang Yang, Chuang Ma +7Mixture-Of-Expert Language Models Hybrid
Advancing multi-site emission control: A physics-informed transfer learning framework with mixture of experts for carbon-pollutant synergy
Apr 29, 2026Yuxuan Ying, Hanqing Yang, Kaige Wang +6Carbon Emissions Waste
Adaptive and Fine-grained Module-wise Expert Pruning for Efficient LoRA-MoE Fine-Tuning
Apr 29, 2026Weihang Li, Jianchun Liu, Hongli XuMixture-of-Experts Models Parameter-Efficient Fine-Tuning
Mixture of Experts Framework in Machine Learning Interatomic Potentials for Atomistic Simulations
Apr 28, 2026Gabriel de Miranda Nascimento, Marc L. Descoteaux, Laura Zichi +9Interatomic Potentials Molecular Dynamics
Preserving Long-Tailed Expert Information in Mixture-of-Experts Tuning
Apr 24, 2026Haoze He, Xingyuan Ding, Xuan Jiang +5Mixture-of-Experts Models Experts
Quadruped Parkour Learning: Sparsely Gated Mixture of Experts with Visual Input
Apr 21, 2026Michael Ziegltrum, Jianhao Jiao, Tianhu Peng +2Parkour Locomotion Mixture-of-Experts Models
Expert Upcycling: Shifting the Compute-Efficient Frontier of Mixture-of-Experts
Apr 21, 2026Chaitanya Dwivedi, Binxuan Huang, Himanshu Gupta +3Mixture-of-Experts Models Experts
Product-of-Experts Training Reduces Dataset Artifacts in Natural Language Inference
Apr 21, 2026Aby Mammen MathewNatural Language Inference Experts
DREAM: Dynamic Retinal Enhancement with Adaptive Multi-modal Fusion for Expert Precision Medical Report Generation
Apr 19, 2026Nagur Shareef Shaik, Teja Krishna Cherukuri, Dong Hye YeRetinal Imaging Medical Report Generation
CoGR-MoE: Concept-Guided Expert Routing with Consistent Selection and Flexible Reasoning for Visual Question Answering
Apr 18, 2026Xiyin Zeng, Yi Lu, Hao WangKnowledge-Based Visual Question Answering Experts
Learning to Trade Like an Expert: Cognitive Fine-Tuning for Stable Financial Reasoning in Language Models
Apr 18, 2026Yuchen Pan, Soung Chang LiewFrontier Models Multiple-Choice Questions
AstroVLM: Expert Multi-agent Collaborative Reasoning for Astronomical Imaging Quality Diagnosis
Apr 17, 2026Yaohui Han, Tianshuo Wang, Zixi Zhao +6Vision-Language Multi-Agent Reasoning
How Annotation Trains Annotators: Competence Development in Social Influence Recognition
Apr 3, 2026Maciej Markiewicz, Beata Bajcar, Wiktoria Mieleszczenko-Kowszewicz +8Human Annotators Human Annotations
Modality-Guided Mixture of Structured Experts with Entropy-Triggered Routing for Multimodal Recommendation
Feb 24, 2026Ji Dai, Quan Fang, DeSheng CaiMultimodal Recommendation Recommendation
ButterflyMoE: Compression-Scalable Ternary Experts via Structured Butterfly Orbits
Jan 20, 2026Aryan KarmoreExperts Expert Demonstrations
Mixtures of SubExperts for Large Language Continual Learning
Nov 9, 2025Haeyong Kang, Hee Suk Yoon, Dahua Feng +1Mixture-Of-Expert Language Models Efficient Continual Learning
Expert Preference-based Evaluation of Automated Related Work Generation
Aug 11, 2025Furkan Şahinuç, Subhabrata Dutta, Iryna GurevychHuman Grounded Evaluation Scientific Discovery
Macro Graph of Experts for Billion-Scale Multi-Task Recommendation
Jun 12, 2025Hongyu Yao, Zijin Hong, Hao Chen +6Multimodal Recommendation Large-Scale Recommendation Systems
"Is This Really a Human Peer Supporter?": Misalignments Between Peer Supporters and Experts in LLM-Supported Interactions
Jun 11, 2025Kellie Yu Hui Sim, Roy Ka-Wei Lee, Kenny Tsu Wei ChooMental Health Support Peers
"You are an expert annotator": Automatic Best-Worst-Scaling Annotations for Emotion Intensity Modeling
Mar 26, 2024Christopher Bagdon, Prathamesh Karmalker, Harsha Gurulingappa +1Human Annotators Annotations
Retrieval of Scientific and Technological Resources for Experts and Scholars
Apr 13, 2022Suyu Ouyang, Yingxia Shao, Ang LiScientific Discovery Experts
Regularized Estimation and Feature Selection in Mixtures of Generalized Linear Experts
Jul 14, 2019Thin Nguyen-Van, Faicel Chamroukhi, Ha Hoang Van +1Mixture-of-Experts Models Lasso
CMoE: Contrastive Mixture of Experts for Motion Control and Terrain Adaptation of Humanoid Robots
Date pendingShihao Ma, Hongjin Chen, Zijun Xu +6Long-Tail Humanoid Whole-Body Control Terrain
Cache-Aware Joint Router Adaptation for Memory-Efficient MoE Inference
Date pendingZhenhe Wu, Yaping Jin, Qinghua Xing +6Mixture-Of-Expert Inference Mixture-of-Experts Models
Dynamic Expert Quantization for Scalable Mixture-of-Experts Inference
Date pendingKexin Chu, Dawei Xiang, Zixu Shen +3Mixture-Of-Expert Inference Mixed-Precision Quantization

May 14, 2026 · cs.LGJ/K move · Enter open · S save

$φ$ -Balancing for Mixture-of-Experts Training

Lizhang Chen, Jonathan Li, Qi Wang, Runlong Liao+4

University of Texas at Austin · 2Northwestern University.

Mixture-of-Experts (MoE) models rely on balanced expert utilization to fully realize their scalability. However, existing load-balancing methods are largely heuristic and operate on noisy mini-batch assignment statistics, introducing bias relative to population-level objectives. We propose

φ

-balancing, a principled framework that directly targets population-level expert balance by minimizing a strictly convex, symmetric, and differentiable potential of the expected routing distribution. Using convex duality, we derive an equivalent min-max formulation and obtain a simple online algorithm via mirror descent, yielding an efficient EMA-based routing adjustment with negligible overhead. Across large-scale pretraining and downstream fine-tuning,

φ

-balancing consistently outperforms prior Switch-style and loss-free baselines, demonstrating more stable and effective expert utilization.

Mixture-of-Experts Models Balanced Learning Experts Mini-Batch Stochastic Gradient Descent Downstream Fine-Tuning

Open paper Ask about it PDF

Recent momentum

Neighbouring directions

Recent digests

6 new papers

2 new papers

4 new papers

Latest in Experts