cs.AINov 4, 2026

AdaFloor: Adaptive Floor Planning and Rank-Time Reuse for Long-Tail MoE RL Rollouts

Authors: Jia Wei, Ziyi Qiu, Yi Xu, Xuehai Qian

Organizations: Department of Computer Science and Technology, Tsinghua University

Abstract

Synchronous MoE reinforcement-learning post-training suffers from response-length imbalance. Existing methods reduce request tails, yet completed ranks remain bound to Expert Parallelism (EP) collectives and expert ownership. In a representative 16-accelerator Qwen3-30B-A3B rollout, this dummy run traps 32.83% of rank-time. AdaFloor converts this EP-coupled capacity into bounded leases for auxiliary work to improve system resource efficiency through rank-time reuse. Historical prompt-tail prediction and a calibrated guard construct the KV-safe layout. AdaFloor then selects the floor-specific safe quorum schedule with the largest predicted release area. Its runtime shrinks the EP group and leases detached ranks until full-world restoration. Across three updated-policy seeds on DeepScaleR mathematical reasoning rollouts, its capacity-oriented Natural floor2 configuration with TailGuard reduces Vanilla's 29393.9 rank-seconds of dummy run by 96.15%. Rollout shortening eliminates 18693.6 rank-seconds, while coordinated exits convert another 9568.6 rank-seconds into reusable capacity, leaving 1131.7 rank-seconds. For sidecar reuse, its Planned residency floor4 configuration without TailGuard serves 3134 Qwen3-8B GSM8K queries per epoch at 99.46% accuracy and incurs only a 2 to 3% primary throughput reduction against the same configuration with sidecar execution disabled. Its latency-oriented Planned residency floor4 configuration with TailGuard reduces rollout time by 24.09% relative to Vanilla and by 2.31% relative to LengthSort with TailGuard. On public Eurus-2-RL code rollouts with the same model and 16-rank topology, AdaFloor reduces rollout time by 20.56% and dummy run by 89.13% while generated work and mean reward remain stable.

Figures & tables

Explore similar work

CardsList
  1. ReFlow: Replay-Driven Decoupling for Efficient RL Post-Training

    Oct 15, 2026Yukun Tian, Junwei Su, Haofeng Xu +6RL Post-TrainingRL for Language Model Reasoning

  2. Adaptive Scheduling Pipeline for Multi-Instance Asynchronous Reinforcement Learning

    Nov 6, 2026Salah Chikhi, Luis Zuin, Entong Li +1LLM Inference SchedulingReinforcement Learning

  3. TideRL: Boosting Agentic RL Goodput with Readiness-Aware Scheduling

    Aug 11, 2026Yanyu Ren, Xizheng Wang, Xiao Liu +8Agentic RLAsynchronous RL