Synchronous MoE reinforcement-learning post-training suffers from response-length imbalance. Existing methods reduce request tails, yet completed ranks remain bound to Expert Parallelism (EP) collectives and expert ownership. In a representative 16-accelerator Qwen3-30B-A3B rollout, this dummy run traps 32.83% of rank-time. AdaFloor converts this EP-coupled capacity into bounded leases for auxiliary work to improve system resource efficiency through rank-time reuse. Historical prompt-tail prediction and a calibrated guard construct the KV-safe layout. AdaFloor then selects the floor-specific safe quorum schedule with the largest predicted release area. Its runtime shrinks the EP group and leases detached ranks until full-world restoration. Across three updated-policy seeds on DeepScaleR mathematical reasoning rollouts, its capacity-oriented Natural floor2 configuration with TailGuard reduces Vanilla's 29393.9 rank-seconds of dummy run by 96.15%. Rollout shortening eliminates 18693.6 rank-seconds, while coordinated exits convert another 9568.6 rank-seconds into reusable capacity, leaving 1131.7 rank-seconds. For sidecar reuse, its Planned residency floor4 configuration without TailGuard serves 3134 Qwen3-8B GSM8K queries per epoch at 99.46% accuracy and incurs only a 2 to 3% primary throughput reduction against the same configuration with sidecar execution disabled. Its latency-oriented Planned residency floor4 configuration with TailGuard reduces rollout time by 24.09% relative to Vanilla and by 2.31% relative to LengthSort with TailGuard. On public Eurus-2-RL code rollouts with the same model and 16-rank topology, AdaFloor reduces rollout time by 20.56% and dummy run by 89.13% while generated work and mean reward remain stable.
Figures & tables
Fig. 1: EP couples token ownership and expert placement. A rank with no unfinished local responses may still host experts selected by active tokens on other ranks, so it cannot leave a fixed EP group independently.
Fig. 2: Per-rank activity during one 1305.886 s Qwen3-30B rollout cycle on 16 ranks. Dummy-run suffixes account for 32.83% of total rank-time but remain EP-coupled until a valid completion quorum enables shrink.
Fig. 3: Prompt-level maximum response lengths for one adjacent epoch pair. Most prompts remain stable, while three rare jumps reach the 16K response limit and motivate a calibrated step-level tail guard.
Fig. 4: Toy four-rank assignment under a KV cap of 144 token units. The complete workload determines pair safety through response overlap. Release-aware matching reaches the 4→2 transition earlier despite larger local skew.
Fig. 5: AdaFloor overview. Historical traces produce length-sorted candidate steps and calibrated response caps. For each fixed step, the planner jointly selects a KV-safe floor, completion quorum schedule, prompt pairing, and physical rank placement. The runtime switches EP groups at the selected quorums and leases detached ranks until full-world restoration.
Floor
Natural
Planned residency
2
131,072
Not supported
4
280,576
133,120
8
315,648
262,656
16
380,800
380,800
TABLE I: Profiled effective per-rank KV token capacities after model, expert, communication, dispatcher, and workspace reservations. Not supported means that group warmup exhausts HBM before usable KV allocation.
Fig. 6: End-to-end results across three seeds, each averaging epochs 1 and 2. Bars show means and whiskers show sample standard deviations. (a) Five-step rollout time and reduction from Vanilla. The bracket compares Ada-P floor4 with LengthSort using TailGuard. LS and LS+TG denote LengthSort without and with TailGuard. (b) Rank-time released from coordinated exits to the rollout boundary, stacked by transition stage with grayscale-safe hatching. Ada-N and Ada-P denote Natural and Planned residency. Planned floor2 is infeasible under the evaluated HBM budget.
Fig. 7: Robust epoch construction for one representative trace. Steps 1 through 5 belong to epoch 1 and steps 6 through 10 belong to epoch 2. Panel (a) reports per-step rollout time for Random, LengthSort, and LengthSort with TailGuard. Panel (b) compares predicted tails and guard caps with realized maxima from guarded and unguarded LengthSort. Markers and line styles remain distinct in grayscale.
Forecast
Time
Release
Reward
TG/P
(ks)
(k rank-s)
×1.0
4.91
8.28
0.8352
2/0
×0.7
4.45
2.21
0.8324
265/55
×1.3
4.94
10.35
0.8375
1/0
Mixed
5.02
6.97
0.8336
146/16
TABLE II: Forecast sensitivity. Perturbation windows freeze the actor. The reused ×1.0 reference includes actor updates.
Fig. 8: Elastic scheduling for one representative trace with LengthSort and TailGuard fixed. Steps 1 through 5 belong to epoch 1 and steps 6 through 10 belong to epoch 2. Shading encodes rank-time from observed coordinated exits to rollout boundaries, while every cell reports the selected floor and released amount for grayscale use.
Planner
Rank 1
Rank 2
Rank 3
Quota-aware ILP
150
0
0
MinSkew
142
8
0
Stepwise-F4
88
2
60
TABLE III: Ranks by release area calculated from original response lengths on 150 Fig. 6 step records. Ties receive the same rank.
Fig. 9: Controlled quorum-aware release. Shaded areas show measured released rank-time. The quota-aware release wave occurs earlier and exposes 22.12% more rank-time.
floor4 policy
Q2.5-1.5B
Q3-8B
Pangu-72B
AICore Rel.
Natural
66,184
3,083
1,441
139.49%
Planned residency
67,272
3,134
1,465
128.94%
Natural+TailGuard
55,164
2,570
1,201
140.09%
Planned residency +TailGuard
21,975
1,024
478
127.44%
TABLE IV: GSM8K sidecar queries completed during one floor4 rollout epoch. Each row is a separate run and each request has a 4096-token response limit. AICore utilization is normalized to the matching floor4 policy with sidecar execution disabled.
Mode
Expert availability
floor8
floor4
floor2
1
Resident replica
93.65
98.12
100.58
2
CPU swap
484.57
496.78
514.71
3
CPU double buffer
197.99
199.72
265.88
4
Remote NPU
144.91
164.35
193.43
5
Dual source
151.44
185.70
201.58
TABLE V: Single-step rollout time for five expert availability mechanisms under the same prompt workload and validated KV headroom. Every cell processes the same configured token workload and lower time is better.
Fig. 10: Offline planning and online control costs for Planned residency floor4. Bars report means and error bars span observed minima and maxima across ten planner runs or three control events. The shared logarithmic axis covers microsecond and second scale components.
Reinforcement learning post-training adapts pretrained models with reward or feedback signals to improve reasoning, alignment, and interactive behavior.