Trajectory transfer from rollout to training affects both learning quality and system efficiency in large language model reinforcement learning (RL). Enforcing batch-wide training requirements can delay the admission of completed trajectories, resulting in higher architectural staleness: the additional parameter-version gap from disaggregated pipelining relative to colocated execution. Beyond admission, misalignment between data access and training consumption causes redundant movement and unnecessary waiting when transferring large trajectory payloads. We present KnitTrans, a system that coordinates the rollout-to-training path through three designs. First, constraint-preserving mini-batch admission allows flexibility in early mini-batch composition, while checking that enough capacity remains to satisfy the remaining quotas. Second, on-demand trajectory data access matches storage units to consumer assignments and overlaps micro-batch data preparation with computation. Third, multimodal encoding deduplication supports reuse across a mini-batch while preserving gradient contributions from all backbone micro-batches. Experiments show up to 93% lower architectural staleness than with train-batch admission, narrowing the learning-quality gap between disaggregated execution and colocated execution. On multimodal workloads, KnitTrans reduces transfer latency by two orders of magnitude and achieves a speedup of up to 5.8 times in transfer plus training over the baseline.
Figures & tables
Figure 1. Timeline of trajectory admission, data delivery, and training computation under KnitTrans.
Figure 2. Architectural staleness under partial rollout. (a) Colocated and disaggregated execution with batch-level streaming, where i denotes the i -th version of parameters, assuming the durations of the two stages are balanced. (b) Average staleness and avg@ n of VeRL (colocated) and slime (disaggregated).
Figure 3. Rollout skew, delayed admission and larger architectural staleness. (a) Per-source rollout durations: bars show means; arrows show ± one standard deviation. (b) Mini-batch admission times under linear cumulative source targets. (c) Delayed (top) versus earlier (bottom) admission at the same mini-batch granularity.
Figure 4. RL step-time breakdown on two Omni RL workloads using VeRL with sample-level transfer.
Figure 5. Overview of KnitTrans.
Figure 6. End-to-end workflow of On-Demand Trajectory Data Access.
Figure 7. Architectural staleness of the disaggregated system on the CodeAgent task.
Figure 8. Exposed transfer and training computation latency per step on Omni RL.
Figure 9. RL convergence: avg@ n per training step on the CodeAgent task with MiMo-V2.5.
Figure 10. Constraint preservation and architectural staleness under different admission policies.
Figure 11. Layer-wise latency ablations: (a) sample-level TQ versus topology-aware distributed storage ( TADS ); (b) eager versus iter fetching on TADS; (c) duplicated ( dup ) versus deduplicated ( dedup ) multimodal encoding with iterator fetching. The eager configuration is TADS from (a), and dup is iter from (b).