Agentic reinforcement learning (RL) rollouts interleave model reasoning with tool calls in persistent workspaces. Scaling these rollouts incurs substantial memory costs for concurrent environments, while slow tool calls delay entire rollout batches. We present BunshinRL, an agentic RL rollout system that exploits shared environment state and recurring tool calls through three techniques: memory sharing across environments, tool-prefix caching, and speculative tool execution. These techniques build on BunshinFork, a novel live whole-workspace forking mechanism for containers. BunshinFork reconstructs process trees, shares memory copy-on-write, and preserves file, communication, and GUI state, enabling cached-state reuse and rollback after incorrect speculation. We implement BunshinRL and BunshinFork through Linux kernel module and CRIU modifications. BunshinRL reduces peak memory by up to 43%, total memory consumption by up to 51%, and end-to-end rollout latency by 23%.
Figures & tables
Figure 1
Figure 3: Illustration of BunshinRL. Right shows two samples in a simplified rollout step; left shows the on-disk and in-memory workspace snapshot stores for tool call prefix. Reasoning steps’ generated tool call signatures are matched with the tool prefix tree. All initial matched tool calls are skipped until the first miss in tool signature (Tool 3 for R1 and Tool 4 for R2), at which point the corresponding snapshots are used to reconstruct the workspaces. As C3 is not cached in memory, Tool 3 is replayed for R2 and its delta write is added to the C0+C1+C2 snapshot. The samples execute actual Tool 3 for R1 and actual Tool 4 for R2, creating new delta CoW pages. After these calls finish, speculation starts from the resulting workspaces: Tool 4 for R1 and Tools 5 and 6 for R2. The Spec Tool Selector selects tool calls that are likely to happen from previous tool call history. When a reasoning step finishes, its output tool signature is compared with the speculative tool signature. For failed speculation, the system rolls back to the workspace snapshot before speculation.
Figure 4: BunshinFork architecture. A brief freeze captures a consistent workspace state. The live fork path reconstructs isolated clones that share unchanged pages copy-on-write and allocate private pages on writes. Retained snapshot holders support asynchronous serialization, allowing the source to resume without waiting for disk writes.
Figure 5: Linux Process Fork vs. BunshinFork Process-Tree Fork . Left: native fork() duplicates one process as a child of the source process. Right: BunshinFork creates a sibling container and reconstructs the whole process tree inside an independent namespace.
Figure 5Figure 6
Figure 12: Memory of Forks. Total memory of a source environment and its forks as the number of forks grows, for BunshinFork, CRIU, and AgentENV.