cs.CLOct 8, 2026
SaveMiMo-V2.6: Scaling Reinforcement Learning Towards Self-Improvement
Abstract
Reinforcement learning (RL) is the central training paradigm for advancing large foundation models towards self-improvement. This report introduces the MiMo-V2.6 series, an omni-modal family that pushes the frontier of model intelligence by scaling RL compute. Prior to RL, we conduct mid-training on a broad multimodal corpus to provide ample exploration space, and build a solid infrastructure on the pretrained hybrid-SWA architecture to support subsequent scale-up. We scale RL compute along three dimensions: (1) larger batches and higher throughput, with an asynchronous training that consumes 1,568 samples and 2.7-3.7B tokens per step at context lengths of up to 1M; (2) more diverse and complex environments, spanning code, general, visual, and cyber domains under a mixture of agent harnesses; and (3) more grader compute, via groupwise agentic grading that yields more accurate reward signals for long-horizon tasks and steers the model towards shorter, more token-efficient solutions. To keep training stable at scale, we freeze the MoE router and establish a multi-layer defense against reward hacking. We further build infrastructure for mixed-task agentic RL, including a unified trajectory representation, high-concurrency multi-framework rollout, decoupled control and data planes, and training-inference consistency. We open-source the training dynamics, RL environments, and RL framework to facilitate reproduction and further research on scaled RL and model self-improvement.
Figures & tables
Figure 2 : Overall architecture of MiMo-V2.6. Audio, visual, and text inputs are mapped into a shared token sequence and processed by the MiMo Hybrid-SWA backbone, followed by the language-modeling head and multi-token prediction (MTP) blocks.
| Block | Configuration | MiMo-V2.6-Flash | MiMo-V2.6-Pro |
| Main Block | Layers (Total/SWA/GA) | ||
| Hidden Size | |||
| SWA Heads (Q/KV) | |||
| Sliding Window Size | |||
| GA Heads (Q/KV) | |||
| Head Dimensions (QK/V) |
Table 1: Detailed model configurations of MiMo-V2.6-Flash and MiMo-V2.6-Pro. Main block layer counts exclude MTP modules. Encoder parameter counts include input embeddings but exclude projectors. The audio tokenizer encoder parameter count excludes EMA codebooks.
Figure 3 : Left: average@3 score on DeepSWE v1.1 versus cumulative RL cost. Right: training, rollout, and grader cost shares for MiMo-V2.6-Pro and MiMo-V2.6-Flash.
Figure 4 : The overall code agent task scaling pipeline. We build tasks from diverse sources and go through rigorous assessment stages to ensure the supervision is accurate and robust.
Figure 5 : The overall general-agent environment and verifiable-task synthesis pipeline. We first construct realistic, resettable environments from real-world files and software mocks. Agents then explore these environments and synthesize tasks with verifiable rubrics.
| Pattern and shortcut | Illustrative case |
| Install and read Uses a newer release of the target package as an answer key, copying a fix absent from the assigned checkout. | Task (pytest): Fix Windows conftest.py imports broken by path lowercasing. Thinking: “Let me check the pytest changelog or GitHub to see if there’s a more recent fix.” Action: pip install pytest==5.4.3 ; inspect the installed source. |
| Fetch upstream source Downloads an upstream file or patch that already contains the fix, exposing the code changes needed to pass the tests. | Task (Astropy): Correct the misleading error when a required TimeSeries column is removed. Thinking: “Let me just look at the file directly from the GitHub raw URL.” Action: curl .../astropy/timeseries/core.py |
| Clone upstream Reads a newer upstream checkout to reconstruct the published fix missing from the assigned historical commit. | Task (Matplotlib): Stop ax.clear() from restoring hidden ticks and labels on shared axes. Thinking: “Let me directly fetch and inspect the relevant files from the latest matplotlib.” Action: git clone .../matplotlib.git ; inspect axis.py . |
| Look up a solution Searches issue discussions, pull requests, or linked commits for the original solution and uses it to guide the patch. | Task (Django): Fix MultiValueField ignoring required subfields. Thinking: “Let me get more info - changesets and the fix that was applied.” Action: Read the change history of Django ticket #29205. |
| Probe versions Finds a newer release to download and compare for its fix. The probe is a precursor to copying the solution. | Task (Sphinx): Remove the spurious return type from class documentation. Thinking: “Let me look for the fix. The issue is likely fixed in later versions.” Action: pip index versions sphinx |
Table 2: Representative reward-hacking cases in repository-repair tasks. In this setting, retrieving existing fixes outside the assigned checkout bypasses the intended repair task. Task queries are summarized, thinking excerpts are quoted verbatim, and actions are abbreviated. Each row comes from a single rollout.
Figure 6 : Reward hacking prevention and monitoring. (a) Environment preparation and iterative hack-agent screening before training, while offline trajectory audits monitor hacking behaviors throughout training. Findings from both stages are used for environment improvements. (b) Top: the fraction of environments found hackable over cleanup rounds. Bottom: the fraction of trajectories with detected reward hacking throughout our final RL run.
Figure 7 : Groupwise agentic grading for code-agent RL. (a) Groupwise reward synthesis combines test outcomes with per-rollout scores from precomputed task-specific rubrics. (b) Groupwise advantage redistribution compares trajectories online, resets confirmed-hack rewards to zero, and redistributes sequence-level advantages. Bars schematically illustrate the redistribution of positive advantage from lower- to higher-quality passing solutions.
Figure 8 : Comparison with and without online groupwise advantage redistribution on DeepSWE v1.1, using code-only RL with MiMo-V2.6-Flash (training batch size 128). Pass rate is avg@n with ; we also report the mean total number of turns and total token length.
Figure 9 : Benchmark scores (top) and total token counts in thousands (bottom) during RL training on DeepSWE v1.1, AutomationBench v1.0.6, and our in-house MiMo Visual Coding benchmark. Light and dark orange curves represent Flash and Pro, respectively.
Figure 10 : Pass@1 on DeepSWE v1.1 during Multi-Harness Training, evaluated with (a) training harnesses and (b) held-out harnesses. Lighter curves show individual harness results; thick orange curves show the mean within each panel.
Figure 11 : MiMo-V2.6-Pro expert-load balance at decoder layer 9 (384 experts) during RL, comparing runs with and without freezing the router. (a) Coefficient of variation of expert load. (b) Peak load factor (max/mean). (c) Fraction of cold experts with load below the mean.
Figure 12 : MiMo-V2.6-Pro and MiMo-V2.6-Flash timelines over 30 training steps, aligned by elapsed time. Light orange denotes completed steps; other colors denote failure and recovery intervals by cause.
Figure 13 : Overview of MiMo MOPD2. (a) Domain-specialized teachers are trained with MixRL on verifiable tasks or with SFT on synthetic demonstrations for open-domain tasks. (b) Standard MOPD uses RL teachers to supervise full student rollouts. (c) Prefix-Conditioned OPD reuses trajectories from teacher rollouts (Teacher-Prefix OPD) or SFT data (SFT-Prefix OPD). A source trajectory with assistant-turn decision points yields complete history prefixes , each of which can initialize a separate student-generated turn for token-level distillation by the relevant domain teacher. SFT data provide prefix contexts rather than fixed continuation targets.
Table 3: Comparison of MiMo-V2.6 with previous-generation and frontier models on agentic benchmarks.