cs.CLOct 22, 2026

IQuest-Q1: Advancing Foundation Capabilities for Agentic CLI Systems

Authors: A. Jian Yang, An Huang, Bo Dan, Chen Mu, Chengran Yang, Chuan Shi, Feng Chang, Hongxin Li, 

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

We present IQuest-Q1, a sparse Mixture-of-Experts (MoE) agentic foundation model with 320B total parameters (15B activated). IQuest-Q1 is developed through a three-stage process: (1) stable and efficient pre-training, combining Stable SplitNorm, Muon, and muP with a two-stage curriculum whose second stage increases the coding-data mixture, preserving a wide stability margin between safe and optimal learning rates; (2) fine-grained mid-training, expanding the data mixture across domains and formats and progressively extending the context; and (3) multi-scaffold with multi-version post-training, combining SFT, domain-specialized RL, and MOPD across various scaffolds to integrate complementary training signals and strengthen agentic coding capabilities. Furthermore, model merging combines different experts and checkpoints from different stages while mitigating conflicts among their parameter updates. The resulting IQuest-Q1 demonstrates capabilities in both coding and general tasks. We release IQuest-Q1 as an open-source model to support community research and development.

Figures & tables

Figure 1: IQuest-Q1 performance across benchmarks. DeepSeek-V4-Flash/Pro refer to the official 0731/0813 releases, respectively. Humanity’s Last Exam is reported without tools. IQuest-CLIBench is our proprietary built-in benchmark for evaluating human-agent collaboration and user experience (see Section 5 for details).
Figure 2: Training pipeline of IQuest-Q1.
Figure 3: Overall architecture of IQuest-Q1.
Figure 4: Scaling efficiency and stability of IQuest-Q1.
ConfigurationValue
Total Parameters320320 B
Activated Parameters1515 B
Transformer Layers8888
Hidden Dimension3,0723{,}072
Attention Heads (Q/KV)48/848/8
Head Dimension128128
Table 1: Summary of the main model architecture parameters.
Figure 5: Overview of coding-agent task instances and their environments, verifiers, and construction methods.
Figure 6: IQuest-Q1 Post-Training Pipeline.
BenchmarkIQuest-Q1GLM-5.3 FlashDS-V4 Flash-0731Tencent Hy3MiniMax M3Claude Opus-5GPT-5.6 Sol
Coding Capabilities
DeepSWE v1.164.663.454.428.0-73.773.0
SWE-bench Multilingual80.379.7*78.7*75.875.0*89.574.1
SWE-Bench Pro57.559.4*57.3*57.959.079.264.6
NL2Repo-Bench63.056.354.245.642.175.356.8
ProgramBench60.958.5*62.1*47.4*33.8*82.377.6
Table 2: Performance across coding and general capability benchmarks. Scores are reported at the highest available reasoning effort for each model. DS denotes DeepSeek. ProgramBench reports raw pass rates. PaperBench is evaluated in the CodeDev setting. IQuest-CLIBench ‡ is our proprietary built-in benchmark for evaluating human-agent collaboration and user experience (see Section 5 for details). Results with * are from our own testing.
Figure 7: Representative interactive frontend applications generated by IQuest-Q1 from natural-language specifications, covering game worlds, scene editing, and scientific exploration.
Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 8: Representative training-loss trajectories for the model under stable, warning, and collapsed training regimes.
ModuleInitialization Std.Learning Rate
EmbeddingO(1)\mathcal{O}(1)O(1)\mathcal{O}(1)
LinearO(d−1/2)\mathcal{O}(d^{-1/2})O(d−1/2L−1/4)\mathcal{O}(d^{-1/2}L^{-1/4})
RMSNorm–O(L−1/4)\mathcal{O}(L^{-1/4})
LM headO(d−1)\mathcal{O}(d^{-1})O(d−1)\mathcal{O}(d^{-1})
Appendix
Table 3: Width- and depth-dependent μ\mu P scaling rules used for different modules. Here, dd denotes the hidden width and LL denotes the model depth.
Figure 9: Optimal learning rate transfer across model widths and depths under μ\mu P scaling.
Figure 10: Overlapping halo exchange with partial attention computation.
Figure 11: Illustration of recursive multi-token prediction. Here, LL denotes the input sequence length and tt denotes the drafting step.
More in this topic

LLM Agent Training

Large language model agent training teaches models to plan, use tools, and learn from multi-step interaction with environments.

See all 581 papers

Explore similar work

CardsList
Mar 17, 2026

IQuest-Coder-V1 Technical Report

In this report, we introduce the IQuest-Coder-V1 series-(7B/14B/40B/40B-Loop), a new family of code large language models (LLMs). Moving beyond static code representations, we propose the code-flow multi-stage training paradigm, which captures the dynamic evolution of software logic through different phases of the pipeline. Our models are developed through the evolutionary pipeline, starting with the initial pre-training consisting of code facts, repository, and completion data. Following that, we implement a specialized mid-training stage that integrates reasoning and agentic trajectories in 32k-context and repository-scale in 128k-context to forge deep logical foundations. The models are then finalized with post-training of specialized coding capabilities, which is bifurcated into two specialized paths: the thinking path (utilizing reasoning-driven RL) and the instruct path (optimized for general assistance). IQuest-Coder-V1 achieves state-of-the-art performance among competitive models across critical dimensions of code intelligence: agentic software engineering, competitive programming, and complex tool use. To address deployment constraints, the IQuest-Coder-V1-Loop variant introduces a recurrent mechanism designed to optimize the trade-off between model capacity and deployment footprint, offering an architecturally enhanced path for efficacy-efficiency trade-off. We believe the release of the IQuest-Coder-V1 series, including the complete white-box chain of checkpoints from pre-training bases to the final thinking and instruction models, will advance research in autonomous code intelligence and real-world agentic systems.

Jian Yang, Wei Zhang, Shawn Guo +35

Recurrent Language ModelsCode Language ModelsLLM Code GenerationLanguage ModelingAgentic Code GenerationSoftware Engineering Agents
AskPDF
Mar 29, 2026

KAT-Coder-V2 Technical Report

We present KAT-Coder-V2, an agentic coding model developed by the KwaiKAT team at Kuaishou. KAT-Coder-V2 adopts a "Specialize-then-Unify" paradigm that decomposes agentic coding into five expert domains - SWE, WebCoding, Terminal, WebSearch, and General - each undergoing independent supervised fine-tuning and reinforcement learning, before being consolidated into a single model via on-policy distillation. We develop KwaiEnv, a modular infrastructure sustaining tens of thousands of concurrent sandbox instances, and scale RL training along task complexity, intent alignment, and scaffold generalization. We further propose MCLA for stabilizing MoE RL training and Tree Training for eliminating redundant computation over tree-structured trajectories with up to 6.2x speedup. KAT-Coder-V2 achieves 79.6% on SWE-bench Verified (vs. Claude Opus 4.6 at 80.8%), 88.7 on PinchBench (surpassing GLM-5 and MiniMax M2.7), ranks first across all three frontend aesthetics scenarios, and maintains strong generalist scores on Terminal-Bench Hard (46.8) and tau^2-Bench (93.9). Our model is publicly available at https://streamlake.com/product/kat-coder.

Fengxiang Li, Han Zhang, Haoyang Huang +43

AI Coding AgentsAgentic Code GenerationLLM Agent TrainingRL for Code Generation
AskPDF
May 26, 2026

The MiniMax-M2 Series: Mini Activations Unleashing Max Real-World Intelligence

We introduce the MiniMax-M2 series, a family of Mixture-of-Experts language models built around the principle that mini activations can unleash maximum real-world intelligence. The flagship M2 contains 229.9B total parameters with only 9.8B activated per token. Designed end-to-end for agentic deployment, the M2 series rests on three components: (i) agent-driven data pipelines producing large-scale, verifiable trajectories across agentic coding and agentic cowork, each grounded in an executable workspace and an artifact-aligned reward; (ii) Forge, a scalable agent-native RL system that adapts to long-horizon agent trajectories, paired with windowed-FIFO scheduling, prefix-tree merging, inference optimization, and a clean training-inference-agent decoupling that supports both white-box and black-box agents; (iii) the latest M2.7 checkpoint takes an early step toward self-evolution -- autonomously debugging training runs and modifying its own scaffold. Across M2 through M2.7, this combination translates a mini-activation footprint into frontier-tier performance on agentic coding, deep search, office-task, and reasoning benchmarks.

MiniMax, :, Aili Chen +204

Mixture-of-Experts Language ModelsAgentic RLAI Coding AgentsLLM Agent TrainingLong-Horizon LLM AgentsAI Agent Architecture
AskPDF