cs.CLOct 22, 2026

IQuest-Q1: Advancing Foundation Capabilities for Agentic CLI Systems

Authors: A. Jian Yang, An Huang, Bo Dan, Chen Mu, Chengran Yang, Chuan Shi, Feng Chang, Hongxin Li, 

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

We present IQuest-Q1, a sparse Mixture-of-Experts (MoE) agentic foundation model with 320B total parameters (15B activated). IQuest-Q1 is developed through a three-stage process: (1) stable and efficient pre-training, combining Stable SplitNorm, Muon, and muP with a two-stage curriculum whose second stage increases the coding-data mixture, preserving a wide stability margin between safe and optimal learning rates; (2) fine-grained mid-training, expanding the data mixture across domains and formats and progressively extending the context; and (3) multi-scaffold with multi-version post-training, combining SFT, domain-specialized RL, and MOPD across various scaffolds to integrate complementary training signals and strengthen agentic coding capabilities. Furthermore, model merging combines different experts and checkpoints from different stages while mitigating conflicts among their parameter updates. The resulting IQuest-Q1 demonstrates capabilities in both coding and general tasks. We release IQuest-Q1 as an open-source model to support community research and development.

Figures & tables

Figure 1: IQuest-Q1 performance across benchmarks. DeepSeek-V4-Flash/Pro refer to the official 0731/0813 releases, respectively. Humanity’s Last Exam is reported without tools. IQuest-CLIBench is our proprietary built-in benchmark for evaluating human-agent collaboration and user experience (see Section 5 for details).
Figure 2: Training pipeline of IQuest-Q1.
Figure 3: Overall architecture of IQuest-Q1.
Figure 4: Scaling efficiency and stability of IQuest-Q1.
ConfigurationValue
Total Parameters320320 B
Activated Parameters1515 B
Transformer Layers8888
Hidden Dimension3,0723{,}072
Attention Heads (Q/KV)48/848/8
Head Dimension128128
Table 1: Summary of the main model architecture parameters.
Figure 5: Overview of coding-agent task instances and their environments, verifiers, and construction methods.
Figure 6: IQuest-Q1 Post-Training Pipeline.
BenchmarkIQuest-Q1GLM-5.3 FlashDS-V4 Flash-0731Tencent Hy3MiniMax M3Claude Opus-5GPT-5.6 Sol
Coding Capabilities
DeepSWE v1.164.663.454.428.0-73.773.0
SWE-bench Multilingual80.379.7*78.7*75.875.0*89.574.1
SWE-Bench Pro57.559.4*57.3*57.959.079.264.6
NL2Repo-Bench63.056.354.245.642.175.356.8
ProgramBench60.958.5*62.1*47.4*33.8*82.377.6
Table 2: Performance across coding and general capability benchmarks. Scores are reported at the highest available reasoning effort for each model. DS denotes DeepSeek. ProgramBench reports raw pass rates. PaperBench is evaluated in the CodeDev setting. IQuest-CLIBench ‡ is our proprietary built-in benchmark for evaluating human-agent collaboration and user experience (see Section 5 for details). Results with * are from our own testing.
Figure 7: Representative interactive frontend applications generated by IQuest-Q1 from natural-language specifications, covering game worlds, scene editing, and scientific exploration.
Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 8: Representative training-loss trajectories for the model under stable, warning, and collapsed training regimes.
ModuleInitialization Std.Learning Rate
EmbeddingO(1)\mathcal{O}(1)O(1)\mathcal{O}(1)
LinearO(d−1/2)\mathcal{O}(d^{-1/2})O(d−1/2L−1/4)\mathcal{O}(d^{-1/2}L^{-1/4})
RMSNorm–O(L−1/4)\mathcal{O}(L^{-1/4})
LM headO(d−1)\mathcal{O}(d^{-1})O(d−1)\mathcal{O}(d^{-1})
Appendix
Table 3: Width- and depth-dependent μ\mu P scaling rules used for different modules. Here, dd denotes the hidden width and LL denotes the model depth.
Figure 9: Optimal learning rate transfer across model widths and depths under μ\mu P scaling.
Figure 10: Overlapping halo exchange with partial attention computation.
Figure 11: Illustration of recursive multi-token prediction. Here, LL denotes the input sequence length and tt denotes the drafting step.
More in this topic

LLM Agent Training

Large language model agent training teaches models to plan, use tools, and learn from multi-step interaction with environments.

See all 581 papers

Explore similar work

CardsList

  1. IQuest-Coder-V1 Technical Report

    Mar 17, 2026Jian Yang, Wei Zhang, Shawn Guo +35Recurrent Language ModelsCode Language Models

  2. KAT-Coder-V2 Technical Report

    Mar 29, 2026Fengxiang Li, Han Zhang, Haoyang Huang +43AI Coding AgentsAgentic Code Generation

  3. The MiniMax-M2 Series: Mini Activations Unleashing Max Real-World Intelligence

    May 26, 2026MiniMax, :, Aili Chen +204Mixture-of-Experts Language ModelsAgentic RL