Organizations: Beijing University of Posts and Telecommunications · Alibaba Group · Minzu University of China · Beijing Institute of Technology · Center for Research on Intelligent Perception and Computing, NLPR, CASIA
LLM-based agents can automate model post-training, accelerating progress toward recursive self-improvement. Despite their promise, existing approaches remain under-explored and face two main challenges: (1) strategy lock-in that limits exploration beyond the initial strategy; and (2) coupled execution that obscures the effects of individual modifications. To address these challanges, we propose TrainExplorer, a multi-agent framework for autonomous post-training, that incorporates strategy planning and execution decoupling, to achieve broaden strategy exploration. First, strategy planning draws on scientific literature to construct a candidate search space of training strategies. Then, decoupled execution enables single-component interventions and attribution to guide strategy exploration. Extensive experiments on PostTrainBench demonstrate that TrainExplorer achieves a weighted aggregate score of 39.02, outperforming Claude Code using the same LLM by 7.32 and surpassing GPT-5.6 Sol-powered Codex, which scores 36.23. Code will be available.
Figures & tables
Figure 1: Comparison between a general autonomous agent and TrainExplorer. Top: The agent exhibits strategy lock-in by restricting exploration to refinements of its initial SFT strategy, and coupled execution by introducing CoT data examples and enabling training packing in the same experiment. Bottom: TrainExplorer uses strategy planning to construct a candidate strategy space and execution decoupling to enable single-component interventions and effect attribution.
Figure 2: Overview of TrainExplorer, a multi-agent framework for autonomous LLM post-training that integrates strategy planning and execution decoupling. First, strategy planning draws on a structured knowledge base of scientific literature to construct a candidate strategy space. Then, the Idea, Builder, and Critic form an iterative loop for ideation, experimentation, and critique. Within this loop, execution decoupling enables single-component interventions and attributes their effects through controlled comparisons to guide subsequent exploration.
Method
Model
AIME
ArenaHard
BFCL
GPQA
GSM8K
HealthBench
HumanEval
AVG
Base Models
1.67
1.26
1.50
8.48
20.43
9.49
12.81
7.53
Official Instruct Models
29.17
70.21
85.00
36.22
87.00
43.32
71.49
51.14
General CLI Agent
Codex
GPT-5.6 Sol
7.50
26.40
94.38
30.14
69.48
27.39
63.95
36.23
Claude Code
Opus 4.8
10.83
36.09
47.13
28.35
68.87
31.68
47.33
32.90
Claude Code
Kimi K3
11.67
36.71
9.67
25.23
76.32
31.84
62.30
31.96
Table 1: Performance (%) of post-training agents on PostTrainBench, averaged across four base models. Base and official instruct models are included as references. AVG denotes the weighted aggregate score across seven tasks. Bold indicates the best score in each column among methods using GLM-5.2 as the agent LLM.
Planning
Decoupling
AIME
ArenaHard
BFCL
GPQA
GSM8K
HealthBench
HumanEval
AVG
✗
✓
1.11
23.80
93.00
28.72
73.89
20.22
57.32
32.51
✓
✗
3.33
17.24
91.00
29.99
75.92
25.04
62.80
34.22
✓
✓
6.67
27.66
94.00
32.07
81.05
27.36
65.24
37.77
Table 2: Ablation of strategy planning (Planning) and execution decoupling (Decoupling) in TrainExplorer using Qwen3-1.7B-Base. ✓/✗ indicates whether each module is enabled. AVG denotes the weighted aggregate score across seven tasks. Bold indicates the best score in each column.
Figure 3: Performance of TrainExplorer and baseline agents on AIME 2025, GPQA Main, and HealthBench under varying time budgets. Curves show the best evaluation scores achieved within each time budget.
Figure 4: Strategy switch rates of TrainExplorer and Claude Code across four base models. Red annotations indicate differences in percentage points.
Figure 5: Tool-use counts of TrainExplorer on AIME 2025, GPQA Main, and HealthBench using Qwen3-1.7B-Base, grouped by behavior type and two-hour intervals.
Figure 6: Search trajectories of TrainExplorer and Claude Code on HumanEval. The curves show best-so-far evaluation scores. The shaded region highlights Claude Code’s continued use of SFT alone after reaching its best score, without further improvement.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Method
Model
AIME
ArenaHard
BFCL
GPQA
GSM8K
HealthBench
HumanEval
AVG
Base Models
0.00
0.91
0.00
14.06
12.66
7.54
7.93
6.66
Official Instruct Models
26.67
50.00
94.00
35.49
88.48
44.92
68.90
49.41
General CLI Agent
Codex
GPT-5.6 Sol
5.00
27.47
94.50
29.13
59.82
26.89
66.77
34.85
Claude Code
Opus 4.7
6.67
31.26
91.33
29.32
50.47
17.03
42.28
30.08
Claude Code
Kimi K3
4.44
32.37
0.00
27.83
77.23
32.47
62.60
30.03
Appendix
Table 3: Performance (%) of post-training agents on PostTrainBench using Qwen3-1.7B-Base as the base model. Base and official instruct models are included as references. AVG denotes the weighted aggregate score across seven tasks. Bold indicates the best score in each column among methods using GLM-5.2 as the agent LLM.
Method
Model
AIME
ArenaHard
BFCL
GPQA
GSM8K
HealthBench
HumanEval
AVG
Base Models
3.33
3.42
0.00
13.39
41.85
13.38
36.59
14.34
Official Instruct Models
53.33
86.84
95.00
44.64
93.78
52.72
77.44
63.75
General CLI Agent
Codex
GPT-5.6 Sol
10.00
32.86
94.50
34.04
80.06
27.07
82.62
41.18
Claude Code
Kimi K3
22.22
48.25
32.67
32.22
86.43
29.51
79.47
41.03
Claude Code
Opus 4.8
23.33
48.67
0.00
31.47
89.88
34.06
69.21
38.78
Appendix
Table 4: Performance (%) of post-training agents on PostTrainBench using Qwen3-4B-Base as the base model. Base and official instruct models are included as references. AVG denotes the weighted aggregate score across seven tasks. Bold indicates the best score in each column among methods using GLM-5.2 as the agent LLM.
Method
Model
AIME
ArenaHard
BFCL
GPQA
GSM8K
HealthBench
HumanEval
AVG
Base Models
3.33
0.42
0.00
4.91
21.08
0.00
6.10
4.52
Official Instruct Models
26.67
49.20
84.00
33.26
82.18
29.58
70.12
44.81
General CLI Agent
Codex
GPT-5.6 Sol
15.00
13.57
93.50
30.47
74.98
27.51
56.10
36.48
Claude Code
Opus 4.8
16.67
37.19
47.00
27.68
51.52
32.63
28.96
30.77
Claude Code
Kimi K3
18.89
28.49
0.00
16.07
76.40
34.92
58.94
30.30
Appendix
Table 5: Performance (%) of post-training agents on PostTrainBench using SmolLM3-3B-Base as the base model. Base and official instruct models are included as references. AVG denotes the weighted aggregate score across seven tasks. Bold indicates the best score in each column among methods using GLM-5.2 as the agent LLM.
Method
Model
AIME
ArenaHard
BFCL
GPQA
GSM8K
HealthBench
HumanEval
AVG
Base Models
0.00
0.29
6.00
1.56
6.14
17.04
0.61
4.60
Official Instruct Models
10.00
94.80
67.00
31.47
83.55
46.06
69.51
46.58
General CLI Agent
Claude Code
Opus 4.8
0.00
47.39
50.50
28.35
69.67
34.77
52.13
32.87
Codex
GPT-5.6 Sol
0.00
31.68
95.00
26.90
63.04
28.08
50.30
32.40
Codex
GPT-5.5
0.00
24.45
86.50
27.46
50.83
21.62
28.05
26.55
Appendix
Table 6: Performance (%) of post-training agents on PostTrainBench using Gemma-3-4B-PT as the base model. Base and official instruct models are included as references. AVG denotes the weighted aggregate score across seven tasks. Bold indicates the best score in each column among methods using GLM-5.2 as the agent LLM.
Large language model post-training adapts pretrained models into more useful, reliable, and specialized assistants through supervised and preference-based learning.
AI agents have become surprisingly proficient at software engineering over the past year, largely due to improvements in reasoning capabilities. This raises a deeper question: can these systems extend their capabilities to automate AI research itself? In this paper, we explore post-training, the critical phase that turns base LLMs into useful assistants. We introduce PostTrainBench to benchmark how well LLM agents can perform post-training autonomously under bounded compute constraints (10 hours on one H100 GPU). We ask frontier agents (e.g., Claude Code with Opus 4.6) to optimize the performance of a base LLM on a particular benchmark (e.g., Qwen3-4B on AIME). Importantly, we do not provide any predefined strategies to the agents and instead give them full autonomy to find necessary information on the web, run experiments, and curate data. We find that frontier agents make substantial progress but generally lag behind instruction-tuned LLMs from leading providers: 23.2% for the best agent vs. 51.1% for official instruction-tuned models. However, agents can exceed instruction-tuned models in targeted scenarios: GPT-5.1 Codex Max achieves 89% on BFCL with Gemma-3-4B vs. 67% for the official model. We also observe several failure modes worth flagging. Agents sometimes engage in reward hacking: training on the test set, downloading existing instruction-tuned checkpoints instead of training their own, and using API keys they find to generate synthetic data without authorization. These behaviors are concerning and highlight the importance of careful sandboxing as these systems become more capable. Overall, we hope PostTrainBench will be useful for tracking progress in AI R&D automation and for studying the risks that come with it. Website and code are available at https://posttrainbench.com/.
Ben Rank, Hardik Bhatnagar, Ameya Prabhu +4
ELLIS Institute Tübingen · Max Planck Institute for Intelligent Systems · Tübingen AI Center +2
Training language models (LMs) remains a highly human-intensive process, even as frontier language model agents become increasingly capable at software engineering and other long-horizon tasks. A central challenge is that autonomous post-training is not just a coding problem: it requires the agent to repeatedly plan iterations, construct benchmark-aligned data, run stable training jobs, evaluate checkpoints, and preserve experiment state across many hours of interaction. We present AutoTrainess, a LM agent that exposes these operations as a repository of agent-computer interfaces for planning, data preparation, training, evaluation, and logging. Rather than leaving the agent to operate in a raw CLI environment with an underspecified action space, AutoTrainess externalizes prior human experience as explicit workflows, rules, and execution constraints that guide the agent toward effective and reliable training behavior. On PostTrainBench, AutoTrainess consistently outperforms CLI-only baselines, achieving 26.94 average score with GPT-5.4 (Codex) versus 23.21 for CLI-only. It also generalizes across models and harnesses, improving DeepSeek-V4-Flash (OpenCode) from 12.13 to 19.58.
Zhaojian Yu, Penghao Yin, Shuzheng Gao +3
1Tsinghua University · 2The Chinese University of Hong Kong · 3Simple Agent Lab
Large language model (LLM) agents can now post-train an LLM end-to-end, raising the prospect of recursive self-improvement (RSI). Yet this progress is measured by aggregate benchmark scores, which cannot tell whether an agent executes a fixed plan well or strategically revises the plan when it fails. We separate these two capabilities: execution-level capability, iterating within an established training strategy, and strategy-level capability, revising that strategy as experimental evidence accumulates. Analyzing 1,338 post-training trajectories of frontier agents, we find that agents reliably execute post-training but lock into a default strategy, which follows the agent rather than the task, and only 2.1% of transitions between adjacent training runs ever change strategy. We then test whether the agent lacks experience, reasoning, or the decision to switch. (1) Experience improves execution but not the strategy. (2) Additional reasoning compute yields front-loaded gains on easier tasks but refines, rather than revises, the committed strategy. (3) Human review before training changes which strategy the agent locks into, not whether it locks in, whereas a single mid-run instruction outperforms the agent's own continuation by up to 17.44 points under the same budget. In conclusion, what the agent lacks is the decision to reopen a committed strategy and try another one. Realizing RSI therefore calls for interaction protocols and training signals that make strategy revision an explicit, rewarded decision.
Joy Jia Yin Lim, Xin Huang, Hao Peng +5
Tsinghua University · University of Electronic Science and Technology of China · Renmin University of China