Fine-tuning large language models for vertical domains remains labor-intensive, requiring practitioners to curate data, configure training, and iteratively diagnose model behavior. Despite growing interest in autonomous machine learning and language agents, end-to-end LLM fine-tuning has not been systematically studied as an interactive agent task. We introduce FT-Dojo, an interactive benchmark environment for autonomous LLM fine-tuning, comprising 13 tasks across 5 domains. Rather than a new collection of static datasets, FT-Dojo standardizes a task interface, shared raw-data repository, sandboxed execution environment, structured feedback protocol, and held-out evaluation procedure. We further develop FT-Agent, a fine-tuning-oriented autonomous framework that uses structured iteration planning, fail-fast validation, and multi-level feedback analysis to refine data and training strategies. Experiments show that FT-Agent provides a strong initial baseline, achieving the best performance on 10 out of 13 tasks, with additional controlled comparisons against frontier agents, open-source planning backbones, and multi-run statistics supporting the main findings. Case studies show that agents can recover from failures through cumulative learning, while still exposing limitations in causal diagnosis and long-horizon planning. The implementation is available at https://github.com/microsoft/rd-agent.
Figures & tables
Figure 1: Overview of FT-Dojo architecture and agent interaction workflow. Agents operate within an isolated sandbox and interact with three core interfaces: Meta API I for task and system information, Data Repository D for training data, and Evaluator F that returns structured feedback. Agents iteratively query information, submit code and configurations, execute training, and refine based on evaluation feedback until achieving satisfactory performance.
Given a patent claim and candidate patents, identify which were cited as prior art for rejecting the claim
No CoT Q&A pairs only High complexity
Acc, Macro-F1
54028
100
Novelty Classification
Classify patent claims as allowed, §102 rejection (lacks novelty), or §103 rejection (obvious)
Acc, Macro-F1
136211
100
Paragraph Identification
Identify the most relevant prior art paragraph from candidates that supports claim rejection
Accuracy
64210
100
Chemistry
Molecular Understanding
Predict molecular properties from SMILES: functional group counting, scaffold extraction, equivalence
Structured CoT Expert validated
MAE, Tanimoto
6319
160
Molecule Editing
Modify molecular structures by adding, deleting, or substituting functional groups
Acc, Tanimoto
4497
50
Table 1: Task suite in FT-Dojo spanning 13 tasks across 5 domains. Data Quality column describes training data characteristics relevant for fine-tuning. Valid/Test indicates the size of both validation and test sets (identical for each task). See Appendix A for metric definitions and abbreviations.
Figure 2: Overview of FT-Agent. The agent iteratively explores configurations, learning from failed trials (red) to reach successful ones (green). Each iteration follows three stages: Strategy Proposal, Implementation & Validation, and Feedback Analysis.
Math
Patent Examination
Table QA
Finance
Chemistry
AIME
PAR4PC
NOC4PC
PI4PC
DA
FC
NR
Vis.
QA
Mol_Und
Mol_Edit
Mol_Opt
Reaction
Method
Acc ↑
F1 ↑
F1 ↑
Acc ↑
Acc ↑
Acc ↑
Acc ↑
P@1 ↑
Acc ↑
MAE ↓
TMS ↑
Acc ↑
Acc ↑
SR ↑
VS ↑
FTS ↑
Acc ↑
Qwen2.5-7B-Instruct
0.00
57.78
26.20
34.00
32.78
75.00
32.00
4.00
65.10
0.53
0.09
63.67
22.20
23.33
68.67
15.59
30.00
Manual SFT
0.00
67.61
33.39
28.00
28.80
70.83
43.00
8.00
71.68
0.65
0.00
36.67
0.00
9.67
31.00
0.00
10.00
OpenHands
0.00
50.51
28.44
44.00
23.22
66.67
40.00
24.00
64.73
0.65
0.10
68.00
40.00
31.00
80.67
35.59
60.00
FT-Agent
11.11 ± 3.82
67.61 ± 2.40
36.81 ± 4.13
49.00 ± 4.36
34.71 ± 0.28
81.25 ± 2.08
45.33 ± 1.15
29.33 ± 6.11
67.20 ± 0.31
0.23 ± 0.16
0.36 ± 0.06
67.55 ± 5.68
54.44 ± 2.94
34.44 ± 2.17
90.00 ± 4.41
30.92 ± 11.22
35.33 ± 3.06
Table 2: Main results comparison. We compare FT-Agent against baselines across 13 tasks in 5 domains. FT-Agent achieves the best performance on 10 out of 13 tasks. Bold indicates best performance. FT-Agent results are reported as mean ± std over 3 independent runs. Manual SFT is a constrained, resource-matched manual baseline without LLM assistance. See Appendix A for metric definitions and abbreviations.
Metric
OpenHands
FT-Agent
Avg. Loops
3.69
8.77
Avg. Improve Rate
17.31%
24.37%
Avg. Cost (iterative only)
$3.92
$5.73
Table 3: Exploration dynamics between OpenHands and FT-Agent.
Figure 3: Ablation study results. (a) Data scaling : Performance comparison between 2k and 5k training samples. (b) Planning backbone : FT-Agent variants with proprietary/open-source planners. (c) Target model scale : Performance across target sizes. Panel (b) shows FT-Agent variants only; cross-agent baselines are in Table 4 , with full ablations in Appendix Table 10 .
Method
Setting
AIME
PI4PC
Mol_Edit
Finance
Vis.
Avg.
Base Model
–
0.00
34.00
22.20
65.10
4.00
25.06
Manual SFT
LLM synthesis
20.00
28.00
50.00
67.57
20.00
37.11
OpenHands
GPT-5.2
0.00
44.00
40.00
64.73
24.00
34.55
Codex
GPT-5.2
6.67
42.00
40.00
65.63
26.00
36.06
Claude Code
Sonnet-4.6-thinking
0.00
51.00
44.00
68.32
36.00
39.86
FT-Agent
Qwen3.5-397B-A17B
6.67
24.00
34.00
67.16
28.00
31.97
Table 4: Five-task controlled comparison. Frontier agents, FT-Agent planning backbones, and an assisted-human reference on representative tasks. Scores are percentages; Avg. is over displayed tasks.
Figure 4: Contrasting Learning Trajectories. (a) On Chemistry, the agent demonstrates cumulative learning , recovering from failure and progressively refining its approach through domain tool integration and iterative optimization. (b) On Patent Classification, it exhibits shotgun debugging , cycling through advanced techniques without identifying the root cause of overfitting.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Task Category
Subtask
Total
Valid
Test
1. ChemCoTBench (Chemistry)
Mol_Und
Murcko_scaffold
40
20
20
equivalence
100
50
50
fg_count
100
50
50
ring_count
20
10
10
ring_system_scaffold
60
30
30
Appendix
Table 5: Detailed statistics of evaluation datasets across FT-Dojo benchmarks.
Method
AIME
PI4PC
Table Vis.
Finance QA
Mol_Edit
Manual SFT (original)
0.00
28.00
8.00
71.68
0.00
Manual SFT (w/ LLM synthesis)
20.00
28.00
20.00
67.57
50.00
FT-Agent
13.33
44.00
36.00
67.53
53.33
Appendix
Table 6: Manual SFT with LLM-assisted synthesis on five representative tasks. All scores are percentages and higher is better. FT-Agent values are representative single-run scores for this focused comparison.
Domain
Task
Metric
OpenHands
FT-Agent
Val
Test
Val
Test
Math
AIME
Acc
0.00
0.00
20.00
13.30
Patent Exam.
PAR4PC
Exact Match
33.00
34.00
54.00
50.00
Macro F1
50.93
50.51
75.43
70.34
NOC4PC
Acc
35.35
34.34
40.00
44.00
Macro F1
27.47
28.44
37.18
41.09
Appendix
Table 7: Full performance comparison on Validation and Test sets. This table supplements the main text by including all auxiliary metrics (e.g., Macro F1, Tanimoto, BLEU) and validation splits to demonstrate robustness. Bold indicates the best performance in each column (for error metrics like MAE, lower is better).
Task
OpenHands
FT-Agent
AIME
2
5
PAR4PC
3
6
Table QA FC
2
7
Mol_Opt
4
6
Average
2.75
6.0
Appendix
Table 8: Number of effective iterations (completing full training) within the 12-hour budget. FT-Agent achieves 2.2 × more effective iterations due to fail-fast validation.
Figure 5: Cross-domain data strategies autonomously discovered by FT-Agent. Left: for Numerical Reasoning, the agent mixes TableInstruct (90%) with out-of-domain DeepScaleR (10%). Right: for Mol_Opt, the agent combines three chemistry sub-tasks with task-informed proportions.
Figure 6: Visualization: training samples and Pass@1 over loops. Loop 8’s multi-stage filtering pipeline reduced training data from ∼ 1,800 to just 27 samples, causing Pass@1 to collapse from 20% to 4%.
Loop
Acc.
Strategy (Before)
Feedback (After)
0
34%
Gold-anchored rationale + LoRA
“accuracy improves from 29% baseline to 34%…eval loss improving early then degrading suggests overfitting”
“shortfall consistent with supervision/distribution mismatch…sampled difficulty still concentrates in very-easy cases”
5
32%
Quota-based difficulty sampler
“hard-case quota did not materialize (bucket ‘ ≥ 7’ ended up 0)…shift likely didn’t target hardest regimes”
Appendix
Table 9: PI4PC: Complete decision trajectory over 8 loops. Each row shows the agent’s strategy (before execution) and feedback (after execution). Despite diverse interventions, the agent returns to baseline performance. Note the hedging language in feedback (“likely,” “appears”) indicating correlation-based rather than causal reasoning.
Domain
Task (Metric)
7B Model Settings
3B Model Settings
Base
Default (2k)
Scaling-5k
w/ GPT-4o
Base
FT
Math
AIME25 (Acc)
0.00/0.00
20.00 / 13.33
6.67/ 13.33
6.67/ 13.33
0.00/0.00
6.67/0.00
Patent
PI4PC (Acc)
29.00/34.00
40.00 /44.00
36.00/ 48.00
38.00/29.00
19.00/21.00
31.00/34.00
Finance
QA (Acc)
73.00/65.10
75.00/67.53
75.55 /66.33
70.33/ 70.95
61.44/56.98
60.73/58.47
Chem
Mol_Edit (Acc)
34.40/22.20
62.22/53.33
63.33 / 58.89
62.22/50.00
13.44/14.44
53.93/54.55
Table
Vis (Pass@1)
8.00/4.00
20.00/ 36.00
20.00/28.00
24.00 /20.00
0.00/8.00
8.00/24.00
Appendix
Table 10: Ablation study and baseline comparison organized by model size (7B vs. 3B). Scores are reported as validation/test . The Default setting represents FT-Agent with Qwen2.5-7B (2k samples, GPT-5.2 backbone). Scaling-5k and w/ GPT-4o are ablations on top of the 7B Default. Bold indicates the best validation and best test performance per row independently.
Math
Patent Examination
Table QA
Finance
Chemistry
AIME
PAR4PC
NOC4PC
PI4PC
DA
FC
NR
Vis.
QA
Mol_Und
Mol_Edit
Mol_Opt
Reaction
Method
Acc ↑
F1 ↑
F1 ↑
Acc ↑
Acc ↑
Acc ↑
Acc ↑
P@1 ↑
Acc ↑
MAE ↓
TMS ↑
Acc ↑
Acc ↑
SR ↑
VS ↑
FTS ↑
Acc ↑
Qwen2.5-7B-Instruct
0
60.63
27.40
29.00
28.87
83.30
40.00
8.00
73.00
0.52
0.09
56.00
34.40
21.00
66.00
14.93
36.00
Manual SFT
0
68.00
27.11
22.00
34.14
72.92
45.00
20.00
73.14
0.44
0.00
46.00
2.23
12.67
31.67
0
10.00
OpenHands
0
50.93
27.47
47.00
33.78
79.17
45.00
8.00
74.40
0.54
0.08
59.00
55.56
34.67
81.67
29.97
64.00
FT-Agent
13.32 ± 6.67
67.50 ± 6.89
34.44 ± 4.82
38.67 ± 2.31
33.56 ± 3.16
82.64 ± 4.82
46.67 ± 4.62
16.00 ± 4.00
74.04 ± 1.10
0.24 ± 0.17
0.26 ± 0.08
62.78 ± 5.96
57.04 ± 5.01
36.22 ± 2.36
88.78 ± 6.38
30.64 ± 11.93
39.33 ± 3.06
Appendix
Table 11: Complete validation set results. We report validation performance across 13 tasks in 5 domains. FT-Agent results are reported as mean ± std over 3 independent runs. Bold indicates the best performance. ↑ : higher is better; ↓ : lower is better. Abbreviations: DA/FC/NR = Data Analysis/Fact Checking/Num. Reasoning; P@1 = Pass@1; MAE = Mean Absolute Error (for FG/Ring Count); TMS = Tanimoto Similarity; SR = Success Rate; VS = Valid SMILES; FTS = Fingerprint Similarity.
While Large Language Models (LLMs) have empowered AI research agents to perform isolated scientific tasks, automating complex, real-world workflows, such as LLM training, remains a significant challenge. In this paper, we introduce TREX, a multi-agent system that automates the entire LLM training life-cycle. By orchestrating collaboration between two core modules-the Researcher and the Executor-the system seamlessly performs requirement analysis, open-domain literature and data research, formulation of training strategies, preparation of data recipes, and model training and evaluation. The multi-round experimental process is modeled as a search tree, enabling the system to efficiently plan exploration paths, reuse historical results, and distill high-level insights from iterative trials. To evaluate the capability of automated LLM training, we construct FT-Bench, a benchmark comprising 10 tasks derived from real-world scenarios, ranging from optimizing fundamental model capabilities to enhancing performance on domain-specific tasks. Experimental results demonstrate that the TREX agent consistently optimizes model performance on target tasks.
Training language models (LMs) remains a highly human-intensive process, even as frontier language model agents become increasingly capable at software engineering and other long-horizon tasks. A central challenge is that autonomous post-training is not just a coding problem: it requires the agent to repeatedly plan iterations, construct benchmark-aligned data, run stable training jobs, evaluate checkpoints, and preserve experiment state across many hours of interaction. We present AutoTrainess, a LM agent that exposes these operations as a repository of agent-computer interfaces for planning, data preparation, training, evaluation, and logging. Rather than leaving the agent to operate in a raw CLI environment with an underspecified action space, AutoTrainess externalizes prior human experience as explicit workflows, rules, and execution constraints that guide the agent toward effective and reliable training behavior. On PostTrainBench, AutoTrainess consistently outperforms CLI-only baselines, achieving 26.94 average score with GPT-5.4 (Codex) versus 23.21 for CLI-only. It also generalizes across models and harnesses, improving DeepSeek-V4-Flash (OpenCode) from 12.13 to 19.58.
Zhaojian Yu, Penghao Yin, Shuzheng Gao +3
1Tsinghua University · 2The Chinese University of Hong Kong · 3Simple Agent Lab
Reinforcement fine-tuning (RFT) has shown promise for training LLM agents to perform multi-turn decision-making based on environment feedback. However, most existing evaluations remain largely in-domain: training and testing are conducted in the same environment or even on the same tasks. In real-world deployment, agents may operate in unseen environments with different background knowledge, observation spaces, and action interfaces. To characterize the generalization profile of RFT under such shifts, we conduct a systematic study along three axes: (1) within-environment generalization across task difficulty, (2) cross-environment transfer to unseen environments, and (3) sequential multi-environment training to quantify transfer and forgetting. Our results show that RFT generalizes well across task difficulty within an environment, but exhibits weaker transfer to unseen environments, which correlates with shifts in both semantic priors and observation/action interfaces. In contrast, sequential training yields promising downstream gains with minimal upstream forgetting, and mixture training across environments improves the overall balance. We further provide detailed analyses and deeper insights, and hope our work helps the community develop and deploy generalizable LLM agents.
Zhiheng Xi, Xin Guo, Jiaqi Liu +11
Fudan University · Shanghai Artificial Intelligence Laboratory · Meituan