Pioneer Agent: Continual Improvement of Small Language Models in Production

Dhruv Atreja; Julia White; Nikhil Nayak; Kelton Zhang; Henrijs Princis; George Hurn-Maloney; Ash Lewis; Urchade Zaratiana

cs.AIApr 10, 2026

Pioneer Agent: Continual Improvement of Small Language Models in Production

Authors: Dhruv Atreja, Julia White, Nikhil Nayak, Kelton Zhang, Henrijs Princis, George Hurn-Maloney, Ash Lewis, Urchade Zaratiana

Organizations: Fastino Labs

Abstract

Small language models are attractive for production deployment due to their low cost, fast inference, and ease of specialization. However, adapting them to a specific task remains a challenging engineering loop, driven not by training itself but by surrounding decisions: data curation, failure diagnosis, regression avoidance, and iteration control. We present Pioneer Agent, a closed-loop system that automates this lifecycle. In cold-start mode, given only a natural-language task description, the agent acquires data, constructs evaluation sets, and iteratively trains models by jointly optimizing data, hyperparameters, and learning strategy. In production mode, given a deployed model with labeled failures, it diagnoses error patterns, constructs targeted training data, and retrains under explicit regression constraints. To evaluate this setting, we introduce AdaptFT-Bench, a benchmark of synthetic inference logs with progressively increasing noise, designed to test the full adaptation loop: diagnosis, curriculum synthesis, retraining, and verification. Across eight cold-start benchmarks spanning reasoning, math, code generation, summarization, and classification, Pioneer Agent improves over base models by 1.6-83.8 points. On AdaptFT-Bench, it improves or preserves performance in all seven scenarios, while naive retraining degrades by up to 43 points. On two production-style deployments built from public benchmark tasks, it raises intent classification from 84.9% to 99.3% and Entity F1 from 0.345 to 0.810. Beyond performance gains, the agent often discovers effective training strategies, including chain-of-thought supervision, task-specific optimization, and quality-focused data curation, purely from downstream feedback.

Figures & tables

Figure 1: Pioneer Agent system architecture. An orchestrator LLM (Claude Sonnet 4.6) drives a LangGraph state machine coordinating task analysis, data curation, training, and evaluation. Cold-start mode (left) starts from a task description; production mode (right) uses judged inference traces. Both share an agent-guided iterative search over training pipelines $\pi=(D,H,S)$ , with explicit graph-structured MCGS in selected runs such as ARC-Challenge, executed in Modal sandboxes with the Tinker SDK.

Figure 2: Cold-start performance: baseline vs. Pioneer Agent. Each pair of bars compares the base model score (grey) with the Pioneer Agent–trained score (orange) on the task-specific metric (accuracy, F1, pass@1, or ROUGE-2). Results span three model families (Llama 3B, Qwen 8B, GLiNER2) and four task categories: reasoning (ARC, TriviaQA, GSM8K), classification (SMS Spam), code generation (HumanEval), and summarization (XSum, SAMSum).

Figure 3: Monte Carlo Graph Search trajectory for ARC-Challenge (Llama 3.2-3B) over 11 iterations. Each node represents a completed training pipeline

(\pi_{i},f(\pi_{i}))

; edges encode the agent’s causal reasoning about why a configuration succeeded or failed. The graph shows three key phases: format learning (Branch A, 5.3%

\to

61.3%), chain-of-thought discovery (Branch B, 68.6%), and cross-branch fusion of DeepSeek-R1 reasoning traces with validation-data expansion (Iteration 6, 72.6%). Post-fusion expansions (Iterations 7–11) all regress and are pruned, confirming convergence.

Figure 4: Pioneer Agent’s two operational modes. Cold-start mode (top) takes a natural-language task specification and autonomously downloads data, constructs evaluation sets, synthesizes training curricula, and iterates via agent-guided search (MCGS in selected runs). Production mode (bottom) takes a deployed model with judged inference failures, constructs a failure taxonomy, synthesizes corrective training data (gold corrections, hard negatives, replay buffer), retrains, and accepts updates only when they pass both the failure evaluation set and a regression gate on previously correct behavior.

Figure 5: Annotated stage-based deployment simulation on GSM8K (Qwen3-8B). At each deployment stage, the annotation shows the agent’s key decision: poison filtering, contrastive pair construction, targeted example generation from failure analysis, and automatic rollback when an iteration regresses. Naive retraining trains on all data indiscriminately and degrades sharply on the first noisy stage.

Figure 6: End-to-end closed-loop pipeline unifying both Pioneer Agent modes. In cold-start mode (left), the agent progresses from a natural-language task description through data acquisition, evaluation setup, curriculum synthesis, and agent-guided iteration. In production mode (right), judged inference failures feed into failure diagnosis, taxonomy construction, targeted curriculum synthesis, retraining with regression gating, and conditional deployment. The two modes share the same data-curation principles (Section 2.3 ) and iteration policy (Section 2.4 ), enabling a single system to handle both initial model creation and continuous post-deployment improvement.

Scenario	Task	Base Model	Domain	Fail	Pass	Tier
Benchmark-derived synthetic-log scenarios
GSM8K	Math generation	Qwen3-8B	Math reasoning	751	577	Hard
GSM8K	Math generation	Llama 3.2-3B	Math reasoning	903	425	Hard
ARC-Challenge	Multiple choice	Llama 3.2-3B	Science reasoning	765	735	Hard
TriviaQA	Open QA	Llama 3.2-3B	Trivia knowledge	798	702	Hard
HumanEval	Code completion	Qwen3-8B	Python functions	395	235	Medium

Table 1: AdaptFT-Bench scenarios used in this paper. The first seven are benchmark-derived synthetic-log scenarios evaluated with the stage-based protocol below; CLINC150 and CoNLL-2003 are production-style deployment case studies.

Benchmark	Model	Baseline	Fine-tuned	$\Delta$	Iters
Decoder models (generation)
ARC-Challenge	Llama 3.2-3B	5.3%	72.6%	+67.3	11
GSM8K	Llama 3.2-3B	$\sim$ 8% *	43.7%	+35.7	10
TriviaQA	Llama 3.2-3B	$\sim$ 0% *	48.6%	+48.6	9
ARC-Challenge	Qwen3-8B	91.7%	93.3%	+1.6	13
HumanEval	Qwen3-8B	71.3%	92.7%	+21.4	4

Table 2: Cold-start benchmark results across eight tasks. The agent receives only a natural-language task name and autonomously downloads data, constructs train/test splits, synthesizes training curricula, and iterates. Baselines are zero-shot on the same held-out validation set. Improvements range from +1.6 pp (Qwen3-8B on ARC, where the base model already scores 91.7%) to +83.8 pp F1 (SMS Spam). * Llama 3.2-3B is a base model that cannot follow standard output formats zero-shot; GSM8K baseline is

\sim

8% with 5-shot prompting.

Benchmark	Key Finding
ARC-Challenge (Llama 3B)	Chain-of-thought training was the largest observed lever (+21pp); DeepSeek-R1 traces outperformed GPT-4.1. Post-peak iterations with more data, higher LoRA rank, or lower learning rate all regressed.
GSM8K (Llama 3B)	Fewer epochs (2) with larger batch size (16) were optimal; the model overfits quickly on math patterns. Final accuracy (43.7%) is competitive for 3B-scale models.
HumanEval (Qwen 8B)	Cross-benchmark generalization: trained on MBPP (374 problems), evaluated on HumanEval (164). Adding GPT-4.1-generated solutions reduced performance (96.9% $\to$ 94.5%), indicating that external outputs can dilute the training signal.
XSum (Qwen 8B)	Summarization overfits rapidly—1 epoch is optimal. Fine-tuning corrected verbosity (32.6-word $\to$ 21-word summaries).
SAMSum (Qwen 8B)	System prompt design was the primary lever: a constrained prompt (“1–3 sentences, main outcome, third person, brief”) outperformed all hyperparameter changes.
SMS Spam (GLiNER2)	55 targeted examples added to a 4,458-example base (+1.2%) increased F1 from 0.9834 to 0.9967 via iterative precision–recall balancing.

Table 3: Key findings per cold-start benchmark. Each row highlights the most significant insight from the agent’s iteration trajectory. Full per-iteration tables appear in Appendix C .

Figure 7: Stage trajectories across the seven AdaptFT-Bench scenarios. The deployed adaptive model (green) moves up or stays flat; naive retraining (red) trends down as poisoned logs accumulate.

Benchmark	Model	Metric	Agent C1	Agent Best	Naive C1	Naive C4	Final Gap
GSM8K	Qwen3-8B	Acc.	75.9%	81.2%	71.6%	64.7%	+16.5%
GSM8K	Llama 3.2-3B	Acc.	27.8%	34.8%	19.8%	14.5%	+20.3%
ARC-Challenge	Llama 3.2-3B	Acc.	74.2%	78.4%	52.9%	55.8%	+22.6%
TriviaQA	Llama 3.2-3B	Acc.	73.5%	76.1%	34.0%	33.1%	+43.0%
HumanEval	Qwen3-8B	pass@1	89.0%	90.3%	74.4%	72.5%	+17.8%
XSum	Qwen3-8B	ROUGE-2	38.9	43.1	10.9	6.5	+36.6

Table 4: Production results across AdaptFT-Bench scenarios. Agent columns show the best deployed checkpoint with rollback, while naive columns correspond to single-shot retraining per stage. Colored cells highlight Agent performance (blue = initial, green = best).

Method	Overall	Noisy	Clean
Base model (no training)	56.6	49.4	66.3
Naive Retrain (40% poison)	64.4	55.1	76.9
Naive Retrain (60% poison)	64.3	56.0	75.2
Random Augmentation	68.5	58.6	81.7
Clean Only (no poison)	71.2	62.6	82.5
Pioneer Agent (filtered)	72.2	61.7	86.2

Table 5: Controlled held-out evaluation on 572 test examples (mixed noisy + clean). Accuracy (%).

Checkpoint	Naive	Agent	$\Delta$	Poison
C1: Clean Base	71.6	75.9	+4.3	0%
C2: +Stage 1	62.6	77.6	+15.0	15%
C3: +Stage 2	63.3	79.1	+15.8	25%
C4: +Stage 3	64.7	81.2	+16.5	40%

Table 6: GSM8K (Qwen3-8B) stage checkpoints. Accuracy (%). Pioneer Agent improves monotonically; naive retraining degrades sharply on the first noisy stage.

Perturbation Type	Count	Fixable	Agent Action
Linguistic / structural noise
Typo / misspelling	74	Yes	Uses contrastive pairs to improve robustness to surface noise
Syntactic / grammatical error	50	Yes	Uses contrastive pairs to enforce correct interpretation
Preamble injection	69	Partial	Trains on clean extraction of relevant content
Truncation	28	Partial	Includes shorter contexts to improve robustness
Number distractor	$\sim$ 50	Yes	Uses contrastive pairs to resolve numerical confusion

Table 7: Failure coverage on the GSM8K/Qwen stage benchmark. The agent separates fixable noise (linguistic and structural) from semantic poison and adversarial inputs. Fixable cases are incorporated into training via targeted supervision, while harmful examples are excluded to prevent learning incorrect behavior. Naive retraining, in contrast, trains on all types indiscriminately.

Figure 8: Failure taxonomy constructed by Pioneer Agent for the CLINC150 intent classification deployment. The agent clusters 453 production failures into actionable categories by querying the inference database, identifying dominant confusion pairs (e.g., flight_status

\to

time , change_language

\leftrightarrow

translate ), and diagnosing root causes: semantically overlapping intents, counterintuitive label semantics, and surface-form similarity across decision boundaries. This taxonomy directly informs the composition of the corrective training curriculum.

Metric	Before	After	$\Delta$
Failures fixed	453	450 (99.3%)	3 remaining
Adjusted (excl. 1 label error)	452	450 (99.6%)	2 remaining
Regression on passing set	198	197 (99.5%)	1 regression

Table 8: CLINC150 deployment results after one improvement cycle. The agent fixes nearly all failures while preserving previously correct behavior.

Figure 9: Agent-guided recovery on CoNLL-2003 NER. Starting from a severe precision-collapse regime dominated by false positives, the agent diagnoses the error pathology, builds a precision-first curriculum with hard negatives, pivots from failed LoRA inference to full fine-tuning, and calibrates the deployment threshold. These interventions reduce sample errors from 2,740 to 821 and improve Entity F1 from 0.345 to 0.810.

Figure 10: Four representative Pioneer Agent trajectories spanning cold-start and production settings. Each panel compresses a full run into its diagnostic signal, the intervention sequence the agent pursued, and the final validated outcome. Despite differences in task family, tooling, and search strategy, all four trajectories follow the same closed-loop pattern: diagnose failures, construct targeted supervision, test competing interventions, and retain only updates that survive evaluation.

Production improvement (AdaptFT-Bench)
		Cost Breakdown
Run	Hours	Claude API	Train + Sandbox	Total
GSM8K (3-stage, run A)	11.5	$22.98	$10.00	$32.98
GSM8K (3-stage, run B)	8.3	$16.62	$9.00	$25.62
CLINC150 (2 iterations)	1.0	$1.98	$1.50	$3.48
Cold-start

Table 9: Estimated cost per benchmark run. Total cost ranges from

3.48 to

54.98. The two GSM8K rows are representative end-to-end three-stage adaptive runs; total cost varies with branching, diagnosis, and rollback overhead. A human ML engineer at $150/hour would cost substantially more for equivalent work.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Tier	Description	Failure count	Failure diversity	Model state
Easy	Few failures, clear patterns	10–50	1–2 clusters	Fine-tuned
Medium	Many failures, systematic patterns	50–500	3–5 clusters	Fine-tuned or base
Hard	Many failures, adversarial + non-fixable	100–500+	5+ clusters, incl. non-fixable	Base model

Appendix

Table 10: Scenario Difficulty Tiers in AdaptFT-Bench.

Scenario	Regime	Base model	Metric	Failure source	Sanitized artifact
CLINC150	Production-style case study	GLiNER2-base	Exact-match pass rate on failing slice	453 judged production failures + 198 passing regression examples	clinc150_trace_and_report_summary.md
CoNLL-2003	Production-style case study	GLiNER2-base / large	Entity F1 + sample pass rate	2,740 judged NER failures from production-style deployment logs	conll2003_ner_trace_and_report_summary.md
GSM8K (Qwen3-8B)	Synthetic 3-stage scenario	Qwen3-8B	Exact match on final #### number	Benchmark-derived noisy inference logs	none in representative bundle
GSM8K (Llama 3.2-3B)	Synthetic 3-stage scenario	Llama 3.2-3B	Exact match on final #### number	Benchmark-derived noisy inference logs	gsm8k_llama_trace_summary.md
ARC-Challenge	Synthetic 3-stage scenario	Llama 3.2-3B	Exact match on answer letter	Benchmark-derived noisy inference logs	arc_challenge_trace_summary.md
TriviaQA	Synthetic 3-stage scenario	Llama 3.2-3B	Case-insensitive answer-alias match	Benchmark-derived noisy inference logs	triviaqa_trace_summary.md

Appendix

Table 11: Scenario-level details for AdaptFT-Bench as used in this paper. The seven synthetic scenarios all use the shared stage-based protocol from Section 3 : three stages with rising poison rates, 70/30 train-test splits per stage, and a held-out evaluation set formed by the union of stage test splits. CLINC150 and CoNLL-2003 are real judged deployment case studies reported separately from the synthetic inference benchmark.

Category	Perturbation Type	Default Rate
Linguistic	Typos (character transposition/insertion/deletion)	10%
	Phonetic misspellings	7%
	Syntactic errors (broken structure, fragments)	6%
	Grammatical errors (tense, articles, agreement)	5%
	Punctuation abuse (missing, excessive)	5%
	Casing errors (all-lower, all-caps, random)	4%

Appendix

Table 12: Noise injection rates for synthetic inference log generation. Each perturbation is applied independently per sample.

Benchmark	Model	Iter 1	Iter 3	Best	Iterations
ARC-Challenge	Llama 3B	48.1%	68.6%	72.6%	11
GSM8K	Llama 3B	11.8%	$\sim$ 35%	43.7%	10
HumanEval	Qwen 8B	72.0%	85.4%	92.7%	4
XSum	Qwen 8B	R2: 13.0	R2: 16.4	R2: 17.9	13
SAMSum	Qwen 8B	R2: 21.8	R2: 24.1	R2: 25.4	11
TriviaQA	Llama 3B	0.0%	44.0%	48.6%	9

Appendix

Table 13: Cold-start performance progression across iterations. The number of iterations varies by task complexity, from 4 (HumanEval) to 13 (XSum).

Benchmark	Model	Train Ex.	Epochs	Batch Size	LR	Notes
ARC-Challenge	Llama 3B	1,119	5	–	–	R1 CoT + val data
GSM8K	Llama 3B	7,473	2	16	2e-4
TriviaQA	Llama 3B	3,000	4	–	1e-4
HumanEval	Qwen 8B	374	3	–	–	Cross-benchmark (MBPP)
XSum	Qwen 8B	3,000	1	4	–
SAMSum	Qwen 8B	500	8	–	–	Constrained system prompt

Appendix

Table 14: Best training pipeline per benchmark, as selected autonomously by the agent. “–” indicates the agent used the platform default or the value was not recorded in the run artifacts.

Component	Configuration
Agent LLM	Claude Sonnet 4.6, 1M context, 32K extended thinking budget
Agent framework	LangGraph + LangChain (Python)
Sandbox	Modal containers, 16GB memory, 24h timeout, 1 CPU
Training backend	Tinker SDK (LoRA fine-tuning, instant inference after training)
Training helpers	tinker_helpers.py – standalone wrappers for train() , infer() , infer_batch()
Data generation LLMs	OpenAI GPT-4.1/4.1-mini, DeepSeek-R1/V3, Fireworks

Appendix

Table 25

Benchmark	Evaluation Method	Code Source
ARC-Challenge	Exact match on answer letter (A/B/C/D)	Agent-written extraction + string match
GSM8K	Exact match on final number after ####	Agent-written regex extraction
TriviaQA	Case-insensitive match against answer aliases	Agent-written alias matching
HumanEval	pass@1 – code execution against unit tests	Official human_eval package
XSum	ROUGE-1, ROUGE-2, ROUGE-L	rouge-score Python package
SAMSum	ROUGE-1, ROUGE-2, ROUGE-L	rouge-score Python package

Appendix

Table 26

Artifact family	Agent mode	Benchmarks	What it preserves
Final reports (PDF)	Cold-start	SMS Spam	Domain research, dataset setup, 4-iteration augmentation loop, final F1 0.9967
Final reports (PDF)	Production	CLINC150	5-cluster failure taxonomy, 1,254-example curriculum, Full FT vs. LoRA comparison, rollback decision
Final reports (PDF)	Production	CoNLL-2003	FP-dominated diagnosis, 6-iteration repair loop, threshold sweep, capacity scale-up to GLiNER2-large
Derived trajectory summaries	Cold-start	ARC, GSM8K, TriviaQA	Search trajectories, iteration-by-iteration configuration changes, evaluation outputs, and intermediate decisions distilled from internal run exports
Derived trajectory summaries	Production	CLINC150, CoNLL-2003	SQL diagnostics, delegated taxonomy analysis, dataset revisions, training/evaluation history, and stopping criteria distilled from internal traces

Appendix

Table 27

LLM Agent Training

Large language model agent training teaches models to plan, use tools, and learn from multi-step interaction with environments.

See all 586 papers

Oct 5, 2026

ImproveAnyTask: An Autonomous Post-Training Harness for Iterative Model Self-Improvement

Adapting general-purpose large language models to specific tasks requires substantial human effort in designing data and training strategies. Sustaining improvement is especially challenging because model updates change the error distribution, requiring strategies to be continually refined. We introduce ImproveAnyTask, an autonomous post-training harness that improves task performance under a limited compute budget. Drawing inspiration from gradient-based parameter optimization, the harness organizes adaptation into error attribution, update-direction selection, and executable model updates. It combines metric-level and case-level analysis to identify a focal problem, then investigates research-backed strategies and compares their reported gains and reproduction difficulty. The selected strategy is translated into training data and a training configuration, with small-scale execution checks preceding full post-training. Subsequent evaluation guides model selection and further adaptation, while validated strategies and scripts are retained for reuse. Across 11 tasks, ImproveAnyTask achieves mean gains of 18.29 and 11.97 percentage points on the Base and Instruct models, respectively, with a maximum gain of 41.96 points, under a 24-hour budget with resources equivalent to eight H20 GPUs.

Xingbo Yao, Xiaoman Wang, Zhengwu Lei +11

HKUST(GZ) · Xiaohongshu Inc. · ECNU +3

LLM Fine-Tuning LLM Post-Training Language Model Post-Training Language Model Self-Improvement

Ask PDF

Jun 30, 2026

AutoTrainess: Teaching Language Models to Improve Language Models Autonomously

Training language models (LMs) remains a highly human-intensive process, even as frontier language model agents become increasingly capable at software engineering and other long-horizon tasks. A central challenge is that autonomous post-training is not just a coding problem: it requires the agent to repeatedly plan iterations, construct benchmark-aligned data, run stable training jobs, evaluate checkpoints, and preserve experiment state across many hours of interaction. We present AutoTrainess, a LM agent that exposes these operations as a repository of agent-computer interfaces for planning, data preparation, training, evaluation, and logging. Rather than leaving the agent to operate in a raw CLI environment with an underspecified action space, AutoTrainess externalizes prior human experience as explicit workflows, rules, and execution constraints that guide the agent toward effective and reliable training behavior. On PostTrainBench, AutoTrainess consistently outperforms CLI-only baselines, achieving 26.94 average score with GPT-5.4 (Codex) versus 23.21 for CLI-only. It also generalizes across models and harnesses, improving DeepSeek-V4-Flash (OpenCode) from 12.13 to 19.58.

Zhaojian Yu, Penghao Yin, Shuzheng Gao +3

1Tsinghua University · 2The Chinese University of Hong Kong · 3Simple Agent Lab

Tool-Augmented Language Model Agents LLM Tool Use LLM Agent Training LLM Post-Training Language Model Self-Improvement

Ask PDF

Oct 10, 2026

Iterate on an LLM, Serve on an SLM

As frontier LLMs gain more capabilities, the cost to run them also goes up. At the same time, small language models (SLMs) also gain more capabilities but have largely been able to keep their cost stable, offering 5-40 times cheaper inference for the narrow, repetitive subtasks that agents decompose work into. However, the standing limiting factor is getting the correct behavior out of a small model in a fast, efficient, and economical way. Unfortunately, in most cases, zero-shot small models fail narrow production tasks. Furthermore, retraining one every time the behavior changes is too slow to iterate on. We propose doing both in sequence. First, develop the desired behavior in prompt space, on a frontier model, with immediate feedback from live traffic. Then, once the behavior converges, distill the recorded production behavior (not synthetic imitations of it) into an SLM through supervised fine-tuning (SFT). We show a proper use case of this method through a scheduling agent for home-care agencies that fans out a single yes/no safety decision per caregiver-shift pair on GPT-5.4-nano. A LoRA-tuned Qwen3-8B trained on 11.6k recorded production decisions reaches 0.913 agreement, while the teacher (GPT-5.4-nano), re-run on its own recorded prompts, agrees with itself only 72.7% of the time, pointing to the fact that the SLM may also be more consistent. Furthermore, when the two disagreed, a blinded professional scheduler sided with the student 48-29. Finally, we find that the SLM has approximately 2.5 times lower median latency and is a fraction of the cost. We believe this lifecycle of quick building, live feedback, and cheap inference is a blueprint for others building similar systems.

Keegan Wang

Carnegie Mellon University · Phoebe

LLM Agent Training Small Language Models Efficient Language Model Inference Language Model Distillation

Ask PDF

Abstract

LLM Agent Training

Explore similar work

ImproveAnyTask: An Autonomous Post-Training Harness for Iterative Model Self-Improvement

AutoTrainess: Teaching Language Models to Improve Language Models Autonomously

Iterate on an LLM, Serve on an SLM