ATTUNE: Agentic Throughput Tuning for LLM Training

Leo Fan; Hongyang Wang; Wenbo Chen; Jingguo Liu; Xian Li; Xuan Wang; Ruochen Wen; Yiming Yan

cs.DCOct 13, 2026

ATTUNE: Agentic Throughput Tuning for LLM Training

Authors: Leo Fan, Hongyang Wang, Wenbo Chen, Jingguo Liu, Xian Li, Xuan Wang, Ruochen Wen, Yiming Yan,

Organizations: Amazon

Abstract

The throughput of an LLM training job depends on its recipe: how the model is split across GPUs, how memory is traded for compute, and which kernels and communication overlaps it uses. The best recipe changes with every model, accelerator and framework version. Automated tuners search a fixed space of knobs and check only that a job runs, so they miss levers outside that space and can ship a configuration that silently changes training. ATTUNE is a coding agent that tunes any working recipe unattended within a GPU-hour budget. Guided by a skill of search rules, scripts and lessons from past tuning campaigns, it diagnoses each run and changes one lever at a time. A correctness gate outside the agent refuses any argument known to change the training objective. ATTUNE tunes Amazon's production post-training recipes on H100, H200 and B200 clusters, raising throughput by 1.7x on average. It beats the published throughput of four of NVIDIA's own benchmark recipes on NVIDIA's AutoModel framework by 6-17%. On models held out from its development, it improves the strongest public recipes most where they were written for another setting, by up to 2.6x on MiniMax-M2, and least where the vendor tuned them for the same hardware. Given the same GPU-hour budget, it outperforms existing automated tuners, including Bayesian optimization, random search and OptiCo, a recent LLM-agent tuner. Its gain on GLM-4.5 is more than four times the best alternative's. The same coding agent without ATTUNE's skill gains less and fails half its launches. We release ATTUNE at https://github.com/amazon-science/attune and OpenRecipes, a leaderboard of tuned and verified training recipes, at https://openrecipes.ai.

Figures & tables

Figure 1: ATTUNE’s architecture. (1) A single coding agent runs the search as a reason–act loop: it states a hypothesis, launches one change, profiles the run and diagnoses the bottleneck. (2) Deterministic gates outside the agent refuse objective-changing flags, decide whether a measurement is valid and whether a candidate beats the leader, and, after the search, check the leader before it ships. (3) The harness keeps the loop running unattended, (4) durable state records every run so a campaign can resume, and (5) the skill passes the lever catalog and the summaries and lessons of past campaigns to later campaigns.

		Throughput (tok/s/GPU)		Gain over
Model	GPUs	Baseline	ATTUNE	Baseline	Published	MFU (%)	GPU-h (budget)
Reproduced on NeMo AutoModel
Qwen3-30B-A3B	8 $\times$ H100	11,963.1	13,151.1	+9.9%	+9.2%	27.9 $\rightarrow$ 30.7	37.2 (70)
GPT-OSS-20B	8 $\times$ H100	12,970.3	15,222.5	+17.4%	+16.6%	30.1 $\rightarrow$ 35.3	18.1 (65)
Nemotron-3-Nano	8 $\times$ H100	16,538.1	17,726.7	+7.2%	+5.6%	33.4 $\rightarrow$ 35.8	23.5 (65)
Llama-3.1-70B	64 $\times$ H100	801.2	1,012.5	+26.4%	+16.8%	36.4 $\rightarrow$ 46.0	133.4 (500)

Table 1: ATTUNE’s gain over the strongest public recipe, at that recipe’s workload shape and GPU count. MiniMax-M2’s recipe, written for 128 GPUs, runs on 64. Baseline is the public recipe on our cluster: for the reproduced set unchanged in NVIDIA’s container, for the ported set on slime and Megatron-LM with the changes of Appendix A (Table A3). In the reproduced set each model has one campaign; the gain over published compares ATTUNE with NVIDIA’s published figure (Table A3), measured on other machines and software versions. In the ported set, values over three campaigns are means, judged by the mean rule (Section 3.1), and no gain over the published figures is given because they come from other stacks. MFU (Appendix D) is given as baseline

\rightarrow

ATTUNE, and GPU-h is the GPU-hours spent per campaign, with its budget in parentheses. Tables D2 and D3 give every campaign.

Model	Tuner	Gain	GPU-h used	Launches	Failed launches (GPU-h)	Wall-clock
GLM-4.5	ATTUNE	+33.9%	669	8.3	24% (60)	6.4 h
	OptiCo	0.0%	1,407	22.7	19% (78)	15.7 h
	BO (TPE)	+1.8%	1,675	19.7	39% (186)	9.6 h
	BO + pre-filter	+3.9%	1,695	13.7	5% (58)	9.5 h
	Random search	+7.8%	1,688	22.7	51% (278)	10.2 h
MiniMax-M2	ATTUNE	+164.2%	203	13.3	23% (25)	3.1 h

Table 2: Tuner comparison on GLM-4.5 / 128

\times

B200 and MiniMax-M2 / 64

\times

B200, means over three campaigns. ATTUNE’s rows are the campaigns of Table 1. Gain is over the public recipe. Failed launches gives the share of launches that ran out of memory, hung or crashed, with their GPU-hours in parentheses; Appendix D defines the columns. Bold marks each model’s best value per column. On both models ATTUNE beats every other tuner under the mean rule, with exact permutation p = 0.05 against each, the lowest that test allows (0.10 against random search on GLM-4.5), and block-stratified p = 0.125, also that test’s lowest. BO: Bayesian optimization; TPE: tree-structured Parzen estimator.

Figure 2: Best-so-far search-time gain over the public recipe’s calibration mean (728.2 tokens/s/GPU) against cumulative charged GPU-hours on GLM-4.5 / 128

\times

B200: the mean over each arm’s three campaigns (line) and their range (shading). Ticks mark each campaign’s last search run; the curve then runs flat to the 1,880 GPU-hour budget (dotted line).

Arm	GLM-4.7-Flash gain, %	p	Failed	GPU-h to +20%	Nemotron-3-Nano gain, %	p	Failed	GPU-h to +20%
Full ATTUNE	+88.0 (81.7–96.6)	–	17%	3.9 (5/5)	+49.4 (47.1–51.4)	–	32%	5.2 (5/5)
Bare agent	+66.9 (54.7–94.5)	0.016	50%	9.0 (5/5)	+42.9 (35.9–48.8)	0.008	50%	7.2 (5/5)
Fixed knobs	+18.1 (15.6–21.4)	0.004	13%	19.8 (2/5)	+48.6 (41.3–51.3)	0.42	33%	3.1 (5/5)
No diagnosis	+73.4 (21.3–96.5)	0.24	19%	4.9 (5/5)	+47.5 (41.9–50.2)	0.14	41%	3.1 (5/5)

Table 3: Ablations on GLM-4.7-Flash and Nemotron-3-Nano-30B-A3B / 8

\times

H200, five campaigns per arm and model. Gain is the mean over each campaign’s own baseline run, with the range in parentheses; p is a one-sided exact permutation test of full ATTUNE against the arm (at least 0.004). Failed is the share of launches that crashed or did not complete; GPU-h to +20% is the median GPU-hours until a run passing the gate first gains 20%, over the campaigns that get there (count in parentheses). Bold marks the best value per column.

Runs with a known answer	Gate passes	Gate rejects
Legitimate: 132 runs on six models	132	0
Legitimate: 328 ablation runs on two models (Section 3.4)	328	0
Faulty: 24 runs of 14 faults	9 runs of four faults that move the first step’s loss by 2% or less: one gradient bucket’s reduction skipped (five models), load-balancing loss zeroed, loss mask shifted by one token, QK-norm gradients summed over tensor-parallel ranks	15 : 6 objective-changing arguments and 4 configuration defects, refused before launch; 5 code faults that change the forward pass (labels offset by one token, attention sinks ignored), caught at step 0
Faulty: 19 ablation runs with an uneven pipeline split on Nemotron-3-Nano-30B-A3B	0	19 , refused as a known stack defect; step-0 loss 215–768% above the baseline’s

Table 4: The correctness gate on runs with a known answer. Bold cells are correct decisions. A fault tested on several models counts once per model. Legitimate ablation runs change only placement, packing or same-math swaps with a passed component test. Table B2 details the first and third rows.

Model	H100	H200	B200
Qwen3.5-122B	2.49 $\times$	1.32 $\times$	1.26 $\times$
Qwen3.5-397B	1.25 $\times$	1.06 $\times$	3.91 $\times$
GLM-5.2	1.07 $\times$	2.54 $\times$	2.21 $\times$

Table 5: Throughput gain of ATTUNE’s recipes in Amazon’s post-training jobs over the recipes they replaced, on 8,192-token rows at global batch 128; the geometric mean is 1.7

\times

. Table D5 gives the GPU counts, the replaced recipes and the cost per million tokens.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Source	Rows $\times$ tokens
Llama-3.1-405B	Project Gutenberg, PG-19 book list (mirror texts); Llama-3.1 tokenizer	6,400 $\times$ 8,192
Mixtral-8x22B	Project Gutenberg, PG-19 book list (mirror texts); Mixtral tokenizer	64,000 $\times$ 4,096
GLM-4.5	HuggingFaceFW/fineweb-edu , sample/10BT shards 000–002; GLM-4.5 tokenizer	225,280 $\times$ 4,096
MiniMax-M2	HuggingFaceFW/fineweb-edu , sample/10BT shards 000–002; MiniMax-M2 tokenizer	28,160 $\times$ 8,192

Appendix

Table A1: Frozen datasets, one per model. Rows

\times

tokens gives the number of packed rows and the tokens per row, which equals the recipe’s sequence length. FineWeb-Edu is released under ODC-By 1.0; the Gutenberg texts are in the public domain.

Model	Changes from the public recipe
Llama-3.1-405B, 128 $\times$ B200	Per-token loss; vocabulary unpadded to match the converted checkpoint; data-parallel and tensor-parallel communication overlap kept on as published, through stack fixes and a gloo bootstrap; learning rate scaled by 1/30 for the released weights
Mixtral-8x22B, 128 $\times$ H100	RoPE base 10 6 as in the released model, which the recipe omits; selective recomputation of the expert activation to fit in memory (the recipe recomputes nothing); userbuffers bootstrap over gloo
GLM-4.5, 128 $\times$ B200	Multi-token-prediction layer removed; B200 instead of the recipe’s H100, with the published layout unchanged; data-parallel overlap kept on as published, through a stack fix
MiniMax-M2, 64 $\times$ B200	Supervised training on fixed 8,192-token rows of FineWeb-Edu instead of the recipe’s reinforcement-learning loop; 64 GPUs instead of 128, with tensor, pipeline, expert-tensor and context parallelism kept; released FP8 weights dequantized to BF16; expert parallelism 4 $\rightarrow$ 8 to fit in memory; QK-norm weight gradients averaged rather than summed over tensor-parallel ranks

Appendix

Table A2: Changes from each public recipe that affect throughput or the baseline; the baseline every tuner starts from includes them. All models read the datasets of Table A1 and start from released weights.

Model	Type	GPUs	Public recipe and workload	Published	Our stack
Reproduced on NeMo AutoModel
Qwen3-30B-A3B	MoE (30B, 3B active)	8 $\times$ H100	AutoModel benchmark; mock tokens, balanced routing, random init; seq 4K, GBS 512, BF16	12,040 (MFU 28%)	11,963.1
GPT-OSS-20B	MoE (21B, 3.6B active)	8 $\times$ H100	AutoModel benchmark; mock tokens, balanced routing, random init; seq 4K, GBS 256, BF16	13,058 (MFU 30%)	12,970.3
Nemotron-3-Nano-30B-A3B	hybrid Mamba MoE (30B, 3.5B active)	8 $\times$ H100	AutoModel benchmark; mock tokens, balanced routing, random init; seq 4K, GBS 512, BF16	16,789 (MFU 34%)	16,538.1
Llama-3.1-70B	dense (70B)	64 $\times$ H100	AutoModel pretrain benchmark; mock tokens, released weights; seq 8K, GBS 128, BF16	866.8 (MFU 39%)	801.2
Ported to slime and Megatron-LM

Appendix

Table A3: Public recipes and their workloads, in tokens/s/GPU unless noted, with 8 GPUs per node (B200: 180 GB and H100: 80 GB of HBM per GPU). “Published” is the throughput the recipe’s authors report ( NVIDIA, 2026b ; Yan, 2026 ; NVIDIA, 2026d ) ; Appendix D defines MFU. For the reproduced set, “Our stack” is the unchanged recipe in NVIDIA’s container (26.08, AutoModel 922e83e4) on our cluster; for the ported set it is the public recipe on slime and Megatron-LM with the changes of Appendix A. Seq: sequence length; GBS: global batch size in rows; –: none published.

Known stack defect	How it shows	Refused on
Fused routing (a same-math swap)	breaks exact BF16 ties differently and routes some tokens to other experts; fails its component test	Mixtral-8x22B, GLM-4.5, MiniMax-M2, GLM-4.7-Flash
Gradient-accumulation fusion off (a same-math swap)	adds a BF16 rounding where main gradients are FP32; fails its component test	Mixtral-8x22B, GLM-4.5, MiniMax-M2
Expert parallelism 1 with expert-tensor parallelism of 4 or more	silently changes training: step 0 matches the baseline, and the loss departs from it after the first update	Mixtral-8x22B, GLM-4.5, MiniMax-M2
Context parallelism 8 with virtual stages	hangs at step 0	Mixtral-8x22B
Context parallelism above 4	a step takes about 1,800 s	GLM-4.5
Tensor parallelism 16	cannot run	MiniMax-M2

Appendix

Table B1: Known stack defects, refused by the configuration check for every tuner on the listed models. On Mixtral-8x22B, expert-tensor parallelism 8 with expert parallelism 1 changes training as it does on GLM-4.5; expert-tensor parallelism 4 with expert parallelism 1 is refused on Mixtral-8x22B because of the GLM-4.5 result.

Run	Models	Configuration check	Step-0 loss shift	Gate
Fault-free: replicates, single placement or packing changes, runs of the public recipe and legitimate recipes whose step-0 numerics differ from the baseline	GLM-4.5 (27), MiniMax-M2 (31), Mixtral-8x22B (28), Llama-3.1-405B (4); development: GPT-OSS-20B (32), GPT-OSS-120B (10)	admitted	at most 1.98%	132 of 132 pass
EP 1 with ETP of 4 or more (a real stack defect that silently changes training)	GLM-4.5, Mixtral-8x22B, MiniMax-M2	refused	at most 0.02%	rejected
PP 32, which halves the load-balancing loss while the language-model loss stays identical (a real stack defect)	GLM-4.5	refused	0.00%	rejected
Argument faults: capacity factor 1, learning rate $\times$ 10, global batch changed	development: Qwen3.5-122B-A10B	refused	+7.6%, $-$ 0.02%, +0.2%	rejected
Argument faults: BF16 gradient reduction, per-token loss off, auxiliary-loss coefficient changed	development: Qwen3.5-122B-A10B	refused	$-$ 0.04%, +0.5%, $-$ 0.01%	rejected
Labels offset by one token	GLM-4.5, MiniMax-M2, Llama-3.1-405B; development: GPT-OSS-120B	passes	+190% to +872%	rejected

Appendix

Table B2: The correctness gate’s validation per fault and model, on runs outside the evaluation campaigns. The step-0 loss shift is relative to the model’s baseline. Counts in parentheses are fault-free runs. “Refused” means the configuration check rejects the candidate before launch, and the step-0 shifts of refused configurations come from running them outside any campaign; “passes” means the configuration check cannot see the fault. EP, ETP, PP: expert, expert-tensor and pipeline parallelism.

Component	Version
OS	Ubuntu 24.04
Python	3.12.3 (training image)
PyTorch	2.11.0 (CUDA 12.9 build)
CUDA	12.9
NCCL / libfabric NCCL plugin	2.28.9 / 1.20.0
Megatron-LM	commit 1dcf0da

Appendix

Table C1: Software in the pinned training images and the tuners’ sandbox.

Model	Calibration mean (tokens/s/GPU)	$\sigma$	Band	Unprofiled launch (GPU-h)	Budget per campaign (GPU-h)
Llama-3.1-405B / 128 $\times$ B200	399.36	1.29%	3.88%	61.38	1,850
Mixtral-8x22B / 128 $\times$ H100	1,091.69	0.95%	3.00%	24.57	740
GLM-4.5 / 128 $\times$ B200	728.19	0.49%	3.00%	62.58	1,880
MiniMax-M2 / 64 $\times$ B200	893.18	0.95%	3.00%	17.58	530

Appendix

Table D1: Per-launch cost and budgets, from at least five calibration runs per model on distinct node allocations. The calibration mean is the denominator during search; reported gains use the final re-measurement.

\sigma

is the calibration runs’ relative standard deviation, and each band is max(3%, 3

\sigma

). The budget is 30 times the unprofiled launch cost, rounded up to a multiple of 10 GPU-hours.

Arm	#	Shipped	Gain	GPU-h used	Stop	Shipped changes
ATTUNE	1	885.7	+22.4%	753.6	agent	PP 8 $\rightarrow$ 16; EP 16 $\rightarrow$ 8; first/last-stage layers 10/10 $\rightarrow$ 4/4; DeepEP dispatcher; shared-expert overlap off
ATTUNE	2	1,119.3	+54.7%	626.3	agent	TP 2 $\rightarrow$ 1; PP 8 $\rightarrow$ 16; EP 16 $\rightarrow$ 8; VP off $\rightarrow$ 2; first/last-stage layers 10/10 $\rightarrow$ 4/4; full recompute
ATTUNE	3	902.1	+24.7%	627.3	agent	EP 16 $\rightarrow$ 8; DeepEP dispatcher; shared-expert overlap off; data-parallel overlaps off
OptiCo	1	723.6	0.0%	1,274.4	no launch	none (public recipe)
OptiCo	2	723.6	0.0%	1,543.8	budget	none (public recipe)
OptiCo	3	723.6	0.0%	1,402.2	no launch	none (public recipe)

Appendix

Table D2: Every evaluation campaign on GLM-4.5, in tokens/s/GPU from the final re-measurement. Gain is over the re-measured public recipe (723.6). GPU-hours include correctness checks. Stop: the agent’s convergence judgment, the GPU-hour budget or our OptiCo driver’s stop after 10 consecutive iterations that launch nothing (no launch). BO: Bayesian optimization; TPE: tree-structured Parzen estimator. Changes are relative to the public recipe. TP, PP, EP, ETP, CP, DP: tensor, pipeline, expert, expert-tensor, context and data parallelism; VP: virtual stages per pipeline rank; first/last-stage layers: transformer layers in the first and last pipeline stages.

Arm	#	Shipped	Gain	GPU-h used	Stop	Shipped changes
MiniMax-M2 / 64 $\times$ B200
ATTUNE	1	2,423.1	+170.7%	200.2	agent	TP 2 $\rightarrow$ 1; PP 2 $\rightarrow$ 4; first/last-stage layers even $\rightarrow$ 15/15; 2 rows per microbatch; data-parallel overlaps on; optimizer offload off
ATTUNE	2	2,235.8	+149.8%	197.3	agent	PP 2 $\rightarrow$ 8; first/last-stage layers even $\rightarrow$ 7/7; 4 rows per microbatch; optimizer offload off
ATTUNE	3	2,436.3	+172.2%	213.0	agent	TP 2 $\rightarrow$ 1; PP 2 $\rightarrow$ 8; VP off $\rightarrow$ 2; first/last-stage layers even $\rightarrow$ 8/6; recompute uniform $\rightarrow$ block, 2 layers; optimizer offload off
OptiCo	1	1,101.3	+23.0%	301.4	no launch	TP 2 $\rightarrow$ 1; data-parallel overlaps on; cross-entropy fusion on
OptiCo	2	1,109.9	+24.0%	384.5	no launch	TP 2 $\rightarrow$ 1; data-parallel overlaps on; cross-entropy fusion on

Appendix

Table D3: Every evaluation campaign on MiniMax-M2, Mixtral-8x22B and Llama-3.1-405B, as in Table D2. Gain is over the model’s re-measured public recipe (MiniMax-M2 895.2, Mixtral-8x22B 1,072.4, Llama-3.1-405B 394.9). Embedding and loss accounting: Megatron counts the embedding and the loss as pipeline layers.

Arm	Gain	GPU-h used	Launches	Failed launches (GPU-h)	Wall-clock
Full ATTUNE	+33.9% (+22.4, +54.7, +24.7%)	669	8.3	24% (60)	6.4 h
Multi-agent committee	+28.3% (+5.5, +55.7, +23.8%)	604	6.0	17% ( 30 )	5.1 h
Public-guide catalog	+29.1% (+28.7, +29.6, +29.0%)	870	9.3	11% (54)	7.0 h
Minimal skill (no lever catalog)	+30.3% (+32.5, +29.6, +28.7%)	773	10.0	13% (41)	6.2 h

Appendix

Table D4: Ablations on GLM-4.5 / 128

\times

B200, three campaigns per arm. Gain over the public recipe is the mean, with each campaign’s in parentheses; the other columns are per-campaign means as in Table 2, and bold marks the best value per column. At GLM-4.5’s band of 3.00%, full ATTUNE is ahead of the committee under the mean rule (mean throughput 4.3% higher; exact permutation p = 0.45 unrestricted, 0.375 block-stratified) and of the public-guide arm (3.7%; p = 0.50, 0.50), and ties the minimal skill (2.8%; p = 0.50, 0.50); with three campaigns per arm, none of these differences is significant.

Recipe	Replaced recipe	Gain	$ per 1M tokens
Qwen3.5-122B, 48 $\times$ H100	Megatron-Bridge	2.49 $\times$	4.86 $\rightarrow$ 1.95
Qwen3.5-122B, 32 $\times$ H200	in-house	1.32 $\times$	1.24 $\rightarrow$ 0.94
Qwen3.5-122B, 16 $\times$ B200	in-house	1.26 $\times$	1.94 $\rightarrow$ 1.54
Qwen3.5-397B, 128 $\times$ H100	verl	1.25 $\times$	7.80 $\rightarrow$ 6.23
Qwen3.5-397B, 48 $\times$ H200	verl †	1.06 $\times$	7.68 $\rightarrow$ 7.22
Qwen3.5-397B, 64 $\times$ B200	verl ‡	3.91 $\times$	12.00 $\rightarrow$ 3.07

Appendix

Table D5: ATTUNE’s recipes in Amazon’s post-training jobs, with their GPU counts and the recipes they replaced. Megatron-Bridge, verl and slime recipes are public. Gain is tokens/s/GPU over the replaced recipe on 8,192-token rows at global batch 128; the geometric mean is 1.7

\times

. † Fitted to the GPU count. ‡ Written for 128 GPUs; the ATTUNE recipe runs on 64. Cost uses AWS EC2 on-demand prices in us-east-1 per GPU (H100

6.88, H200

7.91, B200 $14.24 per hour), not what Amazon pays.

Knob	GLM-4.5 (128 GPUs)	MiniMax-M2 (64 GPUs)
Tensor parallelism	1, 2, 4, 8	1, 2, 4, 8
Pipeline parallelism	1, 2, 4, 8, 16	1, 2, 4, 8, 16
Expert parallelism	1, 2, 4, 8, 16, 32 (1 only with expert-tensor parallelism $\leq$ 2)	1, 2, 4, 8, 16, 32, 64 (1 only with expert-tensor parallelism $\leq$ 2)
Expert-tensor parallelism	1, 2, 4, 8	1, 2, 4, 8
Virtual stages per rank	off, 2, 4, 23	off, 2, 31
Pipeline split	balanced, light first, light last, light both	balanced, light first, light last, light both

Appendix

Table E1: Search spaces. On GLM-4.5 DeepEP is offered only within a node, because it cannot initialize across nodes on our network; on MiniMax-M2 it is offered across nodes too, and those draws fail at startup and score zero. The largest virtual-stage values put one layer in each virtual stage, and the light splits give the first, last or both end stages fewer layers. Megatron sets tensor-parallel communication overlap from the tensor-parallel degree, and the launcher removes it at tensor parallelism 1.

Run	Change $\rightarrow$ result (tokens/s/GPU)	The agent’s reasoning, quoted
pp16ep8	PP 8 $\rightarrow$ 16, EP 16 $\rightarrow$ 8, first/last-stage layers 10/10 $\rightarrow$ 4/4, expert all-to-all inside a node $\rightarrow$ 707.6, tie	“clearly comm/overhead-bound”; “EP placement is throughput-neutral here”
tp1pp8	TP 2 $\rightarrow$ 1 $\rightarrow$ out of memory	“TP1 (no SP, unsharded activations) doesn’t fit at PP8; it needs more PP.”
tp1pp16	TP 2 $\rightarrow$ 1 at PP 16, EP 8 $\rightarrow$ out of memory	“memory-infeasible at both PP8 and PP16 with selective recompute”
tp1pp16fr	+ full recomputation $\rightarrow$ 1,072.7, win (+47.3%)	“our step has many tiny 1-row microbatches $\rightarrow$ lots of launch idle … full recompute may be ~free here AND it enables the TP1 fit”
tp1pp8fr	back to PP 8, EP 16 $\rightarrow$ stopped early, $\approx$ 668	“the cross-node EP a2a (and DP16 comm) is now exposed”
tp1pp16s53	first/last-stage layers 4/4 $\rightarrow$ 5/3 $\rightarrow$ stopped early, $\approx$ 1,083	“the pipeline is not stage-imbalance-bound; it’s uniformly comm/launch-bound”

Appendix

Table F1: The launches of GLM-4.5 ATTUNE campaign 2 in order, starting from the public recipe (728.2 tokens/s/GPU calibrated). Run names are the agent’s labels. Each quote is the agent’s diagnosis after the run; in the quotes, SP is sequence parallelism and a2a is all-to-all. Gains up to and including the winner (tp1pp16fr) are over the public recipe, and later gains are over the winner. Values marked

\approx

are the agent’s estimates for runs it stopped early; the control plane does not score these runs. The shipped recipe re-measured at 1,119.3 tokens/s/GPU in the final stage.

Check	Covers	Result
Lessons checked against the campaigns’ own records	155 lessons of the catalog distilled from development campaigns	71 correct and general; 34 rewritten to their general form; 31 wrong; 11 unsupported; 8 resting on one model or campaign
Current model asked, without the skill, about each entry (Claude Opus 5.5, three answers per question)	106 entries of the curated catalog	19 answered fully; 59 partly; 27 contradicted; 1 not known

Appendix

Table G1: The catalog audit. A lesson is wrong when its own campaign’s records contradict it, unsupported when no record supports it, and partly answered when the model’s answers miss a condition, scope or failure mode the entry states.

Efficient Language Model Training

Efficient language model training studies how to reduce the data, compute, memory, time, energy, and cost needed to build capable language models.

See all 159 papers

torchtune: PyTorch native post-training library
May 20, 2026Mark Obozov, Maxime Griot, Joseph Cummings +8LLM Fine-Tuning Efficient Language Model Training
Work done at PyTorch, Meta · Stanford · Meta-FAIR
Beyond the Trial-and-Error Loop: Hybrid Projection and Automated Tuning for Distributed Training
Oct 14, 2026Anshu Raina, Peyman Razaghi, Yuankai Chen +7Distributed Training LLM Training
Advanced Micro Devices, Inc.
AutoTuneBench: Trustworthy Measurement for Agent Auto-Tuning of LLM Serving Engines
Sep 16, 2026Li ChenGPU Kernel Optimization Benchmark Design

May 20, 2026J/K move · Enter open · S save

torchtune: PyTorch native post-training library

Mark Obozov, Maxime Griot, Joseph Cummings, Evan Smothers+7

Work done at PyTorch, Meta · Stanford · Meta-FAIR

LLM Fine-Tuning Efficient Language Model Training LLM Post-Training Fine-Tuning Language Models

Open paper AskPDF arXiv

Modern LLMs typically require multistage training pipelines to achieve strong downstream performance, with post-training serving as the main interface for adapting open-weight models. We introduce torchtune, a PyTorch-native library designed to streamline the post-training lifecycle of LLMs, enabling efficient fine-tuning, experimentation, and deployment-oriented workflows. Unlike many existing fine-tuning frameworks, which often optimize for ease of use, specialized recipes, or hardware efficiency at the cost of transparency and extensibility, torchtune emphasizes modularity, hackability, and direct access to the underlying PyTorch components. In this paper, we present the design principles behind torchtune, describe how they are reflected in its model builders, training recipes, and distributed training stack, and evaluate the library across representative post-training settings. We compare against popular fine-tuning frameworks, including Axolotl and Unsloth, and show that torchtune provides strong performance and memory efficiency across many settings while remaining flexible enough for rapid research iteration. These results position torchtune as a practical foundation for reproducible LLMs post-training research.

Abstract

Efficient Language Model Training

Explore similar work

torchtune: PyTorch native post-training library

Beyond the Trial-and-Error Loop: Hybrid Projection and Automated Tuning for Distributed Training

AutoTuneBench: Trustworthy Measurement for Agent Auto-Tuning of LLM Serving Engines