InfraRSI: Harnessing Recursive Self-Improvement for Training Infrastructure
Abstract
While Recursive Self-Improvement (RSI) accelerates isolated algorithm and kernel synthesis, distributed LLM training depends on tightly coupled launch settings, operator kernels, and communication schedules, where single edits often hit out-of-memory (OOM) boundaries, compiler conflicts, measurement jitter, or silent numerical drift. We present InfraRSI, the agentic harness that turns open-ended human commands into verified training optimizations via a five-stage workflow: init locks editable scope and a fixed task contract over workload, loss, and timing invariants to prevent reward hacking, doctor profiles baseline phases and calibrates cluster noise, run co-optimizes launch knobs and code patches before re-verifying top candidates against the baseline and logging findings to a cross-hardware experience bank, and report and apply audit source paths, track failed-trial costs, and commit promoted changes. Rather than stepping greedily from a single incumbent, InfraRSI initializes from diverse experience-seeded configurations and fits global execution-time and censored peak-memory models across all trials, including OOMs. Using these models, it prices per-setting memory costs, revises miscalibrated LLM priors from live runs, and pairs memory-hungry optimizations with memory savers and verified code patches. Across four single- and multi-node dense and Mixture-of-Experts (MoE) LLM workloads, InfraRSI achieves a speedup of up to and lifts mean Model FLOPs Utilization (MFU) from to , outperforming existing tuners and coding agents while transferring experience across GPUs. As it improves training infrastructure efficiency, InfraRSI forms a two-level Recursive Self-Improvement loop---revising priors online within runs and distilling experience across clusters---to push foundation model scaling.
Figures & tables
| Dimension | Algorithm / Code RSI [ 33 , 25 , 14 ] | Kernel Synthesis [ 26 , 16 ] | Training Infra RSI (InfraRSI) |
|---|---|---|---|
| Granularity | Single heuristic or script | Isolated CUDA/Triton kernel [ 39 ] | Launch knobs & source-code patches (multi-stack) |
| Cost & Noise | ms–s; – runs; low noise | ms/op; – runs; low noise | Multi-GPU; 10–20 runs; – noise |
| Equivalence | Test suite or val score | Tensor tolerance ( ) | Fixed task contract + source-path audit |
| Hacking Risk | Moderate (val overfitting) | High (caching, skipped sync) | Severe (prefetch skew, timer shift, precision drift) |
| HW Sensitivity | Hardware-agnostic logic | GPU-specific (SM, SRAM) | Roofline/topology-sensitive (H100/H200/H20) |
| Reusable Asset | Task-specific code pool | Op kernel library | Cross-hardware priors + audited patch sites |
| Layer | Invariants | Core criterion | Offline triggers / primary |
|---|---|---|---|
| L1: Execution | completes , no_errors , memory_cap | Exit , full target update count completed, no NaN / OOM / NCCL / import error in main run, VRAM within cap | 503 / 352 ( completes 349, no_errors 153, memory_cap 1) |
| L2: Workload | same_workload , same_data , same_model , frozen_files | Same tokens per update and sequence length, per-update data digest, model digest and parameter count, frozen-file SHA-256 hashes; unverifiable required items reject | 7 / 7 (unverifiable item 7) |
| L3: Schedule | optimizer_steps | Same actual optimizer-step count and synchronous auxiliary state updates inside the timed window | 0 / 0 (offline; deferred updates caught in source audit) |
| L4: Numerical | loss_tol , grad_norm_tol , precision | Pre-declared task loss rule (window-mean within tolerance, or strict per-update max deviation when required); permitted precision menu enforced | 42 / 41 ( loss_tol 42; 39 pass without L4) |
| L5: Timing & Pair | timing_consistent , speed_reproduces | Harness wall-clock gain agrees with reported step-time gain ( , flagging finite-prefetch or log-boundary skew); promotion requires back-to-back fresh re-test beyond calibrated noise | 158 / 0 ( timing_consistent 158; unpromoted runs deliver ) |
| Offline re-verification: 1,212 executions, 400 rejected (33.0%, 710 triggers), 812 valid ; verifier unit suite reproduces 40/40 expected verdicts (28 perturbed rejected, 12 benign passed). Excluded candidates retain full API and GPU costs in the search ledger. | |||
| Model | Mechanism | H20 (148 TF) | H100/200 (989 TF) | Roofline Cause & Rule |
|---|---|---|---|---|
| Qwen3-8B | precision=fp8 | ( TP2) | H200 ( TP2) | Compute-bound H20 vs. comm-bound H200; scaling. |
| Qwen3-8B | tp=2 | ( ) | ( H100) | Sign flip: AllReduce overhead exceeds GEMM gain. |
| Qwen3-8B | te_fp8_attn | ( ) | ( H200) | Sign flip: H200 BF16 FlashAttention-3 beats FP8 cast. |
| Dense LLM | compile | ( TP2) | ( H200) | Amortizes H20 launch; fuses H200 TP2 collectives. |
| Qwen3-30B MoE | alltoall+grouped_gemm | ( ) | (H200) | Consistent kernel gain on high-bandwidth NVLink. |
| Workload & Hardware | Base MFU | OptiCo | Auto-Research | RE-Bench | InfraRSI | Key Discovered Knob + Code Combination |
|---|---|---|---|---|---|---|
| Qwen3.5-9B (8 H100) | ( ) | ( ) | ( ) | ( ) | ( ) | Memory-ledger saver pairing (Liger frees VRAM for recompute-off + micro-batch 2) + fused normalization |
| Gemma-4-26B-A4B MoE (8 H200) | ( ) | ( ) | ( ) | ( ) | ( ) | Synergistic memory completion ( torch.compile + Liger-off + recompute-off) + cached FlexAttention |
| Qwen3.8-27B (8 H200) | ( ) | ( ) | ( ) | ( ) | ( ) | Memory-saver pairing + fused RMSNorm/RoPE/SwiGLU kernels + selective activation recomputation |
| Qwen3.5-35B-A3B MoE (16 H100, 2 Nodes) | ( ) | ( ) | ( ) | ( ) | ( ) | Contrarian 32-channel NCCL tuning + composite patch overlay (fused SwiGLU + bucketed grad norm) |
| 4-Workload Mean (Geomean / Mean MFU) | ( ) | ( ) | ( ) | ( ) | Consistent gains across single- and multi-node dense and MoE training |
| Target Workload & Hardware | Experience Condition | First Trial | Final Speedup | Trials to 90% Peak | GPU-h to 90% Peak |
|---|---|---|---|---|---|
| Qwen3.5-9B (8 H100) | Cold start (no prior packs) | (Trial 2) | (censored) | — | |
| Qwen3.5-9B (8 H100) | All eligible packs | (Trial 1) | (censored) | — | |
| Qwen3.5-9B (8 H100) | Cross-SKU transfer | (Trial 1) |
| Part A: Independent Offline Verifier & Receipt Audit | ||||||
| Suite | Audit Scope & Test Cases | Accepted | Rejected | Agreement | Escapes w/o L4 | Verified Invariants & Failure Modes Caught |
| Receipt Matrix | 40 released system tests (28 single-perturbation + 12 benign) | 12 benign | 28 perturbed | 40/40 (100%) | — | Halved/missing tokens, sequence length, step count, LR schedule, data order, early exit 0, NaN/frozen loss, OOM/traceback, memory cap, edited frozen files, clock skew, model digest |
| HMAC-SHA256 | Cryptographic receipt integrity test | 0 | All invalid | 100% | — | Rejects unsigned receipts, post-signing step-time edits, and mismatched receipt schemas |
| Historical Corpus | Offline re-judge of 1,212 stored executions | 812 (67.0%) | 400 (33.0%) | — | 39 (3.2%) | 39 executions pass non-numerical checks and require the L4 numerical loss/gradient layer to reject |
| Part B: InfraRSI Tolerance & Fixed-Noise Ablation on Qwen3.5-9B (8 H100, ) | ||||||
| Arm | Setting (Qwen3.5-9B, 8 H100) | Valid / Comp. | Final Speedup | 90% Peak | Search Cost | Verification & Resource Accounting Notes |
| System / Paradigm | Primary Role | Stacks | Edit Scope | Contract | Noise Gate | Mem. Model | Escape | Code Audit | X-HW Prior |
|---|---|---|---|---|---|---|---|---|---|
| FunSearch / AlphaEvolve [ 33 , 25 ] | Program evolution | Code | |||||||
| Darwin Gödel / AIDE [ 45 , 14 ] | Agentic tree search | Code | |||||||
| TVM / Triton / Inductor [ 4 , 39 , 2 ] | Compiler / kernel IR | ||||||||
| KernelBench / AI CUDA [ 26 , 16 ] | Isolated kernel synth. | Kernel | |||||||
| Alpa / Galvatron / nnScaler [ 47 , 22 , 20 ] | Auto-parallelism planner | Graph/Plan | |||||||
| OptiCo [ 3 ] | Collaborative knob tuner | Knobs |
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
| Lay | Invariant ID | Mode | Implemented Predicate | Audit (1,212) |
| L1 | completes | Always | Exit code 0, no timeout, and every requested update ran ( ) | 349 (349 pri.) |
| no_errors | Always | No NaN/Inf loss, OOM, NCCL/CUDA error or traceback line (collective lines only after the last update of a clean exit are reported, not failed); loss and gradient norm not frozen across updates | 153 (2 pri.) | |
| memory_cap | Always | Peak reserved device memory | 1 (1 pri.) | |
| L2 | same_workload | Always | Tokens (or samples) per update and the sequence length equal the reference at every update, and every declared static invariant (e.g. the global batch) is unchanged; unverifiable unchecked, which rejects because the item is required | 0 failed; 7 rejected as unverifiable |
| same_data | Always | The per-update data digest equals the reference’s (unchecked where the job logs none) | none | |
| same_model | Always | Model digest and trainable-parameter count equal the reference’s; a candidate that stops reporting a key the reference reported fails | none |
Efficient Language Model Training
Efficient language model training studies how to reduce the data, compute, memory, time, energy, and cost needed to build capable language models.
See all 171 papers