cs.LGOct 24, 2026

Reactive Optimizers

Authors: Xingjian Bai, Jia-Hao Liang, Jayden Teoh, Vincent Sitzmann

Organizations: Massachusetts Institute of Technology

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

We study a reactive optimizer that uses a pretrained language-model agent to inspect training dynamics and rewrite executable optimizer code during a single training run. The agent can change update rules and schedules for individual parameter groups, without optimizer-specific meta-training or memory transferred from other runs. We evaluate this approach, OPAL, on language modeling and image generation when a recipe has not been tuned for the target setting. Across the evaluated settings, it matches or improves on selected fixed recipes. Analysis of its decisions shows changes to learning-rate schedules, optimizer structure, numerical precision, and momentum. Controlled experiments also show adaptation to repeated data corruption and reduced overfitting. We report these behaviors together with the agent's computational cost and implementation requirements.

Figures & tables

Figure 1: OPAL (OPtimizer with an Agent in the Loop), our reactive optimizer, across different settings. In most settings without heavily tuned optimization hyperparameters, reactive optimizers improve the training loss and test performance upon the canonical choices. In modded-nanogpt, #2 is AdamW, #21 is Shampoo, #36 is a tuned Muon, and #30 and #46 are more elaborate Muon-based recipes.
Figure 2: The two programs the agent writes and rewrites iteratively during training. With code as the entire interface, we give the agent maximal freedom to choose what to watch and what to change.
Figure 3: Training trajectories and code edits in four OPAL runs. Blue curves show OPAL’s loss alongside the corresponding fixed-recipe baselines; ticks below each curve mark decisions and distinguish code edits. The edit patterns differ across runs, reflecting module-specific changes made at different stages of training. The curves show the outcomes of the complete runs, rather than isolating the effect of each edit.
Figure 4: DeepSeek-V3’s learning rate, designed by hand (upper left), and learning rates OPAL set in four runs, as multiples of each recipe’s rate. The schedules vary across tasks and parameter groups and include mid-run rate increases as well as cooldowns. These are observed trajectories, rather than schedules specified before training.
Figure 5: Changes of optimizer type and of precision in four OPAL runs. In some runs, the weights are simply moved to standard Muon, while in others, OPAL moves them gradually by module type and chooses high precision for a very small part of the weights, citing the signals it observed.
Figure 6: A variant of the tilted double-well potential, one of the 34 landscapes we test optimizers on. The dashed line uses the blog’s static Preset A, while the blue line traces the reactive optimizer. The reactive optimizer raises its momentum along the gentle slope, lowers it as the gradient falls, and clears it when it overshoots, reaching the optimum where common optimizers at their default settings stall.
Figure 7: ResNet-18 on CIFAR-10 with four stretches of random images and labels, shaded. Left: test loss; markers indicate changes to OPAL’s safeguards. Right: the change in test accuracy in each stretch. Reactive agents can defend against the designed type of loss spikes, induced by data corruption, after seeing it a few times. Clean segments use Gaussian smoothing in log loss ( σ=300\sigma=300 updates); the first 1,000 updates, corruption intervals and two evaluations after each interval remain unsmoothed.
Figure 8: ResNet-18 trained for 120 epochs on 45,000 CIFAR-10 images, 10% with a wrong label. At epochs 24 and 28, dev loss climbs and the agent reacts. OPAL counters the overfitting with progressively higher learning rates and a weight average it chooses on the development set. OPAL ends at a test loss of 0.685, compared with 2.493 for the final AdamW weights and 0.832 for its development-selected weight average; the test set is not used by the agent.
Figure 9: Wall-clock time relative to the same task trained without an agent. Training that waits for the agent at every decision takes much longer (1.30× to 4.39×), while training that does not wait adds only a little time on top of the static run (1.01× to 1.14×). That remaining overhead is the cost of running what the agent wrote: an update rule that costs more per step than the recipe’s (such as Muon’s orthogonalization, or keeping a tensor in float32) and the measurement cost of its analysis program.
More in this topic

Neural Network Optimization

Neural network optimization studies algorithms and training dynamics that help models learn efficiently, reliably, and generalize well.

See all 332 papers

Explore similar work

CardsList

  1. RSI-Opt: Recursive Self-Improvement Optimizer Agents For Model Training

    Oct 23, 2026Binghang Lu, Yige Sun, Guang LinLarge Language Model-Guided OptimizationLearning to Optimize

    Purdue University · Independent Researcher

  2. The Best Optimizer Depends on Batch Size

    Oct 6, 2026Xingyu Dang, Kaiyue Wen, Sadhika MalladiDeep Learning OptimizationHyperparameter Transfer

    Princeton University · Stanford University · University of California, San Diego

  3. Harnessing Language Models as Optimization Modeling Agents

    Oct 22, 2026Yunxiang Luo, Xin Xu, Tianhao Chen +7Large Language Model-Guided OptimizationNatural Language to Optimization Modeling

    The Hong Kong University of Science and Technology · Huawei