Automated Algorithm Discovery

Overview

Automated algorithm discovery studies how to construct or improve executable procedures—such as heuristics, numerical solvers, learning rules, and hardware policies—without specifying every design choice by hand. Common approaches use evolutionary or tree search with language models to modify code, program synthesis over reusable components, reinforcement learning, or learned search controllers; candidates are assessed with benchmarks, simulators, formal checks, and staged evaluation. Persistent memory, modular representations, diversity archives, and adaptive budget allocation help reuse findings and reduce search stagnation. A central challenge is establishing that gains reflect sound, useful algorithms rather than benchmark overfitting, noisy evaluations, recombinations of human designs, or exploitable evaluators—and that they transfer under real-world constraints.

Latest papers

181 papers15 papers in the last four weeks, down 58% on the four weeks before. 0.1% of all new papers.

Feb 10, 2026

Learning to Discover Iterative Spectral Algorithms

We introduce AutoSpec, a neural network framework for discovering iterative spectral algorithms for large-scale numerical linear algebra and numerical optimization. Our self-supervised models adapt to input operators using coarse spectral information (e.g., eigenvalue estimates and residual norms), and predict recurrence coefficients for computing or applying a matrix polynomial tailored to a downstream task. The effectiveness of AutoSpec relies on three ingredients: an architecture whose inference pass implements short, executable numerical linear algebra recurrences; efficient training on small synthetic problems with transfer to large-scale real-world operators; and task-defined objectives that enforce the desired approximation or preconditioning behavior across the range of spectral profiles represented in the training set. We apply AutoSpec to discovering algorithms for representative tasks on spd matrices: accelerating matrix function approximation; accelerating sparse linear solvers; and spectral filtering/preconditioning for eigenvalue computations. On real-world matrices, the learned procedures deliver up to order-of-magnitude improvements in accuracy and/or reductions in iteration count, relative to spectrum-agnostic baselines. We find clear connections to classical theory: the induced polynomials may exhibit equioscillation behavior characteristic of Chebyshev polynomial approximation. The code is available at: https://github.com/zihanghliu/AutoSpec .