cs.AINov 5, 2026
SaveSwarmEvolve: Latent Agent Swarms for Evolutionary Discovery and Optimization
Organizations: DIVISION OF APPLIED MATHEMATICS, BROWN UNIVERSITY
Abstract
LLM-guided evolutionary search uses AI agents to improve programs for mathematical and computational problems by iteratively refining solutions and combining high-scoring methods. However, the process is often slow because the LLM generates a long textual chain of thought for each round and must repeat this generation many times to maintain population diversity. We present SwarmEvolve, a framework in which hundreds of low-cost agents per round work in swarms, reason in latent space without decoding to text, and share information by passing KV caches. Each swarm studies the problem from a different angle. A novelty filter keeps the most distinct agent ideas by directly measuring the distances between the agents' hidden state vectors. Crossover of ideas happens naturally in latent space, as a summarizer agent receives the combined KV caches of all swarm members. On 21 problems in mathematical discovery and scientific computing, SwarmEvolve gets the highest average score and needs a median of only 3 to 8 minutes to reach the best scores that other state-of-the-art frameworks reach in 60 minutes. On several problems, SwarmEvolve with Qwen3 8B improves on the solutions that AlphaEvolve first found with frontier LLMs. Our work shows that agent swarms that communicate in latent space offer a fast, low-cost approach to program search for scientific problems.
Figures & tables
Figure 1: SwarmEvolve (ours) compared with OpenEvolve, ShinkaEvolve, and CodeEvolve on 21 problems, with Qwen3 8B on one GPU for 60 minutes per run. Left: total number of programs generated. Middle: mean best score at 15 minutes. Right: median minutes to reach OpenEvolve’s 60-minute score. SwarmEvolve has a higher throughput and better scores.
Figure 2: Workflow of SwarmEvolve. In each round, SwarmEvolve launches 10 swarms of 45 latent agents. Each swarm draws parent programs from the archive and gets a focus prompt to refine, hybridize, explore, or start fresh. In each swarm, the agents take 40 latent reasoning steps, a novelty filter removes the 15 most similar agents (hollow circles), and the summarizer reads the KV caches of the rest to write one new program. The new programs are evaluated and added to the archive. A parameter tuning agent tunes the hyperparameters of the best program.
Figure 4: Minutes that SwarmEvolve needs to reach the best score that OpenEvolve (left), ShinkaEvolve (middle), and CodeEvolve (right) reach in 60 minutes. All frameworks use Qwen3 8B and run for three seeds. Hollow circles mark problems where SwarmEvolve does not reach the target within 60 minutes. On most problems, SwarmEvolve reaches in a few minutes what the baselines reach in one hour.
Figure 5: Mean hidden states of the latent agents for a circle packing problem with Qwen3 8B, projected onto their first two principal components: the 450 latent agents of all 10 swarms with the swarm centers (left), and the 45 latent agents of one exploration-focused swarm, each reading different parent programs (right). The swarms occupy distinct regions, and the novelty filter keeps latent agents from every cluster.
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
| Qwen3 8B 45 agents | Text reasoning | |||
| Group | Problem | Latent (40 steps) | 40 tokens | 256 tokens |
| Packing | Circles in a square | 0.998 | 0.997 | 0.996 |
| Circles in a rectangle | 0.888 | 0.868 | 0.887 | |
| Hexagons, | 0.627 | 0.509 | 0.544 | |
| Hexagons, | 0.707 | 0.738 | 0.600 | |
| Geometry | Heilbronn, triangle | 0.558 | 0.796 | 0.596 |
Supplementary Table 1 : Best score within 60 minutes, averaged over three runs. Each agent takes 40 latent reasoning steps (Latent, the default) or writes up to 40 or 256 tokens of text reasoning, which the summarizer reads as text in its prompt. Bold: best; underlined: second best.
Supplementary Figure 1 : Mean best score against the number of evaluated programs with Qwen3 8B, for 0 to 75 latent agents per swarm and the three baselines, over the first 150 programs (left) and the full 60-minute runs (right). More latent agents raise the score early in the search, and the settings converge later.
Supplementary Figure 2 : Paired gain in mean best score relative to 0 latent agents per swarm, plotted against the number of evaluated programs with Qwen3 8B, with 95% bootstrap intervals. The gains are largest in the first 100 programs and shrink within their intervals later in the search.
| Stage | Ours | OpenEvolve | ShinkaEvolve | CodeEvolve |
| Reading prompts | 7.1 min (12%) | 1.0 min (2%) | 0.5 min (0.9%) | 0.1 min (0.2%) |
| Text reasoning | – | 52.5 min (87%) | 34.4 min (57%) | 46.8 min (78%) |
| Latent reasoning | 4.1 min (7%) | – | – | – |
| Latent communication | 2.6 min (4%) | – | – | – |
| Writing programs | 39.4 min (66%) | 6.4 min (11%) | 5.7 min (9%) | 4.7 min (8%) |
| Novelty checks | 0.05 min (0.1%) | – | 4.5 min (7%) | – |
Supplementary Table 2 : Minutes (and shares) of a 60-minute run that each framework spends on each stage of its GPU work, averaged over its runs on the 21 problems with Qwen3 8B and three seeds.
| Agent | Temperature | Top- | Top- | Maximum new tokens | |
| Summarizer | Refinement-focused swarm | 0.3 | 0.95 | 20 | 4096 |
| Hybridization-focused swarm | 0.5 | 0.95 | 20 | 4096 | |
| Exploration-focused swarm | 0.8 | 0.95 | 20 | 4096 | |
| Fresh-start swarm | 0.8 | 0.95 | 20 | 4096 | |
| Parameter tuning agent | 1.0 | 0.95 | 20 | 24 | |
| Hyperparameter | Value | ||||
Supplementary Table 3 : Hyperparameters of SwarmEvolve used throughout all experiments in this work.
| Aspects (refinement and hybridization) | Perspectives (exploration and fresh start) |
| The starting point of the solution | The standard textbook method |
| The optimization or solution algorithm | A fundamentally different algorithm |
| Global exploration, restarts, and annealing | A known construction from the literature |
| Symmetries and structure of good solutions | A physics-inspired method |
| Feasibility, tolerances, and a final repair step | A parametrization that satisfies the constraints |
| The use of the time limit | Many cheap starts of one fast local optimizer |
Supplementary Table 4 : Focuses of the swarms. A refinement- or hybridization-focused swarm takes one of the 14 “aspects”, and an exploration-focused or fresh-start swarm takes one of the 14 “perspectives”. Each latent agent receives the focus in one of five phrasings, such as “Pay particular attention to” an aspect or “Consider” a perspective, followed by one of nine ways of thinking, such as “Reason from first principles” or “Question one assumption the program makes” for sample diversity.
Supplementary Figure 16 : Best packings of 26 circles in the unit square. The sums of the radii are 2.636 (our method), 2.615 (OpenEvolve), 2.626 (ShinkaEvolve), and 2.614 (CodeEvolve).
Supplementary Figure 17 : Best placements of 11 points in the equilateral triangle. The shaded triangle has the smallest area among all triangles formed by the points. The smallest areas, divided by the area of the triangle, are 0.0223 (our method), 0.0016 (OpenEvolve), 0.0200 (ShinkaEvolve), and 0.0222 (CodeEvolve).
Supplementary Figure 18 : Solutions of the Burgers equation for (top) and their absolute errors (bottom). Our solution is the most accurate away from the shock at .
Supplementary Figure 19 : Solutions of the convection-diffusion equation for (top) and their absolute errors (bottom). The errors concentrate in the boundary layers along and and along the characteristic line from the corner . The coarser grid of CodeEvolve spreads the error near over a wider region.
Supplementary Figure 20 : Real parts of the solutions of the Helmholtz equation for (top) and their absolute errors (bottom). The error decreases with the number of grid points per wavelength.
Supplementary Figure 21 : Best score so far against wall-clock minutes for each problem with Qwen3 8B. Each line is the best score so far averaged over three runs, and each band spans the lowest and highest run. SwarmEvolve ends the hour with the highest score on 18 of the 21 problems, and its curves rise most steeply in the first few minutes.
Supplementary Figure 22 : Best score so far against wall-clock minutes for each problem with Qwen3 14B. Each line is the best score so far averaged over three runs, and each band spans the lowest and highest run. SwarmEvolve ends the hour with the highest or tied-highest score on 16 of the 21 problems.
Supplementary Figure 23 : Best score so far against wall-clock minutes for each problem with Qwen3 32B-FP8. Each line is the best score so far averaged over three runs, and each band spans the lowest and highest run. SwarmEvolve ends the hour with the highest or tied-highest score on 17 of the 21 problems.