FrontierMath Erdős

Tom Adamczewski; Thomas F. Bloom

cs.CLSep 6, 2026

FrontierMath Erdős

Authors: Tom Adamczewski, Thomas F. Bloom

Organizations: Epoch AI · University of Manchester

Abstract

We introduce FrontierMath Erdős (FME), a benchmark of 68 Erdős problems that are open as of August 2026. To solve a task in FME, AI systems must resolve (prove or disprove) one of the 68 conjectures in the proof assistant Lean. Our 68 problems were selected by the second author among 652 open problems on erdosproblems.com for their mathematical interest and difficulty. AIs have recently resolved several open problems in mathematics, but these demonstrations fall short of a systematic study of AI capabilities. FME evaluates every AI model on the same fixed problems, autonomously and under the same budget. We evaluated five AIs with a budget of $300 per problem. One (GPT-6 Astra) scored 3%, and all others scored 0%.

Figures & tables

Model	Score
GPT-6 Astra (pre-release)	3%
GPT-5.6 Sol	0%
GPT-5.5	0%
Claude Fable 5.1	0%
Claude Fable 5	0%

Table 1: Benchmark scores in the default configuration: one attempt per conjecture, $300 and 72 hours of working time per attempt. The score is the percentage of the 68 conjectures resolved.

Problem	Cost	Working time
#74	$218	15 h
#126	$247	16 h

Table 2: Cost and working time of the two resolutions in the default configuration.

Problem	Result	Resolved in	Cost (working time) of each resolution
#1	disproof	2 of 4 attempts	$405 (27 h),$ 1,384 (84 h)
#74	disproof	6 of 6 attempts	$47 (5 h),$ 84 (6 h), $150 (8 h),$ 183 (12 h), $218 (15 h),$ 271 (19 h)
#126	proof	4 of 4 attempts	$154 (8 h),$ 194 (9 h), $247 (16 h),$ 249 (17 h)
#548	proof	1 of 3 attempts	$363 (20 h)
#571	proof	1 of 3 attempts	$617 (41 h)

Table 3: All resolutions by GPT-6 Astra across every attempt, including the non-systematic additional attempts described in the text. Unlike Table 2 , this is not a benchmark result: attempts differ in budget and agent configuration, and conjectures were attempted different numbers of times. “Resolved in” counts attempts that had reached a verdict by the time of writing.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

ID	Conjecture
Erdos1	If every subset of $A\subseteq\{1,\ldots,N\}$ has a distinct sum, then $N\gg 2^{\|A\|}$ .
Erdos3	If $A\subseteq\mathbb{N}$ has $\sum_{n\in A}1/n=\infty$ , then $A$ contains arbitrarily long arithmetic progressions.
Erdos5	Every $C\geq 0$ is a limit point of the normalised prime gaps $(p_{n+1}-p_{n})/\log n$ .
Erdos7	There is a covering system of the integers all of whose moduli are odd and greater than $1$ .
Erdos20	Sunflower conjecture: the minimal $f(n,k)$ such that every family of $f(n,k)$ many $n$ -element sets contains a $k$ -sunflower satisfies $f(n,k)<c_{k}^{n}$ .
Erdos23	Every triangle-free graph on $5n$ vertices can be made bipartite by deleting at most $n^{2}$ edges.

Appendix

Table 4: The 68 conjectures of FrontierMath Erdős.

Explore similar work

Riemann-Bench: A Benchmark for Moonshot Mathematics
Apr 8, 2026Suhaas Garre, Erik Knutsen, Sushant Mehta +1Mathematical Reasoning Benchmarks Benchmark Design
Surge AI
Measuring Progress in Reasoning Toward Mathematical Discovery with Automatic Verification
Mar 16, 2026Erik Y. Wang, Sumeet R. Motwani, James V. Roggeveen +9Mathematical Reasoning Benchmarks LLM Mathematical Reasoning
Stanford University · Benchmark · University of Oxford +3
Advancing Mathematics Research with AI-Driven Formal Proof Search
May 21, 2026George Tsoukalas, Anton Kovsharov, Sergey Shirobokov +18Automated Theorem Proving AI Agent Evaluation
1Google DeepMind1 · 2Aarhus University · 3Google

Apr 8, 2026J/K move · Enter open · S save

Riemann-Bench: A Benchmark for Moonshot Mathematics

Suhaas Garre, Erik Knutsen, Sushant Mehta, Edwin Chen

Surge AI

Mathematical Reasoning Benchmarks Benchmark Design AI Agent Benchmarks LLM Mathematical Reasoning

Open paper AskPDF arXiv

Recent AI systems have achieved gold-medal-level performance on the International Mathematical Olympiad, demonstrating remarkable proficiency at competition-style problem solving. However, competition mathematics represents only a narrow slice of mathematical reasoning: problems are drawn from limited domains, require minimal advanced machinery, and can often reward insightful tricks over deep theoretical knowledge. We introduce Riemann-Bench, a private benchmark of expert-curated problems designed to evaluate AI systems on research-level mathematics that goes far beyond the olympiad frontier. Problems are authored by Ivy League mathematics professors, graduate students, and PhD-holding IMO medalists, and routinely took their authors weeks to solve independently. Each problem undergoes double-blind verification by two independent domain experts who must solve the problem from scratch, and yields a unique, closed-form solution assessed by programmatic verifiers. We evaluate frontier models as unconstrained research agents, with full access to coding tools, search, and open-ended reasoning, using an unbiased statistical estimator computed over 100 independent runs per problem. Our results reveal that all frontier models currently score below 10%, exposing a substantial gap between olympiad-level problem solving and genuine research-level mathematical reasoning. By keeping the benchmark fully private, we ensure that measured performance reflects authentic mathematical capability rather than memorization of training data.

Abstract

Explore similar work

Riemann-Bench: A Benchmark for Moonshot Mathematics

Measuring Progress in Reasoning Toward Mathematical Discovery with Automatic Verification

Advancing Mathematics Research with AI-Driven Formal Proof Search