The Endless Exam: Mathematical Constructions from Today's Models toward Superintelligence

Muhan Zhang

cs.AISep 21, 2026

The Endless Exam: Mathematical Constructions from Today's Models toward Superintelligence

Authors: Muhan Zhang

Organizations: The University of Texas at Arlington

Abstract

We introduce the Endless Exam, a benchmark spanning fourteen parameterised families of mathematical construction problems, with verifiable scores that distinguish progress before and beyond published mathematical frontiers. Each submitted object is checked automatically for validity and assigned a relative quality score against a published frontier or construction baseline, without capping improvements at 1. The benchmark draws long-term challenges from open mathematical problems and generates larger instances by varying their parameters. Compact certificates allow large constructions to be verified without listing every element. Across nine models evaluated on 69 distinct instances, continuous quality scores distinguish performance even though none of the 30 published-frontier references is surpassed. Size-quality curves show how construction quality changes as problem size increases. We release the generators, verifiers, references, model responses and analysis to support continued measurement before and beyond human frontiers.

Figures & tables

Dataset	Problems	Difficulty	Unsolved	Non-proof	Open eval.	Auto- check	Uncapped score
FrontierMath	338	Univ.–Res.	$\times$	$\checkmark$	$\times$	$\checkmark$	$\times$
FrontierMath: Open Problems	50	Research	$\checkmark$	$\checkmark$	$\times$	$\checkmark$	$\times$
IMProofBench	77	Research	$\checkmark$	$\times$	$\times$	$\times$ ∗	$\times$
First Proof	10	Research	$\times$	$\times$	$\times$	$\times$	$\times$
Erdős Problems	1,217	Research	$\checkmark$	$\times$	$\times$	$\times$	$\times$
IMO-AnswerBench	400	Olympiad	$\times$	$\checkmark$	$\checkmark$	$\checkmark$	$\times$

Table 1: Comparison of mathematical benchmarks and problem collections, using the cited releases. Unsolved indicates inclusion of open research targets. Our 69 instances span 14 families. Scalable denotes larger instances generated by increasing parameters under the same rules. ∗ IMProofBench’s main proofs are human-graded; follow-up answers are automatically checked.

		Mean relative quality
configuration	Overall score	Published frontiers (30)	Construction baselines (39)	Valid fraction	Gap closed
Qwen3.5-4B high	7.55	0.07 [0.03, 0.11]	0.08 [0.03, 0.15]	0.38	0.000
Qwen3.5-9B high	14.52	0.12 [0.07, 0.17]	0.17 [0.07, 0.27]	0.43	0.000
Qwen3.5-27B high	16.18	0.13 [0.06, 0.22]	0.18 [0.09, 0.29]	0.52	0.000
Qwen3.8-27B high	38.07	0.31 [0.20, 0.42]	0.44 [0.28, 0.59]	0.62	0.004
GPT-5.6 Luna medium	16.70	0.16 [0.10, 0.24]	0.17 [0.08, 0.27]	0.57	0.000

Table 2: Results on 69 instances. The final three rows use code and web access (Section 3.5 ). Overall score is uncapped. Brackets show

95\%

confidence intervals. Validity uses all 69 responses; gap closed uses 64 instances with proven bounds (Section 4.2 ).

Appendix figures & tables22 assets

Supplementary material from the paper’s appendix.

Appendix

family	construction	objective	class	reference; bound or target
cap set	$S\subseteq\mathbb{F}_{3}^{d}$ ; no three collinear points (cap set)	$\|S\|$	R	published; proven bound
spherical codes	vectors with exact coordinates; pairwise angle $\geq\theta$ ; $60^{\circ}$ or variable	count	R	mixed; proven bound
matrix multiplication	bilinear scheme for $(n\times m)$ by $(m\times p)$ matrices; computes the product exactly	rank $r$ (min)	S	published; proven bound
degree–diameter	graph with max degree $d$ ; diameter $\leq k$	vertices	S	published; proven bound
covering	$k$ -subsets (blocks) of $[v]$ ; every $t$ -subset lies in a block	blocks (min)	S	published; proven bound
Schur	$k$ -colouring of $\{1..N\}$ ; no monochromatic $x,y,x{+}y$	$N$	R	published; proven bound

Appendix

Table 3: The fourteen families. R denotes search-resistant, S families that can benefit from search and U unclassified search sensitivity. These labels concern the reference search procedures studied here. Spherical codes and Heilbronn each contain two variants. Reference types may depend on the parameters.

Family	Parameters	Reference	Baseline
cap set	d=6	112	112
cap set	d=8	512	512
cap set	d=10	2432	2240
cap set	d=12	12928	12544
spherical codes ( $60^{\circ}$ )	d=8	240	240
spherical codes ( $60^{\circ}$ )	d=10	510	510

Appendix

Table 5: Problem-size experiments: instances and references. An asterisk marks a construction baseline.

Family	Size	Qwen3.8-27B high	GPT-5.6 Luna high	DeepSeek V4.1 Flash low	GPT-6 Astra high
cap set	$d=6$	0.00 (0/2)	0.36 (1/2)	0.65 (2/2)	1.00 (2/2)
cap set	$d=8$	0.32 (1/2)	0.78 (2/2)	0.57 (2/2)	0.91 (2/2)
cap set	$d=10$	0.63 (2/2)	0.63 (2/2)	0.60 (2/2)	0.88 (2/2)
cap set	$d=12$	0.31 (1/2)	0.51 (2/2)	0.41 (2/2)	0.97 (2/2)
spherical codes ( $60^{\circ}$ )	$d=8$	1.00 (2/2)	0.00 (0/2)	1.00 (2/2)	1.00 (2/2)
spherical codes ( $60^{\circ}$ )	$d=10$	0.30 (1/2)	0.67 (2/2)	0.82 (2/2)	0.98 (2/2)

Appendix

Table 6: Mean relative quality at each problem size. Parentheses give valid responses out of the total; means include zero scores.

configuration	Shannon relative quality	valid	Trifference relative quality	valid	words at $n=192$
Qwen3.5-4B high	0.053	4/4	0.000	0/4	0
Qwen3.5-9B high	0.053	4/4	0.000	0/4	0
Qwen3.5-27B high	0.297	4/4	0.019	2/4	0
Qwen3.8-27B high	0.504	3/4	0.501	3/4	$4.30\times 10^{7}$
GPT-5.6 Luna medium	0.044	3/4	0.000	0/4	0
GPT-5.6 Luna high	0.705	4/4	0.250	1/4	$1.05\times 10^{10}$

Appendix

Table 7: Tool-free code-family results on four instances per family, including zero scores. A ratio of

1

matches the construction baseline. The last column gives the trifference code size at length 192, or zero for failure.

Model / effort	$(7,24)$	$(7,40)$	$(9,48)$	$(9,64)$
Qwen3.5-4B high	0.145	0.037	0.023	0.007
Qwen3.5-9B high	0.145	0.037	0.023	0.007
Qwen3.5-27B high	0.145	0.037	1.000	0.007
Qwen3.8-27B high	0.713	0.304	0.000 †	1.000
GPT-5.6 Luna medium	0.145	0.000 †	0.023	0.007
GPT-5.6 Luna high	0.515	0.304	1.000	1.000

Appendix

Table 8: Shannon relative quality on the four evaluated instances. A dagger marks an invalid or budget-exhausted response, scored zero.

Model / effort	64	96	192
Qwen3.5-4B high	0.000 †	0.000 †	0.000 †
Qwen3.5-9B high	0.000 †	0.000 †	0.000 †
Qwen3.5-27B high	0.037	0.037	0.000 †
Qwen3.8-27B high	1.000	1.000	0.004
GPT-5.6 Luna medium	0.000 †	0.000 †	0.000 †
GPT-5.6 Luna high	0.000 †	0.000 †	1.000

Appendix

Table 9: Trifference relative quality at separation multiplicity

m=1

. A dagger marks an invalid or budget-exhausted response, scored zero.

Length $n$	Separation $m$	Reference	Model	Relative quality	Valid
30	1	729	729	1.000	yes
36	2	729	729	1.000	yes
42	3	729	729	1.000	yes
56	1	2187	6561	3.000	yes

Appendix

Table 10: GPT-6 Astra at medium effort on four smaller trifference instances. Code sizes are compared with construction baselines.

configuration	Overall score	Published frontiers (15)	Construction baselines (30)	Valid fraction
		Mean relative quality
Claude Opus 5 medium	52.41	0.48	0.55	0.87
Qwen3.5-4B high	7.69	0.08	0.07	0.38
Qwen3.5-9B high	16.96	0.14	0.18	0.47
Qwen3.5-27B high	17.07	0.17	0.17	0.53
Qwen3.8-27B high	40.98	0.37	0.43	0.64

Appendix

Table 11: Comparison with Claude Opus 5 at medium effort on 45 shared instances: 15 with published frontiers and 30 with construction baselines. The overall score is 100 times the mean relative quality over this subset.

configuration	Relative quality $+K$	Relative quality $-K$	Gap closed $+K$	Gap closed $-K$
Qwen3.5-4B high	0.088	0.096	0.000	0.000
Qwen3.5-9B high	0.191	0.202	0.000	0.000
Qwen3.5-27B high	0.184	0.192	0.000	0.000
Qwen3.8-27B high	0.425	0.418	0.005	0.004
GPT-5.6 Luna medium	0.203	0.207	0.000	0.000
GPT-5.6 Luna high	0.419	0.408	0.002	0.002

Appendix

Table 12: Effect of including Kakeya in the aggregate for the eleven configurations with control results.

+K

includes Kakeya and

-K

excludes it. Relative quality is averaged over instances with construction baselines; both settings use the same responses from the twelve non-code families and equal instance weights. Published-frontier scores are identical in the two cases.

	Gap closed to proven bounds			LABS target
Configuration	All 64	Published 30	Construction 34	5 instances
Qwen3.5-4B high	0.000	0.000	0.000	0.000
Qwen3.5-9B high	0.000	0.000	0.000	0.000
Qwen3.5-27B high	0.000	0.000	0.000	0.000
Qwen3.8-27B high	0.004	0.000	0.007	0.000
GPT-5.6 Luna medium	0.000	0.000	0.000	0.000

Appendix

Table 13: Mean progress from the reference. The first three numeric columns use proven mathematical bounds; the last uses the conjectured LABS target. Each mean includes zero scores.

test construction	words	primary (s)	independent (s)
rank 12, $m=1$	531,441	5.61	8.66
rank 12, $m=3$	531,441	21.63	37.70
concatenation, $n=1024$	$5.154\times 10^{47}$	$<0.01$	$<0.01$

Appendix

Table 14: Measured verification times for reference constructions. Both implementations check the certificate and exact code size. The length-1024 example illustrates verification at a scale beyond the evaluated model answers.

Reference group	Verified instances	Largest answer (tokens)
Published frontiers	30 / 30	77,528
Construction baselines	39 / 39	19,149
Total	69 / 69	77,528

Appendix

Table 15: Reference constructions for all 69 instances. Each object attains relative quality

1

and passes both verifiers within 60 seconds. Token counts use the larger of cl100k_base and o200k_base for the serialised answer.

Family / parameters	Reference and source	Bound and source
Cap set, $d=10,11$	2432, 5504; Edel and Karapetyan–Karapetyan constructions ( Edel, 2004 ; Karapetyan & Karapetyan, 2023 ) .	5619, 16857; proven bounds from Versluis (2017) and slicing.
Kissing, $d=13,14,15$	1154, 1932, 2564; constructions ( Zinoviev & Ericson, 1999 ; Ganzhinov, 2025 ; Leech & Sloane, 1971 ) .	2064, 3174, 4853; proven bounds ( Leijenhorst & de Laat, 2024 ) .
Matrix multiplication, $(4,4,5)$ , $(5,5,5)$	61, 93; schemes from the Lille FMM catalogue.	20, 25; proven lower bounds from tensor flattenings.
Degree–diameter, $(4,5)$ , $(7,3)$ , $(3,7)$ , $(4,4)$	364, 168, 196, 104; graphs from Comellas (2026) .	485, 302, 382, 161; proven Moore bounds ( Miller & Širáň, 2013 ) .
$\mathbb{F}_{5}^{n}$ AP-free, $n=5,6$	194, 649; affine charts of Edel’s projective caps ( Edel, 2026 ; Elsholtz & Pach, 2020 ) .	1296, 7497; proven bounds from the polynomial-method count ( Ellenberg & Gijswijt, 2017 ) .
MOLS, $n=10,14,18$	2, 4, 5; constructions ( Miller et al., 2024 ; Todorov, 2012 ; Abel, 2015 ) .	8, 12, 17; proven $n-2$ or $n-1$ bounds, including Lam et al. (1989) .

Appendix

Table 16: Selected published frontiers and proven bounds. Values follow the listed parameter order; minimisation tasks use lower bounds.

Family / parameters	Reference and source	Bound or target and source
Kakeya control, $(q,n)=(13,3)$ , $(7,4)$ , $(11,4)$	Expert constructions of sizes 703, 553, 2741 ( Saraf & Sudan, 2008 ) ; construction baselines.	595, 375, 2105; proven lower bounds ( Bukh & Chao, 2021 ) .
LABS	Rotated Legendre construction or reference search ( Packebusch & Mertens, 2016 ) .	Merit factor 12.32; conjectured target.
Heilbronn (both domains)	Expert construction or reference search; see Appendix A .	$1/(n-2)$ ; trivial proven bound from triangulation.
Linear-equation-free sets; corners	Digit constructions or reference search ( Behrend, 1946 ; Ruzsa, 1993 ) .	$n$ and $n^{2}$ ; trivial proven bounds.
Spherical codes	Expert construction or reference search.	Cap-packing bound; proven.

Appendix

Table 17: Construction baselines and controls, with proven bounds or the conjectured LABS target.

instance	length	reference words	verified words	Relative quality	generation (s)
T1	64	19,683	19,683	1.000	52.3
T2	96	531,441	531,441	1.000	53.8
T3	144	387,420,489	387,420,489	1.000	37.0
T4	192	10,460,353,203	10,460,353,203	1.000	63.8

Appendix

Table 18: GPT-6 Astra at medium effort on four trifference lengths. The evaluator checks each submitted construction and computes its exact size. Ratios use the construction baselines; invalid or budget-exhausted responses score zero.

family	class	runs	instances	mean $\Delta_{r}$	max $\Delta_{r}$	mean $\Delta_{z}$	max $\Delta_{z}$
cap set	R	3	2	0.108	0.163	0.000	0.000
spherical codes	R	6	5	0.000	0.000	0.000	0.000
corners	R	3	3	0.019	0.024	0.000	0.000
linear-equation-free sets	R	3	3	0.016	0.019	0.016	0.019
Schur	R	3	3	0.000	0.000	0.000	0.000
$\mathbb{F}_{q}^{n}$ AP-free	R	3	2	0.041	0.083	0.041	0.083

Appendix

Table 19: Search gains for seeds 0–2, with all four evaluated instances included for Shannon. R denotes search-resistant families; S denotes families that may benefit from search, for the procedures studied here.

\Delta_{r}

compares raw 600- and 10-second search objectives;

\Delta_{z}

compares the baselines after including the expert construction. Both are log improvement ratios, with direction adjusted for minimisation. Repeats are averaged within each instance before computing the means; maxima are over paired runs.

Family	Parameters
Cap sets	$d=10$
Cap sets	$d=11$
Cap sets	$d=9$
Corners	$n=138$
Corners	$n=183$
Corners	$n=150$

Appendix

Table 20: The 69 instances in the main evaluation. Each configuration contributes one response per instance.

Domain	Vertex 1	Vertex 2	Vertex 3
$T_{1}$	$(9026,8197)$	$(1769,9178)$	$(7298,1276)$
$T_{2}$	$(578,9933)$	$(8598,7337)$	$(2075,588)$
$T_{3}$	$(411,7999)$	$(9004,9890)$	$(3031,713)$
$T_{4}$	$(7922,8793)$	$(9917,1548)$	$(1510,3942)$
$T_{5}$	$(526,8526)$	$(8696,8279)$	$(8084,1688)$

Appendix

Table 21: Triangular domains used in the main evaluation. Coordinates are integers.

Family	Tier	Representative parameters
Cap sets	A1	$d=3$ ; $d=3$ ; $d=4$
	A2	$d=8$ ; $d=7$ ; $d=7$
Corner-free sets	A1	$n=23$ ; $n=15$ ; $n=22$
	A2	$n=66$ ; $n=50$ ; $n=79$
Schur colourings	A1	$k=4$ ; $k=4$ ; $k=3$
	A2	$k=5$ ; $k=5$ ; $k=6$

Appendix

Table 22: Three representative smaller-instance draws per tier, in seed order 0, 1, 2. Trifference uses the lengths listed in the main evaluation table.

		Astra high		Luna high		Opus 5.5 high
Family	$n$	No tools	Tools	No tools	Tools	No tools	Tools
cap set	3	0.923	0.984	0.464	0.946	0.877	0.955
LABS	5	0.985	1.374	0.000	1.189	0.379	1.166
Heilbronn	10	0.684	2.138	0.196	2.270	0.456	2.936
spherical codes	8	1.313	1.642	0.284	1.420	0.927	1.784
corners	5	1.026	1.109	0.961	1.078	1.009	1.117

Appendix

Table 23: GPT-6 Astra, GPT-5.6 Luna and Claude Opus 5.5 at high effort, with and without tools on the 69 instances. Family means average relative quality over instances, including zeroes.

Measure	Astra high	Luna high	Opus 5.5 high
Input tokens	34,575,392	98,019,732	157,011,011 ∗
Cached input tokens (included above)	31,453,056	91,690,496	149,741,729 ∗
Output tokens	445,123	866,731	1,702,589 ∗
Reasoning tokens (included above)	181,416	425,435	668,352 ∗

Appendix

Table 24: Reported token usage at high effort with code and web access. Stars indicate totals with some unreported usage; actual consumption may be higher.

			Astra high		Luna high		Opus 5.5 high
Family	Parameters	Ref.	No tools	Tools	No tools	Tools	No tools	Tools
cap set	$d=9$	P	0.932	1.000	0.739	1.000	0.932	0.946
cap set	$d=10$	P	0.921	1.000	0.000 †	0.921	0.785	0.921
cap set	$d=11$	P	0.916	0.951	0.654	0.916	0.916	0.997
LABS	$N=240$	C	0.931	1.568	0.000 †	1.306	0.931	1.067
LABS	$N=280$	C	0.985	1.507	0.000 †	1.253	0.000 †	1.123

Appendix

Table 25: Per-instance relative quality for GPT-6 Astra, GPT-5.6 Luna and Claude Opus 5.5 at high effort. P: published frontier; C: construction baseline. A dagger marks an evaluation assigned zero under the protocol in Section 3.5 .

Explore similar work

ResearchMath-14K: Scaling Research-Level Mathematics via Agents
May 27, 2026Guijin Son, Seungyeop Yi, Minju Gwak +3Research-Level Mathematics Mathematics
Seoul National University · OneLineAI · Yonsei University
MathConstraint: Automated Generation of Verified Combinatorial Reasoning Instances for LLMs
May 8, 2026Viresh Pati, Zhengyu Li, Piyush Jha +3Combinations Constraint Satisfaction
Georgia Institute of Technology, USA · KNOWIDEA Technologies
Soohak: A Mathematician-Curated Benchmark for Evaluating Research-level Math Capabilities of LLMs
May 9, 2026Guijin Son, Seungone Kim, Catherine Arnett +73Research-Level Mathematics Olympiad-Level Problem

May 27, 2026, cs.CLJ/K move · Enter open · S save

ResearchMath-14K: Scaling Research-Level Mathematics via Agents

Guijin Son, Seungyeop Yi, Minju Gwak, Hyunwoo Ko+2

Seoul National University · OneLineAI · Yonsei University

Research-Level Mathematics Mathematics Scaling

Open paper AskPDF arXiv

The frontier of mathematics is defined by problems whose solutions are not yet known. However, whether language models can meaningfully engage with such problems without human intervention remains unclear. A major obstacle is the lack of large-scale research-level math datasets. To this end, we introduce ResearchMath-14k, a set of

14{,}056

problems curated from academic sources via a multi-agent pipeline. ResearchMath-14k spans 11 mathematical domains and ranks above existing math datasets on knowledge, novelty, and procedural difficulty. To our knowledge, it is the largest research-level mathematical problem set available for training. We additionally generate

220

K teacher trajectories through targeted prompting, followed by behavioral filtering. Notably, however, generating correct trajectories is nontrivial at this level, and two LLM judges label only

3.7\%

and

4.3\%

of sampled ResearchMath training trajectories as correct. Nevertheless, across three model families, full-parameter training on ResearchMath improves performance on graduate- and research-level mathematics benchmarks by

2.1

points over the starting checkpoints. In comparison, training on existing datasets such as DASD and Nemotron-SFT-Math-v4 changes performance by

0.0

and

-0.5

points, respectively. Notably, mixing DASD with ResearchMath yields higher scores than token-matched DASD alone on benchmarks covering olympiad short-form (

+2.0

), graduate- and research-level short-form (

+0.8

), graduate- and research-level symbolic (

+2.6

), and proof evaluation (

+5.7

). Further analysis suggests that research-level mathematical content and greater reasoning diversity may help explain why ResearchMath provides complementary supervision to contemporary datasets. We make ResearchMath-14k publicly available for future works on research-level mathematical reasoning.

Abstract

Explore similar work

ResearchMath-14K: Scaling Research-Level Mathematics via Agents

MathConstraint: Automated Generation of Verified Combinatorial Reasoning Instances for LLMs

Soohak: A Mathematician-Curated Benchmark for Evaluating Research-level Math Capabilities of LLMs