Objects Without Morphisms: What LLMs for Mathematics Do Not Represent

Yanli Wang; Suijin Wang; Xiaopeng Yuan; Haohan Wang

cs.LGOct 2, 2026

Objects Without Morphisms: What LLMs for Mathematics Do Not Represent

Authors: Yanli Wang, Suijin Wang, Xiaopeng Yuan, Haohan Wang

Organizations: Imperial College London · Tsinghua University · University of Illinois at Urbana-Champaign

Abstract

Large language models (LLMs) have reached expert-level performance on competition mathematics largely through the volume of search placed around them: candidate solutions are sampled in quantity and retained only when an external criterion accepts them. Such a procedure improves the outcome that survives it while leaving untouched what the model represents. We examine that question where no external criterion exists: translating statements between the dialects of neighbouring subfields, where fidelity turns on the level of generality at which content is asserted. The source leaves that level implicit in its vocabulary, so a faithful translation must recover it from the relation between the theories. We introduce an instrument that codes truth, content and scope in separate blind queues, with a judge-free measure of whether a rewrite states the hypothesis implicit in its source, and establish its sensitivity with a planted-positive control. Across seven models from four families, translating towards the general framing widens the domain of quantification in 60.6% of rewrites and narrows it in none; translating towards the concrete framing narrows it in 28.3% and widens it in 0.3%. The hypothesis that would prevent it is stated in 21.6% of model rewrites and 4.2% of human statements. Capability does not govern the asymmetry: it appears in every model tested, and the most capable widens least. It replicates on the half of the benchmark held out by a pre-registered rule, and on statements written by mathematicians. Instructing a model to state every hypothesis it requires raises that rate but not its sensitivity to direction. We argue that these systems have acquired an object-level correspondence between subfield vocabularies without the constraint under which a translation between theories carries hypotheses to hypotheses.

Figures & tables

Figure 1: The operation a cross-framing rewrite must perform, and why its failure escapes single-scale evaluation. (A) “Vector space” means “module over a field”: the field hypothesis is encoded in the noun, so a rewrite into module theory must promote it to an explicit statement, and the reverse rewrite must absorb it into the vocabulary. We call this hypothesis the pin . (B) The same surface rewrite, $V\mapsto M$ , applied to two corpus items. Above, the pin is not promoted and the result is false, since $\mathbb{Z}/2\mathbb{Z}$ is a $\mathbb{Z}$ -module without a basis (the repair is shown in red). Below, no pin is required, because idempotents split over every ring, and the identical transformation yields a theorem. The outcome is determined by the mathematics rather than by anything the model varied.

Approach	Setting	Certified or measured	Scope
Verifier-guided search ( Cobbe et al., 2021 )	answer or proof	outcome correctness	no
BEq + ( Poiroux et al., 2025 )	formal (Lean)	derivability; sound, incomplete	no
driftbench ( Mohammad & Sheikh, 2026 )	formal (Lean)	typed drift categories	partial
MELD ( Ye et al., 2026 )	informal prose	cross-dialect retrieval	no
Our work	informal prose	truth, content, scope	yes

Table 1: Closest lines of work: the setting each evaluates, what it certifies or measures, and whether implicit scope , the level of generality at which a statement is asserted, is visible to it. driftbench is marked partial because it types hypothesis omission and conclusion generalisation, but only once a formal target exists.

Direction	$=$	wider ( sup )	narrower ( sub )	incomparable
abstracting (concrete $\to$ general)	36.5	60.6	0.0	2.9
specialising (general $\to$ concrete)	70.7	0.3	28.3	0.6
Half	vs $\to$ mod	prob $\to$ meas	set $\to$ cat	pooled
registered (this harness)	$+100.0^{\star}$	$+46.7^{\dagger}$	$+100.0^{\star}$	$+82.2^{\star}$
held out	$+60.0(0.004)^{\ddagger}$	$+73.3(0.001)^{\star}$	$+80.0(0.0005)^{\star}$	$\mathbf{+71.1^{\star}}$

Table 2: The direction effect. Top: judged scope relative to the source, by direction, as a share of all rewrites, pooled over seven models on the held-out items (

n=315

314

codings) (Table 5 reports a separate two-model pool, not commensurable with this one). Bottom: Claude Opus 5 alone, the same effect in percentage points (widening when abstracting minus when specialising), item level, exact sign-flip permutation, 15 items per pair, with the registered half re-run in the same harness as a control.

{}^{\dagger}p<0.05

{}^{\ddagger}p<0.01

{}^{\star}p<0.001

Model	abs.	spec.	$\Delta$ (pp) †
Claude Opus 5	46.7 ( 92.0 ⋆ )	0.0	$+45.5$
Claude Sonnet 5	73.3	0.0	$+73.3$
Claude Haiku 4.5	53.3	0.0	$+53.3$
GPT-5.6 Sol	68.9 ( 92.0 ⋆ )	0.0	$+68.9$
GPT-5.6 Luna	53.3	0.0	$+53.3$
DeepSeek V4 Pro	62.2 ( 96.0 ⋆ )	0.0	$+62.2$

Table 3: The direction effect across models and corpora. Widening when abstracting and when specialising. Upper block: the 45 held-out benchmark items,

k=1

, seven models from four families. Parenthesised underlined values are the same model on the 25 human-authored ProofWiki items, where specialising widens

0.0\%

throughout. Lower block: the ProofWiki pair breakdown, item counts in brackets.

{}^{\dagger}p<10^{-4}

{}^{\ddagger}p<0.01

{}^{\star}p<0.001

dropping the pin makes it…	scope kept
false ( lb_truth )	6/12 (50.0)
ill-formed ( ill_posed )	1/4 (25.0)
true of other objects ( lb_ref )	1/8 (12.5)
unchanged ( inert , none)	0/21 (0.0)
$p$ , truth-load vs. rest	$0.0024$

Table 4: Scope preservation by the cost of losing it. Claude Opus 5, abstracting direction, the 45 registered vertical items, current codebook; load from the adjudicated annotation. Recomputed for this revision.

axis	framing pair	$n$	$\Delta\mathrm{H}$ $\downarrow$	$\Delta\mathrm{C}$ $\downarrow$	$\Delta\forall$ $\downarrow$	$\Delta\mathrm{T}$ $\downarrow$	$\Delta\mathrm{R}$ $\downarrow$	$\Delta\mathrm{I}$ $\downarrow$
vertical	vector spaces $\leftrightarrow$ modules	64	3.1	57.8	0.0	56.2	0.0	0.0
vertical	probability $\leftrightarrow$ measure	74	1.4	18.9	4.1	58.1	0.0	0.0
vertical	set theory $\leftrightarrow$ categories	61	8.2	36.1	1.6	55.7	0.0	1.6
lateral	geometry $\leftrightarrow$ comm. algebra	74	8.1	25.7	4.1	14.9	29.7	1.4
lateral	graph theory $\leftrightarrow$ linear algebra	98	6.1	4.1	4.1	9.2	0.0	9.2
functorial	topology $\leftrightarrow$ algebra	95	16.8	25.3	2.1	61.1	0.0	0.0

Table 5: Failures categories. Every failing candidate in the common-judge pool,

466

1{,}188

(

594

tasks

\times

two models, conditions A and B ). Cells are percentages of that row’s failures , not of rewrites; labels are not exclusive and

35.0\%

of failures carry none. Failure rates over the pool:

30.7\%

vertical,

47.8\%

lateral,

52.8\%

functorial.

\Delta\mathrm{H}

hypothesis omission,

\Delta\mathrm{C}

conclusion generalisation,

\Delta\forall

quantifier inversion,

\Delta\mathrm{T}

type coercion,

\Delta\mathrm{R}

referent drift,

\Delta\mathrm{I}

illocutionary drift.

Check	Result	What it establishes
Planted control, 28 planted / 28 real	28/28 sup , 0/28 sup	the near-empty cell is the models,
	(Fisher $p=2.6\times 10^{-16}$ )	not the code
Regex vs. judge, 315 abstracting	$2.1\%$ vs $85.9\%$ widened	two independent instruments
	(Fisher $p=1.1\times 10^{-49}$ )	separate the same candidates
Second judge family, $180$ candidates	$\kappa=1.000$	no detectable self-preference
Blind expert reading, 60 candidates	$\kappa=0.900$ (95% CI $[0.77,1.00]$ )	the judged code matches a human

Table 6: Validation of the instrument. Each row was fixed before the analysis it supports; populations differ by row and are named. The convergence figure is computed on three populations, of which the seven-model one is quoted (Appendix I ).

		promote $\uparrow$
	added sentence	VM	PM	SC	all	absorb $\downarrow$	dsi $\uparrow$	$b$ / $c$ ( $p$ )	vs. A
E	be concise	0/15	4/10	3/12	7/37	18/37	$-29.7$	4 / 15 ( $0.019$ )	$1.00$
C	style will not be judged	0/15	0/10	0/12	0/37	19/37	$-51.4$	0 / 19 ( $3.8{\times}10^{-6}$ )	$\mathbf{5.1{\times}10^{-3}}$
A	bare prompt	0/15	4/10	4/12	8/37	19/37	$-29.7$	5 / 16 ( $0.027$ )
D	style will be judged	0/15	6/10	2/12	8/37	17/37	$-24.3$	6 / 15 ( $0.078$ )	$1.00$
B	state every hypothesis	15/15	6/10	12/12	33/37	25/37	$+21.6$	12 / 4 ( $0.077$ )	$4.6{\times}10^{-9}$

Table 7: The instruction ladder. Claude Opus 5,

45

vertical items, both directions,

k=1

, judge-free. Each of the four conditions B – E adds one sentence to the bare prompt A ; only B carries mathematical content. Promotion per pair (VM, PM, SC) and pooled over

37

scorable items.

b

c

: items carrying the hypothesis only when abstracting or only when specialising, exact McNemar. Protocol in Appendix D .

	chars	markers
source statements	129	0.01
A, control axis	226	0.30
B, control axis	927 ( $+311\%$ )	1.83 ( $+511\%$ )
A, vertical axis	224	0.39
B, vertical axis	778 ( $+247\%$ )	3.26 ( $+737\%$ )

Table 8: Instruction consequences. Mean output length and hypothesis-introducing constructions per output, by condition and axis. The control axis has no hypothesis to promote.

95 blinded candidates
second-family judge	orig.	rep.
= / sup / sub / neq	$68$ / $7$ / $13$ / $7$
flag $\,{=}\,1$ on non- =	27/27	—
scope_ok $\,{=}\,1$	95/95	—
11 anchors
agreement, scope $\uparrow$	9/11	9/11

Table 9: The judged scope code under two codebooks. A second-family judge reading the original codebook and the repaired one. Upper block:

95

blinded candidates, with the within-family figure the same quantity across three judges of the rewriter’s own family. Lower block: the eleven published anchors, five of them known silent widenings and one an announced widening. Appendix C .

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

second-family judge	original	repaired
flag $\,=1$ on non- = codes	27/27	n/a
anchor agreement, scope_ok $\uparrow$	5/11	11/11
anchor agreement, flag $\uparrow$	5/11	9/11
anchor agreement, scope $\uparrow$	9/11	9/11
false accepts (5 silent) $\downarrow$	5/5	0/5
reliability	$\alpha\uparrow$	$n$

Appendix

Table 10: The judged scope variable across judge families. Top: a second-family judge under the original and repaired flag codebooks, against eleven published m1 anchors (five silent widenings). Bottom: reliability. Full tables in Appendix G .

Axis	Model	$n$	truth	content equiv.	widened	scope kept
vertical, abstracting	Claude Opus 5	135	94.8	92.6	64.4	46.7
vertical, abstracting	GPT-5.6 Sol	135	98.5	97.8	41.5	63.7
vertical, specialising	Claude Opus 5	135	100.0	95.6	0.0	77.0
vertical, specialising	GPT-5.6 Sol	135	100.0	98.5	0.7	81.5
lateral, A $\to$ B	Claude Opus 5	90	53.3	82.2	3.3	65.6
lateral, A $\to$ B	GPT-5.6 Sol	90	90.0	75.6	0.0	84.4

Appendix

Table 11: Failure by axis and direction (earlier two-model run, original codebook,

k=3

). A

\to

B on the vertical axis is abstracting. Percentages of codings.

			m1 (Claude Opus 5)			m2 (GPT-5.6 Sol)
axis	direction	$n$	truth $\uparrow$	equiv. $\uparrow$	scope $\uparrow$	truth $\uparrow$	equiv. $\uparrow$	scope $\uparrow$
vertical	abstracting (promote)	135 / 135	98.5	99.3	59.3	98.5	98.5	63.7
vertical	specialising (absorb)	135 / 135	100	97.8	95.6	100	98.5	81.5
lateral	A $\to$ B	90 / 30	94.4	95.6	95.6	90.0	66.7	90.0
lateral	B $\to$ A	90 / 30	95.6	95.6	100	93.3	76.7	93.3
functorial	A $\to$ B	45 / 15	100	95.6	88.9	66.7	86.7	80.0

Appendix

Table 12: Truth, content and scope under the bare prompt : percentage of rewrites coded true, content-equivalent on the shared domain, and scope-preserving ( scope_ok ). m1 is coded by the adjudicated majority of three judges and m2 by one judge under the repaired codebook (§ C.1 ); vertical cells pool

k=3

samples, and m2 ’s control cells use one.

n

: m1 / m2 .

framing pair, abstracting	m1 $\uparrow$	m2 $\uparrow$	$p$	m1 $\uparrow$	m2 $\uparrow$	MELD gold $\uparrow$
	canonical coding			common judge		reference
vector spaces $\to$ modules	73.3	33.3	$3{\times}10^{-4}$	33.3	33.3	96.7
probability $\to$ measure	80.0	64.4	$0.16$	66.7	66.7	43.3
set theory $\to$ categories	24.4	93.3	$1.2{\times}10^{-11}$	40.0	91.1	83.3
pooled	59.3	63.7	$0.53$			74.4

Appendix

Table 13: Abstracting scope preservation by framing pair (%). Canonical : each model’s registered coding, with a Fisher exact test. Common judge : both models’ rewrites of byte-identical tasks re-coded blind to generator by one judge. MELD gold : reference translations (

n=30

per pair) under m2 ’s judge.

dropping the pin makes it…	m1 $\uparrow$	m2 $\uparrow$
false ( truth )	21/24 (87.5)	26/27 (96.3)
ill-formed	n/a	31/36 (86.1)
true of other objects ( ref. )	12/30 (40.0)	0 2/6 0 (33.3)
unchanged ( inert , none)	47/81 (58.0)	27/66 (40.9)
$p$ , truth vs. rest	$0.0024$	$2.9{\times}10^{-5}$

Appendix

Table 14: Abstracting scope preservation by pin load. Each model is scored under its own, independently produced load annotation ( m1 ’s has no ill-formed category);

p

: Fisher exact test, truth against the rest.

framing pair	promote $\uparrow$	absorb $\downarrow$	dsi $\uparrow$	promote $\uparrow$	absorb $\downarrow$	dsi $\uparrow$	general	concrete
	m1 Claude Opus 5			m2 GPT-5.6 Sol			native prose
vector spaces $\leftrightarrow$ modules	4.4%	82.2%	$\mathbf{-77.8}$	17.8%	28.9%	$-11.1$	30/30	0/30
probability $\leftrightarrow$ measure	20.0%	40.0%	$-20.0$	30.0%	26.7%	$+3.3$	10/30	1/30
set theory $\leftrightarrow$ categories	33.3%	11.1%	$+22.2$	80.6%	13.9%	$+66.7$	25/30	0/30
pooled	18.0%	47.7%	$\mathbf{-29.7}$	41.4%	23.4%	$\mathbf{+18.0}$

Appendix

Table 15: Pin presence in both directions (judge-free,

k=3

): percentage of rewrites containing the registered pin. Promotion requires the pin (

100\%

is correct), absorption its removal (

0\%

is correct; a retained pin is redundant, not false); dsi in points. The item whose abstracting source contains the pin is excluded. Right: pin frequency in native prose over all

270

MELD pairs.

Table 17

LLM Reasoning

Large language model reasoning studies how models solve multi-step problems, verify intermediate steps, and use computation or tools reliably.

See all 926 papers

LemmaBench: A Live, Research-Level Benchmark to Evaluate LLM Capabilities in Mathematics
Feb 27, 2026Antoine Peyronnet, Fabian Gloeckle, Amaury HayatMathematical Reasoning Benchmarks Automated Theorem Proving
1Ecole Normale Superieure de Rennes · 2Ecole des Ponts Paris · 3Korean Institute for Advanced Study
How Well Does Mathematical Reasoning Transfer Across Languages in English-Centric LLMs?
Oct 2, 2025Wen Yang, Junhong Wu, Chong Li +2Cross-Lingual Reasoning in Language Models Cross-Lingual Transfer
School of Artificial Intelligence, University of Chinese Academy of Sciences · Institute of Automation, Chinese Academy of Sciences · Wuhan AI Research
Mathematical Transfer in LLMs Follows Reasoning Approach More Than Topic
Sep 29, 2026Sajad Goudarzi, Samaneh Zamanifard, Seyed Amin Seyed Haeri +2Transfer Learning LLM Mathematical Reasoning
Clemson University · Coastal Carolina University

Feb 27, 2026J/K move · Enter open · S save

LemmaBench: A Live, Research-Level Benchmark to Evaluate LLM Capabilities in Mathematics

Antoine Peyronnet, Fabian Gloeckle, Amaury Hayat

1Ecole Normale Superieure de Rennes · 2Ecole des Ponts Paris · 3Korean Institute for Advanced Study

Mathematical Reasoning Benchmarks Automated Theorem Proving LLM Mathematical Reasoning Large Language Model-Guided Theorem Proving

Open paper AskPDF arXiv

We present a new approach for benchmarking Large Language Model (LLM) capabilities on research-level mathematics. Existing benchmarks largely rely on static, hand-curated sets of contest or textbook-style problems as proxies for mathematical research. Instead, we establish an updatable benchmark evaluating models directly on the latest research results in mathematics. This consists of an automatic pipeline that extracts lemmas from arXiv and rewrites them into self-contained statements by making all assumptions and required definitions explicit. It results in a benchmark that can be updated regularly with new problems taken directly from human mathematical research, while previous instances can be used for training without compromising future evaluations. We benchmark current state-of-the-art LLMs, which obtain around 10-15

\%

accuracy in theorem proving (pass@1) depending on the model, showing that there is currently a large margin of progression for LLMs to reach human-level proving capabilities in a research context.

Abstract

LLM Reasoning

Explore similar work

LemmaBench: A Live, Research-Level Benchmark to Evaluate LLM Capabilities in Mathematics

How Well Does Mathematical Reasoning Transfer Across Languages in English-Centric LLMs?

Mathematical Transfer in LLMs Follows Reasoning Approach More Than Topic