Authors: Xiaokai Li, Ziyu Zhou, Qibin Cheng, Yangcheng Wu, An Zeng, Ying Fan
Organizations: School of Systems Science, Beijing Normal University, Beijing 100875, China · Microsoft Research Asia, Beijing 100080, China · School of Mathematics, Renmin University of China, Beijing 100872, China
As large language models increasingly generate mathematical proofs, whether they develop a proving style distinct from that of human mathematicians remains an open question. This question is particularly pressing as mathematicians increasingly voice concerns that proofs generated by large language models are difficult to interpret. Here we ask three large language models to independently reprove all 465 propositions of Euclid's Elements and compare their citation choices against the human-proof baseline through Jaccard similarity and complex network analysis. We find that AI-generated proofs differ from human proofs, and this difference is stable and reproducible: mean pairwise Jaccard similarity among AI runs (~0.68) is nearly twice that between AI and human proofs (~0.37), and cross-run classification consistency is also high. By examining citation structure in detail, we identify the concrete signature of this difference: approximately 40% of AI citations point to foundational elements (definitions, common notions, and postulates), compared with only 13% in human proofs; even among theorem-to-theorem citations, AI tends to cite more distant results. Roughly half of AI's extra citations trace to the upstream dependencies of the human proof. Together, these findings suggest that the AI proving style is a tendency to construct arguments on its own from foundational elements rather than directly citing later, more established results, which we term "building from the ground up". Our findings offer a new perspective on how AI systems organise and invoke mathematical knowledge, with implications for understanding the interpretability of AI-generated proofs.
Figures & tables
Figure 1: Experimental pipeline. The left panel shows the main pipeline for the three Claude Opus 4.6 runs, from data extraction through independent proving, automated classification (P/G/D/E), expert validation, error re-proving, and citation network construction. The right panel details the proving and classification protocols.
P
G
D
E
Err%
After AI check
Run A
22
317
115
11 (5)
2.37
Run B
18
320
119
8 (3)
1.72
Run C
24
295
136
10 (3)
2.15
After expert validation
Run A
22
319
115
9
1.94
Table 1: Classification results across three validation stages.
Figure 2: Descriptive statistics of proof citation networks and proof characteristics. a Six topological properties of the citation networks treated as undirected graphs: node count, edge count, density, average degree, average shortest path length (computed on the largest connected component), and average clustering coefficient. Each dot represents one network; colors identify the six proof sources throughout the figure. b–d Complementary cumulative distribution functions (CCDFs) of in-degree, out-degree, and total degree for all six networks. In-degree and total degree are plotted on log-log axes; out-degree is plotted on a semi-logarithmic axis (linear x, logarithmic y). e CCDF of proof length measured in words. AI proof lengths are counted from the step-by-step proof text, excluding citation metadata; human proof lengths are counted from Heath’s English translation. Semi-logarithmic axes. f CCDF of wall-clock proof duration for the five AI runs (no human counterpart), plotted on log-log axes.
Figure 3: Pairwise Jaccard citation similarity across AI runs and human references. Kernel density estimates of the Jaccard similarity between citation sets, computed for each of the 465 propositions. Blue curves show AI–human similarity (each run versus human references; mean ≈0.37 ); red curves show intra-AI similarity (pairwise between runs; mean ≈0.68 ). Vertical solid and dashed lines mark the group mean and median, respectively. The clear separation between the two groups indicates that AI runs agree with each other substantially more than they agree with human references. Cross-model results, including those for Fable 5.1 and GPT-6 Astra, are shown in Supplementary Tab. S10 and Supplementary Tab. S11.
3-run overlap
≥ 2-run overlap
Category
Total
Count
%
Count
%
P
64
45
70.3
57
89.1
G
934
723
77.4
873
93.5
D
371
201
54.2
303
81.7
E
26
0
0.0
8
30.8
Overall
1395
969
69.5
1241
89.0
Table 2: Cross-run classification consistency.
Figure 4: Classification distribution and path-divergence rate across human proof complexity. a–c Violin plots show the distribution of three human proof complexity proxies—proof length, reference count, and available tool count—across expert-verified classification categories (P, G, D, E; three independent runs pooled, n=1,395 ). Red and dark lines indicate the mean and median, respectively. Categories with greater divergence from human proofs (D, E) tend to have higher complexity proxies than those with closer agreement (P, G), indicating that proof complexity is associated with AI–human divergence. d–f Path-divergence (D) rate computed within binned intervals of each complexity proxy. Propositions are grouped into bins of width 100 words, 2 references, and 50 available tools, respectively, with the final bin widened to capture the tail. In each line plot, the horizontal coordinate of each point corresponds to the midpoint of its bin, and the vertical coordinate represents the proportion of D-classified propositions within that bin. The dashed grey line marks the overall D rate. D rate increases with all three proxies, indicating that AI proof paths diverge more frequently on complex propositions. Results for the Fable 5.1 and GPT-6 Astra models are available in Supplementary Fig. S2 and Supplementary Fig. S3, respectively.
Figure 5: Citation structure and human-ancestor overlap of AI proof paths. a Proportion of citation types (theorem, definition, common notion, postulate) in human and AI networks. AI proofs cite a lower share of theorems and a higher share of definitions than human proofs. b–c Citation distance distributions for full and theorem-only networks. Citation distance is the sequential index difference between a proposition and its cited reference. AI distributions shift rightward (full network: AI mean = 111.2 vs Human mean = 86.3), indicating that AI proofs reach further back in the deductive chain. d–e Distribution of the proportion of AI-only citations that appear among the ancestors of human-only citations (the transitive closure of the human-only citation set) for each D-class proposition. f–g Distribution of the proportion of AI-only citations that appear among all human ancestors (the transitive closure of the full human citation set). On average, each D-class proposition has 47% of its AI-only citations falling within the human ancestor set, indicating that many AI path divergences represent recombinations of earlier underlying deductive material rather than direct citations of later mature propositions built on the same material. Similar results for the Fable 5.1 and GPT-6 Astra models are shown in Supplementary Fig. S4 and Supplementary Fig. S5, respectively.
Full Network
Theorem-Only
Metric
AI-Only
Human-Only
AI-Only
Human-Only
All D-class pairs
Sample size
377
352
343
351
Mean node count
6.33
2.87
3.76
2.68
Has internal edges
76.9%
29.5%
51.6%
27.4%
Longest path ≥ 2
28.4%
7.4%
17.5%
6.6%
Table 3: D-class AI-Only and Human-Only subgraph statistics
Mathematical understanding depends on the route from hypotheses to conclusion: how the parts of a proof function and which dependencies carry the argument forward. This structure can persist when wording or notation changes, while a small alteration to a hypothesis or inference can produce a different argument beneath similar prose. Formal verification can establish that a rewrite proves the same theorem, but not whether it follows the same mathematical route. Comparing informal proofs, especially those produced or revised by generative models, therefore requires reconstructing which claims, dependencies, and reasoning steps are preserved, reorganized, or replaced. We present ProofDiff, a system for comparing informal mathematical documents. It is designed to support human review, not to certify mathematical correctness. Given two documents, ProofDiff independently constructs a source-grounded argument map for each and proposes correspondences between the resulting maps. Each map captures the document's claims and reasoning steps and records the typed relations that connect them. An integrated human-review layer allows reviewers to revise the maps and correspondences while preserving an auditable history. Comparing the two argument maps provides a basis for inspecting whether the documents follow the same mathematical route and where their claims or dependencies differ. We demonstrate ProofDiff on four illustrative pairs of AI-generated proofs, ranging from a detailed proof and its concise revision to proofs using different mathematical strategies.
Matan Tsafrir, Esdras Medeiros, Jorge Poco +3
New York University · Universidade Federal do Ceará · Fundação Getulio Vargas +1
The growing use of AI in mathematics raises a question: can we detect AI involvement in proofs, separately from AI assistance with writing? Historical arXiv papers, recent papers disclosing substantive AI use, and manuscripts released by OpenAI provide a rich empirical basis for studying this question. However, several challenges remain: lack of matched case-control pairs; potential incompleteness of disclosure; and lack of ready-to-use features designed to capture signals of AI involvement in proofs. To address these challenges, we construct five cohorts comprising 8,281 papers by combining existing manuscripts with those we produced using generative AI. These cohorts span different levels of AI involvement and include matched pairs of papers addressing the same research problems. We then introduce ProofTrace, a framework that represents each paper as a graph of immediate proof support (GIPS) and extracts interpretable features of mathematical organization. Our study yields three discoveries. First, principal-component projections of our features reveal a qualitative gradient of AI involvement, where AI-reworded papers remain close to human papers, the two AI-generated cohorts depart in broadly shared directions, and papers disclosing substantive AI use occupy intermediate positions which shifts toward AI-generated cohorts over time. Second, we identify recurring signatures of AI-generated proofs, such as fewer externally cited results, greater reliance on intermediate conclusions, and altered graph connectivity; many of these signatures persist under controls for research field, manuscript length, or graph size. Finally, in a separate detection experiment, ProofTrace distinguishes human-written papers from those disclosing substantive AI use. Its performance remains stable under AI rewording, whereas the performance of state-of-the-art text-based detectors declines sharply.
Large language models (LLMs) are increasingly capable of mathematical problem solving and can even assist with research-level proofs, yet we still lack a scalable and reproducible way to measure step-level reasoning in long proofs across diverse sources. This evaluation gap limits trustworthy AI assistance in proof-certified scientific progress. Existing evaluations often emphasize final answers or rely on costly expert grading, while end-to-end proof generation remains open-ended and hard to verify automatically. We introduce Mask-Proof, a pipeline that turns real proofs into automatically checkable masked-step tasks. It masks key formula steps, provides the necessary surrounding context, and evaluates model reconstructions with an LLM-based equivalence judge using repeated votes for stability. The resulting Mask-ProofBench contains 292 curated problems across diverse research areas. Experiments with 17 models show that reasoning-enhanced models outperform standard models by 12% to 27%. Our evaluator achieves 96.8% agreement with expert annotators, enabling faithful, reproducible, and comparable measurement of step-level mathematical reasoning. Benchmark, annotations, and code are available at https://github.com/weating/Mask-Proof.
Jierui Zhang, Siyuan Tan, Xinhang Li +8
School of Computer Science Beijing University of Posts and Telecommunications Beijing, China · Graduate College for Engineers Beijing University of Posts and Telecommunications Beijing, China · School of Mathematical Sciences Fudan University Shanghai, China +6