While neural theorem provers have achieved impressive milestones in formal mathematics, they largely operate on the assumption that faithful Lean 4 formal statements are already provided. Translating informal natural language into a formal language is a critical data bottleneck plagued by an "illusion of rigor": standard type-checkers accept statements that compile but drop hypotheses, introduce vacuous truths, or subtly alter mathematical bounds. To resolve this, we introduce Sage (Semantic Agent-Guided Formalization Engine), an agentic framework that replaces monolithic translation with a four-stage decomposed generation pipeline coupled with a dual-signal semantic correction loop. By pairing Lean 4 compiler diagnostics with multi-dimensional semantic feedback, our correction loop enforces mathematical fidelity alongside syntactic validity. By explicitly accounting for the gap between open-ended queries and declarative formal targets, our pipeline prevents models from achieving high formalization rates by guessing unverified answers (exhibiting a 70.9% answer leakage rate in monolithic baselines). Consequently, Sage suppresses leakage to 2.7% while achieving 73.3% pass@4 joint compilation and semantic fidelity on the Omni-MATH without proofs (compared to 42.0% for a fine-tuned Goedel-Formalizer-V2 baseline). Finally, on IMO-Unformalized, a novel frontier of 175 unformalized International Mathematical Olympiad problems, Sage demonstrates effective zero-shot generalization with 87.4% pass@4 verified fidelity compared to just 19.4% for the baseline, winning over 79% of blind pairwise evaluations.
Figures & tables
Figure 1: Overview of Sage : an autoformalization example ( 1(a) ) , the multi-stage generation chain ( 1(b) ) , and the syntax/semantic feedback loop ( 1(c) ) . Blue: LLM agents; orange: verifiers.
Strategy
Lean 4 Formalization Example ( SNL : Find an object x of type α that satisfies the property P )
Syntactic Validity?
Semantic Fidelity?
Preserves Texture?
Requires Oracle?
Explicit Answer
theorem target : P c := by sorry
Yes
Yes
No
Yes
Placeholder
def x : α := sorry theorem target : P x := by sorry
Breaks Read-Only
Yes
Yes
No
Existential
theorem target : ∃ x : α , P x := by sorry
Yes
Vulnerable
Yes
No
Table 1: The Formalization Feasibility Trilemma across Question Types.
Figure 2: Contextual Integrity Failure Modes.
Table 4
pass@1
pass@4
Method
Compile
Goedel-SM
Compile ∧ Goedel-SM
Compile
Compile ∧ Goedel-SM
IMO-Formalized ( N=223 , Established Benchmark)
Monolithic (Zero-Shot)
72.6
57.4
43.0
83.9
62.3
Goedel-Formalizer-V2 (Baseline)
76.7
44.4
37.2
90.6
50.7
Sage (Ours)
94.2
65.0
61.4
99.6
83.9
IMO-Unformalized ( N=175 , Unformalized Frontier)
Table 4: Formalization quality across the IMO landscape (Answer-Agnostic).
pass@1
pass@4
Method
Compile
Compile ∧ Goedel-SM
Compile
Compile ∧ Goedel-SM
Monolithic †
73.0
40.0
87.3
59.0
Monolithic + Dual †
95.7
55.3
99.0
75.0
Decomposed
62.0
29.7
85.0
50.7
Decomposed + Syntax
95.7
42.7
99.0
61.7
Decomposed + Dual ( Sage )
94.7
50.3
98.7
73.3
Table 5: Formalization quality on Omni-MATH (Answer-Agnostic). † Monolithic variants suffer from severe answer leakage ( >70\text{,}\mathrm{%}$$ , guessing unverified answers; see Table 6 ). Sage is the top-performing valid, leak-free method. Bold indicates best leak-free performance.
Method
Compile ( ↑ )
Goedel-SM ( ↑ )
Answer Leakage ( ↓ )
Multiple Sorries ( ↓ )
Monolithic
73.0
51.0
72.6
0.5
Monolithic + Dual
95.7
57.0
70.9
1.0
Decomposed
62.0
45.3
1.0
2.2
Decomposed + Dual ( Sage )
94.7
54.0
2.7
1.1
Goedel (Finetuned)
74.0
32.0
56.2
2.3
Table 6: Structural fidelity on Omni-MATH (Answer-Agnostic, pass@1).
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Agent
Input
Output
Description & Core Role
Distiller
SNL , PNL ?
Domain + Inferred Goal context
Identifies the mathematical domain and either extracts the Inferred Goal from the proof or reformulates the open query into a declarative statement.
Preprocessor
Distiller output
SDNF , local defs, hyps, G
Standardizes the problem into a unified Declarative Normal Form Statement statement ( SDNF ) and decomposes it into assumptions, local definitions, and goals.
Formalizer
SDNF components
Lean def s + theorem … sorry
Maps assumptions and infrastructure into idiomatic Lean 4 statements, relying on Mathlib concepts while avoiding trivialization.
Formatter
Draft code
Standalone .lean file
Organizes namespaces, imports, and scaffolding into a compilable file without altering the underlying mathematical meaning.
Appendix
Table 7: Sage generation pipeline. The sequential, role-isolated agents composing the generation chain. Each stage processes specialized mathematical components, ensuring clear translation contracts before generating the final compilable Lean 4 code.
Figure 3: Compositional generation walkthrough on Omni-MATH #043. The same informal question is processed under both regimes. With PNL , the Distiller extracts an Inferred Goal ( n=6 ) and the Preprocessor builds a Declarative Normal Form Statement claim with hypothesis h1 and goal G:n=6 . Without a proof, the Distiller commits to a declarative reformulation ( ∃!n ), which flows through to an existential Lean 4 goal.
Model
Role
Temp.
Max tokens
Retries
Qwen3.6-27B
Pipeline generation + in-loop rater
0.6
16,384
2
Goedel-Formalizer-V2-32B
Fine-tuned baseline ( Goedel )
0.7
16,384
1
Gemma4-31B
Post-hoc evaluation ( Goedel-SM / PWR )
0.6
16,384
2
Appendix
Table 8: LLM decoding and sampling hyperparameters.
Method
Base Model
Architecture
Feedback Type
Rounds ( T )
Monolithic Configurations
Monolithic
Qwen3.6-27B
Monolithic
None
1
Monolithic + Syntax
Qwen3.6-27B
Monolithic
Compile
5–10
Monolithic + Dual
Qwen3.6-27B
Monolithic
Compile + Sage-SM
5–10
Decomposed Configurations (Ours)
Decomposed
Qwen3.6-27B
Decomposed
None
1
Appendix
Table 9: Overview of evaluated methods and ablation variants. Iterative variants are evaluated under matching iteration budgets ( T=5 on Omni-MATH , T=10 on IMO-Unformalized ) on a shared base model to ensure strict test-time compute parity.
Dimension
Scale
Cutoff
Semantic Match
0 – 4
≥4
Missing
0 – 3
≤1
Extra
0 – 4
≤1
Wrong
0 – 2
=0
Exactness
0 – 3
≥1
Naturality
0 – 4
≥3
Appendix
Table 10: In-loop Sage-SM gate. Scales are those used by the Rater prompt; cutoffs are conjunctive (all must hold).
Method
Nrejection
Wrong Injection (%)
Mathematically False (%)
Existential (%)
Monolithic
147
69.4
84.4
8.2
Monolithic + Dual
129
68.2
80.6
15.5
Decomposed
164
8.5
22.0
72.0
Decomposed + Dual ( Sage )
138
7.2
18.1
76.8
Goedel (Finetuned)
204
56.9
74.5
19.6
Appendix
Table 11: Goedel-SM reject reasons on Omni-MATH (Answer-Agnostic) at pass@1. Three most common categories among Inappropriate samples (multi-hot; lower is better for defect categories). Bold = lowest rate.
Model Backbone
Compile
Goedel-SM
Compile ∧ Goedel-SM
Joint Count
Compile ∧ Sage-SM ∗
Witness Injection (%)
Qwen3-32B (32B)
57.3
46.3
30.3
91 / 300
52.3
22.0
Qwen3.6-27B (27B)
96.0
80.3
77.3
232 / 300
93.7
90.2
Qwen3.8-27B (27B)
98.0
86.3
85.7
257 / 300
95.7
89.5
Appendix
Table 12: Comprehensive generator backbone ablation on Omni-MATH (Answer-Aware, pass@1, N=300 ). ∗ In-loop optimization target. All methods use Sage with identical prompting, repair budgets ( T=5 ), and independent judge protocols. Joint solved counts reported out of 300 problems.
Benchmark
N
Accuracy (%)
Precision (%)
Recall (%)
F1 (%)
Matthews correlation
Majority baseline (%)
ConsistencyCheck ( Chen et al., 2026 )
859
85.7
88.0
90.9
89.4
0.67
66.4
ProofNetVerif ( Poiroux et al., 2025 )
3752
88.8
82.1
80.9
81.5
0.74
69.6
Appendix
Table 13: External evaluation of the Sage-SM five-core gate against gold faithfulness labels on ConsistencyCheck and ProofNetVerif. Predictor: published Pass/Fail gate; Pass is the positive class.
Preference for Sage over Goedel (%)
Evaluation Type
Much Better
Better
Tie
Worse
Much Worse
Semantic ( PWR )
78.3
1.1
17.7
1.1
1.7
Appendix
Table 14: Blind pairwise preferences on IMO-Unformalized (Answer-Agnostic, pass@1). Percentages may not sum to 100% due to rounding.
Most of mathematical knowledge has been communicated through so-called informal use of mathematics and natural language. With large language models (LLMs) being highly adept in using natural language, they achieve strong performance, yet not perfect, in informal mathematical reasoning. Restraining LLMs to informal reasoning misses out on the opportunity to use the discrete verification abilities that machines offer through machine-checkable proofs. In this paper, we bridge the gap between informal and formal reasoning by integrating Lean signals into the informal reasoning process. We introduce Magenta, a training-free agentic pipeline that, given only a natural-language problem, produces an answer, expresses it as a Lean 4 statement, and constructs a machine-checked proof. A statement judge verifies whether the formalisation preserves the original problem, while an error-attribution judge routes failed attempts either to mathematical re-derivation or local Lean repair. Magenta achieves 100% accuracy across all evaluated olympiad benchmarks, including AIME 2025, AIME 2026, and HMMT February 2026. When paired with the open-weight K2-Horizon-7B reasoner, it solves all six IMO 2026 problems. Our analysis shows that statement adjudication is essential for preventing false certificates and that feedback-guided correction outperforms independent resampling on difficult problems.
Joshua Ong Jun Leang, Haonan Li, Zheng Zhao +6
Institute of Foundation Models · Imperial College London · University of Edinburgh +1
Large Language Models (LLMs) exhibit strong informal mathematical reasoning but struggle to generate mechanically verifiable proofs in formal languages like Lean. We present LEAP, an agentic framework that enables general-purpose foundation models to achieve state-of-the-art performance on automated formal theorem proving. LEAP leverages foundation model capabilities, such as informal reasoning, instruction following, and iterative self-refinement. By decomposing complex problems into smaller units, the system bridges formal proof construction with informal blueprints through continuous interaction with the Lean compiler. To provide a rigorous evaluation beyond increasingly saturated benchmarks, we introduce Lean-IMO-Bench, a benchmark of IMO-style problems formalized in Lean, with short statements yet highly non-routine and multi-step proofs across a wide range of difficulty levels. Empirically, on the latest 2025 Putnam Competition, an annual mathematics competition for undergraduate students in North America, LEAP solves all 12 problems, matching recent breakthroughs by frontier formal mathematical models. On Lean-IMO-Bench, LEAP boosts the one-shot formal solve rate of general-purpose LLMs from below 10% to 70%, notably surpassing the 48% benchmark set by a specialized, gold-medal-caliber IMO system. Furthermore, we demonstrate LEAP's research-level utility by autonomously formalizing complex proofs for open combinatorial challenges, including a verified proof for a key subproblem in Knuth's Hamiltonian decomposition of even-order Cayley graphs.
Po-Nien Kung, Linfeng Song, Dawsen Hwang +10
Google Cloud AI Research · Google Cloud · Google DeepMind
While Large Language Models (LLMs) have demonstrated exceptional capabilities in mathematical reasoning, they frequently produce subtle errors that evade human detection. Formal mathematical languages like Lean 4 offer mechanical proof checking, strongly motivating the need for autoformalization: the automatic translation of natural language mathematics into verifiable code. Recent trends indicate that general-purpose LLMs, heavily optimized for standard programming, now outperform smaller models explicitly fine-tuned for Lean. Leveraging this shift, we introduce an agentic autoformalization framework powered by general coding LLMs. At the core of our system is an orchestrator that manages a multi-agent pipeline tailored for research-level mathematics. Because cutting-edge research frequently relies on concepts outside the scope of existing libraries like Mathlib, our system dynamically extends necessary type definitions and validates them via a novel Auxiliary Lemma technique before formalizing the primary theorems. We applied our approach to PutnamBench, producing machine-checked Lean proofs for a random sample of 32 problems. Furthermore, we evaluate our system on five papers from the ACM Symposium on Theory of Computing (STOC) spanning combinatorics, communication complexity, mechanism design, and learning theory, successfully formalizing their main theorems and validating the generated formalizations with human experts; for all five we also formalize the proofs alongside the statements, and notably two of them are proved with no axioms beyond Lean's kernel. All of our formalizations are available at https://beyondthelibrary.github.io/formal_arxiv .
Arshia Soltani Moakhar, Iman Gholami, Max Springer +2