daVinci-Prover: Agent-Native Mid-Training for Formal Theorem Proving
Organizations: Shanghai Innovation Institute · Shanghai Jiao Tong University · Generative Artificial Intelligence Research Lab (GAIR) · Fudan University
Abstract
Advances in large language models (LLMs) are rapidly transforming formal theorem proving. Yet further progress remains constrained by the lack of a transparent and effective data recipe for systematically improving theorem-proving models. In this work, we argue that effective agentic provers require strong mathematical skill priors to ground theorem proving and agentic trajectories to enable long-horizon, interactive proof construction. To this end, we synthesize an agent-native 33.7B-token mid-training corpus covering decomposition, search, self-verification, and revision, together with verifier-grounded agentic trajectories. We further introduce the daVinci-Prover harness, which enables models to construct proofs and interact with a Lean verifier in an isolated workspace, and use it to collect approximately 36K high-quality agent trajectories for supervised fine-tuning. Our mid-training recipe yields an average improvement of 13.7 points over the matched Qwen3-8B SFT-only baseline across MiniF2F, FATE-M, and ProofNet. When applied to the Qwen3.5 series, our approach also yields strong performance across theorem-proving benchmarks and achieves near-saturating MiniF2F performance with memory compaction. The results highlight agent-native mid-training as an effective foundation for developing stronger interactive theorem-proving agents.
Figures & tables
| Category | # Samples (M) | # Toks. (B) | Share (%) |
|---|---|---|---|
| Pre-Replay Data | 8.131 | 28.6 | 84.8 |
| Self-verfication | 0.083 | 5.70 | 16.9 |
| Theorem Decomposition | 4.662 | 8.43 | 25.0 |
| Proof Revision | 1.027 | 3.84 | 11.4 |
| Proof Search | 1.962 | 3.45 | 10.2 |
| Environment-Aware Agent Trajectories | 0.397 | 7.17 | 21.3 |
| Model | #Params | MathOlympiad | MiniF2F | ProofNet | ProverBench | Putnam |
|---|---|---|---|---|---|---|
| Open-Weights Reasoning Models ( pass@32 ) | ||||||
| DeepSeek-V3.2 | 671B-A37B | 14.7 † | 77.9 † | 20.4 † | 42.8 † | 5.8 † |
| Kimi-K2.5 | 1T-A32B | 7.5 † | 76.6 † | 19.9 † | 44.3 † | 1.2 † |
| Open-Weights Prover Models ( pass@32 ) | ||||||
| DeepSeek-Prover-V2-7B | 7B | 7.9 † | 73.9 † | 18.1 † | 45.7 † | 1.5 † |
| DeepSeek-Prover-V2 | 671B-A37B | 13.9 † | 82.4 † | 30.5 † | 52.9 † | 3.3 † |
| Model | MiniF2F | FATE-M | FATE-H | Putnam |
|---|---|---|---|---|
| Qwen3.6-35B-A3B | 68.0 | 48.7 | 4.0 | 3.0 |
| Qwen3.6-35B-A3B (w/ ret.) | 73.0 | 57.3 | 5.0 | 3.4 |
| DeepSeek-V4-Flash | 90.2 | 94.7 | 19.0 | 18.9 |
| DeepSeek-V4-Flash (w/ ret.) | 91.3 | 99.3 | 29.0 | 22.8 |
| Model | MiniF2F | FATE-M | FATE-H |
|---|---|---|---|
| w/o memory compact | |||
| Qwen3.5-midtrain-9B | 90.1 | 93.3 | 16.7 |
| Qwen3.5-midtrain-35B | 90.1 | 93.7 | 16.3 |
| w/ memory compact | |||
| Qwen3.5-midtrain-9B | 97.1 | 96.7 | 28.0 |
| Qwen3.5-midtrain-35B | 97.1 | 97.3 | 42.0 |
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
| Benchmark | Overlaps | Total | Rate |
|---|---|---|---|
| miniF2F | 16 | 244 | 6.56% |
| MathOlympiadBench | 9 | 360 | 2.50% |
| ProofNet | 2 | 186 | 1.08% |
| DeepSeek-ProverBench | 1 | 325 | 0.31% |
| Putnam | 1 | 672 | 0.15% |
| FATE-H/M/X | 0 | 350 | 0.00% |
| Tool | Category | Description |
|---|---|---|
| file_read | File | Read workspace files or line ranges. |
| file_write | Edit | Write complete workspace files. |
| file_edit | Edit | Replace text spans or line ranges. |
| file_search | Search | Find files by glob patterns. |
| content_search | Search | Search file contents by text or regex. |
| lean_retrieve | Retrieval | Retrieve relevant Mathlib declarations. |
| Hyperparameter | Qwen3.5-9B | Qwen3.5-35B-A3B |